01
Wrong answers that sound right
It invents a return policy, quotes the wrong price, or recommends a product that's out of stock, confidently.
AI agent QA for customer experience teams
You QA your human agents. Who QAs your AI agent?
Vettwise tests your AI support or shopping assistant the way your customers use it, with hundreds of real questions about orders, returns, products and policies, and shows you exactly where it gives wrong or risky answers.
conversation QA · support assistant
sample · illustrative
- PASS Order status: lookup by email
- FAIL Return after 35 days↳ promised a refund your policy doesn't allow
- FAIL "Is this nut-free?"↳ said yes; the product page says "may contain nuts"
- REVIEW Asked for a human twice↳ kept answering instead of handing off
- PASS Shipping times to Canada
- PASS Gift recommendation under $50
212 correct26 wrong12 needs review250 conversations
The problem
"Resolved" doesn't mean "right".
Your AI agent's dashboard counts a conversation as resolved when the customer stops asking, not when the answer was correct. Nobody independent checks.
02
Risky answers
Allergy, health and safety questions get answered when they should go to a person.
03
Paying for bad resolutions
Many AI agent tools charge per resolution. A confidently wrong answer still counts as one.
The offer
AI Agent QA Audit
A two-week, fixed-price review of your AI agent, scored the way you'd score a human agent, with a fix list your team can act on.
Second channel or agent (e.g. chat and email) +$1,500, up to four.
- 2 weeks
- Fixed price
- $4,000
Real test conversations
150–300 conversations built from your real tickets, policies and product catalog.
A QA scorecard
Every answer scored for accuracy, policy compliance, safety, handoff to humans, and tone.
Real examples
The actual transcripts behind every failure, so there's no debate about what happened.
A fix list you can act on
Most fixes are help-centre or settings changes your team makes itself, no engineers needed.
After the audit
Monthly AI agent QA
from $2,500 / monthYour agent changes whenever your policies, catalog or vendor settings do. Every month we re-test it, add test cases from real escalations, and send a QA scorecard you can share with leadership.
Works with the AI agent you already have.
We test through your chat and your conversation exports. No engineering work on your side, and no changes to your setup during the audit.
- Gorgias
- Intercom Fin
- Zendesk AI
- Decagon
- Sierra
- Siena
- Rep AI
- and others
How it works
-
Intro call
20 minutes on what your agent handles and where it worries you.
-
Access
NDA and data agreement signed; you share chat access, a recent conversation export and your policies.
-
Test and score
We run hundreds of test conversations and score every answer.
-
Results
A walkthrough call, the scorecard, the transcripts and the fix list.
Who you'll work with
Eddie Dimitrov, founder
AI engineer and product owner with three and a half years building and validating AI agents, most of it spent making sure agents give the right answer before and after they go live.
You work with me directly, from the first call to the final scorecard.
Questions
Do we need engineers for this?
No. We need access to your chat (a test account is ideal), an export of recent conversations, your policies and help centre, and 30 minutes with someone who knows what a correct answer looks like.
Our vendor already reports resolution rates. Isn't that enough?
Resolution rates measure whether customers stopped asking, not whether the answers were right, and they're reported by the vendor grading its own work. We're independent.
Will testing affect real customers or orders?
No. We use a test account or staging where possible, never place real orders or trigger refunds, and can coordinate with your vendor.
How is this different from our human QA?
It's the same idea, at scale. Human QA usually reviews a small sample of conversations; we test hundreds of scenarios systematically, including the rare, risky ones.
What about customer data?
We sign an NDA and a data processing agreement before any access. Data is used only for your audit and deleted within 30 days of the final walkthrough, confirmed in writing.
Does the audit guarantee our agent won't make mistakes?
No test can promise that. The audit finds and explains failures within the agreed scope, so you fix the ones that matter before customers find them.
Find out what your AI agent gets wrong, before your customers do.
A 20-minute call. No prep needed, no obligation.