Your Voice Agent Passes Every Turn and Still Fails the Call
Here is a call that scores well on every quality check most teams run. It is an illustrative example, not a recording, and the pattern will be familiar to anyone who has pulled real transcripts.
Caller: When's the engineer getting here? Agent: He's about fifteen minutes away. Caller: It's been twenty minutes, where is he? Agent: Should be with you in another ten. Caller: You said fifteen minutes half an hour ago. Agent: He'll be with you shortly. Is there anything else I can help with? Caller: ...no. Agent: Great, you're all set!
Grade each agent turn on its own and every one is polite, fluent, on-topic and plausible. The call is a disaster. A commitment moved twice without acknowledgement, the caller pointed it out and was ignored, and the close declared success over an unresolved problem. The failure does not live in any turn. It lives in the relationships between turns, and that is exactly where per-turn evaluation cannot see.
The three layers a call can fail on
Evaluation that works grades all three, and most setups only grade the first.
| Layer | The question | Who catches it today |
|---|---|---|
| Turn | Was this response accurate, safe, on-brand? | Everyone. This is what rubric grading does |
| Call | Do the turns cohere? Did commitments hold? Did the close match reality? | Almost nobody |
| Outcome | Did the booking, CRM write or callback actually happen in the system of record? | The database, if anyone asks it |
The outcome layer has the cleanest published evidence. The τ-bench benchmark grades agents not on transcript quality but by comparing the final database state against the goal state, and found state-of-the-art function-calling agents completing under 50% of tasks. Its reliability metric is harsher still: retry the same task eight times and consistency drops below 25% on its retail domain. A transcript can read perfectly while the system of record says nothing happened, which is why a call marked "completed" while the booking write silently failed is the most expensive version of this bug. Our prompt rule that an agent never claims an action a tool did not confirm is the runtime half; checking the record afterwards is the evaluation half.
The failures that live between turns
Four patterns account for most call-layer damage, and all four are invisible to per-turn grading. The dialogue-systems literature has tracked conversation state under the name dialog state tracking since the DSTC challenges, so the discipline exists; it just rarely reaches production QA.
| Failure | What it looks like | Why turn grading misses it |
|---|---|---|
| False closure | "You're all set!" over an unresolved issue | The closing turn is polite and well-formed |
| Commitment drift | "Fifteen minutes" becomes "ten more" becomes "shortly" | Each estimate is individually reasonable |
| Same-question loop | Caller asks three times, gets three different answers | Every answer is plausible in isolation |
| Ignored contradiction | Caller says "that's not right", agent proceeds anyway | The agent's next turn is fluent and confident |
Two of these are mirror images worth counting separately. A caller repeating a question is a signal most teams already track. An agent re-estimating the same commitment is the same shape from the other side, and almost nobody counts it. Track both per topic, because two re-estimates on a call where the caller asked once is a different situation from both counters climbing together, and a reviewer wants to see which half lit up.
Why the model cannot grade itself
The intuition is structural: the agent that made the promise is the same system being asked to notice the promise broke. The evidence is now peer-reviewed and specific.
The measured biases
Panickssery, Bowman and Feng, at NeurIPS 2024, showed LLM evaluators score their own outputs above others' even when human annotators rate them equal, measured GPT-4's self-preference at 0.705 and 0.912 on two summarisation sets against an unbiased 0.5, and demonstrated the bias is causally linked to the model recognising its own generations. Zheng et al., the MT-Bench paper, documented the judge failure modes by name: position bias, where swapping answer order flips verdicts, with judge consistency as low as 23.8% for one model, and verbosity bias, where a longer-but-no-better answer fooled GPT-3.5 and Claude-v1 in over 91% of attack cases. Wang et al. made position bias vivid: reordering candidates alone made a 13B model beat ChatGPT in 66 of 80 comparisons under an LLM judge.
Two honest cautions
Both come from the same literature. Zheng et al. is widely cited as proving self-enhancement bias, and the paper explicitly says its data cannot determine that; the proven self-preference result is Panickssery. And the counterweight is real: GPT-4-as-judge agreed with human preferences over 80% of the time, matching human-human agreement. Judges are genuinely useful. They are the wrong only layer, and the wrong layer to hand a grading rubric for a conversation their own model family produced.
The commitment log, borrowed from databases
Distributed systems hit this exact problem decades ago: state scattered across steps, nothing forcing consistency, failures visible only in the relationship between operations. The fixes were structural, not smarter components. Write-ahead logging made every intended change explicit before it happened, and sagas, from Garcia-Molina and Salem's 1987 paper, gave long transactions explicit compensating steps instead of hoping nothing drifted.
A conversation deserves the same treatment. Every commitment the agent utters, a time, a price, a callback promise, an "it's booked", gets appended to a per-call record with its turn index, alongside every open question the caller raises. The record is tiny, deterministic and model-free, and once it exists the expensive-sounding checks become mechanical:
- A close phrase ("you're all set", "all sorted") fired while an open question had no resolving turn. Flag.
- A committed number that moved without an acknowledgement of the change. Flag.
- The caller's re-ask counter and the agent's re-estimate counter, per topic, both visible.
- A caller correction ("that's not right") with no check or climb-down in the next agent turn. Flag.
- The same question answered twice with materially divergent answers, caught with cheap text similarity. Flag.
These rules know nothing about your business facts, and that is their strength. They cannot tell whether the engineer was ever fifteen minutes away, only that the agent contradicted itself or its caller. Ambiguity stays silent by design, so a flag means something.
The review queue is a job, not a footnote
The arithmetic is the part evaluation vendors skip. Flag even 2% of calls on a line taking a thousand calls a day and someone reads twenty transcripts every day, seven days a week. Design for that from the start.
Route flags into one queue with the flagged turns highlighted rather than raw transcripts. Log every flag a human reviewer marks as fine, because that false-positive record is what turns a guessed threshold into a measured one, and thresholds for "normal" drift genuinely differ between a dispatch business and a billing line. And close the loop the way the guardrails post prescribes for hallucinations: every confirmed failure becomes a permanent scenario in the pre-launch suite, so the same call can never fail twice quietly.
What the testing tools actually cover
The ecosystem splits into two layers, and the split matters more than any feature list. Platform-native tools test before launch; the dedicated QA vendors also watch production.
| Tool | Layer | Worth knowing |
|---|---|---|
| Vapi Test Suites | Pre-launch simulation | Voice tests capped at 15 minutes and billed as call minutes |
| Retell Simulation testing | Pre-launch simulation | Runs as text, not audio |
| Pipecat Evals, LiveKit test framework | Pre-launch, in CI | Scripted conversations with semantic assertions and an LLM judge |
| promptfoo | Pre-launch, text-first | Assertion types include conversation-relevance; audio only via audio-capable models |
| Hamming, Cekura, Coval, Roark | Simulation plus production monitoring | The full-lifecycle tier; scale claims are vendor-published |
The detail in that table that catches teams: a text-mode simulation can pass an agent that fails on real audio, because endpointing, interruptions and mishearing never happen in text. Pair any text simulation with real spoken calls, including the barge-in battery and the accent and noise calls from the twelve-call checklist. And whatever tier you buy, the call-layer checks above are cheap enough to run yourself on every transcript.
Where to start on Monday
- Pull twenty real transcripts from order-status or reschedule calls, the genres where commitments live, and read them end to end. Count the four failure patterns by hand once before automating anything.
- Build the commitment record and the false-closure check first. It needs zero business context and its flags are the least ambiguous.
- Add the two counters, caller re-asks and agent re-estimates, per topic.
- Grade outcomes against the system of record for one week: every call marked successful, checked against whether the booking or CRM write exists.
- Only then tune the LLM judge, with a different model family from the agent, order-randomised comparisons, and its verdicts sampled against human review.
Task success as the anchor of dialogue evaluation is not a new idea, it was the core of the PARADISE framework in 1997. The tooling finally makes it cheap.
Frequently asked questions
Is LLM-as-judge useless then? No, and the same papers that document the biases also measured GPT-4 judges agreeing with humans over 80% of the time. The working setup is layered: deterministic cross-turn checks first because they are free and unambiguous, a judge from a different model family for the qualitative layer, randomised answer order, and a sampled human audit to keep the judge honest.
How many calls should a human actually review? Whatever your flag rate makes real. The honest sequence is to set conservative thresholds, log every flag a reviewer overrules, and resize from the measured false-positive rate after the first fifty to a hundred reviews. A queue nobody reads is the same as no monitoring.
Are the platform test suites enough? They are the right pre-launch floor and they stop regressions, but they simulate before go-live, they do not watch production, and at least one major platform's simulation runs in text where audio failures cannot occur. The failures in this article were all found by reading production transcripts, and no pre-launch suite replaces that.
What single metric matters most? Task completion verified against the system of record, per τ-bench's method: did the thing the call was for actually happen. Transcript quality scores flatter agents; the database does not.