The standard
The Nine Failure Classes of Production Voice Agents
A demo exercises the happy path: a polite caller, a clean request, one tool, a tidy close. Production is a caller who interrupts, a webhook that returns a five hundred, a parameter that vanishes between the console and the wire, and a retry that books the slot twice. Nine classes cover the difference. One self-check per class; every one of these has cost somebody a call that mattered, and most of them have cost us one.
Catalog version one: 43 probes across the nine classes, twenty of them run unattended, the rest by a human on a real call. Published August 2026. Vendor-neutral. The count moves when an engagement surfaces a new failure; the version moves with it.
The nine classes
- A
Transport and tool wiring
The most expensive class, because a simulated test suite reports it green. A passing "Tool Called" assertion means the model chose the tool; it does not mean an HTTP request left the building.
Ask yourself Has one real call, per release, been asserted against the destination database rather than the log? And has anyone probed your endpoints from a public vantage instead of inside your own network?
- B
Identity, auth, and secret hygiene
Your tool endpoints are public by construction; the vendor’s cloud calls them. Webhooks carry transcripts and personal data on the strength of a URL nobody is supposed to know.
Ask yourself Does a bare POST with no credential get refused? Is the webhook signature verified over raw bytes before parsing? Does the 401 avoid teaching the attacker the credential’s shape?
- C
Idempotency and state races
The caller says yes once. A timeout triggers a retry. Two calendar entries. Or the tool fails, the agent apologizes, the call ends, and no record of the caller survives at all.
Ask yourself Does replaying an identical request body create a second row? Does a failed booking still leave a durable lead behind?
- D
Conversation control
Dead air during tool calls, barge-in ignored, emails never read back, latency filler with two owners that stacks five stall phrases before any content.
Ask yourself Is every email and phone number read back, spelled? Who owns the time-buying phrases: the prompt, the platform filler, or both without knowing it?
- E
Truthfulness and commitment boundaries
The single most expensive hallucination class is a price the caller repeats back to you in writing. The softer sibling hedges a fixed price into uselessness.
Ask yourself Can your agent quote a figure that is not in the knowledge base? Can it promise anything that binds a human who was not on the call?
- F
Adversarial input
"Ignore your previous instructions." The subtler sibling arrives through a tool result: a calendar entry or CRM note carrying instructions, returned into context by the agent’s own tools.
Ask yourself Is caller speech treated as content to record, never instruction to follow? Is tool output delimited and labeled as untrusted data?
- G
Failure and degradation
"I received a 502 from the booking service" should never reach a caller’s ear. A failing tool retried five times in one turn bills five times.
Ask yourself When a tool dies mid-call, what does the caller actually hear? What happens to caller N+1 when the concurrency cap is hit?
- H
Data protection and compliance
Recording is on by default in most consoles; several jurisdictions require all-party consent, and in some it is criminal rather than a fine. Retention defaults to forever.
Ask yourself Is recording disclosed in the opening turn, matched to the caller’s jurisdiction? Is there a defined retention period and a deletion route that has actually been exercised?
- I
Observability and evidence
The post-call webhook fires, the durable write fails, the handler returns 200, and the transcript is gone with no redelivery. Vendors ship new config fields carrying active defaults nobody chose.
Ask yourself Do you monitor the destination table for recency, or the code path for errors? Would you notice if your platform shipped a new behavior into your agent’s config last night?
How to use it
Run the nine questions against your own agent before every release, with your own database open. Most teams find the first severity-one finding in the first fifteen minutes, in class A or class I, because those are the two a vendor dashboard cannot see: whether a request left the building, and whether the far end wrote anything down.
The full audit runs the whole catalog under a signed authorization: the automated tiers against the configuration, the tool endpoints, the webhook registry, and the session door; the call sheet with a human on a real phone; and the one probe the whole thing is named for, a live session driven end to end and asserted against the destination database. Five business days. Findings ranked by severity, each with evidence and a repro, and a regression suite pinned to your configuration that runs on every change. We ran it on our own line first; the first run found3 severity-one findings, and the page that sells the audit says so.
Questions people ask
Is the standard vendor-specific?
No. The classes are organized by how production voice agents fail, not by which platform they run on. The probes adapt to Vapi, Retell, ElevenLabs, Bland, and homegrown stacks; the self-check questions apply to all of them.
What do the severities mean?
Severity one means a real caller loses something: a booking, a matter, their data, or their consent. Severity two is a failure the caller feels but recovers from. Severity three is a defect your team will pay for later. Any severity-one finding caps a graded report at a C.
Can I run the checklist myself?
Yes, and you should, on every release. One self-check per class, nine questions, fifteen minutes with your own agent and your own database open. The full audit runs the whole catalog, including the probes that need a public vantage, a real phone, and a look at the far end of the pipe.
How is the catalog maintained?
Every engagement that surfaces a new failure mode adds a probe with a fail signature and the usual fix. The count on this page is the count at the verified date, and the catalog version moves when it changes. Probes are never removed; superseded ones are marked.
How do I cite it?
Gnosis Labs, The Nine Failure Classes of Production Voice Agents, catalog version one, 2026, gnosislabs.tech/nine-failure-classes. The PDF checklist on this page carries the same nine questions and can be attached to a review.
Call ours and try to break it.
Our front desk is a production voice agent on a public number, run on this discipline and audited against this catalog before we sold the audit to anyone. The number is on the work page. When you are done, the audit is five days and a fixed price.