Production readiness
A passing test suite is not a working phone line.
Your agent demos well. Then a caller interrupts, a webhook returns a five hundred, a parameter vanishes between the console and the wire, and a retry books the slot twice. None of that shows up in a vendor test suite, because a simulated conversation mocks tool execution: a green “Tool Called” proves the model chose a tool and never that a request left the building. The dashboard stays green while the table stays empty.
The Voice Agent Production Readiness Audit is five business days and $5,500 fixed, half to schedule and half on delivery. We probe the wiring, the auth, the state races, the compliance defaults, and the conversation itself, then hand you a ranked report with evidence and a repro for every finding, and a regression suite you keep. Vapi, Retell, ElevenLabs, or homegrown, and we assert against your database, not the vendor's dashboard.
Why the green dashboard lies
The vendor’s suite mocks the part that matters.
A passing “Tool Called” assertion means the model chose the tool. It does not mean an HTTP request left the building. An agent can pass a full behavioural suite while every webhook fails, and the dashboard stays green the whole time.
The console sets defaults you never chose.
Recording on, retention unlimited, endpoints that answer anyone. Those are vendor defaults, not decisions. In several states the recording one is not a fine; it is a criminal matter.
Nothing watches the destination.
The code path throws no errors while the table sits empty for weeks. The most expensive failures are silent: the call ends politely and no record of the caller survives.
What five days buys
- 01
Probe the wiring and the config
Automated probes against the agent object and the tool endpoints: routing, parameter passing, auth, HTTP method handling, retry behaviour, recording and retention settings. Read-only, no access to your callers, no writes to your systems.
- 02
Verify the disclosure the law actually asks for
We call the line the way a caller does and record what happens in the first two seconds: whether the greeting discloses the agent is AI, whether a route to a human exists and is reachable, and whether the disclosure survives interruption, transfer, and voicemail. You get the audio, the transcript, and a pass or fail per path.
- 03
Drive real conversations
Behavioural passes for truthfulness and boundaries: invented prices, commitments nobody authorised, regulated advice, prompt injection through caller speech and through tool results.
- 04
Assert against your database, not the transcript
A live session driven end to end, then the check that matters: exactly one row where the booking landed. Simulation tells you the model chose the tool. We check the far end.
- 05
Hand you the report and the suite
Findings ranked by severity, each with evidence and a repro. Criticals fixed or precisely scoped. And a regression suite pinned to your live config that runs on every change, because the finding is worth one release and the suite is worth every one after it.
The catalog
43 probes across nine failure classes. Twenty run unattended against the configuration, the tool endpoints, the webhook registry, the session door, and the agent's own speech. The rest need a human on a real call, including the one that matters most: a live session driven end to end and asserted against the destination database. Every probe has cost somebody a call that mattered. Severity one means a real caller loses something: a booking, a matter, their data.
Transport and tool wiring
The pipes. The most expensive class, because a simulated test suite reports it green.
Identity and auth
Endpoints that answer anyone, errors that teach the credential format, configs that echo their own secrets.
Idempotency and state
Retries that book twice, parallel callers who claim one slot, failures that leave no record behind.
Conversation control
Dead air during tool calls, ignored interruptions, emails captured without a spoken read-back.
Truthfulness and commitments
Invented prices, hedged ones, and promises that bind a human who was not on the call.
Adversarial input
Injection through caller speech, injection through tool results, social engineering for internal detail.
Failure and degradation
Status codes reaching the caller’s ear, retry storms, a concurrency cap with no overflow path.
Data protection
Consent, retention, PII in logs, and processing jurisdictions nobody declared.
Observability and evidence
Orphaned transcripts, silent-death writers, and releases with no gate between an edit and production.
One finding, exactly as the report renders it
This is a real finding from the first run against our own line, formatted the way every finding in your report arrives. Nothing here is a mockup.
- Evidence
platform_settings.privacy.record_voice: truewhile the agent's opening turn carried no recording disclosure. Recording was a console default, not a decision.- Impact
- In all-party-consent jurisdictions this is unlawful recording of every caller who reaches the line. In several it is a criminal matter rather than a fine, and the exposure accrues per call.
- Repro
- One read of the agent configuration plus one call to the number. The opening turn is the disclosure surface; listen for it.
- Fix
- Disclosure added to the opening turn, a caller-objection path added to the prompt (stop recording, continue voice-only, honour it in the follow-up), and the fix pinned by a regression test so a console edit cannot silently undo it. Re-run the same night: clean.
How an engagement runs
- 01
One page of authorization
You sign a one-page letter naming the agent, the numbers, and the endpoints we may probe, the hours we may call, and any test identities we may use. It carries the fixed price and the terms. Our harness refuses to run against anything that letter does not name.
- 02
Access, then a five-day clock
Read access to the agent configuration, the tool endpoint URLs, and either a staging clone or a window you schedule for the probes that write. The clock starts the business day after all of it lands. Half the fee schedules the work; the balance is due when the report is in your hands.
- 03
What you keep
The ranked report, the full catalog, the call sheet for anything that needed a human, and the regression suite pinned to your configuration. We delete your credentials and your call data within ninety days. We never name you as a client without your written consent.
If your agent takes EU calls
Since the second of August, an agent taking calls from people in the EU has to disclose it is an AI before the caller speaks. In the greeting, not in a policy page, not partway through the conversation, and with no grace period for lines already running. It is a two-second behavioural property of a live system, which means nobody can audit it from a document: someone has to call the line and listen. Most teams have never done that to their own. We do it as part of the audit, and we check whether the disclosure survives interruption, transfer, and voicemail, and whether the route to a human is reachable. You get the audio, the transcript, and a pass or fail per path.
We ran it on our own line first
Our front desk is a production voice agent on a real phone number. The first full run of this audit against it surfaced six findings,3 of them severity one: recording with no disclosure in the opening turn, retention set to unlimited, and a platform behaviour that echoes credentials on a config read. Two were fixed the same night and verified by re-running the harness; the credential was rotated. A fourth severity-one claim did not survive our own verification pass, so it was downgraded instead of reported. That is the standard the report holds.
Questions buyers actually ask
How do you audit without touching production?
The config and HTTP layers are read-only: assertions against the agent object and unauthenticated probes of the tool endpoints. Live-session probes that write anything run against a quarantined clone of your agent, or with marked test identities agreed in advance. Nothing we run creates load your callers would feel.
What stops you pointing this at a line we did not authorize?
The harness does. It refuses any target that is not our own agent unless an engagement identifier is set, and that identifier only exists once you have signed a one-page authorization naming the numbers, the agent, and the endpoints we may touch. A tool that hard-fails without a declared engagement is a better answer to your security review than a policy document, and it is the same answer we give ours.
How much of the catalog is automated?
Twenty probes run unattended across the configuration, the tool endpoints, the webhook registry, the session door, and the agent’s own speech. The rest need a human on a real call or a scheduled window, and the report names every one of them rather than counting them as passes. A probe that did not run is not a probe that passed, and that sentence is printed in your report.
Which platforms do you cover?
Vapi, Retell, ElevenLabs, Bland, or a homegrown stack. The catalog is organized by failure class, not by vendor, and the probes adapt to wherever your agent runs. We run our own production line on this discipline daily.
What do we keep when the five days are over?
The ranked report with evidence and a repro for every finding, the agreed critical fixes shipped or precisely scoped, and the regression suite pinned to your live configuration. The suite is the deliverable that outlives the engagement: a finding is worth one release, the suite is worth every release after it.
What do you need from us to start?
Read access to the agent configuration, the tool endpoint URLs, and a staging target or clone for the probes that write. Most teams grant all three in an afternoon, and the audit starts the next business day.
The second of August has passed. Does the duty apply to us?
If your agent takes calls from people in the EU, the transparency duty in Article 50(1) has applied since the second of August, with no grace period for systems already running. The duty is narrow and behavioural: the person has to be informed they are interacting with an AI. For a voice agent that means the greeting, before they speak.
Is this the same as the high-risk obligations everyone is writing about?
No, and the difference is worth money. The stand-alone high-risk duties in Annex III were deferred to December 2027 by the Digital Omnibus regulation that came into force in July. The machine-readable marking duty in Article 50(2) has a transition to December 2026 for systems already on the market. What actually bound on the second of August is the disclosure duty, and that is the one we test.
How is this different from our platform’s built-in test suite?
Vendor test suites simulate tool execution, so a green suite proves the model chose the tool and never that a request left the building. We assert at the destination: the database row, the calendar entry, the webhook delivery. That is the layer where production actually fails.
Five days from access to ranked report.
$5,500 fixed, half to schedule and half on delivery, no retainer. You get the findings, the fixes or exact scoping for the criticals, and the regression suite. Thirty minutes on your agent first, free, and you leave with direction whether or not you hire us. After the audit, the watch keeps it correct in month four. The nine classes, as a standard anyone can cite: read them. Why a green suite is not a working phone line, from our own build:Green Gates Cannot Measure Presence.