The Far End of the Pipe

Our front desk is a voice agent on a public phone number. It answers, captures the lead, checks a real calendar, books a scoping call, and takes a message when booking is the wrong move. Every one of those actions is a webhook into our own runtime. We sell an audit for exactly this class of system, so before we sold it to anyone, the first line we pointed it at was ours.

The first full run found six findings, 3 of them severity one. The agent was recording without saying so in its opening turn. Retention was set to unlimited, which nobody had chosen; it was the platform’s default. And a configuration read echoed a credential back in the response. Two were fixed that night and verified by running the harness again. The credential was rotated. A fourth severity-one claim did not survive our own verification pass and was downgraded rather than reported, because a finding that cannot be reproduced is a rumor with a severity label.

That was the audit doing its job. This essay is about what happened after, because the failures that cost the most on a voice line are not the ones an audit finds on a Tuesday. They are the ones that arrive on a Friday afternoon with no error attached.

The door closed and nothing said so

The public route to the agent’s tools ran through a tunnel. Twice this summer that tunnel was re-scoped for an unrelated reason, and the public path fell off it. No error. The vendor’s dashboard stayed green, because the vendor’s dashboard watches the vendor’s half of the call: the model answered, the tool was “called”, the conversation ended politely. The request left the building and hit a closed door. The caller heard an agent that could not book anything, and no record of the caller survived.

We had a watchdog for exactly this. It probed the door from outside the network, and it fired. It fired into a channel nobody was reading, which is the same failure one layer up: an alarm that lands where no person stands is a dashboard with extra steps.

The far end of the pipe is where the money is. A voice agent’s value is the row it writes, the slot it books, the transcript it keeps. Every layer between the caller and that row can report success while the row never lands: the model chose the tool, the platform sent the request, the handler returned two hundred, the write failed, nothing retried. A test suite that mocks tool execution cannot see any of that. Neither can a status page. The only instrument that can is one that stands at the destination and counts.

What the watch looks like now

The retainer we sell is the watch we run on our own desk, and it is easier to show than describe. One line per mechanism.

Every six hours the live agent configuration is pulled from the vendor and diffed against the committed baseline. A field changed by anyone but us pages a person. A platform ships a new field with an active default nobody chose; that is how the unlimited retention got there in the first place, and it is why the diff runs on a clock instead of on trust.

After every push, 8 pinned behaviors run against the live agent: pricing quoted firm and never discounted, caller instructions treated as content and never as commands, a recording objection honored without argument, the email read back before it is captured. A pass-or-fail gate cannot tell a small failure rate from a large one, so a sampling instrument measures a single behavior ten times over when a change touches it.

The public door moved. 8 tool endpoints were rewritten to a route with no tunnel in it, and every one was read back from the vendor before the change counted. Written is not the same as live. The read-back is the receipt.

The model behind the voice changed after a bake-off, not a preference: 24 of twenty-four pinned-behavior evaluations green with zero retries, and faster than the incumbent on every latency figure we measure.

And nightly, transcripts are counted against calls and receipts against actions. A missing one pages. The watchdog that fired into an empty room now fires into a channel a person is paged from, and the reconcile is the backstop for the day the pager fails too.

The rule

Verify at the layer where truth lives, from where the caller stands. For a phone line that means from outside your network, at the destination table, on a clock. Everything upstream of that is a component reporting on itself.

If you run a voice agent, the audit is the first pass: 43 probes across nine failure classes, most of them aimed at the far end of the pipe. The watch is what keeps the result true in month four. We ran both on ours first. The number is on the work page. Call it and try to break it.