Trust is the main barrier to deploying voice agents, because autonomous systems are flexible but hard to predict at production scale.
2
Large-scale simulation, probabilistic evaluation, and continuous regression testing can make voice-agent development more reliable, as they did for autonomous vehicles.
3
Teams should match simulation realism and evaluation metrics to the behavior they need to test, then combine automated evaluation with targeted human review.
Summary
Brooke Hopkins compares voice-agent reliability with the problems Waymo faced in autonomous driving. In both cases, the system interacts with a changing real-world environment, so fixed tests and manual review become expensive and brittle. She argues that large-scale simulation, reference-free metrics, regression suites, CI/CD checks, release evaluation, and live monitoring can create a continuous feedback loop for conversational systems. The right level of simulation depends on the test: text may be enough for workflows and tool calls, while realistic voices matter for accents, background noise, and audio quality. Hopkins also explains how to calibrate LLM judges against human labels before running them across thousands of conversations. Her advice is practical: define what the product must do well, benchmark the components on your own data, create task-level and end-to-end tests, and decide how production failures enter future test sets. She is also candid that manual evaluation still matters for human judgment calls.
Trust is the main reason voice agents remain stuck before production
Hopkins says voice agents promise to automate difficult workflows, but enterprises hesitate because they do not trust them with customer-facing work. Teams can overestimate what agents can automate immediately while underestimating what a narrower system can do well today. One response is to force the agent through a deterministic path, which becomes an expensive IVR tree. The other is to give it autonomy and flexibility, which makes its actions unpredictable. Hopkins calls this a false choice. She argues that voice agents can combine autonomy with reliability if teams build the right evaluation infrastructure.
Large-scale simulation replaced brittle manual and scenario-based evaluation at Waymo
Hopkins describes an evolution in self-driving evaluation. Engineers first drove cars through streets manually, recorded failures, and sent them back to developers. They then created specific scenario tests with exact expected behavior. Those tests were expensive to build and maintain, and they became obsolete quickly. The industry moved toward large-scale evaluation, which measures how often a type of event occurs across many simulations. This approach gives a more durable view of system performance than asking whether one specific scenario produced one expected result.
Conversational systems need simulations because every response changes the next test
Self-driving and conversational agents both interact with an environment one step at a time. In a conversation, the next response differs depending on whether the user gives a name, an email address, or something else. Static tests cannot cover those branching interactions well, and manual testing is expensive. Simulation can generate many possible paths and provide broad coverage. Hopkins says the non-determinism of language models is useful here because repeated runs can reveal the range of user responses and estimate how often an agent succeeds.
Reference-free and probabilistic metrics scale better than exact expected outputs
Traditional prompt evaluation often compares outputs against a golden data set for a fixed input. Hopkins says conversational evaluation should also ask broader questions: how often does the agent resolve an inquiry, repeat itself, or say something it should not say? These reference-free metrics apply across many scenarios instead of requiring a precise expected answer for each one. Teams can then measure probabilities across repeated simulations rather than treating a single pass or failure as definitive.
Continuous evaluation makes production changes safer
Hopkins outlines a loop used in autonomous vehicles and adapted for voice agents. An engineer reproduces a bug, changes the system, reruns focused evaluations, and then runs a larger regression set to check for new failures. Presubmit and postsubmit CI/CD workflows test changes before and after deployment. Release evaluation checks whether the system is ready to launch, while live monitoring feeds new failures back into the test system. Automation provides speed and scale, but Hopkins says manual evaluation remains necessary for judgments that require human understanding.
Simulation realism should match the behavior under test
A simulation does not need to reproduce every detail of the real world for every test. Hopkins recommends controlling the variables that matter for the question being asked. Workflows, tool calls, and instruction following can often be tested with text because it is faster and cheaper. Basic voices can test interruptions, latency, and pauses. Hyperrealistic voices become useful when testing accents, background noise, audio quality, or reproducing a production issue. The test should include the components that affect the behavior under evaluation.
Repeated runs reveal whether a failure is systematic or acceptable
A single failed conversation does not always mean an agent is unusable. Hopkins recommends taking the failed scenario and simulating it repeatedly to estimate its failure probability. A scenario that fails 99 out of 100 times is a clear defect. One that fails once out of 100 may be acceptable for a particular application. Teams therefore need to decide what reliability target applies to each part of the product, much as cloud systems have different reliability goals.
An evaluation strategy starts with the product's actual priorities
Hopkins says evaluation is part of product design because the metrics define what the agent is supposed to do well. Latency matters greatly for outbound sales, where a person may hang up, but it may matter less to someone waiting for a refund. Appointment booking may require strict instruction following and complete details. An interviewer or therapist may need metrics for conversational behavior instead. Teams should choose metrics for their workflow rather than relying only on general model scores.
LLM judges need calibration against human feedback
A judge that simply asks whether a conversation was successful can be noisy. Hopkins says the same conversation may receive different judgments across repeated runs. Coval's Metric Studio lets teams compare automated metrics with human feedback and iterate until the two align closely enough for the intended use. Once calibrated, the metric can be applied to many more conversations than the small set humans labeled. The amount of labeling depends on the reliability confidence the team needs.
"The goal is not to automate all evals, but rather to leverage auto evals for speed and scale and then use the manual time that you have to really focus on those very human touch judgment calls."07:47
Who should watch
You are deploying a voice agent and need a way to catch regressions before customers encounter them.
Your current evaluation process relies on a few scripted conversations or manual checks that do not reflect production behavior.
You are deciding when to use text simulations, realistic audio, automated judges, and human review.
