User satisfaction can improve by changing an assistant's behavior under uncertainty while leaving recognition accuracy unchanged.
2
A cost model should treat a wrong action, a refusal, and a confirmation as different user burdens.
3
Adding stop and confirm behaviors produces thresholds that can be optimized from outcome costs instead of guessed from intuition.
Summary
Amit Desai argues that teams often focus on improving voice accuracy while ignoring a separate decision: what the assistant should do when it is uncertain. In his smart-speaker example, the system correctly identifies 79 percent of requests, but it can still reduce user cost by choosing among acting, stopping, and confirming. A wrong song is expensive because the user must hear it, interrupt it, and ask again. A refusal costs less. Desai turns these differences into a cost function he calls the Outcome User Cost Heuristic, or OUCH. With a stop option, the best confidence threshold in his example is 43 percent, rather than an intuitive 65 percent. Adding confirmation creates two thresholds and reduces the modeled cost again. The same method applies to televisions and other interfaces, where screen controls and current user context change the cost of each outcome.
Voice errors become more expensive when assistants take actions
Desai begins with the appeal and limits of voice. Speaking is natural, but voice systems remain error-prone. The consequences become more serious as assistants move from answering questions to taking physical or digital actions. Playing the wrong song is annoying, while a robot throwing a watch away would be much worse. He makes the same point for digital actions such as making a phone call or sending an email. As systems gain the ability to act, accuracy alone does not describe the user's experience. The system also needs a policy for what to do when its interpretation is uncertain.
Accuracy and system behavior are separate controls
Desai describes two independent ways to improve satisfaction. The familiar one is accuracy, which teams improve across wake-word detection, speech recognition, natural-language understanding, intent classification, entity extraction, and related layers. The second is the system decision made after a hypothesis is produced. In his example, accuracy stays fixed at 79 percent. The assistant can still change how it handles uncertain requests. Desai says this second control is underused and can apply across devices and surfaces. The system does not need to understand more often to make better choices about when to act.
A refusal option reduces the cost of uncertain actions
Desai models 1,000 music requests, with 790 correct songs and 210 wrong songs. The original assistant always acts, so every wrong interpretation produces a bad result. He adds a stop behavior: the assistant can say, "Sorry, I didn't get that" or ask the user to repeat the request. This introduces a confidence score and a threshold. Below the threshold, the system stops. Above it, the system plays the song. The extra behavior leaves the 79 percent accuracy unchanged, but it gives the assistant a way to avoid some costly wrong actions.
Outcome costs produce a better threshold than intuition
Desai says bad outcomes are not equally harmful. If the wrong song plays, the user must hear enough to notice, interrupt the assistant, and request the song again. He assigns that outcome a cost of 10 seconds. A refusal costs 4 seconds because the user can repeat the request. These values turn threshold selection into a cost-minimization problem. Always acting costs 2.1 OUCH points per request in the example. A guessed threshold of 65 percent lowers that to 1.9, but the calculated optimum is 43 percent, which lowers it to 1.27. The improvement comes from behavior and cost modeling, not better recognition.
Confirmation adds a third behavior and two thresholds
Desai then adds confirmation between stopping and acting. The assistant might say, "Play Kiss by Prince" or "Play Kiss by Chris Brown," and the user can affirm or correct it. The confidence range now has three regions: stop, confirm, and act. The costs differ within the confirmation region. Saying yes costs less than saying no and restating the request, so Desai assigns example costs of 2 and 6 seconds. With the wrong-action, stop, and confirmation costs set to 10, 4, 2, and 6, the example's optimal thresholds are 41 and 49. The modeled cost falls to 1.26.
Real systems should learn the decision in real time
Desai is clear that the example is simplified. A production system will not usually rely on one or two fixed offline thresholds. It will use a real-time learned decision model. The principle remains the same: identify possible outcomes, estimate their relative user costs, and choose behavior that minimizes the total. The outcome values can change when the situation changes. A wrong action may be especially costly in one context, while a confirmation may be cheap in another. The assistant's decision policy therefore needs to account for the surface, the available interaction methods, and the consequences of acting.
A television changes the costs because it has a screen
Desai applies the method to television voice assistants. A request such as opening a channel can use the same stop, confirm, and act structure, but the costs change because television is multimodal. Instead of asking verbally, "Did you mean ABC News Live?", the system can display choices and let the user select one with a remote. That can make confirmation less painful. Launching a channel may also remove the user from their current screen, which makes a wrong action more expensive. The same decision framework applies, but the interface and context alter the values used in the cost function.
"There are two ways to improve customer or user satisfaction of a voice AI assistant, and that is by increasing accuracy and the other is a different knob that we have that we are not using adequately."02:17
Who should watch
You are building a voice assistant that already has acceptable recognition accuracy but still frustrates users when it is uncertain.
Your system can take physical or digital actions, and a wrong action costs much more than a short clarification.
You design multimodal assistants for televisions or other devices where screens, remotes, and current UI state change the cost of confirmation.
