Voice is a high-bandwidth way to express intent, while visual responses give AI systems a more forgiving time window than voice responses.
2
Voice-in, visuals-out products need a fast model on latency-focused infrastructure, with larger tasks handed off asynchronously when needed.
3
Frequent inference turns and stable prefix caching help keep responses within the time users can follow and make inference more affordable.
Summary
Allen Pike argues that voice is a strong input method because people speak faster than they type and communicate meaning through tone, while visuals are a natural output because people can process rich information on screen. Forestwalk Labs built a voice agent that can hear an incidental request in a call and file a Linear issue within a second. Pike says latency is the main obstacle. Around 100 milliseconds feels instant for computers, one second is close to the limit before people lose their train of thought, and voice-to-voice conversation needs about 200 milliseconds or less. Voice-in, visuals-out relaxes that constraint because an on-screen response within a second can still feel responsive. His practical advice is to use a small, fast model, send inference every one or two seconds while the user speaks, and keep most of the prompt prefix unchanged so caching reduces cost and latency.
Voice carries more information than typing for human communication
Pike says speaking is the ultimate way humans communicate because people produce more words per minute than they type and convey more with each word. The difference between saying "okay" plainly and saying it with a different delivery can change what the listener understands. When communication matters, people move to a call or an in-person conversation because speech provides higher bandwidth. He applies that idea to AI interfaces, where voice can express intent quickly and naturally without requiring the user to formulate a carefully typed prompt.
A voice agent feels natural when it acts on intent without interrupting the conversation
Forestwalk Labs built an agent that participates in calls and helps in real time. Pike describes mentioning a possible Slack integration bug to his co-founder. After she said she had seen the same issue, Pike said, "Okay, well, let's file that as a Linear issue." The voice agent replied within a second that it had filed the issue. Pike says the experience works because the user can speak incidentally or deliberately, while the agent responds without interrupting. The important behavior is action on intent, rather than making the user conduct a separate voice conversation.
Latency determines whether an AI interaction still fits the user's attention
Pike describes latency as the largest barrier to making these systems feel good. He cites about 100 milliseconds as the response time that feels instant for a computer. In some products, a full second is acceptable, but beyond that people can lose their train of thought. A fully conversational voice-in, voice-out interaction needs roughly 200 milliseconds or less because people interrupt, agree, and interject continuously. A typical chain involving network requests, speech recognition, model inference, and another network response makes that target difficult.
Visual output avoids the hardest timing constraint of voice-to-voice interaction
Pike says teams do not need to wait for new voice-to-voice architectures to build responsive experiences. Voice-in, visuals-out gives the system a more forgiving response window. If something appears on screen within about a second of what the user said, it can still feel attentive and responsive. The output might be generated HTML, a visualization, an interactive control, or an illustration. Visuals can explain a model's answer and let users explore, modify, or direct what the model does.
The response model must be fast, and larger reasoning work should happen separately
Pike's first implementation rule is to use a small model that can respond in a few hundred milliseconds, together with an inference platform that prioritizes latency. He reports seeing GPT-5 mini take 5,000 milliseconds, 7,000 milliseconds at P95, and sometimes 10,000 milliseconds in their testing. He says Haiku performed better for P95 latency, and that small open-source models can fit the same role. A fast model can send heavier work asynchronously to a larger model, then continue interleaving updates while that work proceeds. The real-time model also needs a short enough context to answer quickly.
Sending inference every one or two seconds makes partial user requests feel responsive
Traditional voice applications may wait for the user to finish, detect a second of silence, and only then start inference. Pike says that waiting consumes too much of the one-second response budget. For visual responses, the system should infer eagerly while the person is still speaking, even when it is not fully certain that the utterance has ended. He recommends sending inference every one or two seconds. This allows the system to begin handling a request such as changing one thing and also listing another, rather than waiting for the entire sentence or thought to finish.
Stable prefix caching cuts repeated inference cost and time
Pike's final technique is a stable caching regimen. Prefix caching can make inference up to 90% cheaper and faster when the beginning of the context remains the same across requests, depending on the platform and conditions. He recommends keeping the first 90% of the context window unchanged when possible and using the final 10% for the changing material. Applications should also minimize output tokens. The same design applies to long-running agents and agents that run frequently, because repeated requests can reuse the stable part of their context instead of processing it from scratch each time.
"The models that we have so far, the experiences that people most people have seen are both slow and dumb, which is like not a great combination."02:56
Who should watch
You are building an AI product where users speak naturally and expect the interface to respond while they continue working.
Your current voice interface waits for silence or produces responses too slowly to preserve the user's train of thought.
You are tuning LLM costs and latency and want concrete guidance on model choice, inference frequency, asynchronous handoffs, and prefix caching.
