ChatGPT separates voice conversation from text work, so users cannot easily move between speaking and editing in one interaction.
2
A better design combines voice and text in one interface, with a chat panel for links, drafts, and other detailed responses.
3
The app can route simple requests to a fast model and send complex questions to a reasoning model through tool calls.
Summary
Ben Holmes argues that ChatGPT feels confusing because related capabilities appear to have been built as separate products. Voice input and voice conversation use different controls, and a spoken answer that would work better as an email or link only becomes available as a transcript after the call ends. He connects this to Scott Hanselman's phrase, "shipping the org chart," where separate teams leave users with disconnected interfaces. Holmes builds a different design using off-the-shelf APIs. Voice remains the main interaction, while a chat panel appears during the call for text responses. Tool calls let the model send links and drafts to that panel. Another tool hands detailed questions to a reasoning model. The demo uses simple routing rules based on the request, such as sending questions about pros and cons or deeper research to a slower model. The source code is available in the fixgpt GitHub repository.
ChatGPT splits voice interaction from written work
Holmes demonstrates that ChatGPT exposes two voice controls, one for voice-to-text and another for voice-to-voice. The voice conversation can produce a polished email, but the user cannot edit or collaborate on that email inside the call. To get the written result, the user has to end the call and find the transcript afterward. Holmes says the experience should combine text and voice instead. The problem is practical: a user may want to speak naturally, then receive a link, a draft, or formatted text without changing modes or searching through a transcript.
Disconnected product surfaces can reveal the organization behind them
Holmes uses Scott Hanselman's phrase "shipping the org chart" to describe interfaces assembled by separate teams. Hanselman's example is an electric vehicle whose map, climate controls, and speedometer use different fonts because they are effectively separate Android tablets. Holmes applies the same idea to OpenAI, where technical improvements can appear as disconnected options rather than one coherent product. He compares the model choices to a science fair full of options such as "o3," "4o," and "Xbox 3601 Series S Pro." His criticism is about how the pieces are presented to ordinary users.
A single voice interface can also expose a text conversation
Holmes proposes keeping the familiar voice mode while adding a chat button. The voice screen still has mute and end-call controls, but the new button opens a panel that looks like iMessage. It gives the user a reminder of the relevant request and displays a written answer when text is more useful. In the example, a spoken request for an email can produce a formatted draft in the panel. Holmes compares the interaction to texting a friend during a FaceTime call. The user can stay in the voice conversation while reading and using more detailed material.
Tool calls let the model decide when a response belongs in text
The prototype handles text input through a tool call called send chat message. The tool description tells the model that some details are easier to explain in text. Holmes says this worked without adding a system prompt. In his demo, he asks for a link to a park in California, then asks for more history about Yosemite National Park. The voice model sends the more detailed material to the chat panel and tells the user to check it. This keeps short conversational replies in audio while putting links and longer explanations where the user can read them.
Requests can be routed to different models according to their difficulty
Holmes describes a second design change: choose the model based on what the user asks. In Warp, a request such as undoing the last commit goes to a coding agent that can run terminal commands. A larger request, such as refactoring an entire codebase to Flutter, is detected as complex and sent to a reasoning model to create a plan. The same pattern can apply to ChatGPT. A request for details, comparisons, or pros and cons can be handed to a reasoning model, with the interface telling the user how long it is thinking before returning the fuller answer.
Simple tools and descriptions are enough to build the prototype
Holmes builds the design with off-the-shelf APIs. He uses a realtime API for live audio and tool calls for the actions around it. One tool sends text into the chat panel, and another sends a request to a reasoning model and returns whatever answer the client needs. His demo does not depend on a large system prompt. A short description of the send-chat tool is enough for the model to choose it when a response is better delivered as text. Holmes presents this as a practical way to connect multiple models and modes without rebuilding the underlying services.
"You have some whiz kid who ships a technical improvement. It turns out it's exactly what consumers are craving. Marketing's never consulted and you end up with a science fair full of potential options."01:42
Who should watch
You are designing an AI assistant that needs to handle both spoken requests and written output in the same session.
Your product exposes several models or modes and users are unsure which one to choose.
You want a small implementation pattern for routing detailed requests through tools and handing complex work to a reasoning model.
