Voice gives existing chat agents faster, more accessible interaction through phone lines, Zoom calls, and other interfaces.
2
ElevenLabs Voice Engine wraps an existing chat agent with speech-to-text, text-to-speech, turn-taking, and interruption handling without replacing its underlying logic.
3
The wrapper keeps existing agent tool calling on the back end, while also supporting client-side and server-side tools.
Summary
Luke Harries argues that chat agents were the default AI interface in 2025, but voice is a more natural and interactive way to use them. Voice can make an agent available through phone calls, Zoom, screen readers, and ambient interfaces. The engineering problem is that many teams already have working agents with tuned prompts, evaluations, retrieval, and integrations. Rebuilding those systems around a new voice platform is expensive. ElevenLabs Voice Engine addresses this by wrapping an existing agent. Its voice layer handles speech-to-text, text-to-speech, turn-taking, and emotion-aware interruption detection, while the original agent continues to handle orchestration and tools. Harries shows server and client SDKs, UI components, and a demo that converts a local chat support agent with one prompt. He also explains that tool calls usually remain on the existing back end, with options for client-side tools and future proxying to the wrapped agent.
Harries says chat is a good default because it supports tool calling and retrieval in a declarative interface, but it does not feel like the future to him. Voice is quicker and more interactive, and it helps people who struggle with keyboards or dyslexia. It also opens additional channels. A PostHog-style agent could join a Zoom call and correct incorrect statistics, while a customer support agent could answer a phone line. His conclusion is that existing chat agents should gain a voice interface rather than remain limited to text.
A voice agent has a separate voice layer and orchestration layer
ElevenLabs began by building text-to-speech models, then found that customers needed a larger agent system. Harries describes an architecture with a voice engine for text-to-speech, speech-to-text, and turn-taking. A separate orchestration layer combines the LLM, retrieval, tool calling, and other integrations. Customers such as Revolut were building systems like this. Some wanted an out-of-the-box agent, but others already had an agent with extensive evaluations and transcripts. Replacing that work would force them to rebuild a system they had already tuned.
Voice Engine wraps an existing agent instead of replacing it
Harries introduces Voice Engine as an early product preview. It packages the voice portion as a first-class primitive that can wrap an existing agent. The system uses ElevenLabs speech-to-text through Scribe and text-to-speech models including V3. It includes turn-taking that uses emotional context, detects pauses, and handles semantic boundaries. The product also provides thousands of voices and languages. The important design choice is separation: teams can add the voice layer while keeping their existing chat-agent logic.
The server SDK creates an ElevenLabs client and a voice engine, then attaches the engine to the existing chat agent. When a new session starts, the wrapper begins a loop that proxies the interaction to that agent. Harries presents this as a small integration for teams that have already spent time building a complex agent. The existing agent continues to receive the conversation, while Voice Engine handles the audio-facing work around it.
The client SDK and UI components shorten the path to deployment
Harries says the client SDK can add a site widget in about three lines. Once the wrapper is in place, developers can also add telephony and CSAT out of the box. ElevenLabs provides UI components based on the shadcn and Vercel style. The intent is for a coding agent to use those components directly. Harries says an existing chat agent can be converted with one prompt that analyzes the codebase, determines how the agent is deployed, and works out how to apply the wrapper.
Voice Engine offers a middle layer between raw speech APIs and a full platform
Harries describes two options for developers building voice agents. Voice Engine is for teams that already built a chat agent and want to add voice with a wrapper and SDK. ElevenLabs also offers a full conversational agents platform for teams that want to prompt an agent out of the box. He presents Voice Engine as a higher-level bundle than using separate text-to-speech and speech-to-text APIs, while leaving the existing agent architecture in place.
Tool calling usually stays inside the existing chat agent
In the question period, Harries explains that the wrapped chat agent normally handles most tool calls on the back end. Developers therefore do not need to redesign their tool integrations for the voice wrapper. ElevenLabs also supports client-side and server-side tools. Client-side tools can manipulate the DOM at the front end. Harries says the team plans to add a way to proxy some tool calls to the wrapped agent, although most users will already have their tool calling handled by the chat agent.
"We've basically taken this voice engine bit and wrapped it up into its own first class primitive, which makes it really easy for you to add and wrap any existing agent."02:49
Who should watch
You have a working chat agent with retrieval, evaluations, and tool integrations, but want to add phone or browser voice without rebuilding its logic.
You are choosing between a voice SDK and a full conversational-agent platform and want to understand where each fits.
You need client-side or server-side tools to continue working after adding speech input and output.
