AssemblyAI replaced an off-the-shelf support bot that resolved 10% of conversations with Joey, an agent that resolves 80% end to end.
2
Joey uses local Markdown documentation, Voyage embeddings, agentic file search, Claude Agent SDK instructions, and Railway deployment.
3
Matt argues that forward deployed engineers should automate routine support work and build the same products their customers are building.
Summary
Matt Lawler explains how AssemblyAI built Joey to handle support and onboarding at a company receiving around 1,000 API signups each day. The company's off-the-shelf support bot resolved about 10% of conversations, and the team could not change its prompts, tools, or retrieval system. Joey uses documentation checked out as local Markdown, Voyage embeddings, agentic file search, a large CLAUDE.md file, and deployment on Railway. The team can ship fixes in about 30 seconds. Joey reached an 80% end-to-end resolution rate in its first week at about $700 per month, with humans handling the remaining escalations. Lawler also added AssemblyAI's Voice Agent API, connecting speech-to-text, an LLM, and text-to-speech through one WebSocket. The project gives AssemblyAI a support system while forcing its engineers to use the same voice technology that customers build with.
A forward deployed engineer cannot personally support 1,000 daily signups
AssemblyAI was seeing around 1,000 API signups every day, while Matt Lawler was, until shortly before the talk, the only onboarding engineer. Forward deployed engineers work closely with customers, often committing code to their repositories and meeting several times a month. That model can produce a strong customer relationship, but it cannot cover every inbound customer at this volume. Lawler says the team needs to reserve hands-on work for human-led deals instead of hiring more people to handle every routine question.
The team chose to automate the FDE role rather than only deflect tickets
Lawler tells FDEs to use their customer knowledge to automate themselves out of a job. AssemblyAI first bought an off-the-shelf support bot and pointed it at the documentation. It resolved only about 10% of conversations. With 1,000 conversations, that would remove roughly 100 tickets while leaving the team with 900. The team also lacked access to the bot's system prompt, tools, and retrieval infrastructure, so fixes depended on the vendor's roadmap. They decided to build Joey instead.
Joey behaves more like an engineer because it can use files, tools, and code
Joey is the first line for conversations from AssemblyAI's chat widget, sales contact form, and support alias. Built on the Claude Agent SDK, it has a file system, can write and debug code, call tools, manage infrastructure, and give itself new abilities. Pylon tracks its support conversations and lets Joey remember who customers are. Lawler describes this as closer to an FDE than a standard website chatbot.
Local Markdown gives Joey current documentation even when the docs site is unavailable
AssemblyAI checks its documentation out as local Markdown files and keeps them updated when the documentation system changes. Joey can use those files to answer questions about new releases. Lawler says customers could still get current answers from Joey if the public documentation site went down. Voyage embeddings put relevant documents in front of the agent, while agentic file search lets Joey search across the local files when it needs to combine several resources.
Fast deployment lets the team fix a live support problem during a conversation
Joey runs on Railway because the team did not want to manage EC2 instances or change infrastructure manually. When the team finds a bad conversation in Slack, it can write a pull request and deploy a new version in about 30 seconds. Lawler says they have monitored a live conversation, found a bug, deployed a fix, and had the corrected behavior take effect during the same session. The team controls Joey's behavior and can update it whenever needed.
Joey reached 80% end-to-end resolution at about $700 per month
In the first week after deployment, Joey moved AssemblyAI from a 10% to an 80% end-to-end resolution rate with what Lawler calls a fairly naive implementation. Token and infrastructure costs were about $700 per month. Joey handles all inbound tickets first, and only 20% escalate to a human. Those cases include rate changes, data opt-outs, agreements, and other requests that still need an FDE or legal involvement.
Every escalation becomes a concrete improvement to Joey
Lawler treats unresolved requests as a list of work for the team. If Joey cannot send a BAA, the team can give it a link to the right resource. If it cannot discuss pricing, the team can let customers negotiate with Joey, including quoting a rate based on the customer's requirements. The agent is therefore connected to the team's product and support roadmap. Its limitations show where a new tool, permission, or instruction could remove another human escalation.
Adding voice makes the support agent a product dogfood test
AssemblyAI added its Voice Agent API to Joey so customers can speak to it instead of using text. The API connects speech-to-text, an LLM, and text-to-speech through one WebSocket. It handles real-time latency, pauses, interruptions, and barge-in, and includes voices without requiring another provider. Lawler says building this exposed the same latency, turn-taking, and interruption issues that AssemblyAI customers face, giving him more practical advice for their implementations.
FDEs should build the product their customers are building
Lawler says teaching customers is not enough for an FDE. Engineers should build and ship the same product themselves, including new features they expect customers to use. By building Joey as a voice agent, he had to work through the same problems customers encounter with the Voice Agent API. He argues that this creates better empathy and better technical guidance, while the resulting internal product can also improve customer support.
