Discord found that the hardest part of launching Clyde AI was reducing security, legal, and safety risks at a scale where rare failures become frequent.
2
Small, deterministic evals that run locally helped developers test individual system steps and ship prompt and model changes more often.
3
Predeployment red teaming exposed application-specific jailbreaks that could bypass safeguards, so risk assessment had to happen before launch as well as through live filtering.
Summary
Ian Webster describes how Discord developed and shipped LLM products, including Clyde AI, to a user base of more than 200 million people. The difficult work was managing harmful behavior, legal concerns, and safety failures rather than choosing a model or fine-tuning it. Discord treated evals as unit tests: small, fast, mostly deterministic checks for specific stages such as moderation, tool use, retrieval, and final responses. The team ran them locally, attached them to pull requests, and used Promptfoo for local evaluation and red teaming. Webster also discusses production observability, prompt versioning, model routing, and the limits privacy placed on collecting live user examples. Discord used adversarial models to find jailbreaks that ordinary harmful-input tests missed. His advice is practical and sometimes uncomfortable: simplify prompts, use the observability tools already in place, and test risks before users discover them in production.
Discord launched Clyde AI, an agent and retrieval-augmented chatbot, to more than 200 million users. Webster says the hard part was preventing it from teaching children how to build bombs, along with handling harassment, racism, and other failure modes. Security, legal, safety, and policy concerns repeatedly blocked launches until stakeholders were comfortable. At Discord's scale, a failure that occurs once in a million cases could happen roughly 200 times. That made risk measurement a release requirement rather than an issue to investigate after launch.
Evals should be small tests for specific system steps
Webster defines evals as a systematic way to characterize behavior from inputs and measure outputs. He recommends treating them like unit tests and breaking a system into small checks. A moderation step, each tool use, and an end-to-end response can have separate evals. For a casual-chat personality, Discord checked whether the output began with a lowercase letter. That simple deterministic test ran quickly and got the team more than 80 percent of the way toward the desired behavior for about 1 percent of the work of a more elaborate setup.
Static test cases make retrieval evaluation cheaper and clearer
For web search with retrieval and generation, Discord separated the question of whether an agent should trigger a tool from the question of whether it could summarize retrieved content. The team tested triggering independently, then evaluated summarization against static context. It did not connect the test to a live database or live web searches. Webster presents this split as a way to keep evaluation focused and to avoid mixing tool-selection failures with answer-quality failures. He also notes a trade-off between model cost and accuracy at scale.
Shorter prompts often beat a growing list of exceptions
Discord initially tried to prompt for every failure mode discovered in its evals. Webster says this approach reached diminishing returns and then negative returns. Removing much of the prompt and allowing the model to do what seemed reasonable produced better results. He also calls prompts a form of vendor lock-in. A prompt tuned for GPT-style behavior will not automatically work well with Claude, Llama, or other models, so changing models requires testing and prompt changes rather than a simple swap.
An eval culture depends on local, understandable checks
Webster calls eval culture the most important part of Discord's approach. If evals are tests, developers should be able to run them locally without a cloud service or third party. The metrics should be simple enough to understand, and developers should be able to run dozens of evals per day from the command line. Discord attached an eval link to every pull request and later integrated evaluation into CI/CD. The team built Promptfoo, an open-source local CLI with declarative configuration, around this developer-first workflow.
Existing observability infrastructure can carry LLM metrics
Discord used Datadog for LLM observability because it was already the team's product observability system. Webster's view is that the best observability tool is the one a team already uses. Discord added the metrics it cared about alongside its other product data. The team also ran some online production evals, including simple one-shot model-graded checks, and sent those results to Datadog. Privacy prevented Discord from closing the feedback loop by routinely incorporating live user data into its evaluation set.
A stronger model can pull a drifting conversation back on track
Discord saw lower-powered models such as Llama and GPT-3.5 drift away from their system prompt during very long conversations. The assistant gradually reverted toward a generic ChatGPT-like personality, which users disliked. The team tried occasionally inserting a GPT-4 response at random points in the conversation. Webster compares this to a bowling-alley bumper that nudged the weaker model back toward its intended behavior. He is careful about the evidence: it worked acceptably for Discord, but he does not claim it is a generally smart routing strategy.
Red teaming must test the application built around the base model
Discord treated predeployment risk assessment and red teaming as at least as important as live filtering. Promptfoo included a risk assessment view with categories such as brand and legal risk. Beyond testing obvious harmful inputs, Discord used an attacker model to send adversarial requests and a judge to suggest euphemisms, synonyms, or rewritten approaches. This feed-forward loop found application-specific jailbreaks introduced by the product's prompts and tools. A base model might refuse a direct harmful request while responding to a fictional or indirect version created by the attacker.
"If you put an LLM app on the internet, within 10 minutes people are going to be typing this stuff into it."15:25
Who should watch
You are shipping an LLM feature to a large user base and need a practical release process for safety and quality checks.
Your team has model or prompt changes that are hard to compare, and you want evals that developers can run locally and attach to pull requests.
You are relying on content filters alone and need to test application-specific jailbreaks before deployment.
