A prototype built with prompts, RAG, and agents can reach a demo quickly, but progress stalls when the team cannot measure whether changes help.
2
A useful evaluation system starts with domain-specific assertions, trace logging, human review, and test cases generated from real failure modes.
3
Once the system is working, it can support prompt improvement, fine-tuning data curation, and carefully aligned LLM judges.
Summary
Emil Sedgh and Hamel Husain describe how Rechat developed evaluations for Lucy, an AI assistant for real estate agents. The prototype worked impressively in some cases, but the team could not estimate its success rate or know whether a prompt change fixed one problem while creating another. Husain recommends starting with simple assertions and unit tests based on observed failures, logging traces, and making human review easy through domain-specific viewing and annotation tools. Synthetic user inputs can provide test coverage before enough real users exist. The team can then iterate on prompts while checking whether its evaluation process is itself working. More advanced LLM judges come later, after the team has aligned them with human judgments. Sedgh explains that fine-tuning was needed for Lucy to combine structured interface elements with natural language, gather feedback, and execute complex multi-tool commands. The evaluation framework made those improvements measurable and supported the curation of fine-tuning data.
Vibe checks can get an AI prototype to a demo, but they cannot guide production work
Emil Sedgh says Rechat built an early real estate assistant with GPT-3.5 and a React framework. It was slow and made frequent mistakes, though successful interactions felt remarkably good. The team could reach a prototype, but prompt changes were evaluated by invoking the system a few times and forming an impression. They did not know whether it worked 50 percent or 80 percent of the time. They also saw one use case improve while another broke after a prompt change. Hamel Husain calls this a common way to move from zero to one, but says it leads to stagnation once the team has to make steady, measurable progress.
Assertions based on observed failures are the foundation of an evaluation system
Hamel Husain recommends writing unit tests and assertions for as many known failure modes as possible before reaching for an LLM judge or generic evaluations. Rechat wrote simple checks from problems found in its data, including agents failing to work properly, emails not being sent, invalid placeholders, and details being repeated when they should not be. These checks are cheap to run and give immediate feedback. Husain suggests starting with existing infrastructure, such as CI, and logging results to an existing database and dashboard. Rechat used Metabase to track whether these basic failures were declining over time.
Trace review only works when the team removes friction from looking at data
Husain says logging traces is insufficient if nobody examines them. He recommends using a tracing tool early, and Rechat used LangSmith, but he also argues that teams should build their own viewing and annotation interface when domain-specific details make general tools awkward. Rechat built an application with filters tailored to its data and metadata specific to the product, so reviewers did not have to hunt for context. The same interface supported labeling and human review. Husain calls this the most important part of the process: if reviewing data is difficult, people stop doing it and the evaluation workflow fails.
Synthetic users can provide test cases before a product has enough real usage
When a team lacks users or real examples, Husain suggests using an LLM to generate inputs. Rechat asked an LLM to play the role of a real estate agent and pose questions to Lucy across its features, scenarios, and tools. This approach helped bootstrap test cases and improve coverage. The generated inputs are not the whole evaluation system. They supply situations that can then pass through assertions, trace logging, and human review. Husain presents synthetic generation as a practical way to start exercising the system while the product is still early.
Prompt iteration is also a way to test whether the evaluation workflow works
After setting up basic tests, trace logging, and review, Husain recommends improving the AI through prompt engineering and repeatedly checking the whole loop. The team should ask whether its test coverage is adequate, whether traces are logged correctly, and whether reviewers can inspect data without unnecessary effort. This makes evaluation a working part of development rather than a separate report produced later. It also helps find weaknesses in the evaluation setup itself. The first goal is to remove straightforward failure modes and establish that the team can tell when a change is an improvement.
A good evaluation system creates a path to fine-tuning data
Husain says the evaluation workflow can support data curation with little additional work. Synthetic inputs and evaluation results can help identify promising cases for human review. Failed cases can enter a workflow for correction and later inclusion in fine-tuning data. As the evaluation framework becomes more complete, more checks are automated and human review becomes less expensive. This gives the team a repeatable way to collect useful examples instead of treating fine-tuning as a separate activity. The same records that reveal failures also help decide which examples deserve expert attention.
LLM judges should come after simple checks and be tested against human judgment
Husain says assertions cannot express every quality problem, so an LLM judge can become useful later. He warns against adopting one too early, since close inspection of data often reveals more simple assertions and failure modes first. To assess a judge, he recommends having a domain expert label critiques and comparing those labels with the judge's outputs in a spreadsheet. The team should keep iterating until the LLM judge aligns with the human reviewer and there is confidence that it is doing the intended job. Generic scores such as conciseness or toxicity should not replace domain-specific evaluations.
Lucy needed fine-tuning for mixed outputs, feedback, and multi-tool commands
Sedgh says few-shot prompting never achieved everything Rechat needed, even with newer agents. Lucy had to combine natural language with interface elements in one response, which required structured and unstructured output together. It also needed to ask users for more information when an action could not be completed directly. A third requirement was executing complex commands that involved five or six tools. In the example, Lucy found listings, selected the most expensive one, created a website, rendered an Instagram post, prepared an email with the results, invited Hamel Husain to dinner, and created a follow-up task. Sedgh says this behavior required fine-tuning and a comprehensive evaluation framework.
"If you're having a conversation about evals and the first thing you start thinking about is tools, that's a smell that you're not going to be successful in your evaluations."13:18
Who should watch
You have an LLM prototype that feels good in demos, but prompt changes sometimes fix one workflow and break another.
Your team needs a practical starting point for domain-specific evals before buying a large evaluation platform.
You are deciding whether to use an LLM judge or fine-tune, and need a workflow grounded in human review and observed failures.
