# How to Write Your First AI Eval

Sally-Ann DeLucia, Arize AI | AI Engineer NYC 2026 | 35:20

Source: https://www.youtube.com/watch?v=gv4w23xYj-0
Channel: AI Engineer (https://www.youtube.com/@aiDotEngineer). Summarised by AIE Talks.
Page: https://aietalks.com/talks/how-to-write-your-first-ai-eval
Published: 2026-10-25
Tags: evals, human-in-the-loop, observability, testing

## TL;DR
- Evals give teams a way to detect regressions, compare models, and add quality gates when prompts or agents change.
- A useful eval starts with concrete pass and fail criteria, such as matching the requested trip length and staying within budget.
- Teams should begin with a small dataset and one quality dimension, then test and revise the judge against real traces and human feedback.

## Summary
SallyAnn DeLucia explains how to move from informal vibe checks to repeatable evaluations for AI agents. She distinguishes evals from traditional unit tests because agents can produce many valid outputs and expose their logic through traces. The workshop uses a travel-planning agent to show how to inspect spans, identify the data needed for evaluation, and define specific success criteria. DeLucia recommends judging one quality dimension at a time when using an LLM judge, such as whether an itinerary follows the user's requested days and budget. She then demonstrates how to select traces, build a dataset, turn criteria into a prompt, map variables, run the judge, and revise it in Arize's prompt playground. The final steps are aligning the judge with human annotations, running evaluations continuously on production traces, and involving product, engineering, support, domain, and security stakeholders in defining success.

## Key ideas
### Vibe checks cannot show whether an agent improved
[02:06](https://www.youtube.com/watch?v=gv4w23xYj-0&t=126s)
DeLucia describes a common pattern: a team changes a prompt, ships it, and decides that the system feels better. A later change can break the original use case, but the team has no objective record of what changed. Without evals, teams cannot detect prompt regressions, compare model versions, or add quality gates to CI. The problem is especially difficult when an agent changes in several places over time, because informal impressions do not provide a stable way to prove that a fix worked or that a failure appeared.

### AI evals inspect traces because agents can have many valid outputs
[03:08](https://www.youtube.com/watch?v=gv4w23xYj-0&t=188s)
A traditional unit test usually has one input, one expected output, and logic that can be read in code. An agent can produce several valid itineraries, budget breakdowns, or answers, while its decision process only becomes visible after execution. DeLucia describes traces as the logs for AI and evals as the tests. A trace contains the individual steps, tool calls, agent calls, and model inputs and outputs that led to the final response. This information is needed to evaluate the parts of a multi-step agent.

### Different evaluation methods fit different kinds of checks
[05:15](https://www.youtube.com/watch?v=gv4w23xYj-0&t=315s)
The workshop focuses on an LLM judge, which asks a secondary model to grade an aspect of an agent and return a label with an explanation. DeLucia also describes decision models, which return a probability and label without the explanation of an LLM judge, and code evals for deterministic checks such as JSON structure or the presence of a word. Human annotations can provide labels on golden or synthetic data. An agent can also judge another agent when the success criteria are complex. User feedback and business metrics are more useful in production.

### Reading a trace means following each unit of work
[14:20](https://www.youtube.com/watch?v=gv4w23xYj-0&t=860s)
In the travel agent example, the root span sits at the top and usually contains the request and final answer. Each child span is a unit of work. A research agent may call a tool for weather or destination information, then an LLM may summarize the tool output for a later step. Some traces contain an LLM call that selects a tool and another call that processes the tool's result. Looking down the trace tree helps the evaluator find where the relevant inputs and outputs live.

### Good criteria describe observable pass and fail conditions
[19:03](https://www.youtube.com/watch?v=gv4w23xYj-0&t=1143s)
DeLucia recommends replacing broad terms such as helpful, accurate, or good response with conditions that can be checked. For the travel planner, examples include answering follow-up questions, matching the requested number of days, and keeping the total within the stated budget. The evaluator should record the task, what success means, what failure means, and concrete pass and fail points. DeLucia also recommends judging one quality per LLM-based eval because these judges tend to perform better when they handle one criterion at a time.

### A first eval should start small and grow from real usage
[22:02](https://www.youtube.com/watch?v=gv4w23xYj-0&t=1322s)
DeLucia says evals need iteration rather than a one-time setup. Teams can begin with a small set of criteria and expand as the application gets real usage. She describes starting with around 20 traces, one dimension, and one judge, then reading the explanations and revising the evaluator. Generic templates can help teams get started, but they should become more specific to the application's use case. At Arize, DeLucia says the Alex agent's evaluation set grew from a simple group of checks to 32 evals.

### The judge is built by turning criteria into a tested prompt
[26:53](https://www.youtube.com/watch?v=gv4w23xYj-0&t=1613s)
The practical workflow starts by filtering traces and adding the relevant root spans or tool spans to a dataset. In the prompt playground, the evaluator prompt explains the judge's job, receives mapped variables from the trace or dataset, and defines labels such as correct, incorrect, partially correct, or not applicable. The team runs it over the dataset, reviews whether the outputs make sense, edits the prompt, and runs it again. Once the evaluator is acceptable, it can be saved for use across the platform.

### Human alignment and continuous evaluation come after the first judge
[32:08](https://www.youtube.com/watch?v=gv4w23xYj-0&t=1928s)
After creating an initial evaluator, teams can annotate dataset examples and compare the judge's decisions with human feedback. Online evals can then run continuously whenever new traces arrive. DeLucia says every stakeholder should help define success, including product, engineering, support, domain experts, and security. Metrics can indicate that something may be broken, while evals identify the failure mode. She also suggests comparing an LLM judge with a decision judge for the same task.

## Notable quotes
- "I think the way that I think about evals is that they're our primary signal for our agents." (00:29)
- "Traces are the logs for AI, evals are actual tests." (03:49)
- "The first step of defining what your eval is is defining what good means." (19:03)
- "An eval is better than no eval at all." (21:22)
- "Every stakeholder evaluates the agent." (33:25)

## Tools & references mentioned
- Arize AI
- Arize
- Alex
- Signal
- Anthropic
- OpenAI
- Gemini
- ROUGE
- BLEU
- OpenInference

## Who should watch
- You have an AI agent in production and currently decide whether changes worked by manually reading a few outputs.
- You need to turn product expectations such as accuracy, budget compliance, or follow-up handling into repeatable pass and fail checks.
- You are building an LLM judge and want a workflow for selecting traces, testing prompts, comparing outputs with human feedback, and running evaluations online.

## Editor's note

SallyAnn DeLucia says a later prompt change can break an original use case when a team has no objective record of what changed. Kitaru records each real agent run, including model calls and tool results, then replays it with the same inputs and responses after a prompt, model, or code change. That gives the regression a repeatable test.

Written by the AIE Talks editors (the Kitaru team), not by the speaker.

## Related talks

- [Evals 101](https://aietalks.com/talks/evals-101) (Doug Guthrie, Braintrust, 48:31)
- [If Your AI Scores 100%, Your Eval Is Wrong](https://aietalks.com/talks/if-your-ai-scores-100-your-eval-is-wrong) (Ramakrishnan Lokanathan, eSlams, 18:45)
- [Evals for High-Stakes AI Agents](https://aietalks.com/talks/evals-for-high-stakes-ai-agents) (Doneyli De Jesus & Annabell Schäfer, Langfuse, 53:32)
- [Evals Are Not Unit Tests](https://aietalks.com/talks/evals-are-not-unit-tests) (Ido Pesok, Vercel, 15:22)
- [Evals in AI: A Deep Dive](https://aietalks.com/talks/evals-in-ai-a-deep-dive) (Tejas Kumar, IBM, 59:49)
