# Break It 'Til You Make It: Building the Self-Improving Stack for AI Agents Aparna Dhinakaran, Arize | AI Engineer World's Fair 2025 | 14:25 Source: https://www.youtube.com/watch?v=Qvp9vw4jJQ8 Channel: AI Engineer (https://www.youtube.com/@aiDotEngineer). Summarised by AIE Talks. Page: https://aietalks.com/talks/break-it-til-you-make-it-building-the-self-improving-stack-for-ai-agents Published: 2025-06-10 Tags: evals, observability, tracing ## TL;DR - Agent evaluation needs to inspect tool choice, tool arguments, full trajectories, and multi-turn sessions. - Production traces can reveal where a specific planner, sub-agent, or tool call causes poor results. - The evaluation system needs its own improvement loop, with refined prompts and a growing golden dataset. ## Summary Aparna Dhinakaran explains how to evaluate agents after they leave the prototype stage. She starts with tool-call checks: did the agent choose the right tool, and did it pass the right arguments? From there, teams should evaluate complete trajectories, including whether tools were called in the right order and whether the agent took unnecessary steps. Multi-turn sessions add questions about context, repeated questions, tone, and whether the agent answers every part of the conversation. Dhinakaran then describes a second improvement loop for the evaluators themselves. Evaluation prompts can mislabel outputs, so teams need to review eval results, refine the eval prompts, and build a golden dataset. She demonstrates these ideas with Arize's own copilot and traces, showing how a high-level view can point to weak search performance before engineers inspect individual failures. The talk ends with Arize Phoenix, an open-source tool for running evaluations on application data. ## Key ideas ### Agent development becomes hard when prompt changes are judged from a few examples [00:23](https://www.youtube.com/watch?v=Qvp9vw4jJQ8&t=23s) Dhinakaran describes teams swapping prompts, models, and tool definitions in an Excel sheet. They often decide that one version is better because of how it feels on a small number of examples, then deploy it. This makes it difficult to track where a new prompt improves on the previous one. It also makes it harder to involve product managers and other people in the evaluation process. A systematic method should show which prompt or model performs better and make the improvement process easier to share. ### Tool-call evaluation must check both the selected tool and its arguments [03:18](https://www.youtube.com/watch?v=Qvp9vw4jJQ8&t=198s) An agent first has to choose the tool that fits the current situation. It then has to pass arguments that match the conversation and the tool's required parameters. Dhinakaran shows an example where the agent appears to choose the correct search tool, but its arguments do not align with the required parameters. That failure explains why the interaction received a poor result. Checking only the function name would miss the actual problem, so tool-call evals need separate checks for tool selection and argument correctness. ### A high-level view of agent paths can point engineers toward the failing part of the system [05:11](https://www.youtube.com/watch?v=Qvp9vw4jJQ8&t=311s) Arize's copilot uses a planner, several tools, and sometimes another router or orchestrator. Dhinakaran evaluates the different paths that this architecture can take, rather than opening traces at random. In her example, generic questions perform well, while search questions are correct only about half the time. That result gives her a place to investigate. She can then filter for incorrect search answers and inspect individual traces to find the specific tool-call or argument failure behind the aggregate result. ### Trajectory evaluation checks whether a task's tools are called in the right order [08:49](https://www.youtube.com/watch?v=Qvp9vw4jJQ8&t=529s) Many agent tasks require several tools, including subtools inside a larger path. An agent can call each tool correctly in isolation and still fail because it uses the wrong sequence. Dhinakaran recommends examining the full trace and checking whether the tool calls follow the needed order. Teams should also watch for paths that veer off, use more steps, or spend extra tokens to complete the same task. A wrong sequence can lead to both higher cost and incorrect output. ### Multi-turn evaluation must test whether the agent carries context across the conversation [10:28](https://www.youtube.com/watch?v=Qvp9vw4jJQ8&t=628s) A multi-turn session creates questions that do not appear in a single request. Teams can check whether the agent keeps a consistent tone, repeats questions, and uses information from earlier turns. Dhinakaran describes evaluating a three-part exchange by asking whether all questions were answered and whether earlier context was preserved. The agent should use the previous turns when responding to the current turn. Session-level evaluation is therefore part of evaluating the agent, rather than an optional add-on after single-turn checks. ### The evaluation prompts need their own improvement loop [11:48](https://www.youtube.com/watch?v=Qvp9vw4jJQ8&t=708s) Dhinakaran says teams often understand that application prompts need iteration, but they may leave the eval prompts unchanged. That is a problem because evals determine which failures are surfaced and can label an output incorrectly. Teams should review evaluation outcomes to distinguish an application error from an evaluator error. They can then iterate on the eval prompts and build a golden dataset. The application and evaluation systems need to improve together, with separate loops that inform each other. ### Arize Phoenix provides an open-source way to run these evaluations on application data [13:56](https://www.youtube.com/watch?v=Qvp9vw4jJQ8&t=836s) Dhinakaran closes by pointing viewers to Arize Phoenix, which she describes as a completely open-source product. It can be used to explore traces, run evaluations, and test the approach on a team's own data. The demonstration connects the evaluation ideas to a working product rather than leaving them as an abstract checklist. Engineers can use the tool to inspect individual interactions and broader agent behavior while developing their own evaluation process. ## Notable quotes - "Building agents is incredibly hard." (00:23) - "It's pretty important to actually evaluate, did it call the right tool? And did it pass the right arguments into that tool call itself." (03:55) - "You don't want those evals to remain static, the prompts for those evals." (12:29) - "There really are kind of two iterative loops kind of going on at the same time." (13:37) ## Tools & references mentioned - Arize - Arize Phoenix - Excel ## Who should watch - You are moving an agent from prompt experiments into production and need to find which tool, planner, or path is failing. - Your agent handles multi-step or multi-turn work, and single-request success checks do not explain its failures. - You have application evals in place but have not reviewed whether the eval prompts and labels are still accurate. ## Related talks - [Why (Senior) Engineers Struggle to Build AI Agents](https://aietalks.com/talks/why-senior-engineers-struggle-to-build-ai-agents) (Philipp Schmid, Google DeepMind, 10:40) - [Fighting AI with AI](https://aietalks.com/talks/fighting-ai-with-ai) (Lawrence Jones, Incident.io, 17:29) - [Designing Agents (The Floor Is the Frontier)](https://aietalks.com/talks/designing-agents-the-floor-is-the-frontier) (Ben Hylak, Raindrop, 19:46) - [Taming Rogue AI Agents with Observability-Driven Evaluation](https://aietalks.com/talks/taming-rogue-ai-agents-with-observability-driven-evaluation) (Jim Bennett, Galileo, 16:15) - [Engineering Better Evals: Scalable LLM Evaluation Pipelines That Work](https://aietalks.com/talks/engineering-better-evals-scalable-llm-evaluation-pipelines-that-work) (Dat Ngo & Aman Khan, Arize, 24:46)