An eval should test whether an AI system delivers the promise made to its users, rather than measure performance on one narrow task.
2
High-stakes evals should test every step of the workflow, combine automated checks with human review and adversarial tests, and measure each step according to its purpose.
3
Teams should preserve regression tests, make difficult tests harder over time, and turn real user failures into new cases in the eval pipeline.
Summary
Ramakrishnan Lokanathan explains how to evaluate AI systems used in high-stakes applications. He defines an eval as a set of tasks, standards, and evidence used to check whether system performance matches the promise made to a user. A high score on one test is only one signal, so teams should define the most ambitious version of success, test every part of the workflow, and use automated checks, human review, and adversarial cases. He introduces eSlams, an open-source Python package that lets AI systems play games such as chess, Go, poker, and cooperation games. These matches can expose planning, hidden-information decisions, and long sequences of choices. Lokanathan also argues that teams must preserve tests for tasks that already work, since upgrades can remove old abilities. Customer failures should be reproduced, traced to the failing step, and added to the eval suite.
An eval checks whether the product keeps its promise
Lokanathan defines an eval as an evaluation, similar to an examination for an AI system. It consists of tasks, standards, and evidence that show how a system performs against an expectation. An exam answers a bounded question, so even a familiar benchmark is only one signal of readiness. The SAT, for example, says something about language and mathematics, but does not measure complete college readiness. The job of an eval is to check whether the system's performance matches the promise made to its consumer or user at a particular point in time.
The target should describe the best job the system must do
Before designing tests, teams should define the most ambitious successful outcome. Lokanathan suggests asking what must happen, what must never happen, and when the system should stop. The goal is to describe the job the product needs to perform rather than choose a score that is easy to reach. If an eval scores 100 out of 100, he says the target is probably not ambitious enough or the test is not difficult enough. The evaluation should push beyond the model's current ability and expose where it fails.
A final answer can hide failures earlier in the workflow
Lokanathan uses an image-editing example to show why the final output is not enough. A request to change only the background produced the requested cream background, but also added two handles. A system that checks only the starting image and the final background could miss that error. In a workflow with 100 steps, every step needs checks for its inputs, outputs, and expected result. Tests should match the purpose of the step. A tool-calling step needs measures for tool-call ability and accuracy, rather than a generic end-to-end score.
A useful eval system combines automation, people, and attacks
For persistent agent systems, one score cannot describe all the capabilities involved. Lokanathan recommends automated eval checks inside the continuous integration pipeline, so tests run during each production deployment. Human reviewers should continue checking whether the results make sense, even when agents help speed up the process. Adversarial tests should place the workflow in difficult situations and look for ways to break it. These parts provide different evidence about the system and should be used together rather than reduced to one number.
Games can expose how agents make sequences of decisions
Lokanathan presents eSlams, an open-source Python package that makes AI systems play games against one another. He argues that getting one answer right does not show whether an agent can make a good sequence of decisions. Chess and Go require changing strategy, recovering from mistakes, and dealing with consequences from earlier moves. eSlams supports games involving planning and invisible state, hidden-information decisions such as poker, shared goals, and arcade-style trials. Lokanathan says a match between an old and improved agent is not by itself proof that the winner is the better model, but the way each agent plays can provide additional evidence about its reasoning.
Upgrades need tests for abilities that already worked
Lokanathan describes moving from Google Assistant to Gemini as a regression example. He used Google Assistant for reminders, but after the move to Gemini, the system stopped giving reminders and said that a language model could not set them up. The newer model could be better in many ways while losing an ability the older system handled well. Teams should therefore keep regression cases, known tasks, and past failures in the evaluation workflow. They should also add fresh challenge cases so the system continues to face tasks beyond its current level.
Real user failures should become permanent eval cases
Users provide evidence about failures that existing tests may miss. Lokanathan recommends observing the product through monitoring tools and simple user reports such as thumbs up or thumbs down. When a failure appears, the team should reproduce it, identify where the system failed, and check which part of the eval suite missed it. The failure might involve a tool call or another workflow step. After review, the case should enter the eval pipeline. This feedback loop adds the edge cases that appear in real use and keeps the test suite connected to production behavior.