# E-Values: Evaluating the Values of AI

Sheila Gulati, Tola Capital & Nischal Nadhamuni, Klarity | AI Engineer World's Fair 2024 | 28:13

Source: https://www.youtube.com/watch?v=mpzktdYTcdE
Channel: AI Engineer (https://www.youtube.com/@aiDotEngineer). Summarised by AIE Talks.
Page: https://aietalks.com/talks/e-values-evaluating-the-values-of-ai
Published: 2024-12-31
Tags: evals, multimodal, observability, synthetic-data

## TL;DR
- AI evaluations need to measure whether systems meet their goals, including user needs, safety, context, and values.
- Static leaderboards can reward models for solving known benchmarks without understanding the underlying task.
- Klarity builds evaluations around user outcomes, customer-specific data, synthetic data, feedback, drift monitoring, and fast product development.

## Summary
Sheila Gulati argues that AI evaluations are entering a harder phase as systems become agentic, self-learning, multimodal, and more closely tied to human goals. Existing benchmarks often measure narrow capabilities and can be gamed by models that learn what answer a test expects. Evaluations also need to account for context, users, safety, and whose values a system should reflect. Nischal Nadhamuni explains how Klarity handles these problems while automating document-heavy finance and accounting workflows. Generative AI made their products faster to build, but it also introduced nondeterministic outputs, unfamiliar user experiences, and a growing gap between public benchmarks and real customer performance. Klarity tests features with users early, then builds metrics around business outcomes and use-case-specific accuracy. The company also uses synthetic data, monitors drift, limits architectural complexity, and maintains lightweight evaluations for future capabilities. The speakers end by asking AI builders to replace shallow leaderboards with deeper, community-built evaluations.

## Key ideas
### Agentic systems make evaluation urgent because they act on more of a person's behalf
[00:00](https://www.youtube.com/watch?v=mpzktdYTcdE&t=0s)
Sheila Gulati says evaluations matter more as AI moves into agentic frameworks and becomes more automated. AI reflects human understanding, intentions, preferences, and society, so evaluating a system means asking whether its performance matches the goals set for it. The pressure increases as models move toward self-taught, self-learning, and self-sufficient behavior. She argues that the industry needs to improve evaluations before systems become fully automated. In this setting, a model's technical score alone cannot explain whether it is acting appropriately for its users or society.

### Broad AI needs evaluations for capability, values, safety, context, and modality
[03:12](https://www.youtube.com/watch?v=mpzktdYTcdE&t=192s)
Narrow AI could often be tested against a clear task, such as classifying an image or hitting a target. Sheila describes broad AI evaluation as a radar with several dimensions. It must ask what the system can do, whether it understands a domain, whose values it follows, and what safety means for different people. It also needs end-user awareness, which she says is often treated as an afterthought. These questions must apply across image, text, video, and combinations of modalities. The result is a much larger evaluation problem than the earlier task-specific tests.

### Known benchmarks can reward models for solving the test instead of understanding the task
[07:52](https://www.youtube.com/watch?v=mpzktdYTcdE&t=472s)
Sheila calls this problem benchmark hacking. If a model knows what a benchmark is asking for, it can optimize for the expected answer and appear intelligent without understanding the subject. She compares this with an exam question: passing an AP exam may show that a model was trained on similar questions rather than that it understands the subject. She points to dynamic benchmarks as one response. A Microsoft Research approach generates changing visual data by moving objects, which tests spatial reasoning, visual prompting, and object recognition without letting models memorize published answers.

### Real-world tasks can expose weaknesses hidden by leaderboard scores
[09:31](https://www.youtube.com/watch?v=mpzktdYTcdE&t=571s)
Sheila says benchmark results do not always predict performance in real scenarios. A model can perform well on MMLU and still give wrong answers to specific logic, reasoning, or finance questions. She cites FinanceBench, work associated with Patronus AI and Stanford, as an example of testing basic financial questions in realistic settings. She also says the user should be at the center of evaluation. Teams need stronger user experience and feedback systems because the purpose of these systems is to deliver value to people. Evaluating that experience is difficult, but omitting it leaves a major part of system performance unmeasured.

### Evaluation is a proxy for truth, so interpretability matters for black-box systems
[10:49](https://www.youtube.com/watch?v=mpzktdYTcdE&t=649s)
Sheila describes evaluation as a proxy for a task rather than truth itself. Neural networks cannot yet be mathematically represented in a complete way, which makes it difficult to know what they learned or how they reached an answer. She points to AI interpretability research that traces information through Transformer-based language models. Researchers can inspect how questions are answered and how internal steps connect. She expects more work in interpretability and reinforcement learning around current GPT models because scores alone cannot explain whether a system is behaving in line with its intended goals.

### AI builders must decide whose values a model should reinforce or challenge
[11:53](https://www.youtube.com/watch?v=mpzktdYTcdE&t=713s)
Sheila says models acquire human values whether their creators intend this or not. An assistant tuned closely to one person's beliefs may reinforce that person's views and create a deeper echo chamber. She asks whether systems should appease users or challenge them, whether users should choose, and whether models should follow present social values or future aspirational values. She does not offer a simple framework or final answer. Her point is that these choices belong inside evaluation work, especially as agents represent people more directly and make decisions across longer chains of activity.

### Klarity evaluates generative features from user outcomes backward
[20:08](https://www.youtube.com/watch?v=mpzktdYTcdE&t=1208s)
Nischal Nadhamuni says Klarity learned to frontload user testing and backload expensive evaluation work. Each generative feature can have its own product-market-fit risk because customers are being given experiences they have not had before, such as chatting with documents or asking for natural-language analytics. Once a feature is worth taking into production, the team works backward from the user experience rather than forward from whatever is easiest to measure. They map the end-user outcome to business value, adoption or utilization, and lower-level health signals such as variation in JSON outputs. Individual metrics do not need to capture the whole experience, but the collection should reflect it.

### Klarity combines customer-specific tests with synthetic data and controlled complexity
[21:07](https://www.youtube.com/watch?v=mpzktdYTcdE&t=1267s)
Klarity's customers provide their own labels during user acceptance testing, allowing the company to build accuracy metrics for tasks such as matching and extraction. User feedback helps close the loop, but Nadhamuni warns that a lack of feedback should not be treated as positive feedback. The team monitors data drift and built its own synthetic data generation stack after testing several providers. When a use case becomes large enough, Klarity develops customer-agnostic evaluations and checks whether synthetic data resembles customer data. The company also reduces complexity by using one language model for automated prompt engineering tasks instead of exploring every possible model combination. For future workflows, it keeps lightweight evaluations that test emerging model capabilities quickly.

## Notable quotes
- "Evaluation is a proxy for a task. Evaluation is seeking truth, it is not truth." (10:49)
- "We have to think backwards from the user experience, not forwards from what is easy to measure." (20:58)
- "We try to put the majority of the onus on ourselves to be rigorous about metrics." (22:04)
- "It is too high of a bar to say we want to have MMLU or BBH level metrics for future workflows that nobody has asked us for yet." (24:24)

## Tools & references mentioned
- Tola Capital
- Klarity
- Microsoft Research
- FinanceBench
- Patronus AI
- Stanford
- MMLU
- GPT
- ChatGPT
- The Alignment Problem
- Brian Christensen
- Microsoft
- Azure

## Who should watch
- You are building an agent or other generative system whose behavior cannot be captured by one accuracy score.
- Your product team ships features faster than it can create careful evaluations and needs a way to prioritize user testing.
- You need to connect model metrics with customer outcomes, domain-specific data, synthetic data, or social values.

## Related talks

- [Evals](https://aietalks.com/talks/evals) (Omar Khattab & Vitor Balogh & Rafal Wilinski, Zapier & Ido Pesal, Vercel & Randall, Bolt Foundry & Diego Rodriguez, Krea & Doug Guthrie & Ankur Goyal, Braintrust & Jeff Huber, Chroma & Jason Liu, 567 Studio & John Dickerson, Mozilla AI, 3:59:12)
- [Your Evals Are Meaningless (And Here's How to Fix Them)](https://aietalks.com/talks/your-evals-are-meaningless-and-heres-how-to-fix-them) (, 18:50)
- [Mastering AI Evaluation: From Playground to Production](https://aietalks.com/talks/mastering-ai-evaluation-from-playground-to-production) (Doug & Carlos Essan, Braintrust, 1:25:08)
- [Why should anyone care about Evals?](https://aietalks.com/talks/why-should-anyone-care-about-evals) (Manu Goyal, Braintrust, 05:41)
- [The ROI of AI: Why You Need an Eval Framework](https://aietalks.com/talks/the-roi-of-ai-why-you-need-an-eval-framework) (Beyang Liu, Sourcegraph, 25:28)