# Why 80% Reliability Isn't Good Enough

Felipe Blanes, Amazon AGI Lab | AI Engineer | 17:45

Source: https://www.youtube.com/watch?v=Emo5FGGY-wM
Channel: AI Engineer (https://www.youtube.com/@aiDotEngineer). Summarised by AIE Talks.
Page: https://aietalks.com/talks/why-80-reliability-isnt-good-enough
Published: 2026-10-09
Tags: evals, product-strategy, reliability

## TL;DR
- Static benchmark results can look strong while customers encounter workflows the team never tested.
- An eval flywheel turns customer-defined success, product signals, conversations, diagnosis, and team decisions into a repeating development loop.
- Around 80% reliability still leaves customers monitoring and correcting the agent, while around 90% can make delegation feel trustworthy.

## Summary
Felipe Blanes argues that static evaluations create a benchmark illusion. Teams optimize public benchmarks or synthetic tests, ship the product, and then hope those tests match real customer behavior. Nova Act customers exposed workflows that the team had not imagined, so his team built an eval flywheel around customer feedback. The loop starts by defining success from the customer's perspective, then captures instrumentation and direct conversations, categorizes gaps across the model, engineering, and product, and feeds the results into decisions. Blanes also explains why reliability has a trust cliff: 80% can mean more monitoring and manual work, while around 90% can make customers willing to hand over a workflow. He advises teams to be explicit about what works, what is improving, and what is out of scope. Customer work also revealed needs such as trajectory caching, a non-technical interface, and more flexible actuation.

## Key ideas
### Nova Act was developed through direct customer work
[00:37](https://www.youtube.com/watch?v=Emo5FGGY-wM&t=37s)
Blanes introduces Nova Act as a tool for automating browser tasks and describes its path from research preview to an AWS service. The initial goal was to ship early and gather customer feedback. After the first months, the team changed the developer experience because customers were encountering gaps. Blanes worked day to day with customers to understand their problems and how the product helped solve them. That customer enablement work shaped the talk's central argument: evaluations have to stay connected to what customers actually do.

### Static benchmarks create a gap between product scores and production behavior
[02:22](https://www.youtube.com/watch?v=Emo5FGGY-wM&t=142s)
Blanes calls the problem the benchmark illusion. Teams optimize public benchmarks or their own synthetic data, then ship when the results look good. Customers subsequently try tasks nobody expected, and the product breaks. With a static eval, the team has little left to do after launch except hope that the tests represented customer behavior. Blanes says the answer is not simply better benchmarks. The team needs to close the customer loop and use what customers do to update the evaluation process.

### The eval flywheel starts with the customer's definition of success
[04:26](https://www.youtube.com/watch?v=Emo5FGGY-wM&t=266s)
The flywheel has four steps. First, define success according to what the customer needs, rather than what the team assumes matters. Second, capture signals through instrumentation and data, while also talking directly with customers because usage data can mislead or miss the underlying problem. Third, diagnose gaps and categorize them as model failures, engineering or harness problems, or product issues. Fourth, feed those categories into decisions for science, engineering, and product teams, then repeat the loop.

### Customer needs change through four stages of product use
[07:26](https://www.youtube.com/watch?v=Emo5FGGY-wM&t=446s)
Blanes describes a customer journey that changes the information an eval should capture. Use-case discovery establishes the customer's problem and checks whether the product can solve it. Early adopters reveal what is possible, including unexpected uses and tasks the product cannot yet handle. Scaling reveals patterns and the main use case that many customers share. The work also exposes product gaps that help the team decide what to build next. These stages mean the evaluation target changes as customers become more sophisticated.

### 80% reliability can make an agent feel like extra work
[09:35](https://www.youtube.com/watch?v=Emo5FGGY-wM&t=575s)
Blanes calls this the trust cliff. At 80% reliability, a customer may still need to monitor the agent, correct failures, and perform manual work. The agent then adds work instead of removing it. Around 90% reliability, the customer's interpretation changes and they may feel able to hand over a workflow. Blanes describes reliability as effectively binary from the customer's point of view: either they trust the agent with the task or they do not. The exact threshold depends on whether delegation removes work.

### Admitting limits can build more trust than a higher benchmark score
[11:10](https://www.youtube.com/watch?v=Emo5FGGY-wM&t=670s)
Blanes says teams should explain three things to customers: what works, what the team is improving, and what is out of scope. The last category is difficult because teams do not want to advertise product weaknesses. Still, customers can quickly move to another tool if they discover a capability does not work after being encouraged to try it. Blanes's finding is that being transparent about product limits builds more trust than presenting higher benchmark numbers without explaining where the product fails.

### Customer use cases exposed product changes the team would not have invented
[12:45](https://www.youtube.com/watch?v=Emo5FGGY-wM&t=765s)
Work with Amazon Leo led to trajectory caching. Instead of running inference every time, the product could cache trajectories and fall back to the model when needed, reducing the cost for that customer. Hertz showed that some QA engineers could write scripts while others needed a non-technical experience, which informed a different product interface. Sola, an RPA company with many varied automation workflows, led the team to add flexibility for customizing the tool's actuation stack. These examples show how customer work supplied concrete product requirements.

### Evals should be drawn from production and updated every week
[14:44](https://www.youtube.com/watch?v=Emo5FGGY-wM&t=884s)
Blanes gives four principles for the flywheel. Derive eval scenarios from production rather than imagination, although synthetic data can still help early in development. Categorize findings into actionable research, engineering, and product areas. Share capabilities honestly, especially the things the product does poorly. Run the loop quickly so the product improves faster. He says evals should get smarter every week. If they do not, they are becoming stale because the customer's work and expectations are changing.

## Notable quotes
- "This is not about better benchmarks. It's about closing the customer loop." (04:26)
- "Your data might be a little bit misleading." (05:31)
- "Transparent on product limitation increase trust more than higher benchmarks." (12:04)
- "Evals are not built once. It's a flywheel that evolves as your customer evolves." (16:30)

## Tools & references mentioned
- Amazon AGI Lab
- Nova Act
- AWS
- Amazon Leo
- Hertz
- Sola
- RPA

## Who should watch
- You have an AI product that performs well on internal or public benchmarks but fails on customer workflows after launch.
- Your team needs a practical way to turn customer conversations and production behavior into model, engineering, and product work.
- You are deciding how much to disclose about an agent's limits and want customers to trust its actual capabilities.

## Editor's note

Felipe Blanes says static benchmarks create a "benchmark illusion" because customers try workflows the team never tested. Kitaru records real agent runs, including model calls and tool results, then replays them with the same inputs and responses. A production failure can become a repeatable run for checking a change to the model, prompt, or code.

Written by the AIE Talks editors (the Kitaru team), not by the speaker.

## Related talks

- [Building and evaluating AI Agents](https://aietalks.com/talks/building-and-evaluating-ai-agents) (Sayash Kapoor, AI Snake Oil, 20:00)
- [How Evals and Prompts Shape Agent Behavior](https://aietalks.com/talks/how-evals-and-prompts-shape-agent-behavior) (Preetika Bhateja, Google/YouTube & Daniel Bump & Chris Souza, Google, 19:29)
- [Why (Senior) Engineers Struggle to Build AI Agents](https://aietalks.com/talks/why-senior-engineers-struggle-to-build-ai-agents) (Philipp Schmid, Google DeepMind, 10:40)
- [Practical Tactics to Build Reliable AI Apps](https://aietalks.com/talks/practical-tactics-to-build-reliable-ai-apps) (Dmitry Kuchin, Multinear, 14:55)
- [Turning Fails into Features: Zapier's Hard-Won Eval Lessons](https://aietalks.com/talks/turning-fails-into-features-zapiers-hard-won-eval-lessons) (Rafal Wilinski & Vitor Balocco, Zapier, 16:15)