# Build Agents That Run for Hours

Ash Prabaker & Andrew Wilson, Anthropic | AI Engineer Europe 2026 | 1:15:40

Source: https://www.youtube.com/watch?v=mR-WAvEPRwE
Channel: AI Engineer (https://www.youtube.com/@aiDotEngineer). Summarised by AIE Talks.
Page: https://aietalks.com/talks/build-agents-that-run-for-hours
Published: 2026-05-18
Tags: agents, coding-agents, evals, harness-engineering, long-context

## TL;DR
- Long-running agents need a changing harness that addresses context, planning, and judgment as models improve.
- A separate adversarial evaluator produces better results than asking a generator to judge its own work.
- Structured handoffs, testable contracts, persistent files, and trace reading help agents work coherently for hours.

## Summary
Ash Prabaker and Andrew Wilson describe how Anthropic has built agents that can work for hours or days. They separate improvements in model capability from improvements in the harness around the model. The talk covers context management, planning, checkpoints, sub-agents, skills, compaction, and agent teams. Ash then presents a planner, generator, and adversarial evaluator pattern. The planner turns a broad request into high-level sprints. The generator and evaluator negotiate a specific contract before implementation. The evaluator uses the running application, rather than only reading code or diffs, and sends concrete critiques back to the generator. Rubrics make subjective qualities such as visual design more gradable. The speakers are candid that this approach is expensive and especially suited to greenfield applications. They also describe how harnesses should change with each model release. Some loops become unnecessary as models improve. Reading traces remains the main way to find where model judgment diverges from the intended behavior.

## Key ideas
### Long-running agents fail through context, planning, and self-judgment
[02:24](https://www.youtube.com/watch?v=mR-WAvEPRwE&t=144s)
Andrew Wilson groups the problem into three areas. A context window is finite, and a new session creates amnesia, so the agent needs memory components. Coherence also declines deeper into a session, which he calls context rot. Near the end of a window, a model can show context sense anxiety and hurry to finish. Planning is another weakness: a model may attempt everything in one shot, stop after half a feature, or run out of context with an unfinished application. Models also judge their own output poorly. A button may appear complete even though its backend does not exist, and the model may move on anyway.

### Model releases and harness changes improve agents together
[04:09](https://www.youtube.com/watch?v=mR-WAvEPRwE&t=249s)
The speakers describe two ways to extend agent runs. Models can improve through their weights, while the harness can add scaffolding around the model. Andrew gives a minimal-scaffold comparison that moves from about one hour with Opus 3.7 to about twelve hours with Opus 4.6. The harness includes the agent loop, tools, MCP servers, sub-agents, context files, skills, slash commands, and permissions. Andrew says these pieces co-evolve. A harness can fill a model's current gap, and later model training may make part of that harness unnecessary. The harness changes with each model release instead of disappearing when models improve.

### Fresh sessions, artifacts, and verification created the first long-running pattern
[11:59](https://www.youtube.com/watch?v=mR-WAvEPRwE&t=719s)
The first long-running harness began with an initializer agent that converted a vague request such as 'create a Slack clone' into persistent artifacts. It created a feature list, a progress file, a Git repository, an initialization script, and completion flags. The main loop started each task in a fresh context, checked the working directory and progress file, ran a smoke test, selected one unfinished feature, implemented it, and verified it with the running application. A passing feature was committed and marked complete. Andrew says JSON files were less likely than Markdown files to be overwritten by the model. The pattern combined planning, persistent state, fresh contexts, and a verification loop.

### A separate evaluator is better at criticism than a generator judging itself
[18:34](https://www.youtube.com/watch?v=mR-WAvEPRwE&t=1114s)
Ash Prabaker compares the generator-evaluator setup to a generative adversarial network. The generator builds, while a separate evaluator grades. The evaluator opens live pages with Playwright, clicks around, tries actions, and returns a critique. This differs from telling one Claude Code session to check its own work. Ash argues that tuning a standalone critic to be harsh is more tractable than making a builder self-critical. The evaluator can still prefer language-model-style output, but separating its context, system prompt, and job creates adversarial pressure. If a generator repeatedly fails one criterion, the harness can discard the work and start again instead of endlessly patching the same approach.

### Rubrics can make subjective output testable
[20:59](https://www.youtube.com/watch?v=mR-WAvEPRwE&t=1259s)
For full-stack applications, correctness is only one part of quality. Ash also considers visual appeal, feel, and taste. His rubric uses design, originality, craft, and functionality, with extra weight on design and originality because Opus 4.6 already performs well on functionality. Few-shot examples of reference sites calibrate the evaluator's preferences and help it avoid generic AI aesthetics such as purple gradients. The evaluator launches the application, takes screenshots, scores the four criteria, writes a critique, and hands it back to the generator. Ash says that a strong opinion about what good output looks like can be written down and graded.

### Planner, generator, and evaluator negotiate sprint contracts
[23:49](https://www.youtube.com/watch?v=mR-WAvEPRwE&t=1429s)
The planner takes a one-line request and turns it into a high-level sequence of sprints. It avoids specifying every technical detail because an early mistake could cascade through every later sprint. Before implementation, the generator proposes what it will build and how the evaluator should test it. The evaluator can reject an overly broad scope, strengthen weak tests, or add a missing edge case. They exchange files on disk until they agree. The evaluator then grades against this negotiated contract rather than the planner's original broad specification. Ash describes this as a bridge from user stories to tangible assertions. The planner sets the outer boundaries, while the builder and evaluator decide the exact feature and test details.

### Specific contracts make failures actionable
[32:05](https://www.youtube.com/watch?v=mR-WAvEPRwE&t=1925s)
In the Retro Forge example, the agents settled on 27 contract criteria. Ash says vague criteria produce vague critiques, which the generator can ignore. Granular criteria tell it exactly what to fix. The evaluator found issues that unit tests and a rough loop missed, including FastAPI route ordering that could pass tests but break in production and Boolean logic around the delete key. It also tested the application itself, including play mode. The resulting game had a live debug HUD, working arrow-key movement, and collision with castle walls. Ash attributes the difference between the solo run and the harness run to the scaffolding, while admitting that the harness was expensive and took about six hours for the example.

### Trace reading is the main way to improve an evaluator
[33:08](https://www.youtube.com/watch?v=mR-WAvEPRwE&t=1988s)
Ash is direct that Claude is a poor general QA agent out of the box. Its generosity bias leads it to find a bug and defer it instead of insisting on a fix. The team improved the evaluator by reading traces, locating where its judgment differed from a human judgment, and changing the prompt. He compares this to reading a stack trace. The team sometimes pipes transcripts into files and uses another agent to search or replay them, but manual reading remains the strongest method. Ash says engineers need to understand why the model made a decision, then change prompts, skills, rubrics, or other harness components based on that behavior.

### A stronger model can make parts of the harness unnecessary
[34:27](https://www.youtube.com/watch?v=mR-WAvEPRwE&t=2067s)
Ash says the harness must match the model's current weaknesses. Context resets were important for Opus 4.5 because it showed context anxiety, while Opus 4.6 could maintain a coherent two-hour build with one continuous session and compaction. Earlier versions ran the evaluator after every sprint. Later versions ran it after a complete one-shot generation. The lesson is not that the earlier harness was wrong. It was right for an earlier model. The final pattern kept the planner, generator, and evaluator while removing other components and using the file system for shared state. Ash says the simplified version was roughly half the cost of earlier runs, though still expensive.

## Notable quotes
- "The frontier doesn't really shrink, it just moves." (17:43)
- "Self-evaluation, very much a trap. Just use an adversarial evaluator." (39:03)
- "Compaction doesn't necessarily equal coherence, right? Lossy summaries really drift." (39:03)
- "The primary debugging loop was this, and not necessarily running more experiments. It was reading what the agent actually did." (33:28)
- "You got to read the whole thing. Read the whole thing." (43:36)

## Tools & references mentioned
- Anthropic
- Claude Code
- Claude Code SDK
- Agent SDK
- Claude for Chrome MCP
- Playwright MCP
- MCP
- Model Context Protocol
- Ralph Loop
- Ralph Wiggum technique
- Puppeteer
- Opus 3.7
- Opus 4.5
- Opus 4.6
- Sonnet 3.5
- Sonnet 3.7
- Sonnet 4.5
- Sonnet 4.6
- Haiku 4.5
- GANs
- Git
- FastAPI

## Who should watch
- You are building agents that need to work for several hours and want concrete patterns for state, verification, and context management.
- Your agent produces code that looks finished but fails when someone uses the application, and you need a separate testing and evaluation loop.
- You are maintaining a harness across model releases and need a way to decide which scaffolding to keep or remove.

## Editor's note

From the pack [Harness engineering](https://aietalks.com/packs/harness-engineering):

Templestein reconstructs state, Tahir reruns saved executions, and Govindarajan asks for evidence that survives the run. Kitaru records an agent's inputs, outputs and tool calls so a team can rerun the same task after changing a model, tool or policy, then inspect where the new run took a different path.

Written by the AIE Talks editors (the Kitaru team), not by the speaker.

## Related talks

- [Claude for Long-Horizon Tasks](https://aietalks.com/talks/claude-for-long-horizon-tasks) (Lance Martin, Anthropic, 25:19)
- [Autoresearch & Keynotes](https://aietalks.com/talks/autoresearch-keynotes) (Tariq Shihipar, Anthropic & Tariq Shakat, Sonar & Benois Schillings, Google DeepMind & Gabe Dees Mesa, OpenGov & Stefania Dug, Sakana AI & Tim Sweeney, Weights & Biases and CoreWeave & Arena, Arena & Dominic Tornow, Resonate & Nishan Gupta, Meta & Hio, Elastic & Bash, Visual Labs & Elie, Prime & Raymond Wei, OpenPros & Tis & Victor, Polygraph & Ean, Amnara & Roland & Aparna Dinakaran, Arize & Lakshya Agarwal & Rushabh, Machinecraft & Addy Osmani & George Cameron & Micah Hill-Smith, Artificial Analysis & Wayne Chiang, Arena & Amol, Nori Agentic & Zion, 8:51:56)
- [Beyond the Harness: A Journey Towards Adaptive Engineering](https://aietalks.com/talks/beyond-the-harness-a-journey-towards-adaptive-engineering) (Rajiv Chandegra, Annicha Labs, 37:01)
- [Respect The Process](https://aietalks.com/talks/respect-the-process) (Andrew Dumit, Watershed Technology Inc., 16:43)
- [Harness Engineering & Startup Battlefield](https://aietalks.com/talks/harness-engineering-startup-battlefield) (Garry Tan, Y Combinator & Mike Krieger, Anthropic & Theo Browne, t3.gg & Maxime Rivest & Isaac Miller, DSPy, 9:11:15)