# Smarter Models Killed the Harness. Here's What's Left

Jediah Katz, SpaceXAI | AI Engineer NYC 2026 | 18:15

Source: https://www.youtube.com/watch?v=7Y4sbZgBdnc
Channel: AI Engineer (https://www.youtube.com/@aiDotEngineer). Summarised by AIE Talks.
Page: https://aietalks.com/talks/smarter-models-killed-the-harness-heres-whats-left
Published: 2026-10-19
Tags: agents, cost, harness-engineering, multi-agent, security

## TL;DR
- Smarter models have absorbed many coding skills that agent harnesses once supplied through prompts, tools, and guardrails.
- The harness still matters because it opens new kinds of work and handles infrastructure, trust, security, and efficiency that models cannot absorb.
- Jediah Katz predicts that 90% of agent work will run off the laptop within six months.

## Summary
Jediah Katz traces the shift from prompt engineering to context engineering and then to agent-harness engineering. He divides a harness into a runtime layer, which manages execution and state, and context and orchestration layers, which decide what models see and which tools they use. Smarter models have learned many behaviors that harnesses once patched in, such as codebase exploration, persistent work, and test execution. Katz calls this a data flywheel: harnesses make harder tasks possible, usage creates training data, and models absorb the resulting skills. He argues that this does not make harnesses obsolete. Products such as GrokBot use harnesses to coordinate long-lived agents across new domains. Harnesses also own cloud infrastructure, permissions, safety controls, and cost management. Katz expects harnesses to become easier to change, expose fewer internal details, use reusable agent primitives, and move from coordinating model calls to coordinating agents.

## Key ideas
### An agent harness connects a model to computers and services
[01:29](https://www.youtube.com/watch?v=7Y4sbZgBdnc&t=89s)
Katz describes the harness as the model's eyes and hands. Users reach it through an app, while it connects the model to computers and third-party services such as MCPs. He separates the system into a runtime and context and orchestration layers. The runtime controls where the agent loop runs and how state is stored so work can resume. The other layer chooses which models to call, what context they receive, and which prompts, tools, memory compaction, and behavioral techniques shape the run.

### Smarter models absorbed the coding tricks that harnesses used to provide
[05:43](https://www.youtube.com/watch?v=7Y4sbZgBdnc&t=343s)
Katz accepts the strongest version of the claim that the harness is dead. A fancy harness no longer adds coding intelligence in the same way it once did. Earlier systems used a small model to apply edits, instructions for exploring a codebase, and reminders to work persistently and run tests. He says models have natively absorbed these skills. He cites studies comparing harnesses including Pi, Codex, and Claude Code on SWE-bench Lite, along with an Apple and EPFL study of MLE-bench, where harness choice did not significantly change scores for newer models.

### The data flywheel makes parts of the harness obsolete
[06:32](https://www.youtube.com/watch?v=7Y4sbZgBdnc&t=392s)
The harness first brings harder coding tasks within reach. More users can then attempt those tasks with agents, producing more coding data for model labs. Models learn the skills natively, and the harness can delete the special mechanisms it needed before. This creates a cycle in which the harness helps models improve, then removes some of its own coding-specific behavior. Katz uses this process to explain why people began saying the harness was dead.

### A new frontier requires a new harness
[07:18](https://www.youtube.com/watch?v=7Y4sbZgBdnc&t=438s)
Katz argues that the harness starts the next cycle when an existing capability becomes routine. He uses GrokBot as an example. It relies on long-lived coordinator bots that direct a fleet of worker agents rather than handling one-off chats. Its harness adds prompting and guardrails because tasks such as making slide decks, booking travel, communicating with users, acting without asking every time, and learning a person's preferences remain difficult. The harness makes these tasks feasible while the models continue to learn.

### Agent coordination can cover a software development lifecycle
[10:53](https://www.youtube.com/watch?v=7Y4sbZgBdnc&t=653s)
Katz describes a software-factory pattern in which agents coordinate across the development process. A bug reported in an internal channel can be triaged automatically, sent to a cloud agent for a fix, and checked with before-and-after videos. After deployment, another agent watches production for problems. He calls this a meta harness because it coordinates many agents rather than only shaping one model call. The same pattern can extend software work beyond writing code.

### Cloud infrastructure and security remain harness responsibilities
[12:13](https://www.youtube.com/watch?v=7Y4sbZgBdnc&t=733s)
Autonomous work now runs for minutes, hours, days, or longer, so it cannot depend on a laptop that may go to sleep. A reliable autonomous colleague needs cloud infrastructure and a runtime that stays available. Katz also argues that models should not receive unrestricted access to API keys. Companies need permissions scoped to agents and people, along with controls that prevent dangerous actions or unexpectedly expensive API use.

### Efficiency matters even when harnesses tie on eval scores
[13:13](https://www.youtube.com/watch?v=7Y4sbZgBdnc&t=793s)
The studies Katz cites found a significant difference in cost per task even when harness choice did not significantly change evaluation scores. His work on the GrokBot harness focused on cache hit rate, token efficiency, and model mix. He says these changes reduced cost and speed for users without reducing quality. As people delegate more work to agents, the efficiency of the harness becomes more important than a small difference in coding evaluation results.

### Future harnesses will use primitives and coordinate agents
[14:12](https://www.youtube.com/watch?v=7Y4sbZgBdnc&t=852s)
Katz expects the product layer to move faster than the models for a while, which will make harnesses messy before they are cleaned up. Teams will need systems that let them ship and remove features easily. He points to primitives such as subscriptions, where one interface lets an agent wake up for Slack messages, failed CI, or scheduled events. He also describes centrally tracked shims that can be removed when a model no longer needs them. New harnesses will use agents as their building blocks, rather than composing many individual model calls.

## Notable quotes
- "A fancy harness in this day and age, no longer adds coding intelligence." (05:43)
- "It turns out the harness is actually making itself obsolete through what I call the data flywheel." (06:32)
- "GrokBot, that example, lets us fill in the last arrow in our cycle, which is when a capability like coding gets commoditized by the model, it's actually the harness which unlocks a new frontier for the model to then solve and kicks off that flywheel again." (10:13)
- "Within 6 months, 90% of agent work is going to run off the laptop." (16:59)
- "Yesterday's harness may be dead, but long live the agent harness of tomorrow." (17:41)

## Tools & references mentioned
- Cursor
- SpaceX
- MCPs
- Grind Swarm
- Anthropic
- Claude Code
- Tariq
- Garry Tan
- Y Combinator
- UC Berkeley
- Pi
- Codex
- SWE-bench Lite
- Apple
- EPFL
- MLE-bench
- GrokBot
- Hyrum's Law

## Who should watch
- You are building an agent product and need to decide which behavior belongs in the model versus the harness.
- Your coding agents already work well, but cloud execution, permissions, safety controls, or operating cost are becoming problems.
- You want a practical model for designing systems that coordinate long-running agents across software and knowledge work.

## Editor's note

From the pack [Harness engineering](https://aietalks.com/packs/harness-engineering):

Templestein reconstructs state, Tahir reruns saved executions, and Govindarajan asks for evidence that survives the run. Kitaru records an agent's inputs, outputs and tool calls so a team can rerun the same task after changing a model, tool or policy, then inspect where the new run took a different path.

Written by the AIE Talks editors (the Kitaru team), not by the speaker.

## Related talks

- [What if the harness mattered more than the model?](https://aietalks.com/talks/what-if-the-harness-mattered-more-than-the-model) (Aditya Bhargava, Etsy, 32:04)
- [Harness Engineering: Building the Production Cage for Powerful Domain Agents](https://aietalks.com/talks/harness-engineering-building-the-production-cage-for-powerful-domain-agents) (Mike Chambers, AWS, 20:46)
- [Harnesses in AI: A Deep Dive](https://aietalks.com/talks/harnesses-in-ai-a-deep-dive) (Tejas Kumar, IBM, 20:27)
- [Every Harness Will Become A Claw](https://aietalks.com/talks/every-harness-will-become-a-claw) (Sam Bhagwat, Mastra, 15:36)
- [Harness Engineering is Not Enough: Why Software Factories Fail](https://aietalks.com/talks/harness-engineering-is-not-enough-why-software-factories-fail) (Dex Horthy, HumanLayer, 19:18)
