# Harness Engineering & Startup Battlefield

Garry Tan, Y Combinator & Mike Krieger, Anthropic & Theo Browne, t3.gg & Maxime Rivest & Isaac Miller, DSPy | AI Engineer World's Fair 2026 | 9:11:15

Source: https://www.youtube.com/watch?v=I2cbIws9j10
Channel: AI Engineer (https://www.youtube.com/@aiDotEngineer). Summarised by AIE Talks.
Page: https://aietalks.com/talks/harness-engineering-startup-battlefield
Published: 2026-07-03
Tags: evals, harness-engineering, memory, security

## TL;DR
- AI engineering is shifting from model selection toward harnesses, shared memory, permissions, evaluation, and durable execution.
- Agents can take on larger and longer-running tasks, but reliable autonomy still depends on deterministic verification and human ownership of architecture and risk.
- The most promising startups use agents to combine domain knowledge, distribution, and software execution in industries that were previously difficult to serve.

## Summary
This World's Fair closing-day program connects several views of harness engineering. Bar Yaron's survey reports that agents are moving from reading and drafting toward write-enabled actions, while cost, evaluation, and primitive guardrails remain major concerns. John Ousterhout presents Homa, a network protocol designed for the small, latency-sensitive messages increasingly used by inference and agentic systems. DSPy argues for separating a task's specification, code, and evaluations from its implementation. Other speakers cover durable execution, shared memory, tool selection, security scanning, file-based agents, and company knowledge systems. The loops debate is more cautious: loops work well where outcomes are verifiable, but they do not remove the need to read code, make architectural decisions, or control cost. Garry Tan and Theo argue that agents change the size and shape of viable companies. The Startup Battlefield then shows three applications built around commodity trading, collaborative documents, and creator-led products, with Comment IO winning the competition.

## Key ideas
### AI teams are using more agents with more authority, while their controls remain basic
[16:59](https://www.youtube.com/watch?v=I2cbIws9j10&t=1019s)
Bar Yaron reports that 95% of survey respondents say they use agents, roughly double the previous year. Among teams using agents, the share giving them write access rose from 52% to 89%. Agents are now taking actions inside systems rather than only reading, summarizing, or drafting. Human approvals and permission gates are the most common controls, while teams also try task decomposition, retrieval, memory, and sandboxing. No control layer has become standard. Cost is also shaping product decisions: 40% say it regularly limits how ambitiously they use AI, and another 36% say it sometimes does. Evaluations remain the most commonly reported stack challenge.

### Inference and agentic workloads make network tail latency a product concern
[35:56](https://www.youtube.com/watch?v=I2cbIws9j10&t=2156s)
John Ousterhout explains that older AI workloads moved huge amounts of data, so throughput mattered most. Inference and agentic workloads increasingly exchange small messages for metadata, KV-cache checks, and synchronization. When several machines send data to one destination, a short message can sit behind large messages in a queue. TCP and RDMA also treat data as a byte stream, which prevents message prioritization and can create head-of-line blocking. Homa addresses this with message-based transport, receiver-controlled congestion, shortest-remaining-processing-time prioritization, and switch priority queues. In his benchmark, Homa's short-message P99 latency is under 100 microseconds compared with more than a millisecond for TCP.

### DSPy makes an AI task portable by separating its contract from its implementation
[54:56](https://www.youtube.com/watch?v=I2cbIws9j10&t=3296s)
Maxime Rivest and Isaac Miller describe DSPy as a way to give AI programs the properties of ordinary functions: reusable interfaces, composition, testing, and optimization. A task should have a specification, code that enforces hard requirements, and evaluations that define what good means. The implementation can then change from a simple prompt to chain-of-thought, an agent, a recursive language model, or a learned harness without changing the surrounding integration. They cite a Shopify case where this approach made an implementation 550 times cheaper by moving to a less expensive model while keeping the same business logic and evaluations. DSPy Flex extends optimization toward learned code harnesses, while qualitative learning aims to turn production feedback into better evaluations.

### Reliable agents need infrastructure for persistence, context, and bounded action
[2:06:20](https://www.youtube.com/watch?v=I2cbIws9j10&t=7580s)
Across the program, speakers describe the harness as the part around the model. It includes runtime, tools, context, memory, feedback, permissions, evaluation, identity, and human review. Caitlin Sadowski and Angela describe strategies where tokens have different jobs: an executor performs work, an advisor guides it, a grader checks it against a rubric, and a dreamer writes lessons to memory. Their financial-analysis experiments found different strategies produced different accuracy and token costs under a fixed budget. The point is that more tokens alone are not enough. Giving tokens different jobs can improve results within the same budget, with advice favoring token efficiency and grading or dreaming favoring reliability.

### Loops are useful for verifiable work, but they do not eliminate engineering judgment
[3:40:56](https://www.youtube.com/watch?v=I2cbIws9j10&t=13256s)
The loops debate separates what works now from what is still hype. Jeff Huntley argues that loops turn engineering feedback into an executable process, especially when tests, static analysis, pre-commit hooks, and other checks keep the agent on the rails. Jax and Greg Pastuzka warn that looping can multiply poor output, cost, and nondeterminism. Their view is that agents can handle bounded, well-specified tasks with strong verification, while humans still need to choose architecture, decide what not to build, and inspect qualitative results. The debate converges on a practical position: experiment with loops, build small feedback loops, and keep human ownership of decisions that cannot yet be checked mechanically.

### Shared memory needs structure, provenance, and access control
[4:34:56](https://www.youtube.com/watch?v=I2cbIws9j10&t=16496s)
Several talks treat memory as a core part of an agent system rather than a convenience. Garry Tan describes a company brain as a library plus a librarian. Retrieval alone is easy; the hard work is deciding what gets recorded, enriched, linked, promoted to active context, and reconciled when facts conflict. He warns that an uncured memory becomes a garbage dump with good search. Other speakers propose logs as the durable identity of an agent, shared memory brokers for teams, and graphs that connect repositories, sessions, dependencies, and decisions. These systems preserve context across agents and sessions, but they also create permissions and attribution problems that existing developer tools were not designed to solve.

### Security must be enforced outside the model
[5:45:00](https://www.youtube.com/watch?v=I2cbIws9j10&t=20700s)
Nikita Kothari distinguishes CLI tools, MCP services, and skills. CLIs are transparent and reproducible, MCP provides shared services and server-side isolation, and skills encode a bounded sequence using selected tools. Sarah from PostHog makes the security boundary sharper: prompts steer behavior but do not enforce security. Her Warlock scanner uses deterministic YARA rules to inspect context entering the agent and output leaving it. The scanner detects and recommends an action, while enforcement stays mechanical. PostHog also denies commands by default, uses a vault for secrets, sandboxes execution, and scans its own documentation and skill supply chain because seemingly harmless content can carry prompt injection. The lesson is that attacks compose across systems, while code review often examines only one change at a time.

### Agents expand the startup surface by lowering the cost of specialized software
[7:57:00](https://www.youtube.com/watch?v=I2cbIws9j10&t=28620s)
Theo argues that models have moved many projects down a level of difficulty. A service he once built to triage pull requests can now be a Markdown file run on a cron job. He urges builders to question inherited interfaces and to design for wider product surfaces, because agents can fill in missing features for users. Garry Tan frames this as an organizational change: skill files act like employees, resolver tables act like an org chart, and trigger evaluations act like performance reviews. Mike Krieger gives a more measured startup case. A small group can understand a vertical and iterate faster than a large lab, but the lasting advantage is still domain knowledge, distribution, taste, and user understanding. The battlefield finalists apply this idea to commodity trade, collaborative work documents, and creator businesses.

## Notable quotes
- "Cost is now a first-class engineering constraint." (24:39)
- "Intelligence is very different from being all-knowing." (1:48:53)
- "If it isn't enforced deterministically, it is not enforced." (5:54:25)
- "The agent is its data. It's specifically the log." (6:44:41)
- "If you have to ask for something twice, you failed." (8:14:22)

## Tools & references mentioned
- AI Engineer World's Fair 2026
- Amplify
- Notion
- Vercel
- Homa
- Stanford University
- Ben Moazeni
- DSPy
- DSPy Flex
- recursive language models
- Jepa
- GRPO
- Shopify
- Anthropic
- Claude
- Claude Code
- Fable
- Mythos
- Claude Design
- MCP
- Neo4j
- AWS Glue
- Amazon EventBridge
- Amazon Bedrock Agent Core
- Replit
- WorkOS
- auth.md
- Cloudflare
- NATS.io
- Re-state
- Resonate
- PostHog
- Warlock
- Yara
- TurboCon
- Google Research
- Obsidian
- Readwise
- NotebookLM
- Codex
- Kiro
- Garry Tan
- Y Combinator
- G brain
- OpenClaw
- Hyperagent
- Airtable
- Eve
- Vercel AI SDK
- Komod
- Comet IO
- Built by Foundry
- Comment IO

## Who should watch
- You are building agents that need to run for hours, survive failures, retain context, or operate across many users and services.
- Your team is giving agents write access and wants practical guidance on permissions, evaluation, memory, security, and tool selection.
- You are deciding whether to invest in loops, skills, company knowledge systems, or a vertical AI startup and want both aggressive and skeptical views.

## Related talks

- [Harnesses in AI: A Deep Dive](https://aietalks.com/talks/harnesses-in-ai-a-deep-dive) (Tejas Kumar, IBM, 20:27)
- [Harness Engineering: How to Build Software When Humans Steer, Agents Execute](https://aietalks.com/talks/harness-engineering-how-to-build-software-when-humans-steer-agents-execute) (Ryan Lopopolo, OpenAI & Vibhu Sapra, Latent Space, 46:21)
- [Autoresearch & Keynotes](https://aietalks.com/talks/autoresearch-keynotes) (Tariq Shihipar, Anthropic & Tariq Shakat, Sonar & Benois Schillings, Google DeepMind & Gabe Dees Mesa, OpenGov & Stefania Dug, Sakana AI & Tim Sweeney, Weights & Biases and CoreWeave & Arena, Arena & Dominic Tornow, Resonate & Nishan Gupta, Meta & Hio, Elastic & Bash, Visual Labs & Elie, Prime & Raymond Wei, OpenPros & Tis & Victor, Polygraph & Ean, Amnara & Roland & Aparna Dinakaran, Arize & Lakshya Agarwal & Rushabh, Machinecraft & Addy Osmani & George Cameron & Micah Hill-Smith, Artificial Analysis & Wayne Chiang, Arena & Amol, Nori Agentic & Zion, 8:51:56)
- [Build Agents That Run for Hours](https://aietalks.com/talks/build-agents-that-run-for-hours) (Ash Prabaker & Andrew Wilson, Anthropic, 1:15:40)
- [What if the harness mattered more than the model?](https://aietalks.com/talks/what-if-the-harness-mattered-more-than-the-model) (Aditya Bhargava, Etsy, 32:04)