Harness Engineering: How to Build Software When Humans Steer, Agents Execute

Ryan Lopopolo, OpenAI, Vibhu Sapra, Latent Space46:21 · Apr 2026 · 221K views
Thumbnail for Harness Engineering: How to Build Software When Humans Steer, Agents Execute Watch on YouTube
TL;DR
  1. 1

    Software engineers should spend less time producing code and more time designing systems, writing requirements, and delegating work to agents.

  2. 2

    A repository becomes an agent harness when its documentation, tests, lints, skills, and review agents repeatedly show models what acceptable work looks like.

  3. 3

    Human review feedback should be converted into durable repository rules so agents can correct recurring mistakes without waiting for a person each time.

Summary

Ryan Lopopolo argues that coding agents have made implementation abundant, so engineering work is moving toward system design, delegation, and context management. He describes a team that works through agents rather than editors, with Codex as the entry point to the repository and local development tools. Documentation, skills, lints, source-code tests, review agents, and useful error messages give agents the information they need while they work. The team turns recurring review comments into repository rules, such as requiring retries and timeouts around network calls or limiting files to 350 lines. Lopopolo also explains how repository structure, shared utilities, and package boundaries help agents work in large codebases. He recommends starting by having agents add tests or automate the slow parts of development. His longer-term goal is to give machines a ranked backlog, success measures, and reliability requirements, then let them advance the product with little synchronous human involvement.

Key ideas
01:35

Code production is abundant, so engineering work shifts toward delegation

Lopopolo says coding agents can now take more complex actions over longer periods, making implementation less scarce. He describes each engineer as having access to the capacity of "5, 50, or 5,000 engineers" around the clock, limited by GPU capacity and token budgets. The constraint moves toward human time, human and model attention, and model context windows. Engineers need to decide where human attention goes, automate repetitive work, and coordinate many concurrent agents. This changes the value of skills such as system design, systems thinking, and delegation.

06:42

Repositories need to explain what good work looks like

When agents write the code, the repository must carry the knowledge that experienced engineers normally apply through judgment and review. Lopopolo points to documentation, architecture decision records, persona-oriented guidance, ticket history, and code review history as "breadcrumbs" that lead agents toward acceptable results. Teams should write down non-functional requirements and make systems legible to models. Consistent structures also reduce the context and attention needed to understand a task. Since agents can refactor broadly, he recommends making code more uniform instead of leaving long-running migrations unfinished.

09:25

Recurring mistakes should become automated guardrails

Lopopolo recommends studying failures that agents and humans repeat, then eliminating each class of failure through documentation, lints, tests, or reviewer agents. At OpenAI, security and reliability review agents run in CI and inspect proposed changes against written requirements. One example checks that network calls have retries and timeouts. He also describes a source-code test that limits files to 350 lines because smaller files use context more efficiently. Error messages should explain the problem and give remediation steps, such as instructing an agent to avoid unknown types and follow the repository's parse-at-the-edge approach.

14:49

Prompting can happen throughout the development system

For Lopopolo, prompts are distributed across the repository rather than placed only in an initial user request. They can appear in agents.md files, rules, skills, lint failures, tests, pull-request comments, and review-agent instructions. He recommends refreshing context as long-running tasks continue, especially when automatic compaction removes earlier information. He even used Codex to study OpenAI's prompting cookbooks and create a skill for writing prompts. The resulting system lets the repository repeatedly give agents the instructions they need at the point where those instructions matter.

16:57

A QA plan lets review agents replace some human supervision

The team documented product features, critical user journeys, and how people interact with web apps, APIs, and services. That documentation supports a standard QA plan for user-facing work. A review agent can then check whether a pull request includes the required evidence, including the appropriate media attachments. Lopopolo says this raises his trust in the output, reduces the need to watch an agent closely, and lets him remove himself from more of the workflow. The repository carries one engineer's knowledge about QA so every agent trajectory can use it.

24:06

A good harness supplies instructions at the right time

In the Q&A, Lopopolo defines a harness as a system that gives the model the right text at the right time. It should avoid overwhelming the agent with every requirement at the start. For example, a harness can let an agent prototype a React interface, then use a lint or test failure to require smaller, more stateless components before the change is accepted. He expects coding harnesses such as Codex and Claude Code to keep improving, so he prefers plugging into them through skills, SDKs, or existing interfaces rather than rebuilding their internal mechanics.

33:55

Repository structure helps agents scale across a large codebase

Lopopolo describes moving from a blank repository that became messy to a workspace with 750 PNPM packages separated by business domain or stack layer. Package privacy, reusable utility packages, and source-structure checks give agents clearer boundaries. He recommends one canonical way to perform common operations, such as bounded concurrency, database access, CI scripting, and instrumentation. Consistency makes the tokens an agent needs more predictable across the repository. He also recommends keeping most changes local to a directory subtree and using agents to migrate the codebase toward the agreed structure.

36:53

Review feedback should close a self-healing loop

As agents increased the team's output to three to five pull requests per engineer each day, merge conflicts and review delays became problems. The team reduced conflicts by splitting the code into clearer areas and shortening the time pull requests stayed open. On Fridays, engineers held "garbage collection day" to remove recurring sources of slop. They grouped review comments by personas such as front-end architecture, reliability, and scalability, then created review agents that checked every push for issues at P2 or above. Human feedback became documentation and automated checks instead of repeated synchronous comments.

"A good harness is really operationalized around giving the model text at the right time so it can look at the work it has done and the information around what a good job looks like."24:26
Who should watch
  • You are still reviewing every agent-generated pull request by hand and want to turn repeated comments into tests, lints, or repository documentation.
  • Your team is increasing agent throughput but is running into merge conflicts, slow CI, unclear ownership, or agents that lose context during long tasks.
  • You are deciding whether to build a custom coding harness and want a practical case for focusing on context delivery and repository rules instead of recreating the underlying agent.