# Codex, Behind the Harness Dominik Kundel, OpenAI | AI Engineer World's Fair 2026 | 20:55 Source: https://www.youtube.com/watch?v=shRR1e2HXMk Channel: AI Engineer (https://www.youtube.com/@aiDotEngineer). Summarised by AIE Talks. Page: https://aietalks.com/talks/codex-behind-the-harness Published: 2026-08-10 Tags: agents, computer-use, context-engineering, security ## TL;DR - Codex separates its app server protocol from the Responses API, allowing custom interfaces and compatible model providers to use the same harness. - The harness controls context size with deferred tools and a skills-list cap, then carries out file and computer actions inside platform-specific sandboxes. - WebSocket mode, auto review, goal continuation, and server-side compaction address network delay, unsafe approvals, long-running tasks, and growing context. ## Summary Dominik Kundel explains what happens inside the Codex harness after a user sends a message. The app server connects interfaces to the harness, while the Responses API connects the harness to inference. Much of the engineering work concerns context construction, since size, flexibility, and cachability pull in different directions. Deferred tools stay out of the context until tool search finds them, and the available skills list is limited to 2% of the maximum context window. Kundel also covers sub-agents, browser automation through a persistent Node REPL, file editing with apply patch, shell use, and platform-specific sandboxes. Auto review uses a read-only sub-agent to judge risky actions against the user's authorization and the conversation. WebSocket mode reduces network overhead for fast models, while goal continuation and compaction help agents run for longer. The Codex harness is open source, and several of its capabilities are exposed directly through the Responses API. ## Key ideas ### Codex uses separate protocols for interfaces and inference [01:38](https://www.youtube.com/watch?v=shRR1e2HXMk&t=98s) The Codex app server handles communication from a user interface to the harness. The Responses API handles communication from the harness to model inference. Both are designed for an open ecosystem. Developers can build their own interfaces on the app server protocol, which powers the Codex app and community projects such as T3 Code and Remote X. Kundel also used it to put Codex into Claude Code and Doom. The Responses API adds agent capabilities such as web search and image generation, and an open Responses schema lets compatible providers, including Ollama, LM Studio, and NVIDIA, plug into the harness. ### Context construction balances size, flexibility, and cachability [04:12](https://www.youtube.com/watch?v=shRR1e2HXMk&t=252s) Kundel describes context construction as one of the most important steps in the harness. The context should not waste token budget, because more material can also create contradictory information that confuses the model. At the same time, it needs to work across different numbers of skills, plugins, and MCP installations. Predictable model instructions are easier to manage, while the tool registry and available skills list can grow. Codex marks some tools as deferred, keeping them out of the context window until tool search is needed. It caps the available skills list at 2% of the maximum context window and trims descriptions as the list grows. ### The harness gives agents separate mechanisms for delegated and computer work [07:05](https://www.youtube.com/watch?v=shRR1e2HXMk&t=425s) Codex can create sub-agents with a spawn agent tool, send them more input, wait for them, or shut them down. The same pattern is used for background terminals, which can receive data through standard input. For computer use, newer API shapes let the model use code execution to script interactions with a chosen computer implementation. Codex browser use keeps a persistent Node REPL across turns and writes JavaScript, including Playwright code, to interact with a Chromium browser. This lets the model inspect one page, understand its structure, and reuse scripts on later pages. ### File operations use trained editing patterns inside a sandbox [09:57](https://www.youtube.com/watch?v=shRR1e2HXMk&t=597s) Recent Codex models were trained to edit files with an apply patch tool, supplying a diff to change or create files. For other filesystem work, the model uses a shell and naturally reaches for ripgrep because it encountered that pattern during training. The Codex harness ships ripgrep when it is not already installed. On Windows, the model uses PowerShell natively. Filesystem interactions pass through a sandbox: Seatbelt on macOS, Bubblewrap on Linux, and a custom open source Windows sandbox that the team built because existing alternatives did not meet its needs. ### Auto review judges escalations using authorization and context [11:55](https://www.youtube.com/watch?v=shRR1e2HXMk&t=715s) Approval prompts can create fatigue, while full access can allow mistakes such as uploading a file to a file share or deleting too much data. Codex's auto review mode starts a separate review sub-agent when an action needs escalation. That sub-agent has read-only permissions and cannot create other sub-agents. It receives the conversation transcript, the tool calls, authorization guidance, and risk taxonomies. The same deletion can receive different judgments depending on what the user asked for. Deleting a file explicitly requested by the user is different from deleting a .git folder that was never mentioned. The review can also assess network actions, such as a request to upload a file. ### WebSocket mode moves the bottleneck from inference to the network [15:42](https://www.youtube.com/watch?v=shRR1e2HXMk&t=942s) When GPT-5.3 Codex Spark ran on Cerebras at 1,000 tokens per second, inference was no longer the slowest part of an agent interaction. Tool calls and network traffic became the problem. WebSocket mode replaces server-sent events over HTTP with a persistent connection. It reduces network overhead and keeps stateful context, so a turn can send only what changed. For a tool call, the system can send back the tool result instead of resending every previous item. Kundel's demo contrasts sending one item at a time with sending nine items back, showing how the protocol can improve performance. ### Concrete goals and compaction let agents run through longer tasks [17:34](https://www.youtube.com/watch?v=shRR1e2HXMk&t=1054s) For a long-running goal, the harness injects a continuation prompt containing the objective until the model calls an update goal tool to report that the goal was achieved. This makes concrete, verifiable objectives easier to manage than long essays. Kundel also describes automatic server-side compaction for agents that run for hours or days. Compaction turns the previous context into a new context for later turns and includes a compaction item with the information the model needs. It can happen automatically or be triggered manually, so users do not need to approve every step. ### The open source harness can be reused at different layers [19:39](https://www.youtube.com/watch?v=shRR1e2HXMk&t=1179s) Kundel says the Codex app server and harness are open source and written in Rust. Developers can study the implementation, use the harness directly, or build on top of it. He also points out that many distinctive Codex capabilities are exposed through the Responses API rather than being locked inside one application. Tool search, apply patch, WebSocket mode, and server-side compaction can therefore be used by developers building a different harness. As models and the Responses API change, he recommends watching those changes to decide when an agent should adopt new capabilities. ## Notable quotes - "The first step, arguably one of the most important ones, is context construction." (04:12) - "We noticed this when we launched GPT-5.3 Codex Spark, and it's running on Cerebras at 1,000 tokens per second." (15:42) - "The context matters, right? Like in some cases you want the agent to actually delete a file." (14:40) - "You don't want to, you know, write full essays like I know a lot of you have been trying to into your goal, but instead have very concrete and very verifiable prompts." (18:38) - "The Codex app server and the harness are open source." (19:39) ## Tools & references mentioned - GPT-5.3 Codex Spark - Cerebras - Responses API - Codex app server - Ollama - LM Studio - NVIDIA - T3 Code - Remote X - Claude Code - Doom - MCP - Seatbelt - Bubblewrap - ripgrep - Playwright - Chromium - PowerShell - apply patch - auto review ## Who should watch - You are building an agent and need to decide how tools, context, sandboxes, or long-running work should fit together. - You want to reuse parts of Codex without adopting its entire user interface or harness. - Your agent is fast enough that network overhead, approval prompts, or context growth now dominate the experience.