# Building on the Codex Harness Dominik Kundel, OpenAI | AI Engineer World's Fair 2026 | 18:08 Source: https://www.youtube.com/watch?v=9WiBJRO84yY Channel: AI Engineer (https://www.youtube.com/@aiDotEngineer). Summarised by AIE Talks. Page: https://aietalks.com/talks/building-on-the-codex-harness Published: 2026-10-09 Tags: developer-experience, harness-engineering, open-source, tool-use ## TL;DR - The Codex App Server is an open-source JSON-RPC protocol that wraps the Codex harness and powers first-party and third-party clients. - Developers can customize Codex with appended developer instructions, dynamic or deferred tools, and per-thread model and sandbox settings. - The App Server fits interactive clients and parallel threads, while codex exec fits scripted tasks such as automatic pull request fixes. ## Summary Dominik Kundel explains how to build interfaces on top of the Codex harness without implementing an agent harness from scratch. The Codex App Server is an open-source JSON-RPC protocol used by the Codex app, IDE extensions, and clients for Xcode, JetBrains, and other tools. Kundel shows live protocol events, including thread creation, configuration reads, and streamed deltas. He covers more than 120 client messages, ChatGPT sign-in, developer instructions, dynamic tools, deferred tool loading, per-thread model choices, and sandbox controls. He also explains how to bundle the binary, generate TypeScript bindings and JSON schema, choose STDIO or WebSockets, and use the Python SDK. A Doom demo shows Codex running inside a game and calling tools for game state and shell access. He recommends shipping a fixed harness version, deferring extra tools, and extending the existing instructions instead of replacing the system prompt. ## Key ideas ### The App Server gives clients direct access to the Codex harness [01:10](https://www.youtube.com/watch?v=9WiBJRO84yY&t=70s) Kundel describes the Codex App Server as the protocol that wraps the Codex harness. It uses a JSON-RPC style exchange between a client and the server, with messages sent to perform actions and events returned to the application. The same protocol powers the Codex app, the VS Code extension, and third-party clients for Xcode and JetBrains. It was also used to put Codex into Claude Code. Both the harness and the App Server are open source under the Apache 2 license, and the harness can connect to other model providers with a Responses API-compatible interface. Convenience settings support LM Studio and Ollama. ### The protocol exposes the events and controls inside the Codex app [02:17](https://www.youtube.com/watch?v=9WiBJRO84yY&t=137s) Kundel built an inspector that intercepts traffic between the real Codex app and the App Server. The live view shows thread creation, configuration reads, new threads, and streamed data deltas. The protocol passes through information used by app features such as file search. It currently exposes more than 120 client messages, including initialization, thread changes, turns, goals, plugins, file search, and filesystem modifications. This gives a custom client access to much of the functionality available in the Codex app instead of limiting it to a simple prompt-and-response interface. ### ChatGPT sign-in lets users bring their existing Codex subscription [06:31](https://www.youtube.com/watch?v=9WiBJRO84yY&t=391s) Applications built with the App Server can include sign in with ChatGPT. Users can then use their own ChatGPT subscription, with usage counted against their Codex limits. Kundel gives T3 Code as an example of the experience. This allows a third-party client to use the Codex service without forcing its users into a separate account or billing flow. ### Developer instructions and deferred tools customize the agent without replacing its base behavior [06:56](https://www.youtube.com/watch?v=9WiBJRO84yY&t=416s) Kundel recommends appending developer instructions to the Codex system prompt. This lets an application add behavior for its own interface while retaining the prompt work OpenAI maintains for individual models. Applications can also define dynamic tools directly or use MCP. Tools marked for deferred loading are found through tool search instead of being inserted into the system prompt immediately, which keeps extra tool definitions out of the normal context. A thread can also select a model provider, choose a model, and set sandbox rules for network access and file modifications. ### The binary and generated bindings should be shipped with the client [08:46](https://www.youtube.com/watch?v=9WiBJRO84yY&t=526s) The App Server is included in the Codex CLI binary, so an existing CLI installation can start it. The Codex app bundles its own binary and updates it with each app version. Kundel explains that the app ships both Linux and Windows binaries on Windows systems so WSL and native Windows can use the appropriate version. The CLI can generate TypeScript bindings and JSON schema for a particular App Server version. Since the protocol changes over time, he recommends bundling the expected version with the application rather than depending on whatever CLI a user happens to have installed. ### The Python SDK handles lifecycle work, while Codex can generate a client [10:31](https://www.youtube.com/watch?v=9WiBJRO84yY&t=631s) The recently released Python SDK manages much of the App Server lifecycle and provides a familiar interface, including support for ChatGPT sign-in. Developers can also use Codex itself to implement a client from the documentation. Kundel says he used this approach to build a Visual Basic client for a native Windows application, largely in one pass. The App Server supports STDIO and WebSockets, with the choice depending on the integration. He suggests asking Codex to help decide and implement the protocol rather than learning every low-level detail first. ### The App Server can put Codex inside an application such as Doom [12:00](https://www.youtube.com/watch?v=9WiBJRO84yY&t=720s) Kundel demonstrates an Electron application running Doom through Cloudflare's Doom Wasm renderer. Codex is rendered inside the game and communicates with the App Server through Electron IPC. Dynamic tools expose the game's armor, health, and weapons, including ammunition. Codex can also call a shell tool to inspect the current working directory, which belongs to the game's own workspace. The example shows that an App Server client does not need to look like a chat window. It can be embedded in an existing interface and connected to application-specific functions. ### codex exec is for scripts, while the App Server is for interactive and parallel clients [14:50](https://www.youtube.com/watch?v=9WiBJRO84yY&t=890s) Kundel presents codex exec as a non-interactive command for kicking off tasks from scripts. He gives automatic pull request fixes in CI/CD as an example. The App Server is a better fit when the application needs to show agent activity, expose a richer interface, or run many Codex threads at once. An evaluation harness can create multiple threads by sending new-thread events, rather than building its own parallelization around separate codex exec calls. ### Shipping a fixed harness and extending existing instructions avoids integration problems [15:55](https://www.youtube.com/watch?v=9WiBJRO84yY&t=955s) Kundel closes with three recommendations. Ship the application's own harness version so breaking changes, experimental features, and missing user updates do not disrupt the client. Expose application-specific functions as deferred tools so they remain available without filling the context with every definition. When changing behavior, start with developer instructions instead of replacing the base system prompt. The Codex team continually adjusts that prompt for model performance, and a custom replacement would discard that work. ## Notable quotes - "The app server is actually the protocol that wraps the Codex harness." (01:10) - "We have right now over 120 different client messages that we continuously add more to." (05:19) - "You can really be creative whether it's how you actually trigger it inside of your UI and your existing app." (11:31) - "You should actually ship your own version of the harness." (16:03) - "If you are changing the instructions, try to start with adding developer instructions rather than changing the base instructions." (17:00) ## Tools & references mentioned - Codex App Server - Codex - Agent Client Protocol - ACP - OpenAI - VS Code - Xcode - JetBrains - Claude Code - T3 Code - Remodex - LM Studio - Ollama - MCP - Python SDK - Cloudflare Doom Wasm - FreeDoom - Electron - codex exec ## Who should watch - You are building an editor, desktop app, game, or other interface that needs an agent without owning the whole harness implementation. - Your client needs Codex threads, tool calls, model settings, sandbox controls, or ChatGPT sign-in rather than a single scripted command. - You are deciding between codex exec and the App Server, or need a practical approach to versioning and extending Codex. ## Related talks - [Codex, Behind the Harness](https://aietalks.com/talks/codex-behind-the-harness) (Dominik Kundel, OpenAI, 20:55) - [Future-Proof Coding Agents](https://aietalks.com/talks/future-proof-coding-agents) (Bill Chen & Brian Fioca, OpenAI, 17:48) - [Cooking with Codex](https://aietalks.com/talks/cooking-with-codex) (Charlie Guo & Gabriel Chua, OpenAI, 1:07:24) - [OpenAI Codex Masterclass](https://aietalks.com/talks/openai-codex-masterclass) (Vaibhav Srivastav & Katia Gil Guzman, OpenAI, 1:01:59) - [Hacking Subagents Into Codex CLI](https://aietalks.com/talks/hacking-subagents-into-codex-cli) (Brian John, BetterUp, 13:39)