# The New Code

Sean Grove, OpenAI | AI Engineer World's Fair 2025 | 21:36

Source: https://www.youtube.com/watch?v=8rABwKRsec4
Channel: AI Engineer (https://www.youtube.com/@aiDotEngineer). Summarised by AIE Talks.
Page: https://aietalks.com/talks/the-new-code
Published: 2025-07-11
Tags: evals, prompt-engineering, structured-outputs

## TL;DR
- Specifications capture the intent and values that code leaves out, so they can become the source used to produce code, documentation, tests, and model behavior.
- A specification can align people and models when it is versioned, testable, and tied to challenging examples that grade whether behavior follows its rules.
- AI feature work should start with a written specification, explicit success criteria, and tests that check both the model and the specification.

## Summary
Sean Grove argues that software engineering has always involved more than writing code. Engineers gather requirements, decide what success means, communicate plans, and check whether the result helped users. Code is only a small part of that work, while structured communication carries the rest. He proposes specifications as the durable source of intent. Unlike disposable prompts, a written specification can align teams, preserve values, and generate multiple outputs, including code, documentation, evaluations, and model behavior. Grove uses OpenAI's Model Spec to show how markdown can be versioned, discussed, and linked to tests for specific clauses. He also compares specifications with programming language tools and with the US Constitution, where judicial decisions create precedents that clarify the policy. His practical advice is to begin AI features with a specification, define success, test it against models, and build tools that expose ambiguity before it reaches production.

## Key ideas
### Structured communication carries most of an engineer's work
[02:15](https://www.youtube.com/watch?v=8rABwKRsec4&t=135s)
Grove says code feels like the tangible professional artifact because teams can measure and debate it, but he estimates that code accounts for only 10 to 20 percent of the value engineers bring. The rest involves understanding users, distilling their problems, deciding what to build, planning how to build it, sharing those plans, and checking the effect on the world. Those activities are forms of structured communication. As models improve, he expects this bottleneck to become more visible. His claim is direct: the person who communicates most effectively will become the most valuable programmer, because effective communication can itself program increasingly capable models.

### Prompts should be preserved as source specifications
[04:35](https://www.youtube.com/watch?v=8rABwKRsec4&t=275s)
Vibe coding already puts communication before implementation. A person describes the intended outcome, and a model produces the code. Grove finds the current workflow backwards because developers often discard the prompt and keep the generated code. He compares this with shredding source code while carefully versioning the compiled binary. In ordinary programming, binaries are regenerated from source because the source contains the useful intent. A written specification can preserve the goals and values behind a model-generated artifact. It gives people something to discuss, debate, reference, and use to check whether they agree on what should be built.

### A sufficiently detailed specification can target many outputs
[06:47](https://www.youtube.com/watch?v=8rABwKRsec4&t=407s)
Grove describes code as a lossy projection of a fuller specification. Reading code often requires people to infer the team's goal and the reasons behind its design choices. A specification with enough detail could instead be translated into different targets, much like source code can compile for ARM64, x86, or WebAssembly. He says a strong specification could guide models to produce TypeScript, Rust, servers, clients, documentation, tutorials, blog posts, and podcasts. This matters because a business's codebase does not necessarily contain the information needed to explain to users how to succeed. That information often exists in the team's communication and intent.

### The Model Spec aligns people through a versioned shared document
[09:24](https://www.youtube.com/watch?v=8rABwKRsec4&t=564s)
Grove presents OpenAI's Model Spec as a living document for expressing the intentions and values OpenAI wants its models to follow. He says the public implementation is a collection of Markdown files. Markdown is readable by technical and nontechnical contributors, and the document can be versioned and change logged. Product, legal, safety, research, and policy teams can work from the same source. Each clause also has an ID. A related file can hold challenging prompts for that exact clause, turning the document into a place where people can state both the policy and examples of behavior that would satisfy it.

### Sycophancy shows why a specification can anchor trust
[11:31](https://www.youtube.com/watch?v=8rABwKRsec4&t=691s)
Grove uses a recent sycophancy incident as a case study. The Model Spec already contained a rule against being sycophantic at the expense of impartial truth, along with an explanation that short-term praise can be harmful over time. That gave people a shared reference for judging the behavior. If the model specification is the agreed set of intentions and the model violates it, Grove says the behavior should be treated as a bug. OpenAI rolled back the change, published studies and blog posts, and fixed it. During the incident, the specification still gave people a clear statement of what was expected and provided a trust anchor while the behavior was corrected.

### Specifications can become training and evaluation material
[13:48](https://www.youtube.com/watch?v=8rABwKRsec4&t=828s)
Grove describes deliberative alignment as a way to align models with a specification. The process uses the specification and difficult prompts, samples responses from a model, then gives the prompt, response, and policy to a grader model. The grader scores the response against the specification. The document therefore supplies both training material and evaluation material. A specification can also be placed in the model's context on every request, although Grove says that uses inference-time compute that could otherwise solve the task. Training can push the policy into the model's weights so that it applies the policy more like learned behavior.

### Specification tooling can resemble a programming toolchain
[15:21](https://www.youtube.com/watch?v=8rABwKRsec4&t=921s)
Although the Model Spec is written in Markdown, Grove says it is useful to think of it as code. Specifications can compose, be executed and tested, expose interfaces to the real world, and ship as modules. He imagines tools that work like type checkers: if two departments publish specifications with conflicting assumptions, the conflict should be found before the specification is released. Policies can contain unit tests, and linters could flag ambiguous language because ambiguity confuses both people and models. This creates a toolchain aimed at checking intentions rather than syntax.

### A future IDE could clarify intent before implementation
[19:45](https://www.youtube.com/watch?v=8rABwKRsec4&t=1185s)
Grove imagines a future IDE as an integrated thought clarifier. While someone writes a specification, the tool would find ambiguity and ask for clarification. The goal would be clearer communication among people and between people and models. He asks developers to apply the idea immediately when building an AI feature: write down what should happen, define success criteria, debate whether the specification is clear, make it executable, and test the model against it. He closes with agent alignment at scale, where teams may discover that they never stated what they wanted or fully understood it. For Grove, that problem calls for better specifications.

## Notable quotes
- "Code is sort of 10 to 20% of the value that you bring." (02:35)
- "And this feels like a little bit like you shred the source and then you very carefully version control the binary." (05:53)
- "So the document itself actually encodes success criteria that the model under test has to be able to answer this in a way that actually adheres to that clause." (11:01)
- "Whenever you're working on your next AI feature, start with a specification." (19:42)
- "This is a cry for specification." (21:07)

## Tools & references mentioned
- OpenAI Model Spec
- deliberative alignment
- US Constitution
- TypeScript
- Rust
- ARM64
- x86
- WebAssembly
- V8
- GraphQL
- OneGraph
- Netlify
- Josh

## Who should watch
- You build AI features with prompts and generated code but have no durable record of the intended behavior.
- Your product, engineering, safety, legal, or policy teams need one document that they can discuss, version, test, and use to resolve disagreements.
- You are designing tools for model evaluation, agent behavior, or specification authoring and want a concrete model for connecting policy to tests.

## Related talks

- [Everything is a Spec: The Universal Language of Intent](https://aietalks.com/talks/everything-is-a-spec-the-universal-language-of-intent) (Sean Grove, OpenAI, 00:52)
- [Spec-Driven Development: Agentic Coding at FAANG Scale and Quality](https://aietalks.com/talks/spec-driven-development-agentic-coding-at-faang-scale-and-quality) (Al Harris, Amazon Kiro, 1:03:50)
- [Developer Experience in the Age of AI Coding Agents](https://aietalks.com/talks/developer-experience-in-the-age-of-ai-coding-agents) (Max Kanat-Alexander, Capital One, 18:20)
- [Beyond the Prototype: Using AI to Write High-Quality Code](https://aietalks.com/talks/beyond-the-prototype-using-ai-to-write-high-quality-code) (Josh Albrecht, Imbue, 17:59)
- [Spec-Driven Testing for Agents With A Brain the Size of A Planet](https://aietalks.com/talks/spec-driven-testing-for-agents-with-a-brain-the-size-of-a-planet) (Steven Willmott, Safe Intelligence, 13:03)