The New Code

Sean Grove, OpenAI21:36 · Jul 2025 · 1.1M views
Thumbnail for The New Code Watch on YouTube
TL;DR
  1. 1

    Specifications capture the intent and values that code leaves out, so they can become the source used to produce code, documentation, tests, and model behavior.

  2. 2

    A specification can align people and models when it is versioned, testable, and tied to challenging examples that grade whether behavior follows its rules.

  3. 3

    AI feature work should start with a written specification, explicit success criteria, and tests that check both the model and the specification.

Summary

Sean Grove argues that software engineering has always involved more than writing code. Engineers gather requirements, decide what success means, communicate plans, and check whether the result helped users. Code is only a small part of that work, while structured communication carries the rest. He proposes specifications as the durable source of intent. Unlike disposable prompts, a written specification can align teams, preserve values, and generate multiple outputs, including code, documentation, evaluations, and model behavior. Grove uses OpenAI's Model Spec to show how markdown can be versioned, discussed, and linked to tests for specific clauses. He also compares specifications with programming language tools and with the US Constitution, where judicial decisions create precedents that clarify the policy. His practical advice is to begin AI features with a specification, define success, test it against models, and build tools that expose ambiguity before it reaches production.

Key ideas
02:15

Structured communication carries most of an engineer's work

Grove says code feels like the tangible professional artifact because teams can measure and debate it, but he estimates that code accounts for only 10 to 20 percent of the value engineers bring. The rest involves understanding users, distilling their problems, deciding what to build, planning how to build it, sharing those plans, and checking the effect on the world. Those activities are forms of structured communication. As models improve, he expects this bottleneck to become more visible. His claim is direct: the person who communicates most effectively will become the most valuable programmer, because effective communication can itself program increasingly capable models.

04:35

Prompts should be preserved as source specifications

Vibe coding already puts communication before implementation. A person describes the intended outcome, and a model produces the code. Grove finds the current workflow backwards because developers often discard the prompt and keep the generated code. He compares this with shredding source code while carefully versioning the compiled binary. In ordinary programming, binaries are regenerated from source because the source contains the useful intent. A written specification can preserve the goals and values behind a model-generated artifact. It gives people something to discuss, debate, reference, and use to check whether they agree on what should be built.

06:47

A sufficiently detailed specification can target many outputs

Grove describes code as a lossy projection of a fuller specification. Reading code often requires people to infer the team's goal and the reasons behind its design choices. A specification with enough detail could instead be translated into different targets, much like source code can compile for ARM64, x86, or WebAssembly. He says a strong specification could guide models to produce TypeScript, Rust, servers, clients, documentation, tutorials, blog posts, and podcasts. This matters because a business's codebase does not necessarily contain the information needed to explain to users how to succeed. That information often exists in the team's communication and intent.

09:24

The Model Spec aligns people through a versioned shared document

Grove presents OpenAI's Model Spec as a living document for expressing the intentions and values OpenAI wants its models to follow. He says the public implementation is a collection of Markdown files. Markdown is readable by technical and nontechnical contributors, and the document can be versioned and change logged. Product, legal, safety, research, and policy teams can work from the same source. Each clause also has an ID. A related file can hold challenging prompts for that exact clause, turning the document into a place where people can state both the policy and examples of behavior that would satisfy it.

11:31

Sycophancy shows why a specification can anchor trust

Grove uses a recent sycophancy incident as a case study. The Model Spec already contained a rule against being sycophantic at the expense of impartial truth, along with an explanation that short-term praise can be harmful over time. That gave people a shared reference for judging the behavior. If the model specification is the agreed set of intentions and the model violates it, Grove says the behavior should be treated as a bug. OpenAI rolled back the change, published studies and blog posts, and fixed it. During the incident, the specification still gave people a clear statement of what was expected and provided a trust anchor while the behavior was corrected.

13:48

Specifications can become training and evaluation material

Grove describes deliberative alignment as a way to align models with a specification. The process uses the specification and difficult prompts, samples responses from a model, then gives the prompt, response, and policy to a grader model. The grader scores the response against the specification. The document therefore supplies both training material and evaluation material. A specification can also be placed in the model's context on every request, although Grove says that uses inference-time compute that could otherwise solve the task. Training can push the policy into the model's weights so that it applies the policy more like learned behavior.

15:21

Specification tooling can resemble a programming toolchain

Although the Model Spec is written in Markdown, Grove says it is useful to think of it as code. Specifications can compose, be executed and tested, expose interfaces to the real world, and ship as modules. He imagines tools that work like type checkers: if two departments publish specifications with conflicting assumptions, the conflict should be found before the specification is released. Policies can contain unit tests, and linters could flag ambiguous language because ambiguity confuses both people and models. This creates a toolchain aimed at checking intentions rather than syntax.

19:45

A future IDE could clarify intent before implementation

Grove imagines a future IDE as an integrated thought clarifier. While someone writes a specification, the tool would find ambiguity and ask for clarification. The goal would be clearer communication among people and between people and models. He asks developers to apply the idea immediately when building an AI feature: write down what should happen, define success criteria, debate whether the specification is clear, make it executable, and test the model against it. He closes with agent alignment at scale, where teams may discover that they never stated what they wanted or fully understood it. For Grove, that problem calls for better specifications.

"So the document itself actually encodes success criteria that the model under test has to be able to answer this in a way that actually adheres to that clause."11:01
Who should watch
  • You build AI features with prompts and generated code but have no durable record of the intended behavior.
  • Your product, engineering, safety, legal, or policy teams need one document that they can discuss, version, test, and use to resolve disagreements.
  • You are designing tools for model evaluation, agent behavior, or specification authoring and want a concrete model for connecting policy to tests.