All Code Is Slop (Except Mine)

Lauren Tan, SpaceXSI17:32 · Nov 2026 · 143K views
Thumbnail for All Code Is Slop (Except Mine) Watch on YouTube
TL;DR
  1. 1

    Agents can produce thousands of pull requests when the codebase constrains their choices through architecture, APIs, lint rules, and type systems.

  2. 2

    Verification should become a large part of the agent workflow, including runtime checks, feature maps, formal verification, and UI property-based testing.

  3. 3

    The Grok Bot team shipped 11,000 pull requests in one month with a revert rate of about 0.5%, while several performance measures improved.

Summary

Lauren Tan describes how the Grok Bot team uses software engineering foundations to make AI coding agents more trustworthy. Her central idea is to improve the codebase before adding more written instructions. APIs, directory structure, lint rules, types, compilers, and tests constrain agents in ways that are deterministic and repeatable. The team bans patterns that agents misuse, such as most uses of React's useEffect, and replaces comments with enforceable checks. Tan also treats verification as part of the agent's normal workflow. Agents use feature maps, custom command-line tools, runtime debugging, TLA+, Lean, and property-based UI testing. This produces artifacts that people can review instead of reading every change. The reported result is 11,000 production pull requests in a month, with about 0.5% reverted. Tan is clear that automation comes after these foundations. If the inner coding loop creates bad code, automating it only creates more bad code at higher cost.

Key ideas
01:23

Agent productivity depends on trust in the environment

Tan describes a software factory as an environment where agents can be trusted to do the right thing. She says it took her six months to reach a point where she could ship thousands of pull requests, including while asleep. The first investment is deterministic code infrastructure: API design, architecture, refactoring, lint rules, type systems, compilers, and tests. These mechanisms constrain agent behavior and make the desired behavior the default. Written rules still have a place, but they belong above the deterministic layers and should cover cases that require judgment.

02:50

The codebase is the largest input agents copy from

Tan compares a fast-changing codebase to a garden. When hundreds of agents produce code continuously, existing workarounds, deprecated code, anti-patterns, and technical debt become examples for the next agent. She says engineers therefore need to keep the codebase clean so agents copy good patterns. A correction that can become a code-level check is more reproducible than another instruction in rules or agents.md. The goal is to make the code itself carry the constraints that agents need.

05:00

Architecture and lint rules remove recurring agent mistakes

Dune is a client framework for Grok Bot desktop applications that encodes lessons from building Electron software. Tan says the team bans useEffect in product code because agents misuse an API that requires too much nuance. The team also separates Electron main and renderer code into explicit directories, then forbids invalid imports with a dependency graph analyzer and lint rules. These constraints prevent recurring problems before code review. The same approach turns architectural decisions into checks that agents can follow without needing to infer intent.

06:56

Comments are weak controls because agents can use them as excuses

Tan says agents wrote comments to justify quick fixes, especially around imperative timers such as setTimeout, requestAnimationFrame, and requestIdleCallback. In Cursor, those timers patched over a conflict between an imperative style and React's declarative style, while creating performance problems from repeated creation and destruction. The comments spread along with the workaround. Grok Bot therefore bans 99% of comments and replaces them with lint rules that tell agents to fix the root cause. Each agent correction should become a deterministic check or lead to better API design and architecture.

09:34

Verification artifacts replace much of the old code review

Tan says the team's review process now focuses on verification artifacts such as videos, screenshots, diagrams, and formal proofs. Runtime verification gives agents access to the running application and debugging tools through the Chrome DevTools protocol. A feature map lists application features from the user's perspective, giving agents shared names for UI elements. Custom command-line tools can be reused by the agent and tested separately. These tools let agents inspect their own work rather than relying only on generated code and written explanations.

10:50

Formal methods and UI testing add separate checks

The team uses TLA+ for model checking user flows such as notifications across devices and platforms. It uses Lean to prove parts of algorithms, including a custom virtualization library where the team checks for flickering or jittering problems. Tan says formal verification has already found several bugs, but she does not present it as a complete solution. The team also uses the Bombadil library for property-based UI testing, which has caught UI bugs. These methods become additional layers in the agent environment.

12:01

Verification can consume most of an agent's tokens

Tan cites Cindy's report that 70% of her daily tokens now go into verification, and says her own usage looks similar. The methods are established software engineering practices, but agents reduce the cost of writing, maintaining, and running them. Tan argues that this makes much more testing and verification economical. She also uses a pstack skill called state chart to visualize state machines from the user's point of view, then reviews those diagrams alongside videos and screenshots.

13:22

High volume does not have to mean higher revert rates

The Grok Bot team shipped 11,000 pull requests to production in one month and released a stable client version about once a day. In October, 227 people contributed at least one pull request, including contributors without much context about the project. Despite that volume, the relative revert rate stayed around 0.5%. Tan also says startup, typing, opening chats, and switching chats became faster after launch. She attributes this to the team's software foundations and the constraints around agent-generated code, while acknowledging that Electron still imposes a ceiling.

15:11

Automation should come after the inner coding loop is clean

Tan advises teams to improve the codebase before automating more of the development process. A lint rule should prevent new instances of an anti-pattern, while older instances are cleaned up. The verification process must also be strong. If the inner coding loop produces poor code, automating it produces more poor code and requires more tokens to correct it. Once code quality, static analysis, runtime verification, and formal verification are in place, Grok Bot can run Cursor projects that fuzz the application, open fixes, and auto-merge them because the repository makes good changes the default.

"So our general principle is that every time we correct our agents, it turns into one of these deterministic checks or we think about how to categorically eliminate that problem in the code through better API design, through better architecture."09:15
Who should watch
  • You are adding coding agents to a repository where old workarounds and inconsistent patterns keep being copied.
  • Your team wants agents to make and merge more changes, but human review is already the bottleneck.
  • You need practical examples of runtime checks, formal verification, and UI testing inside an agent workflow.