# All Code Is Slop (Except Mine)

Lauren Tan, SpaceXSI | AI Engineer | 17:32

Source: https://www.youtube.com/watch?v=O_zcc6Xl12g
Channel: AI Engineer (https://www.youtube.com/@aiDotEngineer). Summarised by AIE Talks.
Page: https://aietalks.com/talks/all-code-is-slop-except-mine
Published: 2026-11-14
Tags: agents, coding-agents, reliability, software-factories, testing

## TL;DR
- Agents can produce thousands of pull requests when the codebase constrains their choices through architecture, APIs, lint rules, and type systems.
- Verification should become a large part of the agent workflow, including runtime checks, feature maps, formal verification, and UI property-based testing.
- The Grok Bot team shipped 11,000 pull requests in one month with a revert rate of about 0.5%, while several performance measures improved.

## Summary
Lauren Tan describes how the Grok Bot team uses software engineering foundations to make AI coding agents more trustworthy. Her central idea is to improve the codebase before adding more written instructions. APIs, directory structure, lint rules, types, compilers, and tests constrain agents in ways that are deterministic and repeatable. The team bans patterns that agents misuse, such as most uses of React's useEffect, and replaces comments with enforceable checks. Tan also treats verification as part of the agent's normal workflow. Agents use feature maps, custom command-line tools, runtime debugging, TLA+, Lean, and property-based UI testing. This produces artifacts that people can review instead of reading every change. The reported result is 11,000 production pull requests in a month, with about 0.5% reverted. Tan is clear that automation comes after these foundations. If the inner coding loop creates bad code, automating it only creates more bad code at higher cost.

## Key ideas
### Agent productivity depends on trust in the environment
[01:23](https://www.youtube.com/watch?v=O_zcc6Xl12g&t=83s)
Tan describes a software factory as an environment where agents can be trusted to do the right thing. She says it took her six months to reach a point where she could ship thousands of pull requests, including while asleep. The first investment is deterministic code infrastructure: API design, architecture, refactoring, lint rules, type systems, compilers, and tests. These mechanisms constrain agent behavior and make the desired behavior the default. Written rules still have a place, but they belong above the deterministic layers and should cover cases that require judgment.

### The codebase is the largest input agents copy from
[02:50](https://www.youtube.com/watch?v=O_zcc6Xl12g&t=170s)
Tan compares a fast-changing codebase to a garden. When hundreds of agents produce code continuously, existing workarounds, deprecated code, anti-patterns, and technical debt become examples for the next agent. She says engineers therefore need to keep the codebase clean so agents copy good patterns. A correction that can become a code-level check is more reproducible than another instruction in rules or agents.md. The goal is to make the code itself carry the constraints that agents need.

### Architecture and lint rules remove recurring agent mistakes
[05:00](https://www.youtube.com/watch?v=O_zcc6Xl12g&t=300s)
Dune is a client framework for Grok Bot desktop applications that encodes lessons from building Electron software. Tan says the team bans useEffect in product code because agents misuse an API that requires too much nuance. The team also separates Electron main and renderer code into explicit directories, then forbids invalid imports with a dependency graph analyzer and lint rules. These constraints prevent recurring problems before code review. The same approach turns architectural decisions into checks that agents can follow without needing to infer intent.

### Comments are weak controls because agents can use them as excuses
[06:56](https://www.youtube.com/watch?v=O_zcc6Xl12g&t=416s)
Tan says agents wrote comments to justify quick fixes, especially around imperative timers such as setTimeout, requestAnimationFrame, and requestIdleCallback. In Cursor, those timers patched over a conflict between an imperative style and React's declarative style, while creating performance problems from repeated creation and destruction. The comments spread along with the workaround. Grok Bot therefore bans 99% of comments and replaces them with lint rules that tell agents to fix the root cause. Each agent correction should become a deterministic check or lead to better API design and architecture.

### Verification artifacts replace much of the old code review
[09:34](https://www.youtube.com/watch?v=O_zcc6Xl12g&t=574s)
Tan says the team's review process now focuses on verification artifacts such as videos, screenshots, diagrams, and formal proofs. Runtime verification gives agents access to the running application and debugging tools through the Chrome DevTools protocol. A feature map lists application features from the user's perspective, giving agents shared names for UI elements. Custom command-line tools can be reused by the agent and tested separately. These tools let agents inspect their own work rather than relying only on generated code and written explanations.

### Formal methods and UI testing add separate checks
[10:50](https://www.youtube.com/watch?v=O_zcc6Xl12g&t=650s)
The team uses TLA+ for model checking user flows such as notifications across devices and platforms. It uses Lean to prove parts of algorithms, including a custom virtualization library where the team checks for flickering or jittering problems. Tan says formal verification has already found several bugs, but she does not present it as a complete solution. The team also uses the Bombadil library for property-based UI testing, which has caught UI bugs. These methods become additional layers in the agent environment.

### Verification can consume most of an agent's tokens
[12:01](https://www.youtube.com/watch?v=O_zcc6Xl12g&t=721s)
Tan cites Cindy's report that 70% of her daily tokens now go into verification, and says her own usage looks similar. The methods are established software engineering practices, but agents reduce the cost of writing, maintaining, and running them. Tan argues that this makes much more testing and verification economical. She also uses a pstack skill called state chart to visualize state machines from the user's point of view, then reviews those diagrams alongside videos and screenshots.

### High volume does not have to mean higher revert rates
[13:22](https://www.youtube.com/watch?v=O_zcc6Xl12g&t=802s)
The Grok Bot team shipped 11,000 pull requests to production in one month and released a stable client version about once a day. In October, 227 people contributed at least one pull request, including contributors without much context about the project. Despite that volume, the relative revert rate stayed around 0.5%. Tan also says startup, typing, opening chats, and switching chats became faster after launch. She attributes this to the team's software foundations and the constraints around agent-generated code, while acknowledging that Electron still imposes a ceiling.

### Automation should come after the inner coding loop is clean
[15:11](https://www.youtube.com/watch?v=O_zcc6Xl12g&t=911s)
Tan advises teams to improve the codebase before automating more of the development process. A lint rule should prevent new instances of an anti-pattern, while older instances are cleaned up. The verification process must also be strong. If the inner coding loop produces poor code, automating it produces more poor code and requires more tokens to correct it. Once code quality, static analysis, runtime verification, and formal verification are in place, Grok Bot can run Cursor projects that fuzz the application, open fixes, and auto-merge them because the repository makes good changes the default.

## Notable quotes
- "My thesis is that the job of the software engineer, the meta of being a software engineer is figuring out how to maintain or even improve the quality of your code while massively increasing your productivity." (01:03)
- "So our general principle is that every time we correct our agents, it turns into one of these deterministic checks or we think about how to categorically eliminate that problem in the code through better API design, through better architecture." (09:15)
- "And really, we are entering a golden age of software testing and verification because agents let us test everything to a much greater degree than used to be economical." (12:42)
- "If your inner coding loop produces slop, then automating that loop just produces more slop and you need to throw more and more and more tokens at the problem, which we all know is very expensive." (15:33)

## Tools & references mentioned
- Grok Bot
- SpaceXSI
- Dune
- Next.js
- Electron
- React
- Cursor
- Chrome DevTools Protocol
- TLA+
- Lean
- Bombadil
- pstack
- state chart
- Cindy
- Brett Taylor
- Google Maps

## Who should watch
- You are adding coding agents to a repository where old workarounds and inconsistent patterns keep being copied.
- Your team wants agents to make and merge more changes, but human review is already the bottleneck.
- You need practical examples of runtime checks, formal verification, and UI testing inside an agent workflow.

## Related talks

- [AI-Generated Code Is Already Competing With Human Code](https://aietalks.com/talks/ai-generated-code-is-already-competing-with-human-code) (Daksh Gupta, Greptile, 12:41)
- [Fighting Slop with Slop](https://aietalks.com/talks/fighting-slop-with-slop) (Vaibhav Gupta, Boundary, 21:32)
- [No More Slop](https://aietalks.com/talks/no-more-slop) (swyx, AI Engineer, 09:15)
- [Fixing the PR Bottleneck](https://aietalks.com/talks/fixing-the-pr-bottleneck) (Matt Pocock, AI Hero, 22:35)
- [Making Codebases Agent Ready](https://aietalks.com/talks/making-codebases-agent-ready) (Eno Reyes, Factory, 15:33)
