AI raises average developer productivity by about 15% to 20% after accounting for rework, although the gains vary widely.
2
AI performs best on low-complexity and greenfield work, while high-complexity brownfield tasks can see little gain or lower productivity.
3
Task complexity, project maturity, language popularity, codebase size, and context length all affect whether AI helps.
Summary
Yegor Denisov-Blanch presents Stanford research based on more than 100,000 software engineers across more than 600 companies, with data from private repositories, millions of commits, and billions of lines of code. He argues that common measures such as commit counts, pull requests, and developer surveys give a poor view of productivity. The study instead uses a model that evaluates the functionality of code changes and tracks rework, which is recent code that has to be changed again. AI increases the volume of code delivered by roughly 30% to 40%, but rework reduces the average gain to about 15% to 20%. The results vary by work setting. Low-complexity greenfield tasks gain the most, while high-complexity brownfield tasks gain little and can become slower. Less popular programming languages and larger codebases also reduce the benefit. Denisov-Blanch recommends using AI selectively rather than treating it as a universal replacement for developers.
AI raises productivity overall, but it can also make developers slower
Denisov-Blanch says he does not expect AI to replace developers entirely, at least in the near term. His position is more specific: AI generally increases developer productivity, but some situations produce a decrease. Coding with AI is therefore not a universal solution. He frames this conclusion through Stanford's study, which has followed software engineering productivity for three years. The research combines time-series data, including changes before and after AI adoption, with cross-sectional data from more than 600 enterprises, midsized companies, and startups. The dataset contains more than 100,000 software engineers, tens of millions of commits, and billions of lines of code.
Commit counts and surveys do not measure useful output well
Existing studies often count commits, pull requests, or completed tasks, but task size varies and AI can create follow-up bug-fixing work. A higher commit count may therefore reflect developers correcting AI-generated code rather than delivering more functionality. Denisov-Blanch also criticizes experiments based on simple greenfield tasks, where AI is especially good at producing boilerplate from scratch. Those results do not transfer cleanly to established codebases with dependencies. In a small experiment with 43 developers, self-assessed productivity had very little correlation with measured productivity. Participants misjudged their productivity by about 30 percentile points, and only one in three placed themselves within the correct quartile.
The study measures code functionality and tracks wasteful rework
The ideal measurement would have a panel of 10 or 15 engineering experts assess code quality, maintainability, output, and the time the work would take. Denisov-Blanch says these experts generally agree and that their assessments predict reality, but the process is too slow and expensive to use at scale. Stanford's model automates the assessment by connecting to Git and analyzing source-code changes across dimensions. It focuses on the functionality delivered over time rather than lines of code or raw commit counts. The model also separates refactoring from rework. Rework changes recently written code and is more likely to be wasteful, while refactoring may or may not be wasteful.
In one example, a 120-developer team adopted AI in September. The chart tracks monthly output as added functionality, removed code, refactoring, and rework. After adoption, rework rises substantially. Developers appear to deliver more because more code is written and more commits are pushed, but some of that volume consists of fixing problems introduced earlier. Denisov-Blanch estimates that AI coding can increase the amount of code delivered by roughly 30% to 40%. Once the added rework is included, the average productivity gain across industries and sectors is about 15% to 20%. The distinction changes how leaders should interpret apparent output increases.
The study's distributions show stronger AI gains for low-complexity tasks than for high-complexity tasks. Greenfield work also benefits more than brownfield work, especially when the task is simple. Denisov-Blanch gives an orientative matrix based on 136 teams across 27 companies. Low-complexity greenfield tasks show gains of about 30% to 40%. High-complexity greenfield work shows about 10% to 15%, while low-complexity brownfield work shows about 15% to 20%. High-complexity brownfield work shows only 0% to 10%. In some high-complexity cases, AI lowers productivity. The speaker says the underlying causes of those decreases are not yet clear.
AI helps more with widely used languages such as Python, Java, JavaScript, and TypeScript than with less popular languages such as COBOL, Haskell, and Elixir. For low-popularity languages, even simple tasks receive little benefit, so developers may stop using AI when it helps only occasionally. Complex work in those languages can become slower because the generated code is poor. Denisov-Blanch says this effect affects a smaller share of overall development work, while common languages show gains of about 20% for low-complexity tasks and 10% to 15% for high-complexity tasks. The result depends partly on how much useful training and contextual information exists for a language.
Larger codebases and longer contexts reduce AI performance
Denisov-Blanch presents an illustrative relationship in which AI gains decrease sharply as codebase size grows from 1,000 lines toward 10 million lines. He attributes this to context-window limits, a lower signal-to-noise ratio, more dependencies, and more domain-specific logic. He also cites the NoLiMa paper, whose coding-task results show model performance decreasing as context length grows from 1,000 to 32,000 tokens. Gemini 1.5 Pro may have a two-million-token context window, but the speaker says a model's ability to use a large context does not automatically remain strong. In the example, performance falls from about 90% to about 50% by 32,000 tokens.
AI adoption should depend on the work rather than a blanket policy
Denisov-Blanch's final position is that developers should use AI in most cases, while recognizing that productivity does not rise equally in every setting. He names task complexity, codebase maturity, language popularity, codebase size, and context length as factors that shape the result. The study supports a selective approach: simple greenfield work in common languages is a strong fit, while complex work in mature, large codebases requires more caution. The talk does not claim that the research has identified every cause of negative results. It gives leaders measured ranges and a way to inspect useful output and rework instead of relying on code volume or developer opinion.
"The productivity of a team is basically the functionality of the code they delivered across time, not the lines of code, not the whatever commits, but what that code is doing."08:40
Who should watch
Engineering leaders deciding whether AI coding tools are improving useful output or only increasing code volume.
Teams working in mature codebases who need evidence about rework, task complexity, and context limits before setting an AI policy.
Researchers and engineers comparing productivity metrics, especially commit counts, surveys, and functionality-based measures.
