ReviewDebt: A Practical Framework for Scoring Every Pull Request

Sachin Gupta, Ebay25:00 · Jul 2026 · 1,954 views
Thumbnail for ReviewDebt: A Practical Framework for Scoring Every Pull Request Watch on YouTube
TL;DR
  1. 1

    Review debt measures the gap between code produced by coding agents and code humans have reviewed, trusted, and understood.

  2. 2

    A deterministic score can estimate review burden from diff size, test evidence, ownership spread, AI authorship indicators, and rationale gaps.

  3. 3

    Teams should backfill the score on recent pull requests, show it without blocking merges, and discuss its weekly trend alongside coding-agent throughput.

Summary

Sachin Gupta argues that coding agents are increasing code production faster than humans can review it. The resulting review debt is the accumulated gap between generated code and code that people have understood and trusted. He proposes a 0-to-100 score built from deterministic pull request data, including changed lines and coupling, test evidence, ownership spread, AI authorship signals, and gaps in rationale. The score is intended to be traceable in an engineering review, rather than judged by a changing language model. Three examples show a clean pull request scoring zero, a poorly shaped AI-assisted pull request scoring 60, and a well-structured AI pull request scoring seven. Gupta recommends backfilling the score across recent merges, setting a threshold for author evidence, posting scores as comments, and tracking each team's weekly trend. He also argues that complexity and volume create burden, while AI authorship should only amplify existing review problems.

Key ideas
00:00

Production speed has increased while review attention has fallen

Gupta frames review debt as a measurable gap created when coding agents produce more pull requests than humans can responsibly inspect. He cites GitHub's 2025 report, where commits rose 25% year over year while comments on comments, his proxy for review activity, fell 27%. In a Faros AI 2026 benchmark, median pull request review time rose 441.5%, reviewed pull requests took 5.4 times longer, and 31% more were merged without review. He also points to larger pull requests, from 44 to 72 lines in the cited DX study. PR count and cycle time show production speed, he says, but they do not show the speed of trust.

04:21

Review debt compounds through code, architecture, and staffing feedback loops

Gupta defines review debt as the accumulating gap between code an agent has produced and code humans have reviewed, trusted, and understood. He compares it with financial debt because it compounds through human attention rather than money. Unreviewed code becomes training or retrieval material for future agent output. Reviewers then focus on syntax and obvious bugs while architectural decisions go unexamined. Finally, leadership resets velocity expectations without adding reviewer capacity. These loops leave little slack to repay the debt, even when each individual pull request looks manageable.

06:07

The score uses five deterministic signal families instead of a language-model judge

The proposed score has five families with ten checks: diff size and coupling, test evidence gaps, directory and ownership spread, AI authorship indicators, and evidence and rationale gaps. Gupta insists that every check must be computable from the pull request and repository. A language model judge could give the same pull request different scores after a model change, making the result hard to defend in an engineering review. A deterministic computation ties the number to observable properties such as changed lines, touched files, code-owner teams, test lines, branch names, commit text, and the explanation in the pull request.

07:20

Sprawling changes and weak tests increase the mental work of review

Diff size is not enough on its own. Gupta measures net lines, files touched, and whether changes stay within one module or spread across many. He says agents often fix symptoms at the call site and reach into multiple files, which makes the reviewer hold a larger mental model. The test evidence gap divides test lines added by production lines added. It catches whether tests appeared, but Gupta admits that the ratio cannot judge test quality. Agent-written tests may assert what the code does, including bugs, instead of what the code should do. Ownership spread adds coordination cost when several teams must approve one change.

09:53

AI authorship is an amplifier, not an automatic penalty

The framework detects possible agent assistance through co-authored footers such as Copilot, branch prefixes such as codex, Copilot, or cursor, and phrases such as generated by or assisted by in pull request text. Gupta says this is not meant to blame engineers. In the 60-point example, the AI indicator contributed five points, while the remaining 55 came from size, claim mismatch, and missing tests. The well-shaped AI pull request scored seven out of 100 because its tests were present, CI was green, and the risky path was explained. Its AI indicator still fired, but only as low-severity information.

12:09

The score becomes useful when it produces specific reviewer and author actions

The score runs from zero to 100, with default bands of zero to 24 for very low burden, 25 to 49 for normal burden, 52 to 74 for a request for evidence before senior review, and 75 or above for a high-burden change that should be split or given more context. Gupta recommends calibrating the weights against the last 200 merged pull requests and the team's actual reviewer experience. The output should include what fired, where the reviewer should focus, and what the author can do to reduce the score. A clean pull request produces almost no report, while a score alone does not explain how to improve the change.

17:20

Volume and structural complexity drove burden in the public-repository scan

Gupta describes a scan of 524 pull requests across three public repositories. AI authorship indicators appeared in roughly 5% to 20% of pull requests each week, but those pull requests did not disproportionately enter the high-burden bands. Four pull requests reached the needs-evidence or high bands, and all involved structural work such as large migrations, SDK rewrites, or multi-team refactors. One repository accumulated 186 senior reviewer hours in 27 days, while another accumulated 43 over the same window length. Gupta's conclusion is that complexity drives the score, while AI-driven volume creates the conditions for more complex changes to accumulate.

19:48

Teams should make review debt visible without creating an AI-specific review exception

Gupta recommends one logical change per pull request, tests shipped with the change, work kept within one owner territory, and a human-written explanation of why the change exists. Teams should apply the same review standard to AI and human pull requests. For adoption, he proposes five steps: backfill recent merges, set a threshold such as 50 that requires an author comment, surface the score on every pull request without blocking it, aggregate it weekly by team, and discuss it in retrospectives and planning. He asks teams to compare added throughput with added review debt and to track the weekly slope rather than only the current level.

"These things tell you the speed of production. They do not tell you the speed of trust."03:24
Who should watch
  • You measure coding-agent adoption through pull request counts, cycle time, or lines changed and need a way to account for reviewer workload.
  • Your team is merging agent-assisted changes with weak tests, thin explanations, or ownership spread across several groups.
  • You want to introduce review metrics without making a language model the final judge of engineering quality.