# Tokens Are the Cheap Part Steve Jamison, Jellyfish | AI Engineer NYC 2026 | 17:32 Source: https://www.youtube.com/watch?v=U7ZYBCcdJPg Channel: AI Engineer (https://www.youtube.com/@aiDotEngineer). Summarised by AIE Talks. Page: https://aietalks.com/talks/tokens-are-the-cheap-part Published: 2026-10-13 Tags: cost, evals ## TL;DR - AI spending rose 60-fold while merged pull requests per engineer rose 43% over the period studied. - When developer salaries and AI costs are combined, the cost per merged pull request fell 26%. - Higher AI spending was associated with more throughput and lower cost per pull request, but quality and customer value still need separate measurement. ## Summary Steve Jamison argues that AI budgets should be judged alongside developer salaries, throughput and quality. In Jellyfish data, AI spending rose 60-fold while merged pull requests per engineer rose 43%. Because salary remains about 97.5% of the combined cost of producing a merged pull request, the cost per merged pull request fell 26% over the same period. Companies in the highest AI-spending group saw larger throughput gains and bigger cost reductions than lower-spending groups, although the pattern showed diminishing returns. A comparison of developers with their own lower-use weeks found similar results. Jamison is careful about what the data cannot prove. Revert rates showed no worrying quality pattern, but they were only a limited proxy. More lines of code and more pull requests do not establish customer value. His recommendation is to track total cost per merged pull request, AI spend, throughput and quality together before setting token limits or cutting investment. ## Key ideas ### AI spending and merged pull request throughput rose together [00:02](https://www.youtube.com/watch?v=U7ZYBCcdJPg&t=2s) Jamison opens with two apparently conflicting figures from Jellyfish data. AI costs rose 60-fold over the year studied, while merged pull requests per engineer rose 43%. He says the charts show the inflection points occurring together, with AI spending increasing as throughput began a steady rise. Those figures alone do not answer whether an organization is spending enough or too much. A CFO or board may ask for evidence that the bill produced value, but comparing the AI bill with pull request output leaves out the largest cost in the calculation: developer salary. ### Developer salary belongs in the cost of producing a pull request [02:48](https://www.youtube.com/watch?v=U7ZYBCcdJPg&t=168s) Jamison combines what a company pays its engineers with what it pays for AI and calls the result the intelligence cost of a pull request. In the data from August 2026, AI accounted for about 2.5% of the cost of producing another unit of merged code, while salary accounted for about 97.5%. He uses this split to explain how AI spending can rise sharply while delivery becomes cheaper. The relevant measure is not the AI bill on its own. It is the total cost allocated to each merged pull request, including review time and the fact that the pull request has passed some quality gates. ### The cost per merged pull request fell despite much higher AI costs [04:15](https://www.youtube.com/watch?v=U7ZYBCcdJPg&t=255s) The dollar cost of a merged pull request fell 26% over the year, even though AI costs increased 60-fold. Jamison says 83% of the companies studied showed a drop in cost per pull request. His explanation is that additional throughput spreads developer salary across more merged work, and that effect is larger than the increase in AI costs. He distinguishes merged pull requests from created pull requests because the measure includes review time and some quality screening. The result is an observational cost comparison, not proof that token spending alone caused the decline. ### Heavier AI-spending companies saw larger throughput gains and lower costs [05:00](https://www.youtube.com/watch?v=U7ZYBCcdJPg&t=300s) Jellyfish ranked companies by token usage and divided them into five groups. In the lightest group, AI represented about 0.3% of pull request cost, with roughly 60 million tokens per developer per month. In the heaviest group, AI represented 8.5% of pull request cost, with about 2.5 billion tokens per developer per month. The lower four groups saw throughput gains of about 40%, while the heaviest group saw a 67% increase. Cost per pull request fell about 25% for the lower groups and 32% for the heaviest group. Jamison also shows diminishing returns as usage rises, rather than an immediate point where more tokens have no value. ### Individual developers showed a similar pattern in heavier-use weeks [08:55](https://www.youtube.com/watch?v=U7ZYBCcdJPg&t=535s) Jamison then compares developers with themselves, using weeks with heavy agent-skill use and weeks without those skills. In the heavier-use weeks, developers spent about twice as much on tokens, merged 27% more pull requests and reached a cost per pull request about 18% lower after salary and token costs were combined. This within-developer comparison gives a different view from comparing separate companies. It still does not establish causation, but it shows that the company-level pattern also appeared when the comparison used the same developers across different weeks. ### Reverts provide only a limited check on quality [10:12](https://www.youtube.com/watch?v=U7ZYBCcdJPg&t=612s) Jamison asks whether faster, cheaper work might also be worse work. He uses reverted merged pull requests as one proxy for quality. Revert rates dropped slightly for every spending group, although the changes were just below the noise in the data. He says this is not evidence of an improvement in quality. It also did not show worrying signs of rampant or severe quality problems at higher token spending and faster throughput. Reverts are only one signal, so the cost analysis needs to sit beside broader quality measures. ### Higher throughput did not come from smaller pull requests [11:02](https://www.youtube.com/watch?v=U7ZYBCcdJPg&t=662s) The analysis does not support the idea that more merged pull requests came from splitting work into smaller pull requests. Lines of code generated per pull request increased. Jamison points to verbose comments, unnecessary tests and the ease of combining several changes into one prompt as examples of how AI can increase the amount of code in a pull request. More capable systems may also encourage developers to attempt more ambitious work. He does not settle what caused the increase in lines of code, and he warns that pull request counts and code volume do not by themselves show customer value or reduce review and maintenance work. ### Teams should compare total cost, throughput and quality before limiting tokens [15:02](https://www.youtube.com/watch?v=U7ZYBCcdJPg&t=902s) Jamison says AI spend per pull request shows the intelligence being bought, while pull requests per engineer show the output associated with it. Neither captures the trade-off alone. Adding salary and calculating cost per merged pull request gives a more useful management measure. Teams should track that measure beside AI spend and throughput every month, then watch reverts and other quality metrics as a counterweight. This framework can inform decisions about token limits, model routers, post-training and further AI enablement. His conclusion is that most companies in the observed range should not make the token bill their main spending constraint. ## Notable quotes - "The change is nearly universal, too. 83% of the companies we studied showed a drop in cost per PR." (04:39) - "Spreading the same salary across more PRs in the month has more of an effect than the rising AI costs because tokens are the cheap part." (07:14) - "This isn't to say spend frivolously. Sure, you should watch for runaway jobs." (14:45) - "The bill tells you what you spent. Cost per PR tells you whether it worked." (16:05) ## Tools & references mentioned - Jellyfish - Substack - LinkedIn ## Who should watch - Engineering leaders deciding whether to cap model or agent spending need a way to compare the AI bill with salary, throughput and quality. - Finance and engineering teams preparing an AI budget will benefit from the cost-per-merged-pull-request framing. - Teams measuring AI adoption with pull request counts or lines of code should watch this for its warnings about review burden and weak quality proxies. ## Related talks - [From Tokenmaxxing to Trusted Throughput](https://aietalks.com/talks/from-tokenmaxxing-to-trusted-throughput) (Mingsheng Hong, Ironclad, 23:04) - [Can You Prove AI ROI in Software Engineering?](https://aietalks.com/talks/can-you-prove-ai-roi-in-software-engineering) (Yegor Denisov-Blanch, Stanford, 16:40) - [The State of AI in Software Development: Data from 400+ Orgs](https://aietalks.com/talks/the-state-of-ai-in-software-development-data-from-400-orgs) (Justin Reock, DX, 19:09) - [FinOps for AI Agents: Who Spent All the Tokens?](https://aietalks.com/talks/finops-for-ai-agents-who-spent-all-the-tokens) (Tisha Chawla & Susheem Koul, Microsoft, 21:24) - [The ROI of AI: Why You Need an Eval Framework](https://aietalks.com/talks/the-roi-of-ai-why-you-need-an-eval-framework) (Beyang Liu, Sourcegraph, 25:28)