# The AI Bill Is Too Damn High: Tokenomics 101

Eugene Cheah, Featherless AI | AI Engineer NYC 2026 | 55:25

Source: https://www.youtube.com/watch?v=Ti-BmGU1_mQ
Channel: AI Engineer (https://www.youtube.com/@aiDotEngineer). Summarised by AIE Talks.
Page: https://aietalks.com/talks/the-ai-bill-is-too-damn-high-tokenomics-101
Published: 2026-10-17
Tags: cost, fine-tuning, gpus, open-models

## TL;DR
- AI cost control starts with measuring token use by department, team, and use case instead of sharing one API key.
- Teams can cut inference costs by matching model size, speed, and capacity to the workload, then fine-tuning smaller models for repeated domain tasks.
- Higher token use does not prove higher productivity, because poorly designed processes can make people repeat work or review large piles of generated output.

## Summary
Eugene Cheah presents four steps for reducing AI inference costs at scale: measure usage, right-size models, buy GPU capacity around real traffic, and check the surrounding processes. He separates agentic workloads, which tend to use long contexts and repeated cached input, from process workloads such as message filtering and document processing. For model selection, he recommends using strong models for planning and review while cheaper, faster models handle repetitive implementation. He also discusses classifiers, fine-tuning smaller domain models, shadow traffic, and labels such as 'fast' or 'smart' instead of exposing model names to every department. Capacity planning depends on the speed users actually need, since faster responses can cost much more. The final section is about waste: tokens are not work done, and generated code, slides, or reports can shift the burden to reviewers. Cheah argues that teams should measure outcomes and make the person who generates an artifact responsible for it.

## Key ideas
### Token cost cannot be managed until usage is split by owner and workload
[03:23](https://www.youtube.com/watch?v=Ti-BmGU1_mQ&t=203s)
Cheah says many customers begin with one API key shared across the organization, which makes department, team, and use-case spending invisible. He recommends issuing separate keys and tracking where tokens go. That data can show which teams need fast responses, which workloads can run more slowly, and which departments consume the most tokens. It also supports capacity planning. He calls measurement the highest-return first step and says the rest of the optimization work can be ignored until usage is visible.

### Agentic and process workloads have different cost patterns
[05:12](https://www.youtube.com/watch?v=Ti-BmGU1_mQ&t=312s)
Long-running agents, including coding agents, use large contexts across many turns and can run for long periods. Process automation has shorter, isolated tasks that arrive in high volume, such as call-center message filtering, email routing, transcription, and document processing. Cheah gives the example of a five-person team running eight agents continuously and spending heavily on a frontier API. Moving implementation work to a cheaper model and scheduling background agents overnight reduced that workload without reported disruption.

### Model choice should follow the task rather than a general ranking
[09:27](https://www.youtube.com/watch?v=Ti-BmGU1_mQ&t=567s)
Cheah says open models are becoming competitive with closed models on specific tasks, including specialized financial and user-interface benchmarks. He describes cases where a much smaller model performed near a frontier model for a narrow use case. He also introduces classifiers, which focus on making decisions rather than producing chat responses. These models can be much cheaper and faster, so teams should identify classification traffic inside larger workflows instead of sending every request to a general-purpose model.

### Planning and review can use a strong model while repetitive implementation uses a cheaper one
[14:37](https://www.youtube.com/watch?v=Ti-BmGU1_mQ&t=877s)
A common coding-agent pattern is to use the strongest model for planning and review, then assign implementation to a faster, cheaper model. Implementation involves repeated file edits, tests, and checks, so it can consume many tokens. Cheah says teams are more likely to switch when the cheaper model is faster, because speed gives users a direct incentive. He also recommends exposing department-level labels such as 'fast' and 'smart' rather than model names, which lets platform teams change the underlying model without retraining every user.

### Fine-tuning can turn an expensive successful workflow into a smaller specialized model
[25:27](https://www.youtube.com/watch?v=Ti-BmGU1_mQ&t=1527s)
Teams can start with a strong model, use it on a limited deployment, and collect data for a domain-specific model, subject to the provider's terms. Cheah describes organizations fine-tuning smaller models for call-center work, proprietary programming languages, and telecom tasks. Traffic can then be split by difficulty, with a smaller model handling routine requests and a larger one handling cases escalated by a classifier. He recommends sending some shadow traffic to a stronger current model so the specialized model's data keeps improving.

### Inference speed has a direct cost trade-off
[30:07](https://www.youtube.com/watch?v=Ti-BmGU1_mQ&t=1807s)
Cheah warns that tokens per second and cost do not scale linearly. His rule of thumb is that doubling speed can cost three times as much, with the effect compounding. Running more requests in parallel can lower the cost per output while reducing the speed of each individual request. Teams should pay for very high speed when a person is waiting, while background jobs can use slower, cheaper capacity. Traffic measurements also help decide between dedicated instances, public-cloud spillover, and capacity sized around a base workload.

### Token volume does not prove that useful work happened
[37:07](https://www.youtube.com/watch?v=Ti-BmGU1_mQ&t=2227s)
Cheah describes teams that increase agent activity but end up repeating the same work. In coding, a large generated change can overwhelm reviewers, who may ask another agent to identify the relevant edits and then rebuild the change. Similar problems occur when people generate slides or reports and leave someone else to find inconsistencies. Cheah recommends smaller, more modular changes, stronger tests, and processes that make review easier. The right measure is the outcome produced per unit of AI use, not token consumption alone.

### The person who generates an artifact should own its consequences
[42:07](https://www.youtube.com/watch?v=Ti-BmGU1_mQ&t=2527s)
Cheah calls an unchecked pile of generated code, slides, or reports a 'slop grenade.' His proposed rule is that the person handing over generated work remains responsible for it. They must own the text, code, or other output instead of blaming the agent. He also asks teams to compare outcomes with usage by department and team. That can reveal whether additional tokens improve performance or simply increase the amount of material that other people must verify.

## Notable quotes
- "If you are not measuring, you can pretty much ignore everything else I say and start with measuring first because all your biggest ROI is just by measuring." (09:10)
- "Nobody really switches their model to save the company money." (15:47)
- "For every two x of speed, it's three x the cost and it compounds." (30:07)
- "Tokens are not work done." (37:07)
- "Do not let slop grenades happen." (42:07)

## Tools & references mentioned
- Featherless AI
- Linux Foundation
- Hugging Face
- Gemini
- DeepSeek
- AT&T
- Shopify
- Devin
- Cerebras
- Claude Code
- Pi
- MCP

## Who should watch
- Your organization spends enough on inference that department, team, and use-case attribution would change deployment decisions.
- You run coding agents, background jobs, or high-volume classification and need to decide where speed, model quality, or dedicated capacity is worth paying for.
- Your AI output is growing faster than your review, testing, or approval processes, so more tokens are creating work for people downstream.

## Related talks

- [Tokens Are the Cheap Part](https://aietalks.com/talks/tokens-are-the-cheap-part) (Steve Jamison, Jellyfish, 17:32)
- [Why No One Agrees on What Their AI Tokens Cost](https://aietalks.com/talks/why-no-one-agrees-on-what-their-ai-tokens-cost) (Pratyusha Duvvuri, Cisco, 18:57)
- [FinOps for AI Agents: Who Spent All the Tokens?](https://aietalks.com/talks/finops-for-ai-agents-who-spent-all-the-tokens) (Tisha Chawla & Susheem Koul, Microsoft, 21:24)
- [From Tokenmaxxing to Trusted Throughput](https://aietalks.com/talks/from-tokenmaxxing-to-trusted-throughput) (Mingsheng Hong, Ironclad, 23:04)
- [The Cheaper AI Model That Costs You More](https://aietalks.com/talks/the-cheaper-ai-model-that-costs-you-more) (Rayan Krishnan, Vals AI, 15:28)
