Same GPU, 50% More Output: Why the Operator Matters

Oliver Kennon, SemiAnalysis17:42 · Oct 2026 · 4,976 views
Thumbnail for Same GPU, 50% More Output: Why the Operator Matters Watch on YouTube
TL;DR
  1. 1

    GPU hours are an input, while useful intelligence is the output that matters to customers.

  2. 2

    The same B200 deployment can produce about 50% more output when the operator uses a better serving stack and settings.

  3. 3

    A credible compute market needs grades that describe the service, the operator, and the rental price being benchmarked.

Summary

Oliver Kennon argues that buying GPUs only creates potential. The operator determines how much useful work those GPUs deliver, at what cost, and with what security and reliability. He uses a B200 example in which two serving stacks run the same model and meet the same interactivity target, yet one produces about 300 output tokens per second compared with 200 for the other. That difference lowers rental spend per acceptable token when the hourly price is the same. Useful intelligence must also meet the task's quality, speed, availability, and security requirements. Kennon describes SemiAnalysis's ClusterMAX ratings for managed GPU clusters, which assess provisioning, reliability, monitoring, and security. He then separates rental price from production cost and explains why cost position depends on competitors. The talk closes with a case for comparable compute grades and operator ratings as foundations for credible compute futures, while noting that a price hedge cannot repair a failed deployment.

Key ideas
01:11

Compute is a commodity when buyers can compare usable output

Kennon frames compute as a commodity market in which producers compete to deliver comparable output. A cost curve ranks producers by what they spend to deliver the same usable work. If two producers receive the same price, the one with the lower production cost keeps more margin and has more room to cut prices. The eventual metric should be price per successfully completed task at the required quality, although that is difficult to standardize. For now, price per token is a practical proxy for comparable workloads and quality requirements.

01:51

GPU hours are inputs, while intelligence is the output

Kennon separates the price of renting a GPU from the output a customer receives. Hardware and electricity provide the basic inputs. Software and operations multiply what those inputs can produce. The operator selects and implements the serving software, keeps requests moving, and makes the machines deliver responses that customers can use. The relevant question is how much work meets the specified task requirements, rather than how many GPU hours were purchased.

03:28

The same B200 can produce about 50% more output

Using SemiAnalysis's InferenceX calculator, Kennon compares two serving stacks running the same model on B200 GPUs, with roughly 8,000 input tokens and 1,000 output tokens. The P50 target is 100 output tokens per second for each user. In the P90 scenario, one stack produces about 200 generated output tokens per second and the other about 300. The difference comes from software, settings, and the number of requests running. With the same hourly bill and acceptable quality, the faster stack gives roughly 1.5 times as much output per hour.

04:39

Useful intelligence includes security and production ownership

A response is useful only when it is good enough for the task, arrives quickly enough, remains available, and stays within the agreed access boundaries. Kennon says security is missing from many chip performance metrics but is essential to usability. SemiAnalysis found monitoring access that exposed another customer's logs and metrics. The operator also has to test serving updates on customer workloads, check responses under load, deploy changes safely, keep rollback paths, and maintain the system after release. Responsibility may sit with the provider, the customer's team, or both, but it must be clear who owns it.

06:27

ClusterMAX assesses managed operators rather than bare-metal products

SemiAnalysis's ClusterMAX, described as version 3.0, assesses operating capability on managed GPU clusters. Its evaluation covers provisioning, reliability, monitoring, and security, using hands-on testing, documentation, and customer feedback. The assessment considers how a service behaves when a machine fails, not only on a good day. Kennon stresses that managed-cluster ratings do not automatically apply to bare metal, where more operational work usually belongs to the customer. The tiers are relative to peers, and pricing and availability also matter.

07:49

Production cost is different from rental price

A GPU provider's rental bill is revenue for the provider and an input cost for an inference producer. The provider covers equipment, power, space, and support. The inference producer adds rent plus software, staff, and other services, or spreads owned-equipment investment over its working life and adds running costs. Cost per acceptable token comes from dividing all spending over a period by the useful output delivered in that period. Better operations can spread the same rental expense or equipment investment over more acceptable responses, although testing and safely deploying improvements also costs people and money.

09:51

Cost advantage depends on what competitors can deliver

Kennon gives a hypothetical example in which one producer's cost per successful task falls from 10 to 8 while a competitor's cost falls from 6 to 12. The first producer improves but moves from being cheaper to being more expensive. Its competitor then has more room to lower its selling price if customers find both services acceptable. Strong demand and limited supply can hide these differences because producers with very different costs can still find customers. Kennon argues that operators should build cost advantages while capital and demand are available, before customers have more choices and prices come under pressure.

12:29

Comparable compute needs service grades and careful benchmarks

Two offers for B200 GPU hours may include different services and leave different operating tasks with the customer. A compute grade should describe the service and what buyers can expect, so buyers know which offers are comparable. A rental-price benchmark summarizes GPU-hour prices within its defined market. It does not measure production cost or intelligence delivered. Advertised offers and completed trades are different, and service scope, contract terms, sampled prices, and aggregation affect the reference. A futures hedge works only when the reference tracks the rent a buyer pays or a seller earns.

14:13

Operator ratings help build compute futures, but cannot fix deployments

Kennon calls operator ratings an essential building block for a credible compute futures market. A rating helps describe the service behind a price and supports the definition of comparable grades. It does not define a futures grade, create a sound benchmark, produce liquidity, or guarantee an effective hedge. A rental-price hedge can offset changes in rent, but it cannot repair a failed deployment or ensure that a customer pays. The producer still has to convert rented or owned compute into work that customers can use.

"Buying hardware gives you potential. Operating capability is what determines how much useful work you actually are able to deliver."00:30
Who should watch
  • You run inference infrastructure and need to understand how serving software, operations, and security affect the cost customers actually experience.
  • You buy GPU capacity and need a clearer way to compare managed services, bare metal, and rental-price offers.
  • You are assessing compute benchmarks or futures and want to separate rental-price risk from production performance and deployment risk.