Agents create a measurement problem because token usage does not tell customers whether the work was worth the cost.
2
Horsepower made steam engines legible to people who understood work in horses, and agents need an equivalent way to connect their output to value.
3
The best agent tasks have uncertain steps but clear acceptance criteria, so their work is harder to perform than to verify.
Summary
Maximillian Piras argues that agents have a measurement problem. Early adopters can run many agents in parallel, but their experience does not describe people who have never used one. Token counts show what an internal system consumed, not whether it closed bugs, resolved support requests, or advanced a business objective. Piras uses James Watt's horsepower as a historical model. The measure was imperfect, but it helped people who thought in horses understand the possible gain from steam engines. He proposes "mousepower" as a similar idea for agents, while admitting that cursor movement cannot capture the complexity of computer work. The practical framework is an entropy matrix. Tasks should have enough uncertainty to benefit from an agent, while retaining clear criteria that make the result cheaper to check than to produce. This also moves verification into the scope of agent design.
Parallel agents make work feel free until the bill arrives
Piras describes backgrounding agents while he focuses on a main task, such as giving the talk. He starts several agents in parallel to explore slide layouts and design directions because there always seems to be more research and exploration to do. The pattern is attractive because it appears to maximize idle time. The doubt arrives with the bill: was the work worth the tokens, and could the agents have been sequenced more efficiently? This everyday experience motivates his question about how to evaluate token cost and help customers judge the value of agent work.
Computer-use models reach work that has no API or MCP
At Yutori, Piras works on computer-use models. These models learn to operate a computer as a person would, which lets an agent gather or manipulate information when an API or MCP is unavailable. He describes this as less efficient than using an API or MCP, but useful as a last resort. His customer conversations focus on making agents intuitive and understanding how customers value possible use cases. Customers are excited, yet many feel they are only scratching the surface and do not know which tasks justify the token cost.
Agents are hard to measure because early adopters are not typical users
Piras's thesis is that agents have a measurement problem. People in the room may already have fleets of agents and may feel comfortable measuring them, but he warns that they are early adopters. Their enthusiasm and habits do not represent people who have never touched an agent. Some users still copy and paste into ChatGPT. A measure that works for heavy users may fail to explain value to those customers. Piras connects this problem to the challenge of selling tokens, whether the tokens are sold directly or through a product built on them.
Horsepower made an unfamiliar technology understandable
James Watt studied the horse gins that powered mills because he wanted to sell steam engines to people who understood productive work in horses. A horse walked in a circle while pulling a rotary arm, producing power for tasks such as grinding grain. Watt created horsepower as a baseline for comparing the steam engine with a familiar source of power. Piras says the measure was not especially scientific or accurate, but it communicated an increase in value. That was enough to help people estimate the possible return and try a technology whose benefits were otherwise hard to imagine.
Piras describes a cycle of overspending and underusing. People max out tokens, become austere or stop using agents, then return when fear of missing out pushes them to try again. He says token use is an output of the system, not the outcome customers care about. Teams should trace spending to results such as bugs closed or support requests resolved, then connect those results to progress on their objectives. Without that link, customers cannot tell whether the spend produced enough value.
Agent coding moved the bottleneck into human review
Coding agents can produce code faster, but review has not scaled at the same rate. Piras says this leaves teams dying by a thousand pull requests, with the human review process becoming the bottleneck. Anthropic team members have said that coding may be solved while code review is not. The result is a measurement problem: teams generate code, but cannot judge its quality quickly enough to know whether the additional output justifies the spend. Piras argues that agent builders must help customers verify the work, rather than focusing only on execution.
Mousepower is a mental model, not a cursor-speed benchmark
Piras introduces "mousepower" as a possible equivalent to horsepower for computer agents. He briefly jokes about asking Claude to build a device that measures cursor movement, then rejects that approach because computer work takes place in a space with too many dimensions for cursor speed to capture it. Mousepower is an idea about creating a baseline for how people use computers and showing where an agent performs better. The useful requirement is a rubric that lets customers verify the work and decide whether the tokens were worth spending.
The best agent tasks are harder to do than to check
Piras's entropy matrix maps uncertainty in the steps of a task against uncertainty in its acceptance criteria. When the steps are predictable, a script is usually better than an agent. When the steps are extremely uncertain, the task may be out of distribution and have sparse rewards. When acceptance criteria are highly uncertain, checking the result can require doing the task again. The promising middle has enough step uncertainty to make an agent useful, while keeping the result relatively easy to validate. Piras compares this to an NP-style problem, where verification is easier than execution, and suggests building agents that verify other agents' work.
