Model routing should follow a session's changing subtasks, because an agent can move from codebase exploration to implementation and debugging.
2
Cognition keeps a frontier model in charge of planning while cheaper models do implementation work, reducing the cost of Fable-level intelligence by 40%.
3
Small models can cost more than larger ones on tasks outside their training distribution, while caching and continuous context change the economics of multi-model systems.
Summary
The panel argues that model routing is still an early field, especially for agents whose work changes over time. Walden Yan describes Cognition's Fusion approach: a frontier model remains responsible for planning and difficult decisions, while cheaper models handle implementation and can explore a codebase in parallel. Alex Atallah explains why choosing a model from a static task benchmark is fragile. A small model can thrash on an unfamiliar task, repeatedly calling tools until it costs more than a stronger model. The panel also covers context sharing, KV-cache costs, compaction, local versus cloud inference, and ways to detect when a model is out of its depth. Cognition uses one continuous sidekick context instead of many short-lived subagents. OpenRouter's router gained adoption after OpenClaw introduced frequent heartbeats with very different intelligence requirements from its main tasks. The speakers expect routing logic to be distributed across models, agents, and harnesses, with some controller still needed for arbitration.
A frontier model can make cheaper delegation more comprehensive
Walden Yan says Cognition's Fusion keeps the frontier model responsible for planning and hard decisions, then delegates implementation to a cheaper model. This reduced the cost of Fable-level intelligence by 40%. The cheaper model can use more tokens within the same budget and can spin off three subagents to explore a codebase. That broader exploration can be more comprehensive than asking one frontier model to inspect the codebase along a single path. The system therefore saves money while allowing the implementation work to go deeper.
Models have different strengths inside the same engineering task
Tanay Varshney describes model capabilities as jagged. Coding is made up of smaller areas such as scikit-learn, matplotlib, and model building, and the training data for each model gives it different strengths. A model that scores higher on a coding benchmark is not automatically better at every coding task. Routing requires understanding those strengths and weaknesses, then applying them to subtasks. Alex Atallah extends the point to scientific discovery, where many subdomains and task-specific training histories can create useful complementary abilities across models.
Static task-based routing breaks down during agent sessions
Walden Yan calls naive routing by initial task type extremely fragile for agentic work. A developer session can begin with a request to explain a codebase, move into feature implementation, and then become live debugging. The task type and its complexity change over time, so a model selected at the beginning can become a poor fit later. Cognition's approach keeps a frontier agent watching the work even when another model is executing it. That agent can recognize when the delegated model is out of its depth and move the work elsewhere.
Continuous sidekick context can reduce repeated context costs
Cognition avoids a simple main-agent-and-many-subagents design. Yan says Fusion uses one sidekick with a continuously running context, so the main agent does not need to provide earlier context again. The context remains in the KV cache, where cached tokens cost roughly ten times less. The system still has to decide what information returns to the main model. Possible signals include the files the sidekick reads and a high-level account of its work. Context compaction can also produce a shorter representation for the main agent.
Small models can thrash outside their training distribution
Alex Atallah says model routing should distinguish between tasks that are in distribution and those that are out of distribution for a model's training data. On an in-domain classification task, a small model is usually the sensible choice. On an unfamiliar task, it may call tools repeatedly and enter costly loops. Atallah gives the example that running Terminal-Bench on Opus and Haiku produced about three times better performance from Opus at one-tenth the cost, even though Haiku is cheaper per token. A cheap token price does not guarantee a cheap completed task.
Walden Yan says compaction does not necessarily solve cost or throughput problems. Compaction creates a cache miss, so previously cached input tokens can become about ten times more expensive. Cognition uses compaction mainly because long contexts lose intelligence quality. Yan recommends avoiding the advertised million-token windows when possible, saying model intelligence can fall off a cliff before that limit, often below 200,000 tokens and preferably below 100,000. A file system and other external stores let an agent retain full information outside its immediate context, then retrieve only the relevant parts.
OpenClaw made model routing useful through heartbeats
Alex Atallah says OpenRouter had an auto router for almost two years with little adoption. Users generally wanted to select specific models, and the router was mostly a way to discover suitable options. Adoption changed when OpenClaw began sending heartbeats about every ten minutes. If Opus was the default model, those heartbeats consumed many expensive tokens even though they needed little intelligence. One application therefore created two distinct workload types, periodic activity checks and more demanding agent work. That gave users a reason to route the workloads separately.
Routing can use local compute for privacy and cost
Carter Abdallah describes local-versus-cloud routing as another important use case. A system could detect sensitive information and handle that part on device, then anonymize information before sending a more advanced workload to a cloud model. A developer with a DGX Spark could also send suitable work to local hardware instead of paying full cloud token prices. The panel notes that self-hosting changes the economics because the operator controls cache duration and can optimize hardware for a known workload. Cloud providers instead amortize capacity across many users and price for more general usage.
"If you run terminal bench on Opus and Haiku, like Opus will do about three times better at 1/10 the cost of Haiku, even though Haiku's significantly cheaper per token."15:25
Who should watch
You are building an agent that moves through several phases, such as codebase exploration, implementation, and debugging, and need routing that can adapt during one session.
Your cloud model bill is growing because agents repeat context, call tools in loops, or run frequent background heartbeats.
You are deciding whether to self-host models, use local hardware for sensitive work, or combine local inference with frontier cloud models.
