# How to Get Frontier Coding Results at 70% Lower Cost

Nouha Dziri, Cohere | AI Engineer | 31:22

Source: https://www.youtube.com/watch?v=maitMe9HCvw
Channel: AI Engineer (https://www.youtube.com/@aiDotEngineer). Summarised by AIE Talks.
Page: https://aietalks.com/talks/how-to-get-frontier-coding-results-at-70-lower-cost
Published: 2026-11-07
Tags: agents, coding-agents, cost, inference, multi-agent

## TL;DR
- A coding-agent team can outperform a solo agent on difficult software tasks, but the team can cost four to seven times more.
- Extra workers should be assigned to the stage that limits progress, such as planning on ProgramBench and implementation on Commit0.
- A strong lead paired with cheaper workers can approach the performance of an all-strong team at much lower cost.

## Summary
Nouha Dziri studies how coding-agent teams should divide work, choose models, and spend inference compute. Her harness gives a lead agent a shared workspace and tools to delegate, resume, supervise, or stop workers. On ProgramBench, teams beat solo agents, but the gain costs several times more. A blocking lead performs better than an asynchronous lead because it waits for worker results before continuing. The best allocation depends on the task. Planning workers help most on ProgramBench, where agents must discover the behavior of a hidden executable. On Commit0, implementation is the bottleneck because the program skeleton, documentation, and tests are already available. Dziri also finds that a strong lead with cheaper workers can match an all-strong team at much lower cost. Agents mostly communicate through the lead, and extra peer messaging did not improve results in these experiments. She ends with open questions about larger teams, communication, prompt-free coordination, and safety.

## Key ideas
### Agent teams split long coding tasks into fresh contexts
[00:24](https://www.youtube.com/watch?v=maitMe9HCvw&t=24s)
Dziri contrasts two familiar ways to spend inference compute: let one agent reason for longer, or run several independent attempts and aggregate them. Long coding tasks create a context problem because the agent must inspect repositories, edit code, run tests, and debug over hours. Independent sampling does not remove that burden because every attempt repeats the whole task. A lead-worker team gives each worker a subtask and a fresh context. Workers can inspect files, run tests, and examine logs, then return a compact result while the lead retains the overall picture. This can reduce completion time when the task has separable parts, although the token budget grows quickly as more workers are added.

### The harness exposes team structure as a compute-allocation problem
[02:05](https://www.youtube.com/watch?v=maitMe9HCvw&t=125s)
Dziri's minimal harness uses a top-down hierarchy. A lead receives the task, dynamically chooses the number of workers, assigns subtasks, and consolidates their results. Every agent works through a bash tool in one shared workspace. The lead can delegate work, resume or kill a worker, and intervene when a worker makes an error. The experiments vary one design choice at a time: team organization, communication, team size, model assignment, and the software-engineering stage where workers operate. Dziri measures efficiency as the reward gained per added dollar over a solo agent. The aim is to find where the next unit of inference compute produces the most improvement rather than simply maximizing the number of agents.

### A blocking lead outperforms an asynchronous lead because it waits
[06:46](https://www.youtube.com/watch?v=maitMe9HCvw&t=406s)
On ProgramBench, a team with a blocking lead beats a solo agent by almost 10 points, but costs about four to seven times as much. The blocking lead assigns subtasks and waits for every worker to report before consolidating the work. An asynchronous lead keeps coding while workers explore. Dziri found that the asynchronous lead often starts building on guesses before worker findings arrive. When the reports come back, it may ignore them, redo work, or submit while workers are still running. The blocking lead avoids this failure because it cannot submit until the workers have returned. More detailed coordination instructions can help the asynchronous version, but the resulting teams still cost six to seven times as much as a solo agent.

### Workers pay off most at the task's bottleneck stage
[11:20](https://www.youtube.com/watch?v=maitMe9HCvw&t=680s)
Dziri tests workers assigned only to planning, only to implementation, or only to verification. On ProgramBench, planning workers provide the best return. The agent must probe a hidden executable, discover its interface and edge cases, and build a detailed implementation plan before writing code. Several workers can explore different parts of the program in fresh contexts and send compact findings to the lead. Planning-focused teams reach almost the same performance as a heavily prompted team at half the cost. Dziri measures each planning dollar as buying almost 1.6 times more reward than a dollar spent on implementation or verification in these experiments. Extra implementation or verification workers mainly raise cost because they build on an incomplete understanding of the task.

### The best stage changes with the benchmark
[18:24](https://www.youtube.com/watch?v=maitMe9HCvw&t=1104s)
The planning result does not generalize automatically to every coding task. On Commit0, the documentation, program skeleton, and unit tests are already available, so planning is not the main obstacle. Implementation is the bottleneck instead. Doubling the number of implementation agents produces the largest improvement there. Dziri's conclusion is that team allocation must follow the stage that limits the particular task. ProgramBench rewards parallel discovery because the executable is a black box. Commit0 gives agents more of the necessary structure, so the difficult work shifts toward producing a correct implementation. A team that allocates workers uniformly can spend heavily on stages that do not improve the result.

### A strong lead sets the performance ceiling while cheaper workers set the price
[19:19](https://www.youtube.com/watch?v=maitMe9HCvw&t=1159s)
Dziri compares teams built from models with different capability and cost levels. Keeping the strongest model as the lead and replacing strong workers with cheaper workers preserves nearly the same performance while reducing cost by about 60 percent. A cheap lead with stronger workers performs worse because the lead controls the plan, delegates the work, and decides how to use the returned results. Dziri summarizes this relationship as: the lead sets the ceiling and the workers set the price. Scaling the number of cheaper planning workers also reduces cost. A team with 16 cheaper planners is 38 percent cheaper than an all-strong planning team and 70 percent cheaper than using strong workers at every stage in the tested setup.

### Agents mostly report through the lead instead of collaborating directly
[23:35](https://www.youtube.com/watch?v=maitMe9HCvw&t=1415s)
Even when every agent receives a tool for messaging any other agent, the workers rarely communicate directly. About 84 percent of messages go from workers to the lead, while worker-to-worker communication is less than one message per task. Additional messaging does not improve the tested results. Stronger agents send more message text, but more communication does not reliably produce more reward. Cheaper teams can lose four to seven points when messaging is added. Dziri attributes this partly to the harness: the lead assigns independent tasks, and workers cannot see which other agents exist or what they are doing. She does not conclude that peer communication is unnecessary. Larger teams will need a better way to discover relevant teammates and coordinate without routing every exchange through one lead.

### Larger agent teams need new coordination and safety mechanisms
[28:27](https://www.youtube.com/watch?v=maitMe9HCvw&t=1707s)
Dziri tested up to 16 workers in a lead-worker hierarchy and leaves open how teams of thousands should organize and communicate. She expects future systems to choose their own structure based on the task, budget, and deadline, but says this needs careful study. Detailed prompt tuning for every task is also unlikely to scale, so leads should learn how to organize teams without extensive hand-written instructions. She raises safety and security concerns because autonomous teams can run continuously with little supervision. The problem changes when systems contain teams or swarms rather than one agent. Securing the infrastructure and aligning these agents become necessary research problems.

## Notable quotes
- "So, basically, if we To answer this question, we actually design our multi-agent harness as a top-down hierarchy." (05:05)
- "So, the lead sets the ceiling and the workers set the price." (21:47)
- "Where you put agent matters more than how many." (28:11)
- "The first thing is that teams are now a real new axis for inference compute but the catch here is that they cost several times more." (28:10)

## Tools & references mentioned
- Cohere
- Mila
- Allen Institute
- ProgramBench
- Commit0
- GPT-5.6
- Cloud Opus 4.7
- Gemini
- Meta

## Who should watch
- You are building coding-agent workflows and need to decide whether parallel workers justify their token cost.
- Your tasks have distinct planning, implementation, and verification stages, and you want to place workers where they can change the result.
- You are designing a multi-agent harness and need evidence about lead behavior, model selection, communication, and scaling risks.

## Related talks

- [Benchmarking Coding Agents on New vs Legacy Codebases](https://aietalks.com/talks/benchmarking-coding-agents-on-new-vs-legacy-codebases) (Denys Linkov, Wisedocs, 18:08)
- [Develop at Idea Velocity](https://aietalks.com/talks/develop-at-idea-velocity) (Jeffrey Lee-Chan, Snapchat, 15:28)
- [Without Caching, Your AI Agent Costs 10x More](https://aietalks.com/talks/without-caching-your-ai-agent-costs-10x-more) (George Cameron, Artificial Analysis, 19:43)
- [Coding Agents Don't Scale Themselves. Neither Do Your Teams.](https://aietalks.com/talks/coding-agents-dont-scale-themselves-neither-do-your-teams) (Patrick Debois, Tessl, 22:06)
- [From AI-Assisted to AI-Native: Building a Frontier Development Team](https://aietalks.com/talks/from-ai-assisted-to-ai-native-building-a-frontier-development-team) (Clare Liguori, AWS, 20:57)
