# Building Cursor Composer Lee Robinson, Cursor | AI Engineer CODE 2025 | 15:36 Source: https://www.youtube.com/watch?v=fL1iJHtl51Q Channel: AI Engineer (https://www.youtube.com/@aiDotEngineer). Summarised by AIE Talks. Page: https://aietalks.com/talks/building-cursor-composer Published: 2025-12-02 Tags: agents, coding-agents, inference, reinforcement-learning, tool-use ## TL;DR - Cursor Composer was trained to combine low-latency token generation with the ability to handle real software engineering tasks. - Cursor matched its training environment to production by connecting inference, rollout, training, and sandboxed environment servers, then scaling them across large VM and GPU fleets. - Reinforcement learning improved Composer's tool use, including parallel calls and semantic search, while reducing unnecessary edits. ## Summary Lee Robinson explains how Cursor built Composer, its first agent model for software engineering. The target was a model that felt fast during interactive work while becoming capable enough for daily coding. Cursor trained it against internal tasks based on the company's own repositories and production usage. The system runs rollouts where the model chooses tools, including file operations, code search, linting, and shell commands, then uses scored results to update the model. Scaling this process required matching training and inference, handling rollouts with very different lengths, and supporting bursty workloads. Cursor connected inference, training, rollout, and environment servers, added custom low-precision kernels, and used cloud virtual machines as secure training sandboxes. Composer improved at parallel tool calls, semantic search, and deciding when to inspect code before editing it. Robinson describes the model as a faster, more synchronous option between slow background agents and weaker fast models. ## Key ideas ### Composer was built to make agent coding feel fast without giving up capability [00:36](https://www.youtube.com/watch?v=fL1iJHtl51Q&t=36s) Cursor designed Composer for real-world software engineering and aimed for both speed and intelligence. Robinson says it performs better than the best open-source models and sits somewhat below the latest frontier models, while generating tokens about four times more efficiently than models with a similar level of intelligence. Cursor first tested a low-latency prototype called Cheetah. Users liked its speed, but many did not find it smart enough for daily coding. The team therefore built an internal benchmark around its own repositories and development practices. A checkpoint that Cursor developers would use every day became the practical test for whether the model was good enough. ### The agent learns through rollouts that mirror Cursor's actual tool environment [03:17](https://www.youtube.com/watch?v=fL1iJHtl51Q&t=197s) A user query reaches Cursor's backend, where the agent decides which tools to call. The tools include reading and editing files, searching the codebase, checking lints, and running shell commands. The agent can choose serial or parallel calls. During reinforcement learning, Cursor runs multiple rollouts from the same starting point. One rollout might read and edit files, while another also searches the codebase. Cursor scores the outputs, selects the better result, and updates the model parameters. The training process is designed to resemble real Cursor queries rather than a simplified text-only task. ### Scaling agent training created infrastructure problems around speed, workload variation, and consistency [04:40](https://www.youtube.com/watch?v=fL1iJHtl51Q&t=280s) Composer is a large mixture-of-experts model spread across thousands of GPUs, so training and sampling need to run quickly and in closely matched environments. Realistic rollouts can consume hundreds of thousands to millions of tokens and make hundreds of tool calls. Their durations vary widely because some agents install packages or perform many operations. Training also creates bursty compute demand, unlike ordinary production inference. Robinson says the team needed identical tool formats and responses between training and production, while also handling uneven completion times. These requirements made infrastructure part of the machine learning work. ### Cursor connected separate servers to train, roll out, and simulate the product [05:59](https://www.youtube.com/watch?v=fL1iJHtl51Q&t=359s) The system has inference servers, rollout servers using Ray, and environment servers that simulate the Cursor environment. These components communicate during training. Rollouts call tools and receive results from the environment, while the trainer updates the model and sends new parameters back to inference. Cursor developed custom kernels for low-precision training so the large model could train faster and move more easily to inference. Robinson reports that the mixture-of-experts layer became about three and a half times faster on Nvidia Blackwell chips. Load balancing across threads and processes also reduced idle time when individual rollouts took different amounts of time. ### Cloud agents gave Cursor a production-like sandbox for reinforcement learning [08:12](https://www.youtube.com/watch?v=fL1iJHtl51Q&t=492s) Cursor was building its cloud agents product alongside Composer. Cloud agents run Cursor agents away from the desktop, including from a phone, the web, or Slack. Each task starts a virtual machine that loads the user's code and lets the agent edit files and run tools inside a secure sandbox. Cursor reused this fleet as a training environment because it closely matched the production product. The workload was still difficult to orchestrate because reinforcement learning is spiky, while cloud-agent inference has a different pattern. Robinson describes many clusters and hundreds of thousands of virtual machines, with internal dashboards used to view the fleet. ### Training with semantic search turned Composer into a stronger user of that tool [09:39](https://www.youtube.com/watch?v=fL1iJHtl51Q&t=579s) Cursor indexes a user's codebase with its own embedding model. The agent can then issue natural-language queries to find files it may need to edit. Robinson says Cursor's research found that semantic search helped every model tested inside the agent harness, and helped Composer especially. He attributes this partly to the matching training and inference environments. Since Composer was trained while using the same tool it would receive in production, it learned how to use semantic search effectively. The production-like environment therefore gave Cursor a way to train specific tool behavior directly rather than adding the tool only after model training. ### Reinforcement learning changed both how fast Composer worked and how it edited code [11:00](https://www.youtube.com/watch?v=fL1iJHtl51Q&t=660s) Cursor saw reinforcement learning work when additional rollouts kept improving the model. Composer began around the performance of the best open model and moved closer to frontier coding agents as Cursor added training compute. The process also changed the agent's behavior. Parallel tool calls let it read multiple files at once, improving the end-to-end experience instead of only increasing token speed. Earlier versions made too many edits, including edits that were unnecessary. With more training, Composer searched and read more before changing files, which made its actions more effective. ### Composer fills the uncomfortable middle between slow powerful agents and fast weak ones [12:13](https://www.youtube.com/watch?v=fL1iJHtl51Q&t=733s) Robinson compares early coding-agent use to airplane Wi-Fi. It works, but delays make the experience frustrating. Agents that take 10 or 20 minutes can leave a programmer in what he calls a 'semi async valley of death': the task is too slow for synchronous work, yet not powerful enough to justify waiting like a long background job. Composer is intended to bring coding agents back into a more immediate loop. Robinson says he uses a frontier model such as GPT-5.1 Codex for planning, then gives the plan to Composer to build. His broader conclusion is that reinforcement learning can work well for narrowly focused coding models, and that their training depends heavily on infrastructure. ## Notable quotes - "So we needed it to be smart and fast. Definitely needed to be smart." (01:48) - "All of these servers talk to each other." (06:30) - "The model kind of becomes a power user of this tool which is really effective." (10:19) - "I kind of describe this problem as like airplane Wi-Fi." (12:24) - "At Cursor we're not trying to build general intelligence. We're not trying to build AGI. We're trying to build very good coding models." (13:31) ## Tools & references mentioned - Cursor - Cursor Composer - Cursor 2.0 - Cheetah - Ray - PyTorch - Nvidia Blackwell - GPT-5.1 Codex - Claude Sonnet 4.5 - Sasha - Dex - Vercel - Slack ## Who should watch - You are building coding agents and need a concrete example of how tool calls, environments, and reinforcement learning fit together. - You run model training or inference workloads with uneven, bursty jobs and want to see why rollout orchestration becomes a systems problem. - You are deciding whether to use semantic search, parallel tool calls, or cloud sandboxes in an agent product. ## Related talks - [Recursive Model Improvement](https://aietalks.com/talks/recursive-model-improvement) (Lee Robinson, Cursor, 20:32) - [Replacing 12K LoC with a 200 LoC Skill](https://aietalks.com/talks/replacing-12k-loc-with-a-200-loc-skill) (David Gomes, Cursor, 19:22) - [Computer-Use 2.0: Agents Just Got Multi-Cursor](https://aietalks.com/talks/computer-use-2-0-agents-just-got-multi-cursor) (Francesco Bonacci, Cua, 16:41) - [Building a Chess Coach](https://aietalks.com/talks/building-a-chess-coach) (Anant Dole & Asbjørn Steinskog, Take Take Take, 18:22) - [Rethinking how we Scaffold AI Agents](https://aietalks.com/talks/rethinking-how-we-scaffold-ai-agents) (Rahul Sengottuvelu, Ramp, 16:32)