Coding agents become more useful when their context, tools, skills, and verification loops are designed around the task.
2
Reinforcement learning is being applied to coding agents, environments, prompts, kernels, and model-specific products.
3
Benchmark scores can improve while experienced developers still lose time, because real codebases contain context, coordination, and reliability problems that tests may miss.
Summary
This full-day summit examines how AI is changing software development, from coding agents and model training to code evaluation and organizational practice. Anthropic argues that general agents need domain expertise packaged as progressively disclosed skills. Dex Horthy and Jake Nations focus on context management, research, planning, and the danger of generating code faster than teams can understand it. Cursor, Meta, Prime Intellect, OpenAI, and Cline describe reinforcement learning systems built around tools, sandboxes, verifiers, and real coding tasks. Other talks cover prompt learning, AI-generated PyTorch kernels, agent-first products, and the limits of benchmark scores. METR presents a counterpoint: in a study of highly experienced open-source developers, allowing AI increased completion time by 19 percent. The consistent concern across the day is that better models do not remove the need for human judgment, reliable tests, useful environments, and clear software design.
Barry Zhang and Mahesh Murag argue that agents are increasingly general because code is a universal interface to the digital world. A coding agent can use APIs, the file system, Python, and shell commands to produce a financial report, for example. The missing piece is domain expertise. Anthropic's skills are organized folders containing procedural knowledge, scripts, assets, and instructions. They can be versioned in Git, shared through Google Drive, and progressively disclosed so that hundreds of skills do not fill the context window. MCP provides outside connectivity, while skills provide expertise. Anthropic has seen skills used by finance, legal, recruiting, accounting, and developer productivity teams.
Dex Horthy says coding agents work well on greenfield projects but struggle with old, complex codebases because their context fills with searches, file contents, tool output, and wrong turns. He recommends intentional compaction, where an agent compresses its understanding into a reviewed Markdown file before work continues. Subagents should control context rather than imitate organizational roles. His preferred workflow is research, plan, implement. Research identifies the relevant files and dependencies, planning records exact steps and code snippets, and implementation follows the reviewed plan. Horthy warns that a bad research assumption can send the whole task in the wrong direction.
Coding models improve when reinforcement learning matches the production environment
Lee Robinson describes Cursor Composer as a model trained for real software engineering with an emphasis on both speed and intelligence. Cursor's reinforcement learning setup reproduces the product environment, including file reading, editing, semantic search, linting, shell commands, and parallel tool calls. The infrastructure includes inference, rollout, training, and environment servers. Rollouts can involve hundreds of tool calls and very different completion times, so load balancing is needed to avoid idle compute. Cursor found that Composer learned to search and read more before editing, and that parallel tool calls made the agent feel more synchronous.
Useful coding evaluations must change as models and tasks change
Naman Jain presents coding evaluations across several time horizons, from autocomplete snippets to repository tasks and multi-hour optimization work. Static benchmarks suffer from contamination, brittle tests, and poor difficulty calibration. LiveCodeBench addresses this by updating problems over time and using newer problems to reduce contamination. For software optimization, Jain's group extracts real performance-related commits, builds workloads, and checks both correctness and speed. Models sometimes reward-hack these tests by changing the environment or exploiting test assumptions, so evaluation needs stronger infrastructure and model-based hack detection. He also argues for intermediate signals such as the fraction of code translated or refactored on long tasks.
RL environments are reusable units for evaluation, training, and product improvement
Will Brown describes an environment as a harness containing tasks and rewards. The same abstraction can support evaluation, synthetic data, supervised fine-tuning, reinforcement learning, and deployed agents. Prime Intellect's Verifiers toolkit lets builders compose environments for question answering, games, tool use, sandboxes, CLI coding agents, and math. In one example, a Qwen 3 4B model improved on a Wikipedia search task from 55 percent to 89 percent after training. Brown's broader aim is to make model research accessible through open-source environments, shared tooling, and platforms such as the Environments Hub and Lab.
Verification infrastructure is the limiting factor for agent autonomy
Eno Reyes argues that software is unusually suitable for automation because much of it can be checked. Linters, unit tests, end-to-end tests, documentation, API specifications, and other validators give agents feedback. Many organizations tolerate flaky builds or partial test coverage because humans fill the gaps, but those gaps become serious when agents work across the software lifecycle. Strong validation allows teams to parallelize agents and break large modernization projects into smaller tasks. Reyes says organizations should improve their validation criteria instead of spending weeks comparing tools. The better the environment becomes, the more useful the agents become, which creates a feedback loop.
AI makes easy code cheaper while leaving simplicity and understanding to humans
Jake Nations distinguishes simple from easy. Simple code has little entanglement and clear structure. Easy code is close at hand, such as copied, generated, or packaged code. AI makes the easy path nearly frictionless, so teams can accumulate accidental complexity faster than they can understand it. At Netflix, Nations describes an authorization migration involving millions of lines and tightly coupled logic. The successful approach began with manual work to expose hidden constraints, followed by research, a detailed implementation plan, and agent-assisted coding. He says passing tests is not enough if nobody can explain or safely change the system.
Benchmark capability and real developer productivity can diverge
Joel Becker contrasts METR's long-task capability measurements with a randomized study of experienced developers working on large open-source repositories. Benchmark-style tests suggest rapid progress on difficult autonomous tasks. In the field study, 16 experienced developers completed real issues with AI allowed or disallowed, and the AI-allowed condition took 19 percent more time. Becker offers several possible explanations: developers already understood their repositories, AI output needed checking, the tasks were messier than benchmarks, and benchmark scoring often ignores future maintainability. He treats the result cautiously because the study was small, concentrated in March 2025, and focused on unusually experienced developers.
