Software factories are becoming practical because models now have better vision, tools, memory, reasoning, security controls, and computer-use environments.
2
Long-running agent systems need durable context, delegation, triggers, sandboxes, evaluation, and human review at the points where decisions carry risk.
3
The speakers argue that engineering is moving toward designing and operating loops, while maintainability, product judgment, cost control, and safety still require deliberate human involvement.
Summary
This day of keynote programming examines what it takes to build software factories around coding and general-purpose agents. Speakers describe systems that collect work, plan it, delegate to agents, run code in sandboxes, validate results, learn from production feedback, and send focused decisions back to humans. Microsoft presents knowledge as intrinsic model knowledge, organizational grounding, and learned knowledge from repeated work. OpenAI focuses on persistent managers, agent teams, open harnesses, and agents that can work across local and cloud environments. Other talks cover replayable agent runs, browser-agent infrastructure, multimodal open models, model routing, autonomous code validation, generative interfaces, and mobile development in cloud sandboxes. The day also contains warnings. Large autonomous loops can damage codebases, hide failures, consume too many tokens, or create unsafe side effects. The practical advice is to build incrementally, measure outcomes, preserve optionality across models, and keep humans responsible for product direction and high-risk changes.
AI engineering is about choosing the right loop level
Swix describes software work as a stack of nested loops. Engineers move up a loop when they want more productivity and scale, then move down when they need to inspect reliability or fix a bug. He connects this to human work, which moves from individual activity to teams and larger collective projects. His point is that the conference itself is a loop for deciding what the next loop should be. The same idea applies to personal workflows and organizations: define the loop you are operating, notice where it is blocked, and change levels instead of treating every problem as a prompt-writing problem.
Software factories now have the ingredients to work in production
The opening software-factory introduction points to several changes that make autonomous development more practical. Models can see and verify work, agents have richer tools and access to production data, context windows and memory are larger, reasoning has improved, and security practices for autonomous agents are emerging. A Ralph Loop previously came with a greenfield-only recommendation and an expectation of reaching roughly 90% completion. The claim is that these improvements now make longer-running factories a practical engineering problem. The hard question becomes how to separate useful autonomy from hype and produce results that are actually deployable.
Useful agent knowledge has intrinsic, organizational, and learned parts
Pablo Castro divides knowledge into three categories. Intrinsic knowledge is what the model learned during training. Extrinsic knowledge comes from company data such as documents, email, chats, analytics, and web sources. Learned knowledge comes from observing work and improving the agent over time. Microsoft Foundry combines these ideas through model hosting, grounding, retrieval, and an agent optimizer. Castro shows a layered retrieval system that can hide complexity for ordinary users while exposing indexes, vector settings, and ranking controls to experts. The optimizer evaluates a baseline, generates candidate configurations, and applies the best one to the agent's externalized instructions, tools, and skills.
Agents should manage work across persistent threads and delegated workers
Peter Steinberger says his workflow changed from managing ten terminal windows to managing a long-running manager that delegates work to other agents. Three changes enabled this: server-side compaction, coordination across projects, and automation triggers. His manager can decide whether an issue fits a project's goals, create a worker, run tests, request review, and return a pull request with supporting evidence. Steinberger separates the agent's inner execution loop from the human's outer decision loop. He also argues that attention becomes the limiting resource once tokens and compute are available, so people should stop watching every generated line and spend attention on direction, review, and decisions.
A software factory is a full development lifecycle, not a collection of coding agents
Theresa defines a software factory as the whole autonomous lifecycle: collecting signals, reacting to feedback and logs, prioritizing work, orchestrating agents, executing changes, validating them, testing in production, and gaining new skills. She says a swarm of coding agents is insufficient because writing code is only one part of engineering. Her factory uses an orchestrator, sequential workers, and validators. Validators include code scrutiny and user testing in a virtual computer. She also describes deferred context, which progressively reveals tools instead of loading hundreds of schemas and descriptions at once. The factory should be model-agnostic, autonomous under governance, and always improving.
Production evaluation must measure system behavior and replay failures
The production-evaluation talks argue that model benchmarks do not capture tool failures, API outages, changing context, user variation, long-running workflows, or multi-agent coordination. Evaluation therefore needs to move from answers to workflows. Useful signals include execution traces, user outcomes, escalations, failures, and human feedback. A separate replayability talk makes the operational point: temperature zero does not guarantee reproducibility, and exact token matching is less useful than reconstructing the same state transition. Recording inputs and outputs at each agent boundary, along with model and code versions, allows teams to replay failures offline without making new model calls.
Incremental control loops are safer than blind autonomous code generation
Kyle applies control theory to agentic software development. A codebase has a desired state, sensors measure the current state, a controller chooses an incremental change, and an actuator applies it. His example migrates procedures to Effect one at a time. A deterministic scan identifies violations, a controller selects the next procedure, and an agent applies a skill containing handwritten golden patterns. Each run creates one pull request, and a feedback file lets humans redirect the loop through a comment trigger. Flow control prevents another run when the previous pull request is still open, avoiding duplicated work and review overload.
Agents need upstream design help, not only implementation help
Dominic Tornow argues that reusable value is moving from implementations to specifications. In a Resonate example, asking an agent to jump directly from an abstract durable-execution specification to Rust and Postgres produced a happy-path prototype that failed under concurrency, process failure, and network failure. The revised process inserts a deterministic simulation and a concrete target-specific specification. The agent first discovers algorithms in simulation, where stale reads and failed optimistic-concurrency writes can be reproduced and explained. It then derives the concrete specification and implementation. This lets the agent participate in system design while humans remain involved in the design process.
Model training rewards passing tests, so maintainability still needs human judgment
Jack argues that more loops and more harness engineering cannot fully solve a model-training problem. Coding models are often trained on tasks with binary rewards for passing tests, so they receive no direct penalty for poor architecture, duplicated logic, or declining maintainability. Those costs appear months later, which makes the reward signal difficult to propagate. His recommendation is to restore product review, architecture review, program design, and vertical implementation slices before asking an agent to code. Thirty minutes of alignment can reduce review time while preserving human ownership of the code. Agents can accelerate the work, but they do not remove the need to understand it.
"We don't need the model to return the exact same token back every time. We just need our system to execute the exact same state transition."1:50:11
Who should watch
You are building agent workflows that need to run for hours or days and want practical patterns for delegation, memory, sandboxes, triggers, and validation.
Your team is tempted to remove code review or send every task to the most expensive model, and you need arguments for incremental loops, model routing, and human checkpoints.
You work on production or regulated systems where replayable failures, security boundaries, maintainability, and measurable outcomes matter more than impressive demos.
