# Stateful environments for vertical agents

Josh Purtell, Synth Labs | AI Engineer World's Fair 2025 | 06:51

Source: https://www.youtube.com/watch?v=5rMc-moNVx0
Channel: AI Engineer (https://www.youtube.com/@aiDotEngineer). Summarised by AIE Talks.
Page: https://aietalks.com/talks/stateful-environments-for-vertical-agents
Published: 2025-07-22
Tags: agents, harness-engineering, multi-agent, reinforcement-learning

## TL;DR
- Stateful environments keep the logic and state of a vertical application outside the agent code, so the agent sees a useful representation instead of an entire operating system or application.
- Network boundaries let multiple agents and models work on one product across time, while making asynchronous work easier to handle in production.
- Resettable environments support rollbacks and language-agent tree search, allowing an agent to compare branches and continue from the better state.

## Summary
Josh Purtell explains why vertical agents need stateful environments. An environment packages the logic behind a task and the external state that the agent changes. The agent receives a focused representation of that environment, rather than having to operate an entire application such as Excel. Keeping this logic outside the agent implementation makes it easier to replace or improve the agent when a new model arrives. Separate processes and network boundaries also allow multiple agents to work on the same product across time. The largest practical benefit is state control. Developers can reset or roll back an environment after an agent goes off course. Purtell connects this capability to language-agent tree search, where an agent explores multiple branches, compares their outcomes, and continues from the strongest branch. He presents the approach as an extension of earlier reinforcement-learning environment work, adapted for long-running tasks in areas such as finance, accounting, and health.

## Key ideas
### Stateful environments package the task outside the agent
[00:38](https://www.youtube.com/watch?v=5rMc-moNVx0&t=38s)
Purtell uses the term stateful environment for an environment that captures state for an agent. He connects the idea to reinforcement learning, where environments keep the logic of a task separate from the AI algorithm. RL Glue and OpenAI Gym established this pattern, while SWE-bench and its agent-computer-interface work applied related ideas to more vertical tasks. His proposal builds on that history rather than starting from scratch. The environment holds the task state and computes results outside the agent implementation.

### Long-running artifacts made stateful abstractions more useful
[02:08](https://www.youtube.com/watch?v=5rMc-moNVx0&t=128s)
Earlier tool use often involved simple operations such as calculating a sum or checking the weather. As models improved, agents began using API-based tools and working on products or artifacts over many steps. Purtell points to Claude 3.5 Sonnet and Claude Artifacts as examples of this change. Once an agent needs to iterate on an artifact over a long horizon, it needs an abstraction for the workspace, its state, and the changes made from one step to the next.

### The environment gives the agent a focused view of a larger system
[02:50](https://www.youtube.com/watch?v=5rMc-moNVx0&t=170s)
A stateful environment is an engine that computes results outside the agent code. It may access an API, work on an Excel document, or update an external source of truth. The full application can be too much for an agent to operate directly. The environment therefore exposes a representation that the agent can observe and manipulate. Purtell gives the example of showing an agent what it needs in a terminal instead of exposing the whole operating system.

### Network boundaries separate agents from the state they change
[03:56](https://www.youtube.com/watch?v=5rMc-moNVx0&t=236s)
Purtell says the environment can run outside the agent process behind a network boundary. This matters for reinforcement-learning training and can also help multi-agent systems. With the environment separated, developers can replace the agent, start new models, or have several agents work on one product across time. He also describes network boundaries as the production answer for reliable asynchronous work. The stateful application remains available while agents interact with it through a defined interface.

### Externalized logic makes agent replacement easier
[04:15](https://www.youtube.com/watch?v=5rMc-moNVx0&t=255s)
When the logic of a vertical application is containerized in code that does not change, the agent can be revamped independently. Purtell contrasts this with systems where the application logic and agent behavior are clumped together. In the latter case, changing the model or redesigning the agent also risks changing the application. A separate environment gives the agent a stable task interface while allowing the agent implementation to change.

### Reset and rollback prevent long-horizon work from staying derailed
[04:54](https://www.youtube.com/watch?v=5rMc-moNVx0&t=294s)
A stateful environment can reset the state of the product an agent is working on or roll it back after a bad step. Purtell says this is especially useful in coding settings, where an agent can get derailed after making several changes. He connects the approach to the Language Agent Tree Search paper, which produced strong results but was difficult to put into production because suitable abstractions were missing. Resettable state makes these long-horizon techniques easier to implement.

### Tree search becomes practical when branches can be restored
[05:46](https://www.youtube.com/watch?v=5rMc-moNVx0&t=346s)
Purtell describes an agent playing Minecraft and branching into two possible action paths. One branch performs better, so the system can select it and continue from that state. He says a resettable environment gives language-agent tree search almost for free. The same pattern applies beyond Minecraft to tasks where an agent takes hundreds or thousands of steps. Developers can explore alternatives without permanently committing the environment to every attempted path.

### The open-source repository applies one interface across environments
[06:24](https://www.youtube.com/watch?v=5rMc-moNVx0&t=384s)
Purtell points viewers to Synth Labs' open-source Environments repository. It contains the abstractions discussed in the talk and implementations across academic benchmarks. The description names environments such as Pokémon Red, Minecraft, and SWE-bench. The goal is to give different environments a common interface for development and agent training, so the same style of agent interaction can be used across very different tasks.

## Notable quotes
- "A stateful environment is an engine that computes results external to the agent implementation." (02:50)
- "If you have a resetable environment, you sort of get language agent tree search for free." (05:46)
- "The answer to that question of how to do asynchronous work in a reliable way in production is network boundaries." (04:54)
- "This is not reinventing the wheel. We're just building on top of what people have already thought about." (01:41)

## Tools & references mentioned
- Synth Labs
- RL Glue
- OpenAI Gym
- SWE-bench
- Claude 3.5 Sonnet
- Claude Artifacts
- Language Agent Tree Search
- Minecraft
- Pokémon Red
- Excel
- MCP
- Synth Labs Environments repository

## Who should watch
- You are building a vertical agent that needs to work on a persistent product, document, or external system over many steps.
- Your agent and application logic are tightly coupled, and you want to change models without rewriting the task implementation.
- You are exploring multi-agent workflows, asynchronous work, or tree search and need a way to reset and share application state.

## Related talks

- [Stateful Agents](https://aietalks.com/talks/stateful-agents) (Charles Packer, Letta, 1:19:34)
- [Your agent architecture has a half-life of 6 months](https://aietalks.com/talks/your-agent-architecture-has-a-half-life-of-6-months) (Dan Farrelly, Inngest, 19:20)
- [The Evolution of Agentic Surfaces](https://aietalks.com/talks/the-evolution-of-agentic-surfaces) (Gagan Bhat & Isabella Kai He, Anthropic, 31:24)
- [Everything Is a Rollout](https://aietalks.com/talks/everything-is-a-rollout) (Alex Shaw & Ryan Marten, Laude Institute, 21:11)
- [The Multi-Agent Architecture That Actually Ships](https://aietalks.com/talks/the-multi-agent-architecture-that-actually-ships) (Luke Alvoeiro, Factory, 18:31)