Don't Vibe Code Your Agent Infrastructure

Akshat Bubna, Modal17:35 · Nov 2026 · 8,934 views
Thumbnail for Don't Vibe Code Your Agent Infrastructure Watch on YouTube
TL;DR
  1. 1

    An agent needs a persistent workspace, elastic compute, suitable accelerators, shared files, network controls, and observability.

  2. 2

    A sandbox platform is difficult to build safely because reliability, security, and scale require ongoing systems engineering.

  3. 3

    Agent infrastructure is moving toward a separation between the harness that reasons and dynamic runtimes that provide the hands-on environment.

Summary

Akshat Bubna explains the infrastructure behind applications where many agents work at once, using a spreadsheet whose cells each have their own agent as the running example. Each agent needs an isolated workspace containing its code, dependencies, documents, or assets. That workspace must persist for days without keeping compute running, while CPU, memory, and accelerators expand and shrink with the workload. Agents also need shared files for collaboration, controls over network access and credentials, and observability that they can inspect themselves. Bubna argues against vibe coding a sandbox platform because reliability failures, security boundaries, and large-scale scheduling require sustained engineering. He describes workloads involving millions of sandboxes, large snapshot and volume systems, and interactive launches at high rates. He expects the harness, or agent brain, to create and manage dynamic runtimes, or hands, as needed.

Key ideas
00:12

A spreadsheet can turn every cell into an independent agent

Akshat Bubna uses a financial analysis spreadsheet to show why agent infrastructure needs to operate at high concurrency. A fund could ask for complex analysis across every company in its portfolio. Heavy AI has built a product where each spreadsheet cell is backed by a different agent. A query may require thousands of agents to start at once and then disappear when the work is finished. The architecture therefore needs fast startup, isolation, communication between agents, and a way to scale down after the query ends.

01:33

The agent loop should be separated from the environment that runs its tools

Each agent runs in a Modal sandbox, which Bubna describes as a secure environment for untrusted code. The sandbox can contain multiple containers or micro VMs. One container runs the harness, the loop that calls the inference API, invokes tools, and repeats. Other containers run the tool calls themselves. This separation provides security and isolation. The design can also include a sidecar and shared storage between the main container, sidecar, and other agents.

04:03

An agent workspace must contain the materials for its job

Bubna gives different examples of what a workspace holds. Ramp Inspect, Ramp's internal coding agent, needs the repository, dependencies, and development services together. Basis, an accounting agent, needs documents, accounting work papers, and scripts. A game development agent needs its game engine and assets. The common requirement is one workspace for each agent, with the files and services needed to complete the task available in the same environment.

05:03

Persistent state should pause compute while an agent waits

An accounting agent may receive documents, work on them, and then wait days for an accountant's response. When the response arrives, it should continue from where it stopped. Bubna says the state may need to persist indefinitely, but compute should run only while the agent is working because idle compute is expensive. Persistence can mean a snapshot of the whole root file system, a single directory such as node modules, or a shared file system such as a volume.

06:21

Agent workloads need resources that change during execution

Data from Ramp Inspect shows large variation in CPU and memory use across sandboxes and over time within one sandbox. Compilation and analysis can create short periods of high demand. Provisioning every sandbox for its peak would mean giving each one 15 CPU cores and 15 gigabytes of RAM, which Bubna says does not make economic sense. The infrastructure should provide resources when the sandbox needs them and charge for them only during that period. Agents may also need GPUs, TPUs, or AMD hardware for scientific models, game design, video editing, 3D development, and CAD.

09:02

Collaboration and self-improvement require fast shared file systems

Agents increasingly work with sub-agents and humans on the same files. Bubna says the shared file system must be fast, consistent, and understandable to developers. Agents also save and update memory and run background tasks, sometimes called dreaming. These activities can also be represented as files in a file system, so storage performance affects both collaboration and the agent's ability to maintain memory over time.

09:57

Security controls must reach beyond domain allow lists

Agents can make mistakes and leak sensitive data, so infrastructure must control which IPs and domains they can contact and how credentials are provided. Credentials should be injected in a way that prevents the agent from seeing them and sending them elsewhere. Bubna says users increasingly want rules for specific paths and operations on services such as Datadog or GitHub. Some teams use a sidecar proxy with custom rules to enforce those limits.

11:20

Agents need to inspect failures, metrics, and budgets themselves

Observability is part of the agent's working environment. If an automated training run fails, the agent should read the logs, reason about the failure, and try another set of parameters. During a run, it may inspect metrics, GPU utilization, and spending to detect a stalled or overly expensive sandbox. Bubna says these views matter to humans as well, but exposing them to agents lets the agents respond to problems while they are driving the compute.

14:54

The harness will create dynamic runtimes instead of managing fixed servers

Bubna separates the brain, meaning the harness, from the hands, meaning the sandbox or runtime. The harness can create an environment for its current task, snapshot it, keep the snapshot as an artifact, and adjust the environment later. He compares this with the older pets model, where each service was bespoke, and the cattle model, where services use homogeneous templates. He expects agent systems to move toward an ocean of microbes, with many dynamic environments being created, changed, and used together.

"You want to persist the state over time, potentially indefinitely. But you don't want to run the compute the entire time because compute is expensive."05:37
Who should watch
  • You are building coding, accounting, research, or creative agents that need their own files, dependencies, and services.
  • Your agents wait for human input, run at uneven resource levels, or need GPUs and shared storage during a task.
  • You are considering building your own sandbox platform and need to think through isolation, credential handling, scheduling, and large-scale operations.