Your AI Agents Shouldn't Care Where They Run

Nikhil Harithas, Factory17:13 · Nov 2026 · 1,574 views
Thumbnail for Your AI Agents Shouldn't Care Where They Run Watch on YouTube
TL;DR
  1. 1

    Agents need their own identity and controlled compute instead of borrowing a user's permissions and environment.

  2. 2

    Factory's Open Compute layer separates the agent product from compute management across EC2, Kubernetes, micro VMs, and sandbox providers.

  3. 3

    Snapshots, forking, drivers, and outbound relay connections let customers run agent workloads inside their VPC while managing compute life cycles.

Summary

Nikhil Harithas explains how Factory rebuilt Droid for customers whose security and deployment requirements made the usual agent setup unusable. He uses Blackstone as the example: the company wanted its own compute, inference, identity, and single-tenant deployment. Factory's answer is Open Compute, a layer that manages compute separately from the agent product. A compute gateway asks a driver to create the chosen environment, while a relay lets the environment maintain an outbound WebSocket connection without requiring inbound access. Customers can keep model calls inside their VPC through bring-your-own-key setups. The system also handles pausing, resuming, snapshots, forking, retries, and replacement of compute when providers impose limits. Harithas says the layer is planned for open source because its abstractions apply beyond Factory and Droid.

Key ideas
00:12

Enterprise customers need control over both identity and compute

Harithas uses Blackstone to describe an environment where Factory's defaults did not work. The company could not run agents on employee laptops and wanted to bring its own compute, inference, identity, and single-tenant deployment. Droid can receive work through Slack, Jira, or another client and return a pull request, so the deployment must answer who the agent acts for and where it runs.

02:44

Agents should have application identities instead of borrowing users' permissions

Harithas argues that agents are slightly non-deterministic applications, so they should use the systems built for software rather than permanently borrowing a person's identity. The question kept returning across deployments because customers wanted to run agents on Kubernetes, EC2, Google Cloud, or other infrastructure. A setup that only installs an agent on a machine still leaves the customer responsible for the compute life cycle.

04:26

Open Compute separates agent behavior from compute management

Factory built Open Compute to abstract the compute life cycle for Droid and, eventually, any agent. The unit of work is a computer rather than a Factory session. This lets customers choose their compute environment while keeping the agent product's control layer separate. Harithas presents it as a way to deploy inside a customer's VPC and fit the customer's security constraints without moving the entire control plane on premises.

05:42

A gateway, driver, and relay connect control software to customer-owned compute

A client asks the backend for a new agent session. The compute gateway passes that request to Open Compute, whose driver knows how to create a particular type of environment, such as an EC2 machine, Kubernetes pod, or sandbox from Daytona or Modal. The new machine starts the agent daemon and opens a WebSocket connection to Factory's relay. The customer environment makes the outward connection, so the system does not need inbound access.

09:34

Bring-your-own model keys can keep inference inside the VPC

In a bring-your-own-key setup, a mature engineering organization can use an existing relationship with a model provider and make calls through something such as Bedrock. The model calls stay inside the customer's VPC while Factory keeps the control layer outside it. Harithas says this gives customers observability and control over the non-deterministic part of the agent loop, while containing the consequences of actions they consider too risky.

11:21

Open Compute is meant to be useful beyond Factory

Harithas says Factory plans to open source the compute layer because its abstractions concern managing compute rather than Factory-specific behavior. The layer supports starting, pausing, resuming, and destroying environments from customer-controlled templates. Customers can define installed software, image or container details, and snapshots. Harithas also describes storing snapshots in S3 or another chosen system and hydrating them with secrets from a customer-selected secret store.

12:24

Snapshots and forks support delegated work across agent fleets

Forking lets one agent save its state and hand a subtask to another agent. Harithas connects this to systems with sub-agents and larger agent fleets. Snapshots can preserve the exact build requirements and later create another environment. Drivers provide the adapter for each compute type, so adding support for a new environment means implementing the Open Compute specification rather than rewriting the agent product.

14:22

Compute management requires bookkeeping and replacement of expired environments

The compute layer tracks whether a requested sandbox was created, what it was called, and how to route requests to it. It must retry when a provider acknowledges a request but never creates the environment, or when a request is dropped mid-session. Harithas also mentions an eight-hour session cap for AWS micro VMs. For multi-day missions, the system must snapshot a sandbox, create another one, and swap it in without disrupting the work.

"Agents can't just keep borrowing your identity and they're ultimately they're just applications."03:03
Who should watch
  • You are deploying coding agents inside a bank or another company with strict identity, network, and tenancy requirements.
  • Your agent can run on several compute providers, but your team does not want every product integration to manage sandbox life cycles independently.
  • You are designing multi-agent systems that need snapshots, forks, long-running sessions, or customer-owned model and secret infrastructure.