We Gave an Agent Production Code Access and Then Tried to Sleep at Night

Moritz Johner, Form321:57 · Jul 2026 · 772 views
Thumbnail for We Gave an Agent Production Code Access and Then Tried to Sleep at Night Watch on YouTube
TL;DR
  1. 1

    Dependency patching across containerized systems requires reasoning about linked changes, not only version bumps.

  2. 2

    PatchPilot keeps GitHub write access and CI control in a deterministic Go layer, away from the coding agent.

  3. 3

    A Docker socket can let a prompt-injected agent escape a normal sandbox, so PatchPilot uses a Firecracker microVM and separate network policies.

Summary

Moritz Johner describes PatchPilot, a production system Form3 built to remediate CVEs across thousands of repositories. Standard dependency tools can update manifests, but they miss vulnerabilities in base images and build-time downloads. They also fail when one version change requires related updates and code fixes. PatchPilot combines a deterministic Go controller with agents that reason about repository changes and CI failures. The agents edit files on disk, while the controller handles commits, pushes, pull requests, and CI triggers. This split limits what a prompt-injected agent can do. Johner is direct about the remaining danger: giving an agent a Docker socket can allow it to escape a conventional container sandbox. Form3 moved the system into a Firecracker microVM, then added separate network controls for the controller and agent. Johner says the surrounding tooling is still early, with enterprise deployment requiring more orchestration and access control.

Key ideas
00:01

Container CVE patching requires reasoning beyond manifest updates

Johner starts with the scale problem: Form3 has thousands of repositories, so the backlog of dependency issues never empties. Dependabot and Renovate can bump a version in a manifest, but vulnerabilities may live in an operating system package inside a base image or in a binary downloaded during a Docker build. Patches also interact. A Go runtime update may require a linter update, and the newer linter can invalidate existing code. The system therefore needs to locate where the CVE actually lives, determine which related components must change, and get the repository back to green CI.

02:52

PatchPilot separates orchestration from agent reasoning

PatchPilot has two layers. A deterministic Go application discovers vulnerable OCI images, scans them, and maps each image to the repository that builds it. Agents handle reasoning, such as deciding whether a CI failure comes from the change, a flaky test, or an infrastructure timeout. This division gives the controller predictable responsibilities while allowing the agent to deal with repository-specific problems. Johner describes the deterministic layer as deliberately simple and boring, because the system relies on it for actions that affect external services.

05:13

The agent edits files, while the controller owns repository actions

The remediation agent receives an assessment manifest, repository context, and instructions to make the smallest effective change set. It should fix the reported CVEs rather than upgrade dependencies to the newest versions. It can build and rescan its work, but it only modifies files on disk. It does not commit, push, open a pull request, or watch CI. The controller checks for accidental empty files and binaries, then commits the change, pushes it, opens the PR, and monitors CI. A second agent can address CI failures, with a retry limit before a human takes over.

08:37

Retrospectives expose infrastructure and repository-specific failures

After each agent invocation, PatchPilot asks for a short retrospective covering what went well, what went wrong, missing tools, and useful context for the next run. Johner says aggregated retrospectives reveal two recurring classes of problems. Infrastructure failures include network issues and insufficient permission to clone a repository. Complexity failures arise when a repository needs context the agent does not have. Form3 responds by changing the system prompt or adding repository-specific instructions. Johner presents this as a way to understand behavior across many pull requests rather than debugging each one in isolation.

11:12

Credentials belong to the deterministic layer when the agent does not need them

PatchPilot receives GitHub read and write access, registry credentials, runtimes, linters, a shell, and network access at the application level. Form3 does not give all of those capabilities to the agent. GitHub write access and the ability to trigger CI remain in the deterministic layer, which performs commits, pushes, pull requests, and CI operations. Johner's security model is based on this boundary. If an agent is prompt-injected by content in a repository, its access does not automatically include the credentials needed to change external systems.

12:32

Prompt injection cannot be removed, so the system limits its blast radius

Johner says prompt injection is not solved. The team uses prompt steering for known untrusted locations, such as vendor directories and stored CI logs, telling the agent not to trust instructions found there. They also maintain a crafted test repository with a deprecated function and a migration guide that attempt to recruit the agent into malicious behavior. PatchPilot runs against this repository as an end-to-end test. These checks cover injection patterns the team knows about, but Johner acknowledges that unknown vectors remain.

14:32

A Docker socket defeats an ordinary agent sandbox

The hardest problem appeared when the agent needed to build or run Docker containers to verify its changes. Giving it the host Docker socket meant a prompt-injected agent could spawn a privileged container, escape it, read other processes' environment variables or memory, and plant SSH keys. Form3 ran this design in production before moving away from it. Johner says technologies such as Landlock, bubblewrap, seccomp, and related unprivileged build tools did not solve the combination of containers and a host Docker daemon. A box drawn around the agent is not enough when the socket crosses that boundary.

15:51

A Firecracker microVM gives the Docker daemon its own kernel

Form3's replacement puts the agent and Docker socket inside a Firecracker microVM. The Docker socket is backed by a Docker daemon using the microVM's own kernel, so an escape attempt remains inside that VM rather than reaching the host system. The two-layer application also gets two network policies. The deterministic layer can receive the GitHub access it needs, while the agent's network needs vary by repository and language. A DNS and TCP forwarder inside the VM sends traffic through a host process over Vsock, where Form3 can filter by host name, port, or address range.

18:03

Agent containment tooling exists, but production orchestration is still early

Johner says the microVM design is ordinary virtual machine isolation with additional plumbing, and he points to sandbox-as-a-service products and the open source microsandbox project. He says microsandbox would be his choice if he rebuilt PatchPilot because it includes network controls and related components. At the same time, he describes most available tools as beta-level for enterprise use. Kubernetes' agent sandbox special interest group and OpenSandbox are also working on orchestration. His conclusion is that the agent's blast radius is set by architecture, especially by deciding which operations are deterministic and which are agentic.

"A useful coding agent is a supply chain actor, whether you plan for that or not."02:52
Who should watch
  • You are considering giving a coding agent repository write access, CI control, or production credentials and need a concrete way to split those permissions.
  • Your agent must build Docker images or run containers, so a normal process or container sandbox may leave the host exposed through the Docker socket.
  • You operate a large vulnerability backlog and need agents to reason about container contents, linked dependency updates, and CI failures without handing them control of GitHub.