How Secrets Leak Through AI Agents (and How to Stop It)

Joshua Mo, Venice AI15:22 · Oct 2026 · 7,070 views
Thumbnail for How Secrets Leak Through AI Agents (and How to Stop It) Watch on YouTube
TL;DR
  1. 1

    Privacy reduces the damage from a breach because storing less user data creates a smaller blast radius.

  2. 2

    Agent systems can leak secrets through logs, reasoning layers, and tool calls, even when the underlying model is private.

  3. 3

    Credentials should stay in the execution layer, with narrow agent scope, permission monitoring, and confidential compute added where appropriate.

Summary

Joshua Mo explains why a private model does not automatically produce a private AI product. Venice AI avoids storing prompts and conversations, anonymizes requests sent to closed providers, runs some models in trusted execution environments, and uses split-key custody for encrypted data. Mo then follows a secret through a typical agent: a runtime fetches a credential from a vault, sends it to the model, and lets the model use it in a tool call. That design can expose the credential through logs, training data, or an external request. His proposed alternatives include hashed identifiers, revocable handles, narrow tool access, prompt-injection screening, and keeping credentials in a deterministic execution layer. Sandboxes and virtual machines help, but internet access still requires permission monitoring. He also describes confidential compute with verifiable attestations and says confidential tool calls still need better APIs.

Key ideas
01:50

Privacy limits the damage when a system is breached

Mo argues that privacy matters because a single breach can make users leave, and an enterprise breach can put a major customer relationship at risk. Sensitive data increases the impact of a security incident. Storing less user data reduces the blast radius directly, because there is less information for an attacker to obtain. He frames privacy as a product and business concern as well as a model property. A system can use a capable model and still create serious exposure through the data it stores around that model.

02:15

Venice combines data minimization with protected inference

Venice does not store prompts or conversations, so requests made to its models remain stored locally on the user's side. For requests sent to providers such as OpenAI and Claude, Venice says it anonymizes the user's footprint so the provider cannot identify them from the request alone. Some models run in trusted execution environments, which allow the inference process to be verified as private. For generated media endpoints that require polling, Venice stores results in encrypted payloads using split-key custody. The read and write halves of the key are separated, so both are needed to decrypt the data.

04:00

The model is only one part of an agent system

Mo warns that model privacy does not make the surrounding product private. An agent needs a complete process around the model, including a runtime, tools, logging, and data stores. Those surrounding components can be more exposed to leaks and breaches than the model itself. He uses observability as an example. Metrics, traces, and logs are useful for production debugging, but logs can quietly capture prompts, secrets, and personal information. Teams therefore need to examine the full system that carries a request, rather than treating the model as the whole security boundary.

05:54

Sending vault secrets into the reasoning layer creates an exfiltration path

In Mo's example, a user request reaches an agent runtime, which retrieves a credential from a vault such as 1Password or HashiCorp Vault. The credential then goes into the LLM reasoning layer. The model makes an API request through a tool call, and the result returns to the user. Once the secret is in the reasoning layer, it may be included in training data or exposed through a tool call. Mo's warning applies to the architecture itself: a secret fetched safely from a vault can still become unsafe when it is handed to the model.

06:49

Hashes, split keys, and revocable handles reduce exposure

Mo recommends storing hashes for identifiers when an application only needs to recognize a user, such as for rate limiting. A hash can identify the user without storing the direct personal data. He also describes split-key custody, where encryption keys are divided into a read half and a write half. An attacker needs both halves to decrypt the data. Handles provide another boundary. A handle grants access to an object, such as a database pool, and access ends when the handle is removed or revoked. Mo connects this pattern to authorization systems that use JSON Web Tokens or session keys.

08:39

Narrow tool access reduces both security risk and context pressure

Mo recommends giving an agent only the tools it needs. If an MCP server exposes 15 or 20 tools and the task needs one, loading all of them increases the risk. An API key fetched for the useful tool could be exfiltrated through another tool call to a malicious web server. Narrow scope reduces that opportunity. Mo also says a smaller context can improve agent performance. Near the context limit, performance can degrade considerably, while a mostly empty context gives the agent more room to work.

09:12

Screening before the agent can catch prompt injection

Venice places a classification layer in front of an agent to check for prompt injection. Mo gives several possible implementations. The classifier can use another LLM, semantic search with vector search tools such as Qdrant or LanceDB, or an in-memory process. This layer sits before the agent receives the request. It is part of a broader design in which the agent has limited tools and a controlled context. Mo presents prompt-injection screening as an additional check, rather than as a replacement for limiting what the agent can access.

10:30

Credentials belong in the execution layer, not in the model

Mo's preferred architecture sends the user request to the reasoning layer, where the model produces non-deterministic inputs for an agent runtime. The password vault connects to that runtime instead of to the reasoning layer. The runtime then acts as a more deterministic execution layer that can handle operations the model should not see. For example, a browser tool can log in to a website using credentials without exposing the password directly to the model. Mo says this structure makes it more reasonable to describe the operation as private and secure, including when the runtime calls external APIs.

12:04

Sandboxes help, while confidential tool calls remain unfinished

Firecracker, virtual machines, and AWS EC2 in a VPC can improve agent security, but Mo says they do not solve every problem. An agent with internet access can still exfiltrate data through an external API, so permission monitoring remains necessary. He suggests owning the agent runtime and coding harness so teams control the execution layer, reasoning layer, tools, and observability. At the hardware level, confidential compute can run inference in a trusted execution environment and return an attestation that users can verify. Mo says confidential tool calls still need work because encrypted calls must also have an ergonomic API.

"By tying the password vault to the agentic runtime instead of the reasoning layer, what happens then is that the agent runtime is essentially turned into an execution layer."10:58
Who should watch
  • You are building an agent that can read credentials, call external APIs, or use a large tool server, and you need concrete ways to reduce what it can access.
  • Your production logs may contain prompts, secrets, or personal information, and you want a design that limits the data collected in the first place.
  • You are evaluating confidential compute or sandboxing and need to understand where those protections stop, especially when agents have internet access.