Multi-agent systems create security risks because agents, tasks, tools, credentials, and third-party libraries often share one process.
2
Agents need security-focused evaluations, enforcement controls, and observability before and after they reach production.
3
Enterprise agents must propagate user identity through every task and tool so authentication, authorization, approvals, and data policies remain in force.
Summary
Don Bosco Durai explains why autonomous agents need a different security approach from ordinary software. A typical agent system may place agents, tasks, tools, credentials, prompts, and libraries in one process. That creates opportunities for unauthorized access, data leakage, and misuse of shared credentials. Autonomy also makes behavior harder to predict because agents choose their own workflows and tool calls. Durai proposes three layers of protection. Security and safety evaluations should produce a risk score before an agent enters production. Enforcement should apply authentication, authorization, identity propagation, approvals, thresholds, and sandboxing. Observability should track failures, user inputs, confidential data movement, and unusual behavior after deployment. He describes an enterprise customer that treats agents like human users, with onboarding, training, and regional data rules. Durai also introduces PAIG.ai, an open-source project for security, safety, and governance in generative AI applications and agents.
Shared processes give agents access to more than they need
Durai says many agent frameworks run the agent, its tasks, and its tools inside one process. A tool that connects to a database needs credentials, while an API call may need a token. These are often service-user credentials with super-admin privileges. Because the components share a process, one tool may technically access credentials belonging to another. Prompts and other data in the process may also be available to third-party libraries. He describes this as a zero-trust problem: the system gives components too much access by default.
Autonomy creates security behavior that is hard to predict
An agent is autonomous because it can choose its own workflow for a task. It may decide which tools to call and what actions to take. Durai calls the resulting risks 'unknown unknowns' because developers cannot fully know what the agent will do in advance. The behavior is non-deterministic, so the attack surface can be larger than in traditional software. A poorly designed agent can create unauthorized access, leak sensitive or confidential data, or produce wrong results after someone changes a prompt or compromises the environment.
Enterprise agents need the same governance attention as human users
Durai describes a customer that is among the top three credit bureaus. The company has built many agents, but moving them into production is difficult because it treats an agent like a human user. Human users receive training and follow regulations, and agents must meet similar requirements. Rules can depend on the person's consent, location, and role. For example, California resident data cannot be used for marketing without consent, and European data has its own access requirements. The customer therefore needs an onboarding and training process for agents before production use.
Security evaluations should produce a production risk score
Durai says ordinary model evaluations measure response quality, but agents also need security and safety evaluations. These evaluations should result in a risk score that determines whether an agent can be promoted to production. The same criteria should apply to an agent built by a third party. Tests can cover use cases, prompt changes, new libraries, new frameworks, and new language models. Teams should also scan models and libraries for vulnerabilities, test for prompt injection, check data leakage, verify unauthorized actions, and look for agents stuck in runaway loops.
Authentication must follow the request through every agent component
A user request may move from an agent to a task, then to a tool, and finally to a database or service API. Durai says authentication at the entry point is not enough. The user's identity must be propagated across every step until the final data access or API call. At that point, the system must enforce the correct policies and access controls. Authorization also has to cover the agent's own role and any action it takes on behalf of another user. The agent must not access data or perform actions that the user is not allowed to access or perform.
Approvals can combine automation with human limits
Agents can perform many tasks automatically, so approval workflows need to be designed for this setting. Durai suggests using a separate agent to check whether results are correct and to manage approvals. Teams can define thresholds for what an agent may approve on its own. If an action exceeds a limit, the workflow can bring in a human. This approach keeps routine actions automatic while placing human review around higher-impact decisions.
Observability has to account for changing models and inputs
Durai says agent behavior changes as models, frameworks, and third-party libraries change. User input is another source of variation. A system tested with a straightforward request can behave differently when a user phrases the same request in another way. Teams should monitor responses, failure rates, personally identifiable information, and other confidential data sent out of the system. Since monitoring every request is impractical at scale, they should define thresholds and metrics. Alerts can be triggered when failures rise above an accepted level or when behavior suggests misuse.
Production safety is a cycle of evaluation, enforcement, and observation
Durai recaps his approach as three connected layers. Preemptive vulnerability evaluation produces a risk score before an agent enters production. Proactive enforcement adds guardrails, access controls, and a sandbox so the agent runs within defined limits. Observability provides real-time or near-real-time information about performance and anomalies after deployment. Teams can then investigate and tune the system when behavior changes. Durai says his company has open-sourced its safety and security solution and is looking for design partners and contributors.
"You can't monitor each and every request. As the number of requests increases, it's just not possible."15:29
Who should watch
You are taking an agent from a prototype toward production and need a concrete security review process.
Your agents call databases or APIs on behalf of users, and you need identity and permissions to survive across those calls.
You operate an enterprise system where data residency, consent, auditability, or human approval rules apply.
