An agent combines a foundation model with the ability to reason, act, communicate, and adapt, but more agency is useful only when it preserves system effectiveness.
2
Simple chains and branching workflows are easier to measure and maintain, while multi-agent designs can manage growing tool collections by routing tasks to specialized agents.
3
Evaluation, observability, and safety need to be designed into the system before deployment, with test sets, tracing, red teaming, and human fallback for critical cases.
Summary
Michael Albada presents agents as systems built around foundation models that can reason, act, communicate, and adapt. He argues that agency is a spectrum and should be judged alongside effectiveness. Tool use is the first major building block, but exposing too many tools creates semantic overlap and reduces accuracy. Albada recommends starting with chains and branching workflows, then using more autonomous patterns when fixed workflows become too complex. Multi-agent systems can divide tools into groups and route work through a coordinator. The talk spends substantial time on evaluation. Teams should define expected inputs and outputs, build evaluation sets, review failures, and run automated improvement loops. Once deployed, agents also need detailed logs, traces, clustering, and summaries to reveal failures that are otherwise hard to see. Albada is direct about safety risks. Agentic systems introduce a new class of vulnerabilities, so teams should add red-team testing, detectors, trip wires, and human review paths.
Agentic systems should increase flexibility without reducing effectiveness
Albada defines an agent as an entity that can reason, act, communicate, and adapt to solve tasks, with a foundation model at its center. He describes agency as a continuum rather than a binary property and adds effectiveness as a second axis. Robotic process automation has low agency but can deliver high value, although it is fixed and brittle when inputs change. Agents can adapt to changing inputs, but adding autonomy does not automatically improve a system. Teams should check that each increase in agency preserves performance. Albada warns against both ineffective chatbots and ambitious systems that are difficult to control or do useful work.
Tool design should reduce choice and present clear human-facing actions
Foundation models can output function calls, allowing an agent to invoke APIs and receive observations in a repeated tool-use loop. Albada warns against mapping every API directly to a tool. An organization with 300 APIs should not register 300 tools with one agent, because more exposed tools create semantic collisions and reduce accuracy. Tools should be grouped logically, with a narrow scope, clear names, and precise descriptions. Each tool should feel like one action a person would take. The design also requires judgment about which capabilities to expose and under what conditions, since tool access expands both what the agent can do and the risks it can create.
Simple workflows are preferable until their complexity becomes harder to maintain
Albada recommends using a single chain whenever the task fits one. Chains are easier to measure, cheaper to run, more reliable, and easier to deliver. Branching trees can handle decisions such as assigning incident severity or choosing enrichment steps in cybersecurity workflows. A fully agentic pattern gives the model more control over which actions to invoke and repeat, but it is harder to measure and optimize. When chains and trees become convoluted and difficult to maintain, that complexity is a reason to consider a more agentic design. For fixed business rules, Albada recommends keeping the logic outside the model, storing state externally, and validating conditions before actions are allowed.
Multi-agent systems can control tool growth through specialization and routing
The strongest reason Albada gives for splitting one agent into several is tool overload. Instead of placing a large collection of tools in one prompt, teams can group semantically related tools under separate agents. A coordinator then routes each task to the appropriate specialist. This lets the system cover more scenarios as the tool collection grows. Albada also discusses agent-to-agent protocols as a possible way for agents built by different teams to discover and coordinate with one another. He describes that direction as early, with technical and security questions still open. The multi-agent systems he has seen work best so far were coordinated by one team.
Evaluation should define the agent through expected inputs and outputs
Albada's main recommendation is to invest more in evaluation because it is easy to reach an early level of accuracy while leaving many design choices unresolved. The number of agents, exposed tools, model choice, and memory design cannot be chosen well without a rigorous evaluation set. He compares this to test-driven development. Teams should define expected inputs and outputs, run real cases through the agent, include human review, and add new examples to the set. They can then analyze failures, cluster them, summarize patterns, and suggest improvements. Albada says AI engineers need to take ownership of this definition rather than treating labeling as work to outsource.
Evaluation tools can automate test generation, red teaming, and improvement
Albada names several tools for building an evaluation process. Intel Agent can generate synthetic inputs when raw user data is unavailable because of privacy or security concerns, or when the product has not launched. Microsoft's PyRIT can red-team agents by attempting jailbreaks and other compromises. Label Studio can help build evaluation sets. He also mentions automatic prompt optimization and tools including Trace, TextGrad, and DSPy, which can use a foundation model as a judge to inspect failures and suggest changes to a workflow. Running cases in batches and reviewing aggregate patterns is preferable to making isolated changes based on anecdotes.
Observability is needed to understand failures after deployment
Generative systems produce varied outputs, which makes it difficult to understand all their uses and failures once customers begin using them. Albada uses an iceberg metaphor for the gap between what teams can see and the full range of behavior underneath. He recommends detailed logs and tracing, including OpenTelemetry integrations. Teams should also cluster events and automatically summarize them so recurring failure modes become visible. This information supports system improvement after deployment. Without it, developers may see individual bad outputs without knowing whether they are isolated incidents, symptoms of a tool-selection problem, or part of a larger category of failure.
Safety requires controls throughout the agent stack and human fallback
Coming from Microsoft's cybersecurity division, Albada describes agentic systems as a new class of potential vulnerability. He recommends designing for safety at every layer and combining red teaming with ordinary software engineering principles. Trip wires and detectors should appear at different stages of the agent stack. Critical cases need a way to eject from the automated flow and fall back to human review. The warning applies especially when agents can invoke tools or repeat actions without direct supervision. Safety is therefore part of the architecture and deployment process, rather than a final check after the system has been built.
"Agency or the agenticness of your system is not a goal or an end in and of itself. It is a tool to help you solve problems."03:16
Who should watch
You are building an agent prototype and need to decide how much autonomy, tool access, or workflow complexity to add.
Your team is choosing between a single agent, a workflow, and a multi-agent architecture as the number of tools grows.
You have deployed an agent but lack evaluation data, traces, failure analysis, or safe fallback paths.
