A single agentic loop should be reliable before a team splits work across multiple agents.
2
Navan treats skills as pluggable units of context and uses one master agent that loads them progressively.
3
Agent systems need hooks, trajectory scoring, and guardrails around every tool call because logs, output assertions, and old authorization models are insufficient.
Summary
Roberto Milev and Uday Kanagala compare the current agent wave with the microservices wave. Their advice is to learn the basic unit before adding orchestration. At Navan, that means running a single master agent that progressively loads reusable skills. A skill contains domain instructions, setup, and tool execution, so it can be tested and reused independently. The talk also covers the operational problems that appear when agents produce too much reasoning for ordinary logs. Navan uses hooks around decisions and tool calls to capture goals, reasoning, confidence, and traces. Since agents are nondeterministic, evaluations score trajectories and inferred answers rather than checking one fixed output. Authorization is harder because an agent can act for a user or with a service account, so guardrails run before and after each tool call. Runtime, memory, MCP, and some orchestration patterns are maturing. Cost prediction, replay, debugging, and observability remain unresolved.
Agent systems should earn the right to become multi-agent
Milev compares agents with the microservices shift. Microservices brought useful practices such as container orchestration, Kubernetes, service meshes, and circuit breakers, but teams had to learn how to operate them. He applies the old advice, "If you can't build a well-structured monolith, why even try to build microservices?" to agents: if a team cannot make one agentic loop work, it should not start with a multi-agent orchestrated system. A reference architecture is beginning to emerge from production experience, with layers for runtime, memory, context management, operational concerns, and orchestration.
Agent runtimes need persistent state and isolated sessions
Agents differ from the stateless services that teams traditionally scaled. Milev says agents are stateful by nature, so they need persistent sessions, isolation, and a lifecycle that differs from a normal API service. AWS, Google Cloud, and Azure have each introduced an agentic runtime. Navan runs on AWS and uses its AgentCore runtime, while adding capabilities such as session persistence and rehydration. These runtimes are generally framework-agnostic, although each cloud provider favors its own framework. The runtime layer is therefore becoming a standard part of the architecture, with teams still filling practical gaps in production.
Memory moves from retrieval to a set of time-based memories
Navan began with retrieval-augmented generation because an agent cannot hold unlimited context. Memory systems now commonly use ingestion, extraction, consolidation, and retrieval. Milev describes several forms of memory: short-term conversational memory, long-term memory managed by the system, and episodic memory about particular instances that worked well or poorly. Long-term memory can carry semantic characteristics, while episodic memory records experience over time. Navan uses AWS AgentCore Memory and adapts it to its own use case. The speaker presents memory as a managed pipeline that helps agents retain useful information without placing every past interaction in the active context.
Skills give the agent a manageable unit of context
Milev says Navan focuses context management around skills. Each skill contains context, such as instructions and setup for a domain or task, along with the tool execution and agentic behavior for that work. Skills are pluggable units that Navan can compose dynamically, test independently, and reuse. Progressive disclosure lets the agent begin with a limited scope and expand it through additional metadata when needed. This gives a master agent a way to load only the material relevant to the current task instead of placing a large collection of instructions and tools in every context window.
Hooks produce more useful operational signals than ordinary logs
Uday Kanagala says conventional logs stop being useful when an agent produces a large amount of thinking. With an agent such as Claude, hooks can intercept actions at several points, including before and after a tool call and before and after a decision. Those hooks can block an operation, emit a metric, or create a trace. Navan uses these interception points to capture spans and identify where an agent becomes stuck. The traces include the agent's current goal, the reasons behind its actions, its belief status, and the tools it calls. This gives operators signals they can use during day-to-day production work.
Confidence scores can route inferred answers to people
The traces also include a confidence score for each decision. Kanagala describes this as a way to distinguish a choice supported by multiple paths from an inferred answer. When the answer is inferred, a human can review it and guide the agent. The score is therefore an operational signal, rather than just a model explanation. It can help determine when an agent should continue on its own and when a person should inspect or adjust the result. The talk connects this with the broader need to capture critical signals at decision and tool-call boundaries, where a system can still block or redirect an action.
Agent tests should score trajectories rather than assert one output
Agents are nondeterministic, so a fixed expected output does not describe every valid run. Kanagala contrasts this with ordinary programs, where engineers can often explain the exact algorithm and sequence of operations. An agent may take different steps each time while still reaching the goal. Navan therefore evaluates the trajectory from the starting point to the goal. The team measures how far the run progressed and how far it moved from source to destination, using those signals to assess efficiency and completeness. They also track inferred answers and classify regressions so changes can be corrected without assuming a single deterministic path.
Authorization must account for who an agent is acting for
The flight example exposes a problem with traditional identity models. A user can tell an agent to book a flight whenever the price falls below 200 dollars, and the agent may make the purchase later without the user being present. Kanagala asks whether the user made the purchase or whether the agent made it on the user's behalf. An agent can act on behalf of a user, or it can use a service account. That blurs the old distinction between a user and a service principal. Navan puts policy and guardrails around this problem, checking before and after every tool call so the system can block an action or make a fine-grained authorization decision.
Navan uses one master agent with progressively loaded sub-skills
Navan chose a single master agent with sub-skills rather than starting with a collection of cooperating agents. The master agent decides which skills need to enter its context and navigates the use case through those skills. Kanagala still describes other patterns, including agent-to-agent communication across organizational boundaries. The A2A protocol can define contracts in terms of skills between agents owned by different teams. The advice remains to avoid over-engineering: teams should first learn from failures in a single agent before adding more orchestration. The architecture leaves room for multiple agents later, but does not make them the default starting point.
"Can we have a trajectory of its starting from an end to a goal and then see how much how far it went in the trajectory and how far it went from the source to the destination is what we can compute to evaluate the efficiency or the completeness of the agent agent evaluation."10:56
Who should watch
You are building a production agent and ordinary logs no longer let you explain why a long run failed.
Your team is deciding whether to split one agent into several and needs a practical architecture starting point.
You need to test nondeterministic workflows or authorize actions that an agent takes on behalf of a user.
