# Effective agent design patterns in production

Laurie Voss, LlamaIndex | AI Engineer World's Fair 2025 | 15:38

Source: https://www.youtube.com/watch?v=72XxWkd8Jrk
Channel: AI Engineer (https://www.youtube.com/@aiDotEngineer). Summarised by AIE Talks.
Page: https://aietalks.com/talks/effective-agent-design-patterns-in-production
Published: 2025-06-27
Tags: agents, planning, rag, workflows

## TL;DR
- Agents are useful when an LLM must turn messy, unstructured input into a smaller, structured result or an action.
- RAG reduces cost and latency by retrieving relevant context, while agents can improve retrieval through decomposition, retries, and self-checking.
- Chaining, routing, parallelization, orchestrator-workers, and evaluator-optimizers can be combined into complex production workflows.

## Summary
Laurie Voss presents LlamaIndex briefly, then explains when agents are worth using and how to build them. He defines an agent as semi-autonomous software that uses tools to achieve a goal without a developer specifying every step. He recommends applications where an LLM turns a large body of messy text into a smaller result, such as a decision, report, calendar event, or answer. Voss describes RAG as a way to retrieve relevant context instead of sending an entire data corpus to the model. Agents can improve RAG by breaking down difficult questions, retrying bad extraction, and checking their own answers. The talk then covers five patterns from Anthropic: chains, routing, parallelization, orchestrator-workers, and evaluator-optimizers. LlamaIndex Workflows implements these with Python functions, type annotations, events, concurrency, and loops. Voss closes with brief examples of tools and multi-agent systems.

## Key ideas
### Agents are most useful when they turn messy text into a smaller result
[03:12](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=192s)
Voss defines an agent as semi-autonomous software that can use tools to achieve a goal without being told every step. The flexibility matters when inputs are unexpected or unstructured. He recommends cases where an LLM turns a large body of text into a smaller body of text, such as interpreting a contract, processing an invoice, applying regulations, summarizing documents, or producing a decision, report, calendar event, or answer. He also argues that teams should look beyond chat interfaces. An LLM can handle messy input, turn it into structured data, and pass that data into ordinary software that makes decisions or takes action.

### RAG supplies the specific context that general-purpose models lack
[05:29](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=329s)
Voss says that most useful applications need data from the company or domain, rather than a bare question sent to an LLM. RAG addresses this by embedding the data as vectors, embedding the query in the same space, and retrieving nearby context that is likely to be relevant. The application sends that smaller context to the model instead of sending the entire corpus on every request. This is faster and cheaper, and Voss says more specific context also produces better answers. He expects RAG to remain useful even as context windows and models grow because sending less data still reduces what the model must process.

### Agents can make RAG handle difficult questions and bad intermediate results
[07:09](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=429s)
Voss describes the relationship between agents and RAG as mutual. An agent can use RAG as one of its tools, while an agent layered over naive top-k retrieval can improve the RAG system itself. The agent can ask whether a complicated question should become several simpler questions. It can decide to extract data again when the first result is nonsense. It can inspect its answer and decide whether it made sense or should try again. In his account, these capabilities improve RAG in speed and accuracy, because retrieval and answer generation become steps that the system can inspect and revise.

### A chain passes one model's output into the next step
[08:48](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=528s)
The chain is the simplest pattern Voss covers. One LLM does some work, its output goes to another LLM, and that output can go to another step. In LlamaIndex, Workflows express this with ordinary Python functions and type annotations that define how events move between steps. Voss calls the pattern simple and flexible, and says LlamaIndex Workflows includes a visualizer for the resulting flow. A chain can be useful when a task has a fixed sequence of transformations, although Voss places it alongside more flexible patterns rather than treating it as the complete model for LLM applications.

### Routing lets the model choose among specialised paths
[09:30](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=570s)
Routing creates several LLM-based tools or paths, each designed for a different kind of problem or a different way of solving the same problem. The LLM decides which tool to call or which path to follow. Voss describes this as a straightforward pattern to build in LlamaIndex, where branches can split from the original decision into another chain of work. The routing decision gives the application a way to send different inputs to different prompts, tools, or workflows without forcing every request through the same sequence.

### Parallelization uses independent model calls for checks or agreement
[10:09](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=609s)
Voss describes two forms of parallelization. Sectioning sends the same input through different tracks that perform different jobs. His example runs the main processing path alongside a guardrail that checks whether the request is illegal or against the application's rules. The guardrail can stop the answer if needed. Voting sends the same query to several tracks, which can use the same nondeterministic model or different models with different capabilities. The system compares their answers using a majority or unanimous vote. Voss says this can reduce hallucination because models tend to make different mistakes. LlamaIndex Workflows handles the pattern with concurrency and event collection.

### Orchestrator-workers split a complex question into parallel subquestions
[12:18](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=738s)
In the orchestrator-workers pattern, an LLM examines a complex task, such as a multi-part question, and divides it into simpler questions. The worker steps answer those questions in parallel. A later step aggregates the results into one coherent answer. Voss uses deep research as the basic example: the system identifies the possible questions hidden inside a larger question, answers them at the same time, and combines the answers. He says this pattern is also implemented through parallelization in LlamaIndex Workflows. It gives the application a way to handle a broad research task without requiring one model call to solve the entire problem at once.

### Evaluator-optimizers create a feedback loop around model output
[13:18](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=798s)
The evaluator-optimizer pattern, which Voss also calls self-reflection, uses an LLM to judge whether another LLM reached its goal. The evaluator receives the original input or question, the intended goal, and the generated output. If the result is weak, it can produce feedback such as identifying a hallucinated detail or a missed part of the question. That feedback goes back to the first step so the system can try again. In LlamaIndex Workflows, Voss says this is implemented as a loop that returns to the earlier step. He also stresses that these patterns can be combined into arbitrarily complex workflows.

## Notable quotes
- "An agent is a bit of semi-autonomous software that can use tools to achieve a goal without you having to explicitly specify what steps it's going to take to achieve that goal." (03:12)
- "In general, I regard a good agent use case as any situation where an LLM is required to turn a large body of text into a smaller body of text." (04:12)
- "It's always going to be cheaper and faster to send less data that the LLM has to think about less." (06:46)
- "LLMs hallucinate, but they hallucinate in different ways." (11:39)

## Tools & references mentioned
- LlamaIndex
- LlamaParse
- LlamaCloud
- LlamaHub
- RAG
- Llama 3
- Anthropic
- AI Engineer World's Fair

## Who should watch
- You are deciding whether an agent is appropriate for a production feature and need a practical test based on the shape of the input and output.
- You are building RAG and want patterns for decomposition, parallel retrieval or processing, guardrails, voting, and retry loops.
- You use Python or TypeScript and want a short overview of how LlamaIndex Workflows expresses tools, events, concurrency, and multi-agent systems.

## Related talks

- [The Future of Knowledge Assistants](https://aietalks.com/talks/the-future-of-knowledge-assistants) (Jerry Liu, LlamaIndex, 16:55)
- [Agent Engineering (Day 2)](https://aietalks.com/talks/agent-engineering-day-2) (swyx, AI Engineer Summit & Shashank Kapoors, Princeton University & Mukun Sudar & Arush, Google & Barry Zhang, Anthropic & Zach Renau, Sierra & Will Brown, Morgan Stanley & John Křizí, Jane Street & Anu Koduri, Bloomberg & Mike Conover, Brightwave & Kevin Hou, Windsurf & Mustafa Ali, Method Financial & Kyle Corbett, OpenPipe & Nick Kotakis, SuperDial & Rahul Sanghvi, Ramp & Karina Nguyen, OpenAI & Stefania Druga, Google & Soumith Chintala, Meta and PyTorch, 8:26:36)
- [How to Build Enterprise-Aware Agents](https://aietalks.com/talks/how-to-build-enterprise-aware-agents) (Chau Tran, Glean, 19:53)
- [12-Factor Agents: Patterns of Reliable LLM Applications](https://aietalks.com/talks/12-factor-agents-patterns-of-reliable-llm-applications) (Dex Horthy, HumanLayer, 17:06)
- [Stop AI Agent Hallucinations: 5 Techniques + Production Patterns](https://aietalks.com/talks/stop-ai-agent-hallucinations-5-techniques-production-patterns) (Elizabeth Fuentes, AWS, 55:19)