# Building Reliable Agentic Systems Eno Reyes, Factory.ai | AI Engineer World's Fair 2024 | 18:14 Source: https://www.youtube.com/watch?v=bjNYEc908oQ Channel: AI Engineer (https://www.youtube.com/@aiDotEngineer). Summarised by AIE Talks. Page: https://aietalks.com/talks/building-reliable-agentic-systems Published: 2024-08-20 Tags: human-in-the-loop, planning, reliability, tool-use ## TL;DR - Reliable agentic systems need planning, decision-making, and the ability to read from and write to an external environment. - Hardcoded plan criteria, structured subtasks, replanning, and feedback processing can reduce errors, even though they limit generality and add engineering work. - Human guidance can raise reliability substantially when the system asks for help at carefully chosen points in the workflow. ## Summary Eno Reyes describes the design lessons Factory.ai learned while building Droids for code review, documentation, testing, refactoring, migrations, and other software tasks. He defines agentic systems through planning, decision-making, and environmental grounding. For planning, he discusses carrying intermediate reasoning through a plan, choosing the right subtask resolution, replanning from current feedback, and defining explicit criteria for successful plans. He argues that hardcoded logic is practical when a team needs useful results in a specific domain soon. For decisions, he covers consensus methods, explicit reasoning structures, fine-tuning, and simulation. For environmental grounding, he recommends custom interfaces for repeated tool workflows, explicit processing of logs and other feedback, bounded exploration, and carefully designed human intervention. The talk is a practical catalogue of system design choices, including their costs and failure modes. ## Key ideas ### An agentic system plans, makes decisions, and interacts with an environment [01:36](https://www.youtube.com/watch?v=bjNYEc908oQ&t=96s) Reyes uses three characteristics to define an agentic system. It plans by deciding which actions to take in the future. It makes decisions according to criteria or an algorithm, which matters when the system faces a broad action space. It also has environmental grounding, meaning it can read from and write to an external environment. That interaction gives agentic systems properties that are different from systems that only generate an answer. At Factory.ai, these ideas apply to Droids for code review, documentation, testing, refactoring, migrations, and larger coding tasks. Different tasks use different cognitive architectures. ### Passing intermediate reasoning through a plan can improve consistency and spread errors [02:39](https://www.youtube.com/watch?v=bjNYEc908oQ&t=159s) Reyes describes a planning pattern inspired by control systems and robotics. As an agent works through a long plan, its reasoning can change quickly and send it off course. Passing intermediate reasoning between plan steps can help individual decisions converge toward consistent reasoning. The cost is error propagation. A small mistake early in a plan can produce strong downstream effects. This matters in software work because even a small code migration may involve hundreds of steps. The pattern therefore gives later steps more continuity, while also making early mistakes more consequential. ### Subtask resolution controls the action space but can overwhelm the model [04:07](https://www.youtube.com/watch?v=bjNYEc908oQ&t=247s) Factory.ai found that different forms of subtask decomposition produce different downstream effects. Increasing the resolution of subtasks gives the system finer control and defines the action space more clearly. The downside is that every additional small task creates another decision for the language model. As the number of tiny tasks grows, deciding what to do becomes harder. Reyes presents this as a design balance. More detailed decomposition can make execution easier to control, but it can also make the overall plan more difficult for the agent to manage. ### Replanning and explicit criteria keep long-running plans connected to reality [05:16](https://www.youtube.com/watch?v=bjNYEc908oQ&t=316s) Model predictive control lets an agent evaluate subtask outcomes and its current state, then replan from real-time feedback. Reyes says this matters when the environment changes or humans are working in the same process while an agent runs. Teams do not need to implement advanced Bayesian predictive modeling to benefit from the general idea. Updating the trajectory and replanning can already improve quality. He also recommends explicit criteria for successful plans or successful initial states. These criteria can come from instruction tuning, few-shot examples, or hard-coded validation logic. ### Hardcoded logic is a practical choice for near-term domain systems [06:53](https://www.youtube.com/watch?v=bjNYEc908oQ&t=413s) Reyes argues that explicit criteria and hardcoded logic are difficult to build and scale, but useful when a team has a domain problem to solve soon. He does not expect general artificial intelligence in the next few months, so Factory.ai focuses on delivering value to customers with concrete constraints. Plan validation and predefined logic reduce errors and help keep a trajectory successful for longer. This approach reduces generality, and Reyes acknowledges that some builders may resist admitting how much symbolic logic their systems contain. His recommendation is to use it when the product goal requires dependable behavior in a specific domain. ### Inference-time consensus can make decisions more consistent at a direct cost [07:48](https://www.youtube.com/watch?v=bjNYEc908oQ&t=468s) For decision-making, Reyes recommends consensus mechanisms such as self-consistency, prompt ensembles, and cluster sampling. The system runs multiple inferences and selects an ideal or representative result. More inference at runtime can improve decision accuracy and consistency, but it costs more and may increase wait times when the calls are not parallelized. He treats this as an important engineering trade-off rather than a free improvement. The same decision process can also use explicit structures such as checklists, Chain of Thought, chain of density, or analogical prompting to make the criteria easier for the model to follow. ### Simulation and fine-tuning address different decision problems [09:29](https://www.youtube.com/watch?v=bjNYEc908oQ&t=569s) Fine-tuning becomes useful when a team has data for specific decisions, especially decisions that are far outside the distribution handled well by general models. Reyes suggests generating training data with a strong model, validating it with team members, and fine-tuning an open source model. This can lock in system quality, while sampling newer models keeps a system open to future improvements. Simulation is especially workable in software development because the system can execute code and reason through code trajectories. Factory.ai samples multiple decision paths and can combine real execution with language-model-imagined paths before evaluating which decisions work. ### Custom interfaces and processed feedback give agents usable environmental grounding [11:52](https://www.youtube.com/watch?v=bjNYEc908oQ&t=712s) Basic tools such as calculators and sandboxed Python execution are widely available. Reyes says the harder problem begins when a task repeatedly combines several tools, such as calculating a value, passing it to another system, parsing logs, and transforming the result. In that situation, the repeated workflow should become a custom tool and interface rather than a sequence the model must rediscover every time. This is particularly useful in software development, where valuable tools often have command-line or editor-specific interfaces. Agents also need explicit feedback processing. For example, a system can extract failing tests from CI logs and provide a short explanation of the remaining output. ### Bounded exploration and human guidance can improve the whole trajectory [16:05](https://www.youtube.com/watch?v=bjNYEc908oQ&t=965s) Agents benefit from collecting context before starting a problem, especially when long-context models can hold more information. They still need a bound on exploration, because gathering context forever does not help. Reyes says the right exploration time is difficult to set in advance and requires evaluation. The system can gather data, decide what is relevant, and begin with a better chance of success. Human guidance provides another control point. With careful user-experience and interaction design, deciding when to ask for intervention can move a system from roughly 30 or 40 percent reliability to 90 or 100 percent reliability, at the cost of reducing full autonomy. ## Notable quotes - "We think that there's three characteristics which are most representative of an agentic system." (01:36) - "Ultimately we're probably not building AGI tomorrow or next week or in the next six months." (06:53) - "The more inference at run time that you run and the more that you can build a clever way of selecting ideal or optimal samples from those many number of inferences, the higher the accuracy it's just going to cost more." (07:48) - "A lot of the time the LLM is not great at actually criticizing its own action, it's just good at listening to you tell it to criticize itself." (15:20) - "With careful UX design and interaction design I think that this can be extremely effective at allowing your systems to go from 30 or 40% reliability to 90 or 100% reliability." (17:30) ## Tools & references mentioned - Factory.ai - Droids - LangChain - Claude Artifacts - Gemini Pro 1.5 - Sonnet 3.5 - Chain of Thought - chain of density - Language Agent Tree Search - Monte Carlo Tree Search - Hugging Face ## Who should watch - You are building software agents that run multi-step coding or development workflows and need practical ways to reduce drift and error propagation. - Your agent uses tools or CI output but still has trouble turning raw logs and tool results into useful next actions. - You need to choose between more autonomy, more hardcoded control, and asking a human for help during long-running tasks. ## Related talks - [3 ingredients for building reliable enterprise agents](https://aietalks.com/talks/3-ingredients-for-building-reliable-enterprise-agents) (Harrison Chase, LangChain/LangGraph, 20:55) - [Building Applications with AI Agents](https://aietalks.com/talks/building-applications-with-ai-agents) (Michael Albada, Microsoft, 15:50) - [Deterministic Infra for Non-Deterministic AI Agents](https://aietalks.com/talks/deterministic-infra-for-non-deterministic-ai-agents) (Nishant Gupta, Meta Superintelligence Labs, 07:14) - [Why (Senior) Engineers Struggle to Build AI Agents](https://aietalks.com/talks/why-senior-engineers-struggle-to-build-ai-agents) (Philipp Schmid, Google DeepMind, 10:40) - [Build Systems, Not Code](https://aietalks.com/talks/build-systems-not-code) (Angie Jones, Agentic AI Foundation, 19:39)