# Real AI Agents Need Planning, Not Just Prompting

Yuval Belfer, AI21 Labs | AI Engineer World's Fair 2025 | 07:58

Source: https://www.youtube.com/watch?v=Th5e4h-oVmc
Channel: AI Engineer (https://www.youtube.com/@aiDotEngineer). Summarised by AIE Talks.
Page: https://aietalks.com/talks/real-ai-agents-need-planning-not-just-prompting
Published: 2025-06-03
Tags: cost, planning, workflows

## TL;DR
- Language models struggle with instruction following when developers put all context, constraints, and requirements into one prompt.
- Planning lets an agent break a complex task into steps, reconsider its plan, and run independent work in parallel.
- A planner and execution engine can improve answer quality by comparing candidates and validating results, although this costs more time and money.

## Summary
Yuval Belfer argues that prompting alone is no longer enough for many instruction-following tasks. Modern prompts often contain context, requirements, constraints, and formatting rules all at once, which language models can fail to satisfy. He presents planning as a way for agents to work through complex tasks. A planner can choose steps, reconsider the plan during execution, and identify dependencies that allow parallel work. An execution engine can compare candidates by expected cost, latency, and success probability, then validate and improve promising results. Belfer illustrates this with a simplified version of AI21 Maestro, where requirements are separated and checked through an execution graph. He is also clear that planning is not always needed. Teams should start with a simple model, tools, or ReAct, and add planning only when task complexity justifies the extra runtime and cost.

## Key ideas
### Large prompts make instruction following harder than the original demos suggested
[00:00](https://www.youtube.com/watch?v=Th5e4h-oVmc&t=0s)
Belfer contrasts the early InstructGPT example, "Explain the moon landing to a six-year-old," with current prompts that combine context, constraints, requirements, and formatting rules. Developers often put everything into one prompt and hope the model follows it. He argues that language models alone are no longer enough even for apparently simple instruction-following tasks. The problem applies across language models, including GPT-4.1, which was released three and a half years after InstructGPT but still has trouble following instructions.

### An agent is whatever system works for the task
[01:34](https://www.youtube.com/watch?v=Th5e4h-oVmc&t=94s)
Belfer avoids a strict philosophical definition of an agent. He says engineers should call a system an agent, workflow, or something else based on whether it works. He mentions several systems that people may label as agents: an LLM router that directs queries to specialized models, function calling that connects a model to APIs and tools, and systems built around MCP. The label does not matter to him. The useful question is what capabilities the system has and whether it solves the task.

### ReAct handles one step at a time without looking ahead
[02:39](https://www.youtube.com/watch?v=Th5e4h-oVmc&t=159s)
ReAct means reasoning and acting in a loop. The model forms a thought, takes an action, observes the result, and repeats the process. Belfer says this framework can work with any language model, but each step focuses on the current situation and the next action. It does not look ahead across the entire plan. For complex tasks, this makes it difficult to understand why the system followed a particular route. You can inspect the sequence of thoughts and actions, but that does not reveal a complete plan.

### Planning maps the steps needed to reach a goal
[03:02](https://www.youtube.com/watch?v=Th5e4h-oVmc&t=182s)
Belfer defines planning as figuring out which steps are needed to reach a goal. He recommends it for complex problems, especially when the work needs parallelization or explainability. He mentions text-based planners such as Microsoft's Magentic-One and code-based planners associated with Hugging Face's smolagents. A planner can make the work easier to inspect because it describes the route toward the goal instead of producing only a sequence of local reactions.

### Dynamic planning lets the system reconsider its route
[03:43](https://www.youtube.com/watch?v=Th5e4h-oVmc&t=223s)
Dynamic planning does not create one fixed plan and follow it to the end. During execution, the system can ask whether the current plan is still good and choose another route. This matters when a step produces new information or when the expected cost, latency, or likelihood of success changes. Belfer pairs the planner with an execution engine. The engine analyzes dependencies between steps, which can enable parallel execution, and can trade speed against cost. He also mentions branch prediction as one option for faster systems.

### Separating requirements makes them easier to validate
[04:31](https://www.youtube.com/watch?v=Th5e4h-oVmc&t=271s)
In Belfer's simplified description of AI21 Maestro, the original prompt contains context, a task, and requirements such as keeping the answer under ten paragraphs, using a formal tone, and keeping brand mentions positive. Instead of leaving these requirements mixed together in one prompt, the system separates them so they can be checked. The planner and execution engine then build an execution tree or graph. At each step, they select candidates and continue improving the ones that look promising.

### Candidate search and validation trade extra cost for quality
[05:36](https://www.youtube.com/watch?v=Th5e4h-oVmc&t=336s)
Belfer describes several techniques for improving an answer. Best-of-n samples multiple generations from a language model at high temperature, or uses different models. The system discards candidates that do not look promising and continues with the strongest candidates within a predefined budget. Validation supports repeated fixing and improvement. In a more complex workflow, paths carry expected cost, latency, and success probability. A final reduce step can select the best result or combine several results into one answer.

### Planning should be added only when simpler systems fail
[07:03](https://www.youtube.com/watch?v=Th5e4h-oVmc&t=423s)
Belfer says that language models are not always enough, even for instruction following, but he does not recommend starting with the most complex architecture. If a small language model works, use it. If a model with tools works, use that. ReAct is also appropriate when the task allows it. Planning and an execution engine are for tasks complex enough to justify more runtime and money. His advice is to start simple and use the approach that works for the actual task.

## Notable quotes
- "And agents don't just use prompting, they need planning." (01:10)
- "Whatever works, just make it work. Call it an agent. Call it a workflow. Call it whatever." (01:34)
- "And really, even for simple tasks such as instruction following, LMs alone are no longer enough." (01:10)
- "I will always say start simple and then use whatever works." (07:03)

## Tools & references mentioned
- OpenAI
- InstructGPT
- GPT-4.1
- AI21 Labs
- AI21 Maestro
- MCP
- ReAct
- Magentic-One
- Microsoft
- Hugging Face
- smolagents

## Who should watch
- You are building an agent that must satisfy several instructions at once and need a way to check whether its output meets them.
- Your ReAct workflow keeps making local decisions without a clear route through a complex task.
- You want to compare the extra quality from planning against its added runtime and cost before adopting a more involved architecture.

## Related talks

- [Reasoning + RL](https://aietalks.com/talks/reasoning-rl) (Will Brown & Greg Kamradt, ARC Prize Foundation & Aakanksha Chowdhery, Reflection AI & Ryan Marten, Bespoke Labs & Kyle Corbitt, OpenPipe & Nathan Lambert, AI2 & Christian Szegedy, Former co-founder of xAI, 3:54:58)
- [How to Build Planning Agents without Losing Control](https://aietalks.com/talks/how-to-build-planning-agents-without-losing-control) (Yogendra Miraje, FactSet, 15:58)
- [Build a Prompt Learning Loop](https://aietalks.com/talks/build-a-prompt-learning-loop) (SallyAnn DeLucia & Fuad Ali, Arize, 52:08)
- [Software Engineering Is Becoming Plan and Review](https://aietalks.com/talks/software-engineering-is-becoming-plan-and-review) (Louis Knight-Webb, Vibe Kanban, 20:23)
- [Building Reliable Agentic Systems](https://aietalks.com/talks/building-reliable-agentic-systems) (Eno Reyes, Factory.ai, 18:14)