# Why people think "agent" is a buzzword but it isn't

Chip Huyen | AI Engineer Summit 2025 | 28:07

Source: https://www.youtube.com/watch?v=D6v5rlqUIc8
Channel: AI Engineer (https://www.youtube.com/@aiDotEngineer). Summarised by AIE Talks.
Page: https://aietalks.com/talks/why-people-think-agent-is-a-buzzword-but-it-isnt
Published: 2025-02-22
Tags: context-engineering, memory, planning, tool-use

## TL;DR
- An agent is a model that can perceive and act on an environment, with its available actions defining what it can do.
- Agent reliability drops quickly as tasks require more steps, so systems need task decomposition, stronger models, or more test-time computation.
- Useful agents need carefully designed tools, clear documentation, and memory systems that divide information between short-term context, external storage, and model knowledge.

## Summary
Chip Huyen argues that "agent" is an old and useful term for a system that can perceive an environment and act on it. Modern language models become agents when they can use tools such as web browsers, calculators, code editors, inboxes, and calendars. The engineering problem is that agent tasks often require several dependent actions, and failure probability rises with every step. Huyen discusses three main difficulties: task complexity, translating ambiguous human instructions into API calls, and managing the growing amount of context produced by tools and reasoning. She recommends breaking large tasks into smaller ones, using stronger models or test-time compute, exposing a small set of narrow tools, documenting parameters and error meanings, and asking users to clarify ambiguous requests. For context, she describes short-term memory, external long-term storage, and knowledge learned into the model as separate options.

## Key ideas
### An agent is defined by its ability to perceive and act
[01:26](https://www.youtube.com/watch?v=D6v5rlqUIc8&t=86s)
Huyen uses the definition from Stuart Russell and Peter Norvig: an agent is "anything that can perceive the environment and that act on the environment." The environment and available actions depend on the system. A chess agent sees the chessboard and makes chess moves. ChatGPT can interact with the internet, browse the web, use a calculator, and process text and images. In a coding-agent example from the SWE-agent paper, the environment is a computer with a terminal and file system. Its actions include navigating a repository, searching files, viewing files, and editing files. Giving a model web access expands its environment to include the internet.

### Tools let models work around limits and enter existing workflows
[02:58](https://www.youtube.com/watch?v=D6v5rlqUIc8&t=178s)
Huyen gives several reasons to give models access to actions. A news API or web browser can provide information newer than the model's cutoff date. A calculator can handle arithmetic instead of requiring the model to become much better at math. Tools can also turn a text-only model into a multimodal system. A language model can send an image to an image-captioning model, then use the caption to answer the user. The larger shift is putting the model inside a workflow. Access to an inbox, calendar, or code editor lets people use the model in their daily work instead of opening a separate browser for every AI interaction.

### Reliability falls rapidly as an agent takes more steps
[05:05](https://www.youtube.com/watch?v=D6v5rlqUIc8&t=305s)
Huyen explains that task failure rates increase with task complexity for both people and models. If a model makes a mistake 2% of the time on one step, a ten-step task can have an error rate of about 18%. At 100 steps, the model can become nearly useless because it makes a mistake most of the time. Even a simple question such as how many people bought a company's products last week can expand into several actions: list the products, get orders for each product, add the numbers, and produce a response. Huyen says it is rare to see agents consistently solve tasks involving more than five steps.

### Complexity should determine which tasks an agent receives
[07:39](https://www.youtube.com/watch?v=D6v5rlqUIc8&t=459s)
Different applications measure complexity in different ways. A common measure is the number of steps needed to solve a task. Huyen describes a planning benchmark she is working on that uses a dataset allowing her to control task complexity. Most models in her benchmark perform well only on tasks requiring at most five steps, and most fail after ten steps. She compares this with an older coding study that measured complexity by the number of transformations in a prompt, such as converting a string to lowercase and removing half its characters. The success rate also fell rapidly as the number of steps increased. Newer reasoning models perform better, but Huyen still recommends giving an agent tasks at the level it can handle.

### More computation can help an agent plan longer tasks
[12:29](https://www.youtube.com/watch?v=D6v5rlqUIc8&t=749s)
Huyen describes several ways to increase the complexity an agent can handle. One is to break a task into subtasks. If an agent can reliably perform three steps but a larger task needs five or six, the system can divide the work. Another is test-time compute scaling, where the model receives more computation during inference. It can generate more tokens to think through a problem, or produce several candidate solutions and select one. A stronger model is another option, which Huyen calls train-time compute scaling because more computation was invested during training. Her benchmark shows newer reasoning models solving more complex tasks than several non-reasoning models.

### Natural-language instructions and APIs create problems on both sides
[13:55](https://www.youtube.com/watch?v=D6v5rlqUIc8&t=835s)
Agent tool use is a translation from a human instruction into API calls. For an instruction such as creating an order from a customer email, the agent may need to extract a customer ID, extract an order ID, and call the order-creation function. Natural language can leave important details unstated. If a user asks for five bestselling products under $10, the agent may not know whether to search yesterday, last week, or last month. APIs can also have poor names and documentation. Huyen says tool documentation should explain what a function does, its parameters and types, returned values, and error codes. It should also explain what an error or return value means and how the agent should respond.

### Tools should match how models work, rather than copying human procedures
[17:40](https://www.youtube.com/watch?v=D6v5rlqUIc8&t=1060s)
Humans and models can prefer different interfaces and work patterns. A person may use the Salesforce graphical interface, while an AI system may perform better through a direct API. People usually handle steps sequentially, while an agent can perform many independent actions in parallel. Huyen compares browsing 100 websites, which is tedious for a person, with sending many queries and receiving summaries. This difference affects training data. Supervised fine-tuning teaches a model to copy human behavior, which may not be optimal for an AI system. Reinforcement learning can let the model try actions and discover a better procedure. Huyen also recommends a small set of narrow, well-defined tools and asking users to clarify ambiguous requests.

### Long context is different from effective context
[21:08](https://www.youtube.com/watch?v=D6v5rlqUIc8&t=1268s)
Agents accumulate system instructions, examples, tool documentation, tool outputs, plans, and reasoning tokens. Text-to-SQL systems may also need to choose among thousands of table schemas. Huyen says models that plan well are not always the same models that handle long inputs well. Planning is output-heavy because it requires reasoning tokens, while long-context work is input-heavy. A model may accept a million tokens but behave poorly once given more than 30,000. Huyen separates memory into three levels. Short-term memory holds information relevant to the current task. External databases or files provide long-term memory, allowing intermediate results to leave the context and return when needed. Internal knowledge is information learned during training or fine-tuning.

## Notable quotes
- "An agent is anything that can perceive the environment and that act on the environment." (01:51)
- "The more steps there are, the more complex queries, the higher number of steps and the more likely the agent is going to fail." (07:01)
- "You will need to provide necessary documentation as a list, you need to explain what the function does, what parameters it take in, what is the type of parameter, what does the parameter stand for." (16:17)
- "A model might have very long context but then it might not use that context effectively." (25:01)

## Tools & references mentioned
- Chip Huyen
- Stuart Russell
- Peter Norvig
- NVIDIA
- Snug AI
- Stanford
- SWE-agent paper
- CTIC dataset
- DeepSeek-R1
- Gemini 2.0 Flash Thinking
- o1-preview
- Claude 3.5 Sonnet
- Gemini 2.0 Pro
- GPT-4
- ZebraLogic
- Z3

## Who should watch
- You are deciding whether a workflow needs an agent or can be handled by a simpler model call.
- Your agent works on short tasks but starts failing when it chains several tool calls.
- You are designing tool APIs, documentation, or memory for a model that must operate inside a real application.

## Related talks

- [Why (Senior) Engineers Struggle to Build AI Agents](https://aietalks.com/talks/why-senior-engineers-struggle-to-build-ai-agents) (Philipp Schmid, Google DeepMind, 10:40)
- [How We Build Effective Agents](https://aietalks.com/talks/how-we-build-effective-agents) (Barry Zhang, Anthropic, 15:09)
- [Agents Need More Than a Chat](https://aietalks.com/talks/agents-need-more-than-a-chat) (Jacob Lauritzen, Legora, 14:21)
- [Building Applications with AI Agents](https://aietalks.com/talks/building-applications-with-ai-agents) (Michael Albada, Microsoft, 15:50)
- [Let's Build an Agent from Scratch](https://aietalks.com/talks/lets-build-an-agent-from-scratch) (Kam Lasater, 19:29)