A minimal agent combines an LLM, memory, planning, tools, and a while loop.
2
Tool calling requires local code to execute the requested function and return its result to the model.
3
A to-do list gives the model a way to plan, track completed work, and keep iterating toward a goal.
Summary
Kam Lasater builds an agent incrementally, starting with a basic LLM call and adding a condition, tool calling, parallel tool execution, and planning. Each step exposes which part of the behavior comes from ordinary application code and which part comes from the model. The tool-calling example uses Google search and web browsing, with local code handling function arguments, execution, tool-call IDs, and returned text. The largest change comes when the agent gets a to-do list. The model creates tasks, marks them complete, checks whether the goal is done, and responds to feedback by adding more work. Lasater is honest that the early versions feel deterministic and that the final system is imperfect. He also warns that an LLM can keep calling tools without converging unless the client adds boundaries. The talk ends with possible extensions such as storing browsed pages in a vector database.
An agent can be reduced to a small set of moving parts
Lasater defines an agent with an LLM, memory, planning, tools, and a while loop. He then breaks memory into read and write operations and the loop into a condition plus repeated execution. This decomposition gives a practical implementation plan. The talk is deliberately aimed at making the system understandable enough to run and break. Lasater wants viewers to see where a simple script starts to feel like an agent, rather than treating an agent framework as a black box. The initial code begins with a standard OpenAI chat completion, which he calls Step Zero.
A condition adds evaluation after the model answers
In Step One, the program makes the original completion call and then sends the answer to a second LLM acting as a judge. The judge receives a question and answer and must return a strict JSON object containing a Boolean that says whether the answer is complete. Lasater demonstrates this with the question, "what is the average Wing speed of a swallow". The program therefore has a basic completion, an evaluation step, and a response to the user. At this point, he describes the behavior as deterministic, mechanistic, and straightforward.
Tool calling moves execution into local application code
Step Two adds a Google search tool through SerpAPI. The model can request a search with a query and location, but the OpenAI SDK does not execute that request for the application. Local code must inspect the tool call, read its arguments, invoke the matching function, and append both the call and its result to the conversation. The model then receives the tool response and makes another decision. Lasater shows the function schema, including its name, description, strict setting, query, location, required fields, and restrictions on additional fields. A validation error exposes how tool schemas can fail in practice.
Parallel tool calls require matching each result to its call
Step Three refactors the completion-with-tools logic into a utility and changes it to handle multiple tool calls. The returned calls are treated as an array, and the application loops over them, passes each function its arguments, and adds every result back into the conversation. Each response includes a tool-call ID, allowing the model to associate a result with the request that produced it. Lasater notes that the same tool can be requested more than once with different parameters. His mental model is that tools are usually text transformations, often taking structured parameters and returning a string.
A to-do list gives the model an explicit planning mechanism
Step Four adds planning through a to-do list. The agent can add tasks, mark tasks as done, inspect remaining tasks, and check whether all work is complete. Lasater connects this list to memory because it supports both reading and writing state. The prompt instructs the assistant to make a plan before doing work, use available tools, mark tasks complete, check the goal, and create a report. Feedback from the goal check becomes another to-do item. The loop is now driven by the model's repeated tool calls, while the client still performs the requested operations.
The model can keep calling tools unless the client sets boundaries
Lasater warns that an LLM may continue iterating without reaching an answer, especially when the context is repeatedly pruned and no error or stopping condition appears. One possible guardrail is a limit on the number of iterations before client code stops the process. His example agent does not yet include that protection. This is an important limitation of the simple design: the model decides what to do next, but the surrounding application still needs to control runaway execution. The to-do list gives the model structure, but it does not guarantee convergence.
Planning makes the simple loop produce useful multi-step work
When asked to learn about building agents without a framework, the agent creates tasks to search for information, summarize the results, and ask whether the explanation is sufficient. It calls the search tool, browses pages, marks tasks complete, checks the goal, and lets the LLM judge approve the result. Lasater then tries a more personal request about finding activities and dinner for a date. The agent makes a plan, searches for local activities and restaurants, browses results, and returns options. He does not claim the system is perfect, but says the combined components produce something interesting.
The system can grow toward stored retrieval without changing its basic shape
Lasater suggests adding a vector database as a next step. Browsed web pages could be inserted into ChromaDB or another memory store and used in a more retrieval-augmented system. This extension follows directly from the existing design: tools return text, the agent can write information into storage, and later steps can read it. The talk keeps the first implementation small so that these additions remain visible as separate engineering choices. The result is a basic agent that calls tools, tracks work, evaluates progress, and can later gain longer-lived memory.
"There are cases when that LLM can just iterate forever calling tools and never converging."11:00
Who should watch
You are building an agent with an SDK or framework and want to understand the application code underneath it.
You want a small example that separates model calls, evaluation, tool execution, state, and planning so you can experiment by changing one part at a time.
You need to decide whether your current workflow really needs an agent, or whether a simpler completion and a few explicit functions would be enough.
