# Pydantic is STILL all you need

Jason Liu, Independent AI consultant, advisor, writer, and educator | AI Engineer World's Fair 2024 | 15:21

Source: https://www.youtube.com/watch?v=pZ4DIH2BVqg
Channel: AI Engineer (https://www.youtube.com/@aiDotEngineer). Summarised by AIE Talks.
Page: https://aietalks.com/talks/pydantic-is-still-all-you-need
Published: 2024-09-06
Tags: guardrails, rag, structured-outputs, tool-use

## TL;DR
- Pydantic gives language-model applications typed, validated outputs instead of unstructured strings that require manual JSON parsing.
- Validators can catch errors, feed useful error messages back to a model, and support retries for cases such as invalid totals or hallucinated URLs.
- RAG and agent systems can use structured response models to generate searches, workflows, tables, and other data structures while keeping control flow in ordinary code.

## Summary
Jason Liu argues that Pydantic remains a practical interface between language models and normal software. The basic pattern is to define a response model, pass it to a client, and use validators to check the result. Streaming, partial objects, iterable responses, and support for several model providers fit around that same pattern. He shows how validators can correct names, check receipt totals, reject dead URLs, and constrain extracted data. For RAG, structured models can produce date-bounded searches, select a backend, and split a comparison into parallel queries. For extraction, the same approach handles labels, transcript summaries, tables, and data frames. Liu says complex systems still have failure modes, but useful validation errors and retries can reduce them. He also describes agents as generators of workflows and DAGs that ordinary execution engines can run. His central claim is that language models become easier to use when they generate typed data structures while the programmer owns the functions and control flow.

## Key ideas
### Schemas keep language-model integrations compatible with ordinary software
[01:01](https://www.youtube.com/watch?v=pZ4DIH2BVqg&t=61s)
Liu says unstructured responses create problems with compatibility, composability, and reliability when code interacts with external systems. Pydantic models and function calling provide nested objects, nested models, and validators. The result is closer to programming with data structures than parsing a string and hoping its contents survived. He presents this as the same kind of discipline developers already use in Python and other language ecosystems, including tools such as FastAPI. The model output becomes something the rest of the application can consume directly.

### The Pydantic API stays small while supporting multiple model providers
[01:53](https://www.youtube.com/watch?v=pZ4DIH2BVqg&t=113s)
Liu says the library's core API has changed very little. The pattern is to define a Pydantic object, patch an OpenAI-compatible or other client, and pass a response model such as User. The current library supports Python, TypeScript, Ruby, Go, Elixir, and Rust, along with providers including Anthropic, Cohere, Gemini, Groq, Ollama, and llama.cpp. He describes the interface as one client with create, iterable creation, and partial creation, while response models, validators, and the messages array contain the application-specific behavior.

### Streaming can return typed objects before the whole response is complete
[02:27](https://www.youtube.com/watch?v=pZ4DIH2BVqg&t=147s)
The iterable mode returns individual objects as they arrive, which Liu says can improve latency while preserving structure. Partial responses take a different approach: the entire object can be validated as it is built. He connects this to generative user interfaces, where an application can render a structured result during streaming without maintaining custom JSON stack logic. The distinction is whether the application wants multiple complete objects or one object that becomes progressively available.

### Validators turn model errors into actionable correction loops
[05:28](https://www.youtube.com/watch?v=pZ4DIH2BVqg&t=328s)
A field validator can express rules that the prompt does not need to state. Liu demonstrates a validator that uppercases or rejects names, then sends the resulting error message back to the language model when retrying is enabled. A model-level validator can check whether receipt prices and quantities add up to the total cost. He says these failures are uncommon, but the useful outcome is a visible error in monitoring and a chance to request a corrected result. The error message becomes part of the conditional prompt.

### Structured search objects make RAG applications easier to compose
[07:45](https://www.youtube.com/watch?v=pZ4DIH2BVqg&t=465s)
Liu says embeddings do not capture every search requirement, such as the meaning of 'latest news', and different questions may need different indices. A structured Search object can contain a query, optional start and end dates, a result limit, and a source that selects the backend. The application then implements the search function separately. Iterable output can split a comparison into one query for each subject, allowing parallel searches before the results are passed into an answer function. He describes the resulting RAG system as two models and two ordinary functions.

### The same type-and-validator pattern supports extraction and custom data types
[10:04](https://www.youtube.com/watch?v=pZ4DIH2BVqg&t=604s)
For classification, a literal field can restrict the result to values such as spam or not spam. Liu says adding chain of thought can improve accuracy by about 15 percent in his example, while validation still checks the returned value. A transcript extraction model can contain a meeting type, title, action items, summary, and owners, with a possible validator checking owners against participants. He also shows a custom table type that parses markdown into a data frame before validation and serializes it back to markdown, while exposing data-frame methods to the IDE.

### Retries and better error messages address the remaining failure modes
[12:19](https://www.youtube.com/watch?v=pZ4DIH2BVqg&t=739s)
Liu says one retry is often enough for models such as OpenAI and Anthropic when the validation error is informative. He reports seeing four to five percent failure modes in complex validations during consulting work, and says fine-tuning models on function calling reduced those failures to zero in examples involving smaller models such as Mistral or GPT-3.5. Faster models make it easier to accept the latency cost of retries. His practical advice is to write error messages that are useful both to the programmer and to the language model.

### Agents can generate executable workflows while code retains control
[13:31](https://www.youtube.com/watch?v=pZ4DIH2BVqg&t=811s)
Liu extends the approach beyond question-answering RAG. In enterprise applications, RAG can generate reports that support decisions. Agents can generate workflows and DAGs for an execution engine instead of repeatedly running a ReAct loop and hoping it terminates. In both cases, the application defines the types, functions, and control flow. Prompting, response models, and constrained sampling remain implementation choices behind that interface. His conclusion is that generative AI can fit existing software by generating data structures that ordinary code owns and executes.

## Notable quotes
- "The problem is that by not having schemas and structured responses we tend to lose compatibility composability and reliability when we build tools and write code that interact with external systems." (01:01)
- "We are going to be programming with data structures which is something everyone knows how to do rather than trying to beg and pray to the LLM gods." (05:10)
- "Prompting is an implementation detail, the response model is an implementation detail." (13:31)
- "What we've done is we've turned generative AI just to becoming generating data structures." (14:22)

## Tools & references mentioned
- Pydantic
- Instructor
- FastAPI
- OpenAI
- Anthropic
- Cohere
- Gemini
- Groq
- Ollama
- llama.cpp
- Mistral
- GPT-3.5
- DataDog
- Google Calendar
- ReAct

## Who should watch
- You are building LLM features that still return strings and require hand-written JSON parsing before the rest of the application can use them.
- Your RAG system needs filters, multiple search backends, parallel queries, or validation around generated search plans.
- You are designing extraction or agent workflows and want the model to produce typed objects while your existing code controls execution.

## Related talks

- [Pydantic is all you need](https://aietalks.com/talks/pydantic-is-all-you-need) (Jason Liu, Independent consultant, 17:55)
- [Human seeded Evals](https://aietalks.com/talks/human-seeded-evals) (Samuel Colvin, Pydantic, 12:02)
- [DSPy: The End of Prompt Engineering](https://aietalks.com/talks/dspy-the-end-of-prompt-engineering) (Kevin Madura, AlixPartners, 1:13:13)
- [Pragmatic AI with TypeChat](https://aietalks.com/talks/pragmatic-ai-with-typechat) (Daniel Rosenwasser, Microsoft, 18:34)
- [Why Agentic Systems Need Ontologies](https://aietalks.com/talks/why-agentic-systems-need-ontologies) (Frank Coyle, UC Berkeley, 21:18)