Designing AI-Intensive Applications

swyx13:02 · Aug 2025 · 28K views
Thumbnail for Designing AI-Intensive Applications Watch on YouTube
TL;DR
  1. 1

    AI engineering needs reusable models like ETL, MVC, CRUD, and MapReduce to guide how applications are built.

  2. 2

    The useful distinction between workflows and agents is the ratio of human input to valuable AI output.

  3. 3

    SPADE describes an AI-intensive application as one that syncs data, plans and parallelizes work, analyzes and reduces results, delivers output, and evaluates it.

Summary

swyx uses the conference opening to ask what the standard models of AI engineering might be. Traditional software has patterns such as ETL, MVC, CRUD, and MapReduce, but AI applications still lack an equivalent model that covers the whole system. He argues that debates over whether a system is a workflow or an agent are less useful than measuring human input against valuable AI output. AI News provides his example: scripts scrape sources, plan work, recursively summarize material, format results, and evaluate the output. When this pattern is expanded to thousands of model calls, it becomes an AI-intensive application. swyx proposes SPADE as a working model: sync, plan and parallelize, analyze and reduce, deliver, and evaluate. He presents it as a current hypothesis and asks engineers to develop better models that help them add useful intelligence without making products annoying.

Key ideas
05:23

AI engineering is still searching for its reusable building patterns

swyx compares AI engineering with established areas of software engineering, where people use familiar models such as ETL, MVC, CRUD, and MapReduce. He asks what the equivalent standard model for AI engineering might be. Retrieval-augmented generation is one candidate, but he does not see it as a complete answer. He mentions arguments that long context or fine-tuning might replace it, then says the field needs other models to guide how engineers think about applications. His proposal is exploratory rather than final. He wants conference attendees to consider which patterns could help teams build AI applications beyond individual demos.

10:57

AI applications become harder when model calls multiply

swyx describes AI-intensive applications as systems that make thousands of AI calls for a particular purpose. His example is AI News, which he built with separate scripts for Discord, Reddit, and Twitter. Each source follows the same general process: scrape the material, plan the work, recursively summarize it, format the result, and evaluate it. The system runs every day and has been improved incrementally while he also works on the conference. The point is that an application can contain many model calls even when the user sees only one finished result. This changes the engineering problem from a single prompt-response interaction into a coordinated processing pipeline.

09:18

Human input and valuable AI output are a better lens than agent labels

Instead of arguing over whether a system is a workflow or an agent, swyx proposes comparing human input with valuable AI output. Copilots may react after every few typed characters, while chat systems respond after each query. Reasoning systems can produce more output from less direct input, and agentic systems such as deep research can perform many steps from one request. He extends the idea to ambient agents, where there may be no human input at all. This gives engineers a way to discuss how much work the system performs between user interactions without depending on disputed definitions of 'agent.'

08:48

AI News is a workflow that still creates useful value

Sum, whom swyx identifies as the lead of PyTorch, told him that AI News is not an agent. swyx initially disagreed because he called it an agent, but accepted the distinction. AI News is a collection of scripts that repeatedly perform a fixed process over different sources. That does not make it less useful. swyx says the product delivers value even though it fits better under the label workflow. He uses this example to explain why conference tracks do not all need to be branded as agents. The engineering question is whether the system produces something useful for people, rather than whether its terminology sounds more advanced.

03:46

AI engineering often improves through simpler systems

swyx says a consistent lesson from Anthropic, the Latent Space podcast, Eric Sun, Greg Brockman, and the AMP team is to avoid unnecessary complexity. He cites a simple scaffold associated with beating SWE-bench and describes Deep Research as another example of a system that does not need an elaborate structure to work. He also says the field is still early, which means engineers should feel encouraged rather than assume the main design space has been settled. His view is practical: simple systems can produce strong results, and the difficult engineering work appears when applications move toward real production use.

06:57

Production work shifts toward evaluation, security, and orchestration

In the software development life cycle, swyx sees the early parts becoming increasingly commoditized. He names language models, monitoring, and retrieval-augmented generation as areas with free tiers or low initial costs. The harder work begins when a product makes real money from customers. At that point teams need evaluations, security, and orchestration. He connects these concerns to the conference tracks added that year. His argument is that AI engineering has to move past demos and handle the operational work that determines whether a system can support a real product.

11:21

SPADE organizes the flow of an AI-intensive application

swyx proposes SPADE as a model for applications that coordinate many AI calls. The process starts by syncing information, then planning and parallelizing work. The system analyzes the resulting material and reduces many pieces into one result before delivering content to the user. It then evaluates the output. He also mentions related implementation choices, including turning processed information into a knowledge graph, producing structured outputs, and generating code. Canvas in ChatGPT and artifacts in Claude are examples of delivering a code artifact instead of only returning text. SPADE is his current mental model, not a settled standard.

"The question that I want to phrase here is what is the standard model in AI engineering?"05:23
Who should watch
  • You are designing a system where one user request triggers many model calls and need a way to describe its stages.
  • Your team is debating whether a feature is an agent or a workflow, while the more practical issue is how much useful output it produces from limited user input.
  • You are moving an AI prototype toward production and need to think about evaluation, security, orchestration, and structured output.