# The LLM Triangle: Engineering Principles for Robust AI Applications

Almog Baku | AI Engineer Summit 2025 | 26:19

Source: https://www.youtube.com/watch?v=ro5HkZvzfiQ
Channel: AI Engineer (https://www.youtube.com/@aiDotEngineer). Summarised by AIE Talks.
Page: https://aietalks.com/talks/the-llm-triangle-engineering-principles-for-robust-ai-applications
Published: 2025-02-22
Tags: agents, context-engineering, workflows

## TL;DR
- Production LLM applications require far more experimentation and data-driven engineering than model selection alone.
- Standard operating procedures turn expert workflows into explicit steps that an LLM can follow with more consistent quality.
- Teams should start with a large model, collect data, and optimize the model and architecture over time.

## Summary
Almog Baku presents the LLM Triangle as a way to build production AI applications. The triangle has three working parts: the model, engineering techniques, and contextual data. A standard operating procedure guides all three. Baku argues that an LLM application is roughly 10% sophisticated models and 90% experimentation and data-driven engineering. He recommends extracting an expert's implicit workflow, expressing it as a graph or sequence of steps, and implementing those steps with a mix of LLM agents and deterministic software. Autonomous agents are fast and creative, but they are difficult to debug and can produce inconsistent results. Crafted flows are easier to inspect and control, although they require more development work. Model choice depends on task complexity, infrastructure, latency, cost, and available data. For context, Baku recommends using examples and selecting only relevant data rather than filling the prompt with everything available.

## Key ideas
### Production quality comes from engineering work around the model
[02:23](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=143s)
Baku says that building a proof of concept is easy, while a production application that maintains a required quality level is hard. The problem appears when a demo meets the one customer use case that the team did not anticipate. From his experience building applications, he describes LLM-native systems as 10% sophisticated models and 90% hard experimentation and data-driven engineering work. This changes where teams should spend their time. The model matters, but repeatable behavior comes from the surrounding process, data, and software decisions.

### A standard operating procedure turns expert judgment into explicit steps
[04:21](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=261s)
The triangle is guided by a standard operating procedure, which Baku defines as step-by-step instructions that let workers perform routine tasks at a consistent quality. He compares an LLM with a smart but inexperienced intern. For a landing-page builder, the team should interview an expert, ask how they approach the task, and break the work into smaller steps. The important work is finding the implicit reasoning between visible actions. An expert may know that a landing page needs a compelling story, but may not mention that step unless asked directly. Baku recommends compiling the steps and drawing them as a graph when the process contains loops or recursion.

### An LLM-native architecture splits the procedure across models and software
[08:15](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=495s)
Engineering techniques implement the procedure. Baku groups them into prompt-level methods, software techniques, and methods between the two. His main example is an LLM-native architecture, also called flow engineering, chains, or a workflow. The team splits the procedure into multiple steps and assigns each step to an agent or ordinary code. In a Wikipedia-writing system, one component normalizes the request, another categorizes the subject, software performs a search and formats the results as Markdown, and another agent writes the table of contents. The model handles work suited to it, while deterministic code handles work that does not need an LLM.

### Autonomous agents trade easy development for uncertain behavior
[12:09](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=729s)
Baku defines an agent as an LLM plus a prompt template that performs a task from A to B. An autonomous agent also has tools and can decide how to plan the work. This is attractive because a developer can provide functions and let the model choose how to call them. Baku says autonomous agents are creative and fast to build, but difficult to debug, with quality that may not hold up in production. A crafted flow is more predictable and easier to inspect, though it costs more time and resources to develop. His recommendation is to give autonomous agents clear boundaries, so their creative choices stay inside a defined area.

### Model selection should follow the task and the available evidence
[16:20](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=980s)
Baku lists task complexity, infrastructure, performance, latency, cost, budget, and available data as model-selection factors. Summarization and entity extraction are simpler than tasks that require complex reasoning. Smaller models can work well for simple tasks or when the system can provide substantial data, while complex tasks with limited data may require a larger model. His practical advice is to start an LLM-native application with a large model, collect data, understand its behavior, and optimize incrementally. Teams do not need to begin with fine-tuning. They can later change the model or fine-tune it after learning more about the task.

### The model receives one text input, so context has to be prepared carefully
[19:50](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=1190s)
Baku explains that the model does not inherently understand the distinction between context, user prompts, system prompts, and assistant prompts. Software combines those pieces into a template and sends the model a single text input. Prompt templates let developers insert variables while keeping the surrounding structure consistent, but the model itself remains stateless. The practical problem is deciding which data belongs in that text. Too little information leaves the model unable to answer, while too much information makes it harder to find the relevant material. Baku describes this as a needle-in-a-haystack problem, made harder when the prompt contains many relevant needles rather than one isolated fact.

### Examples teach concepts that are difficult to describe directly
[22:54](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=1374s)
Few-shot examples let a team teach the model a classification style through demonstrations. Baku gives a product-classification example in which examples such as TV and printer establish the desired categories. The PlayStation example produces a label closer to the intended terminology after the demonstrations define the preferred style. He summarizes the approach as 'show don't tell.' Teams with many examples can select only a small relevant set for each request. Selection can use similarity methods such as vector search or simpler rules based on keywords and categories. This keeps the context focused while using the model's ability to learn from the prompt.

## Notable quotes
- "LLM native applications are 10% sophisticated models and 90% of hard experimentation and data driven engineering work." (02:23)
- "We should look of the LLM like an intern." (06:10)
- "Autonomous agents are elegant software, seriously amazing, very creative, very fast to build." (14:59)
- "Start with the big model, collect data, understand what's going on, and then optimize it incrementally." (18:56)
- "Show don't tell." (24:15)

## Tools & references mentioned
- GPT
- Python
- JavaScript
- PHP
- Go
- Google
- OpenAI
- Anthropic
- Markdown
- HTML
- few-shot prompting
- fine-tuning
- vector search
- retrieval-augmented generation

## Who should watch
- You are moving an LLM proof of concept toward a customer-facing system and need a way to make its behavior repeatable.
- Your team is deciding between an autonomous agent and a hand-built workflow with explicit steps.
- You need to choose a model and prepare context without committing to fine-tuning before you understand the task.

## Related talks

- [Real ROI: Lessons from Enterprises That Have Already Succeeded with LLMs at Scale](https://aietalks.com/talks/real-roi-lessons-from-enterprises-that-have-already-succeeded-with-llms-at-scale) (Raza Habib, Humanloop, 20:01)
- [Lessons From A Year Building With LLMs](https://aietalks.com/talks/lessons-from-a-year-building-with-llms) (Eugene Yan & Shreya Shankar, UC Berkeley & Hamel Husain & Jason Liu & Bryan Bischof, Hex & Charles Frye, 35:21)
- [Make Your LLM App a Domain Expert: How to Build an Expert System](https://aietalks.com/talks/make-your-llm-app-a-domain-expert-how-to-build-an-expert-system) (Christopher Lovejoy, Anterior, 19:18)
- [12-Factor Agents: Patterns of Reliable LLM Applications](https://aietalks.com/talks/12-factor-agents-patterns-of-reliable-llm-applications) (Dex Horthy, HumanLayer, 17:06)
- [Navigating Challenges and Technical Debt in LLMs Deployment](https://aietalks.com/talks/navigating-challenges-and-technical-debt-in-llms-deployment) (Ahmed Menshawy, Mastercard, 16:15)