# Agent Output Is Not UX: Rendering Layer Your LLM Pipeline Is Missing

Bala Ramdoss, Amazon | AI Engineer World's Fair 2026 | 14:13

Source: https://www.youtube.com/watch?v=maTp79FD9gI
Channel: AI Engineer (https://www.youtube.com/@aiDotEngineer). Summarised by AIE Talks.
Page: https://aietalks.com/talks/agent-output-is-not-ux-rendering-layer-your-llm-pipeline-is-missing
Published: 2026-07-20
Tags: latency, multimodal, product-strategy, structured-outputs

## TL;DR
- A correct model response can still produce a poor product experience if users must turn the response into actions themselves.
- A typed, versioned rendering contract lets the model choose from supported UI components while allowing mobile clients to fall back safely.
- Streaming structured UI and a Backend-for-Frontend layer help teams show useful progress, preserve context, and keep model unpredictability away from clients.

## Summary
Bala Ramdoss argues that model output is only the raw material for an AI product experience. A useful feature needs a rendering layer between the model and the interface. He describes three patterns for building that layer. A rendering contract gives the model version-aware client capabilities and a fixed set of components to choose from. Streaming lets the client progressively render structured elements instead of waiting for a complete response, which changes the main latency measure from total response time to time to first useful content. A Backend-for-Frontend absorbs platform rules, hydrates components with actions and metrics, and carries conversational context across turns. This approach matters especially for mobile apps, where old clients may run for days or weeks and cannot be patched immediately. Ramdoss recommends reusing familiar components from existing apps so agentic features retain the product's native look and behavior.

## Key ideas
### The layer between model output and interaction decides whether an AI feature is usable
[00:01](https://www.youtube.com/watch?v=maTp79FD9gI&t=1s)
Ramdoss opens with a restaurant reservation example. The assistant provides the phone number, hours, and information about the walk-in oyster bar, so its answer is factually useful. The user still has to research the restaurant and complete the booking themselves. A better experience turns the same information into a date selector, a time, and a few taps. The agent already has tools that can support the task. The missing part is the layer between the model and the thing a person can interact with. Ramdoss says models and agents are already here, so product teams need to make their output friendly to humans.

### Generative UI gives teams a name for the delivery work they used to build from scratch
[03:04](https://www.youtube.com/watch?v=maTp79FD9gI&t=184s)
Ramdoss says the problems around agentic UX are delivery problems. Teams must decide whether an experience feels fast, whether information arrives all at once or progressively, and how the feature works across mobile versions and device capabilities. He used to solve this part separately for each AI feature because there was no shared vocabulary. He now calls the layer generative UI and points to Google's open A2UI specification. Instead of returning raw text or HTML, the agent describes UI as data, such as a list of components, and the client renders those components with native widgets.

### Production systems usually keep the model inside a controlled component catalog
[03:57](https://www.youtube.com/watch?v=maTp79FD9gI&t=237s)
Ramdoss describes generative UI as a spectrum. At the controlled end, the model selects a pre-built component such as a product card. In the declarative middle, it composes a screen from a catalog containing elements such as a date field, time field, and submit button. At the open-ended end, it generates a novel interface on the fly, as in MCP apps. The more freedom the model has, the more the client must trust its output. Ramdoss says most production mobile apps stay in the first two levels because they are safer. The talk focuses on those controlled patterns.

### Mobile clients need version-aware rendering and safe fallback because teams cannot patch every install
[05:17](https://www.youtube.com/watch?v=maTp79FD9gI&t=317s)
Web clients can receive a rendering fix soon after a breakage is found. Mobile apps have hundreds of millions of installs, and teams do not control when every device updates. If an old client receives a content type it has never seen and has no graceful fallback, it can crash for days or weeks. Ramdoss states the rule plainly: teams cannot meaningfully patch the client. His pipeline therefore includes version-aware context, typed UI intent from the model, a Backend-for-Frontend, and a client renderer that falls back safely when it cannot handle an element.

### A rendering contract tells the model which UI a particular client can support
[06:32](https://www.youtube.com/watch?v=maTp79FD9gI&t=392s)
The contract makes the model aware of client capabilities. A version map records which components exist in each client release. If a flight card arrives in version 2.0, the system exposes it to the model only when it builds context for a version 2.0 client or later. The model should return typed UI blocks rather than leaving the client to infer an interface from token output. The contract can also encode layout rules. Ramdoss gives the example of choosing a swipeable carousel for one to three flights and a vertical list for four or more. The model selects intent from a fixed menu and never invents a component.

### Streaming structured components makes waiting feel different from waiting for a complete answer
[08:46](https://www.youtube.com/watch?v=maTp79FD9gI&t=526s)
Traditional applications wait for an API response before rendering. Model calls often take longer because of generation and additional safety checks. Streaming lets the client render chunks as they arrive. Ramdoss describes a product card that first appears as a skeleton, then fills partially, and finally completes after three to four seconds. The experience changes the metric teams should watch. Instead of focusing only on total latency, they should measure time to first chunk, meaning the first useful thing the user sees. A loading spinner is often a poor fit for AI features because it says nothing about what the system is doing.

### The interface can keep users oriented while the model is still working
[10:03](https://www.youtube.com/watch?v=maTp79FD9gI&t=603s)
Ramdoss gives examples of designing around a long response. Lens Live lets users focus on different objects and tap the one that interests them while they wait for results. When the product does not control the full screen, it can show a thinking experience, although he recommends using that sparingly. He also points to Gemini, which gives users a glimpse of the work an agent is doing. A ten-second task can feel acceptable when users understand what is happening and can trust the final result. Streaming therefore carries both UI content and meaningful intermediate states.

### A Backend-for-Frontend turns streamed output into native components with actions and context
[11:43](https://www.youtube.com/watch?v=maTp79FD9gI&t=703s)
The Backend-for-Frontend decides how streamed chunks become useful UI elements. It owns platform-specific rules, such as differences between Android and iOS, and reduces the number of decisions made by the client. It also handles more than layout. The BFF hydrates elements with action payloads for taps and deep links, and it can name the impression metric to log. It carries conversational context across turns so the next response knows what came before. Teams can reuse existing flight rows, product cards, and other production components. The result keeps the same brand, density, and familiar native feel.

## Notable quotes
- "The only thing that you need to focus on is the layer between the model and something that a human can interact with." (00:51)
- "These are delivery problems and they live in between the model output and what's on the screen." (02:44)
- "You cannot meaningfully patch the client." (05:17)
- "You start chasing time to first chunk. You know, the first useful thing that your user sees." (09:37)
- "Your agent output is not the CX. You build on it." (13:33)

## Tools & references mentioned
- Amazon Lens
- Google
- A2UI
- Co-pilot Kit
- MCP apps
- Gemini
- Android
- iOS

## Who should watch
- You are shipping an agentic feature in a mobile app and need to support old client versions without immediate updates.
- Your model responses are accurate, but users still have to translate them into actions or wait through an opaque loading state.
- You want to reuse existing product cards and interaction patterns while adding structured, conversational AI features.