# The Era of Unbounded Products: Designing for Multimodal IO

Ben Hylak, Dawn | AI Engineer World's Fair 2024 | 20:32

Source: https://www.youtube.com/watch?v=5nOLb27hQ5w
Channel: AI Engineer (https://www.youtube.com/@aiDotEngineer). Summarised by AIE Talks.
Page: https://aietalks.com/talks/the-era-of-unbounded-products-designing-for-multimodal-io
Published: 2024-09-25
Tags: design, multimodal, product-strategy

## TL;DR
- AI products accept many forms of input, which makes them powerful but difficult for users to understand.
- Unbounded products become clearer when they foreground what matters, establish hierarchy, and use familiar patterns.
- Future interfaces will reduce prompt engineering through ranked presets, personalization, and analytics based on user needs.

## Summary
Ben Hylak argues that AI products are becoming unbounded because users can type, talk, show images, share video, and express intentions such as bargaining or confiding. This flexibility creates a product design problem: users often assume an app can do something, fail once, and conclude that it cannot do anything useful. Drawing on his work on Apple Vision Pro and AI products, Hylak explains how structure can make open-ended systems easier to understand. Vision OS used a clear home screen, windows, hierarchy, and familiar conventions. AI products can pull structure out of chat through timelines, sources, artifacts, versions, projects, spreadsheets, examples, and presets. Looking ahead, he expects less prompt engineering, more granular controls, ranked and searchable presets, developer-defined personalization, and a shift from fixed evaluations toward analytics that show whether each user is getting the result they need.

## Key ideas
### AI's many input modes make products powerful and unpredictable
[01:43](https://www.youtube.com/watch?v=5nOLb27hQ5w&t=103s)
Hylak says AI products are less bounded because people can type, talk, show images, share video, plead, bargain, or confide. That freedom creates confusion. Users may assume a product can do something, try it, fail, and then walk away believing the product cannot do capabilities it actually has. People outside technical communities often learn through word of mouth, such as hearing that ChatGPT helped with travel planning. They may not keep experimenting after a failed first attempt, so the product has to communicate its abilities more clearly.

### Unbounded products need structure to give users clarity
[06:30](https://www.youtube.com/watch?v=5nOLb27hQ5w&t=390s)
Hylak describes unbounded products as systems defined by an endless series of "what if" situations. Vision Pro designers had to consider people moving from a living room to a bedroom, lying down, sitting on a plane, sitting beside a friend, or using the product with a disability. AI builders face similar uncertainty with different inputs and evaluations. Without structure, the product becomes a blank slate with too many possible actions. Hylak says designers need to add structure because structure creates clarity.

### Vision OS used hierarchy and familiar patterns to make a new device understandable
[07:07](https://www.youtube.com/watch?v=5nOLb27hQ5w&t=427s)
The Vision OS home screen put apps, people, and environments in front of users because the team judged them to be central to the experience. Its hierarchy began with the home menu, continued through movable and resizable windows, and allowed an individual window to become full screen. Hylak also stresses familiarity. The TV app looked like the TV app people already knew from tvOS, and familiar controls gave users signs of home in an unfamiliar environment. His three design moves were to foreground what matters, establish hierarchy, and use familiar patterns.

### AI products become clearer when they pull structure out of chat
[10:17](https://www.youtube.com/watch?v=5nOLb27hQ5w&t=617s)
Hylak gives Dot and Perplexity as examples of products that shape an open-ended interaction. Dot lets users pinch out to see journal days and tap people to view structured information and a timeline of mentions. Perplexity presents the query as a title, shows sources, and places the answer below, making the experience feel more like search than an ongoing chat. He criticizes ephemeral controls embedded in a chat flow because they drift away as follow-up messages arrive. His preferred approach is to move the structured work area beside the conversation, as Claude does with Artifacts.

### Versions, projects, and spreadsheets make unfamiliar AI behavior easier to grasp
[12:58](https://www.youtube.com/watch?v=5nOLb27hQ5w&t=778s)
Hylak points to version control as a useful pattern for AI products. ChatGPT's message editing lets users move between versions, while v0 makes iteration feel familiar because users can return to earlier interface states without losing work. Projects also provide a recognizable structure for sharing context across tasks. Agents are harder for most people to understand, but spreadsheets give them a familiar shape. In Clay, each column represents an agent step, rows accumulate context, and users can run a small set of rows before scaling to a much larger set.

### Examples and presets reduce the blank-page problem
[15:08](https://www.youtube.com/watch?v=5nOLb27hQ5w&t=908s)
Examples and presets help users understand what an AI app is for without requiring prompt hacking or prompt engineering. ChatGPT offered suggestions such as messaging a friend or planning a relaxing day. v0 added an Explore page where users could see what others were doing, and Notion provided controls for changing the tone of text. Hylak says these patterns use ideas that the product can validate instead of forcing users to invent instructions such as a long role prompt.

### Future interfaces will replace broad prompts with ranked personalization
[16:27](https://www.youtube.com/watch?v=5nOLb27hQ5w&t=987s)
Hylak expects less prompt engineering as interfaces expose more direct controls, such as mixing image concepts or adjusting text between professional, casual, expanded, and concise. Broad labels are still reductive because casual means different things to different companies, brands, and relationships. He proposes many more granular presets, potentially informed by sparse autoencoders that isolate controllable features. Presets could be ranked, personalized, searchable, and invoked through natural language. Developers could tune these features per user rather than relying on fragile text prompts.

### AI products will increasingly measure user fit through analytics
[19:17](https://www.youtube.com/watch?v=5nOLb27hQ5w&t=1157s)
Hylak expects AI teams to move from fixed evaluations toward analytics. Some questions have an objectively correct answer, such as identifying the first president. Other outputs, including the right tone for a summary, depend on the individual user. For those cases, he says product teams need to understand whether they are meeting users' needs and responding to what users ask for. This changes the product goal from finding one universally correct output to tuning the experience for different people.

## Notable quotes
- "Without structure you just have chaos." (06:50)
- "Structure is what creates Clarity." (06:50)
- "I think the answer is you pull it out." (12:20)
- "The future has a lot less prompt engineering." (16:27)
- "I think that increasingly it's going to be about how do you understand if you're meeting the needs of your users and what they're asking for." (19:54)

## Tools & references mentioned
- Dawn
- GitHub
- Apple Vision Pro
- Vision OS
- ChatGPT
- ChatGPT memory
- Dot
- Perplexity
- Vercel
- Claude
- Artifacts
- v0
- Google Slides
- Clay
- Notion
- sparse autoencoders
- Golden Gate Claude
- Figma
- Llama

## Who should watch
- You are designing an AI product where users can provide several kinds of input and need help understanding what the product can do.
- Your interface puts controls or generated artifacts inside a chat stream, and follow-up messages make those controls hard to find.
- You are building agents, presets, or personalization and need patterns that make these features more familiar to users.

## Related talks

- [The Intelligent Interface](https://aietalks.com/talks/the-intelligent-interface) (Samantha Whitmore & Jason Yuan, New Computer, 18:56)
- [Keynotes & Multimodality Track](https://aietalks.com/talks/keynotes-multimodality-track) (Benjamin Duny & Simon Sturmer, AI Engineer & Chris Lattner, Modular & Auntie Aart & Mike Chambers, AWS & Alex Albert, Anthropic & Harrison Chase, LangChain & Rob Chandra, Substrate & Vic Kapati, M87 Labs & Ben Hilac, Dawn & Karan Goel, Cartesia & Chang She, LanceDB & Noah Schnapp, Character.AI & Steph Duenas & Quinn Dombrowski, Daily & Roman Huet, OpenAI & Brian Bischof, Charles Fry, Hamza Hussein, Jason Liu, Shreya Shankar & Eugene Yan & Thomas Dohmke, GitHub, 8:34:14)
- [Designing AI To Scale Human Thought](https://aietalks.com/talks/designing-ai-to-scale-human-thought) (Jun Yu Tan, Tusk, 12:24)
- [Climbing the Ladder of Abstraction](https://aietalks.com/talks/climbing-the-ladder-of-abstraction) (Amelia Wattenberger, Adept.ai, 16:47)
- [Everything is ugly, so go build something that isn't](https://aietalks.com/talks/everything-is-ugly-so-go-build-something-that-isnt) (Raiza Martin, Huxe, 25:15)