# Let's Go Bananas with GenMedia

Guillaume Vernade, Google DeepMind | AI Engineer Europe 2026 | 1:17:14

Source: https://www.youtube.com/watch?v=BcWFc3H7Khg
Channel: AI Engineer (https://www.youtube.com/@aiDotEngineer). Summarised by AIE Talks.
Page: https://aietalks.com/talks/lets-go-bananas-with-genmedia
Published: 2026-05-18
Tags: long-context, multimodal, prompt-engineering, structured-outputs

## TL;DR
- Gemini can read an entire book and create prompts for character portraits, chapter illustrations, video scenes, music, and dialogue.
- Interactions API keeps conversation context on the server, which avoids resending a full book and makes chained calls cheaper.
- Lyria Realtime generates music continuously and changes it when new prompts arrive, while TTS can make one voice sound like several characters through speaking-style instructions.

## Summary
Guillaume Vernade demonstrates a complete generative media workflow built around The Wind in the Willows. Gemini reads the book, extracts characters and chapters, and writes structured prompts. Imagen, referred to in the demo through the Nano Banana image model, creates character portraits and chapter illustrations. Veo uses those images as first frames for animated clips. Lyria creates music for each chapter, with prompts controlling instruments, structure, timing, lyrics, and language. Vernade also shows a text-to-speech technique that assigns different speaking styles to characters while using the same voice. The talk explains why Gemini is effective at writing prompts for other Google media models, since those models were trained partly on Gemini-written prompts. It also introduces Interactions API, which stores context server-side and supports branching conversations, plus Lyria Realtime, which changes an ongoing music stream in response to new prompts.

## Key ideas
### Google's media models are separate releases on the path toward one multimodal world model
[02:03](https://www.youtube.com/watch?v=BcWFc3H7Khg&t=123s)
Vernade describes DeepMind's long-term goal as a model that can understand many kinds of input, including sound, video, audio, sensors, and other modalities, then communicate through text, audio, and more. Google currently ships separate image, video, and music models because updating one large model could break unrelated capabilities. He says Gemini models have always been intended to be multimodal, although some early releases had image and music understanding disabled or incomplete. The wider set of Google systems includes Gemini, image and video generation, music generation, robotics, agents, open models, and research models such as AlphaGo and AlphaGenome.

### The workshop turns a public-domain book into a chain of generated media
[11:20](https://www.youtube.com/watch?v=BcWFc3H7Khg&t=680s)
The notebook uses Kenneth Grahame's The Wind in the Willows from Project Gutenberg. Gemini receives the whole book through file upload and a large context window. It then produces structured JSON containing a name and prompt for each character and chapter. The workflow generates portraits, chapter illustrations, animated clips, music, and spoken dialogue. Vernade says the accompanying Cookbook GitHub repository contains quick starts and larger examples that combine several model capabilities. The notebook includes limits because books can contain many chapters and characters, and the live run costs about one dollar when video generation is included.

### Gemini's context and structured output make image consistency easier to manage
[19:09](https://www.youtube.com/watch?v=BcWFc3H7Khg&t=1149s)
Vernade uses chat mode so Gemini retains earlier messages, including the book, the chosen visual style, and previous generated images. He asks for a colorful building-block style and adds instructions to avoid book-cover layouts and panelled compositions. For chapter images, he first relies on conversation history to remember the characters. He then shows a more scalable approach: Gemini returns the characters appearing in each chapter along with the chapter prompt. Code searches the saved character images and passes only the relevant references to image generation. Vernade says a production version would create several views for each character, such as portraits, full-body images, and side or rear views.

### Veo works better when Gemini writes a separate prompt for the action after the first frame
[34:48](https://www.youtube.com/watch?v=BcWFc3H7Khg&t=2088s)
Veo uses a chapter illustration as the first frame of a video and receives a prompt describing the scene. Vernade explains that image generation is an important part of video training because the model needs a strong starting frame. In one result, the wrong character speaks because the same prompt was reused for the image, video, and audio. He improves the process by asking Gemini to review the starting image and describe what should happen in the next few seconds. The generated prompt says the Water Rat lowers his pistols, reassures the Mole, and walks away with him. The extra action description produces a better result, although the clip still lacks dialogue.

### Lyria puts musical structure and control inside the prompt
[40:07](https://www.youtube.com/watch?v=BcWFc3H7Khg&t=2407s)
Vernade uses Gemini to write a different Lyria prompt for each chapter, preserving some consistency while giving each scene its own musical character. Lyria can generate short clips or longer songs. Its controls are expressed in natural language rather than separate parameters. A prompt can specify duration, scale, BPM, instruments, an intro, verses, a chorus, an outro, or changes at particular points in the song. Lyrics can be supplied directly or invented by the model. The output includes lyrics and timing information, which Vernade says could support a karaoke application. He also demonstrates songs in different languages and prompts that combine more than one language.

### Interactions API caches context and supports branches from one conversation
[27:23](https://www.youtube.com/watch?v=BcWFc3H7Khg&t=1643s)
Vernade explains that chat mode resends the conversation history on every request, which becomes expensive and slow when the history contains an entire book. The Interactions API returns an interaction ID that can be reused in later calls. The server retrieves the earlier context, so the client does not upload the book repeatedly. Vernade says this also makes it easier to fork a workflow. A developer could generate lyrics, then branch from that context to create an image on one path and a song on another. The API automatically caches context that will be reused, reducing the cost of chained calls.

### One TTS voice can sound like several characters when the script carries speaking styles
[48:00](https://www.youtube.com/watch?v=BcWFc3H7Khg&t=2880s)
Vernade asks Gemini to extract a passage from the book and format it as a play. The script labels the narrator and characters, then places a speaking style in parentheses for each line. The narrator uses one voice and all characters use another voice, but the style instructions differ. One character has long poetic pauses, another is breathless and boastful, and the model changes delivery accordingly. The resulting audio makes the characters sound distinct even though the same character voice is reused. Vernade also says prompts can request whispering, emotion, or accents. He found that the TTS model needs an instruction such as "read this text" before the script, or it may not interpret the input correctly.

### Lyria Realtime behaves like a DJ that changes an ongoing stream
[56:01](https://www.youtube.com/watch?v=BcWFc3H7Khg&t=3361s)
Lyria Realtime continuously generates music until it is stopped. New prompts can arrive during playback, and the model shifts toward the requested sound in roughly real time. Vernade changes a stream by asking for more K-pop, more drums, and then a more relaxed style. He imagines using it in games, where the music could respond to the player's region, activity, or health. He also describes a space-themed example in which each planet has a prompt and moving between planets changes the music. The session shown in the demo ends after ten minutes.

## Notable quotes
- "The goal is really to have like one model that encompass all of that." (03:52)
- "The longer your prompt, the more interesting it's going to be." (42:00)
- "You can use this trick to actually create multiple characters, multiple voices for those characters." (52:59)
- "That's my favorite model because I think it's underused." (57:20)

## Tools & references mentioned
- Google DeepMind
- Gemini
- Stadia
- Nano Banana
- Nano Banana 2
- Imagen
- Veo 3.1
- Veo 3.1 Lite
- Lyria
- Lyria Realtime
- Gemma 4
- AlphaGo
- AlphaGenome
- Project Gutenberg
- The Wind in the Willows
- Kenneth Grahame
- The Adventures of Chatterer the Red Squirrel
- AI Studio
- Vertex AI
- Gemini Developer API
- Interactions API
- NotebookLM
- Google Cloud
- Colab

## Who should watch
- You are building a pipeline that combines text understanding with image, video, music, and speech generation, and want a concrete end-to-end example.
- You need to keep a long source document available across many model calls without uploading it repeatedly.
- You are experimenting with generative music or synthetic dialogue and want prompt-based control over structure, timing, language, or character delivery.

## Related talks

- [From Transcription to Live Music: Gemini's Audio Stack](https://aietalks.com/talks/from-transcription-to-live-music-geminis-audio-stack) (Thor Schaeff, Google DeepMind, 19:34)
- [Any-to-Any: Building Native Multimodal Agents](https://aietalks.com/talks/any-to-any-building-native-multimodal-agents) (Patrick Löber, Google DeepMind, 16:21)
- [Building Conversational Agents](https://aietalks.com/talks/building-conversational-agents) (Thor Schaeff & Philipp Schmid, Google DeepMind, 1:47:34)
- [Prompt to Pipeline: Building with Google's Gen Media Stack](https://aietalks.com/talks/prompt-to-pipeline-building-with-googles-gen-media-stack) (Paige Bailey, Guillaume Vernade & Ian Valentine, Google DeepMind, 1:54:35)
- [Building in the Gemini Era](https://aietalks.com/talks/building-in-the-gemini-era) (Kat Kampf & Ammaar Reshi, Google, 17:57)