# Books reimagined: AI to create new experiences for things you know Lukasz Gandecki, TheBrain.pro | AI Engineer World's Fair 2025 | 09:44 Source: https://www.youtube.com/watch?v=Kcka7rzcxLk Channel: AI Engineer (https://www.youtube.com/@aiDotEngineer). Summarised by AIE Talks. Page: https://aietalks.com/talks/books-reimagined-ai-to-create-new-experiences-for-things-you-know Published: 2025-07-22 Tags: code-generation, design, embeddings, multimodal ## TL;DR - Lukasz Gandecki built an AI reading companion that adds character context, images, summaries, voice questions, semantic search, and research without spoiling the book. - The product combines generated scenes, graphics, music, and metadata with human review, because AI cannot reliably judge whether the creative output fits the book. - Fast AI-assisted coding helped him explore the product, while the codebase became more conventional only after the large waves of experimentation had settled. ## Summary Lukasz Gandecki describes BookGenius, an AI-assisted reading experience that adds context and media to books. He started with a tool for understanding the people in a book about Donald Trump's re-election, then built richer experiences for The Snow Queen and 1984. Readers can ask questions by voice, find scenes with natural-language search, and ask for research over the part of the book they have read. The system uses language models for scene and character analysis, mood detection, music generation, and structured metadata. Gandecki says the AI should disappear behind the experience. People still need to judge the music, graphics, and avatars. He also explains how he built the project with rapid AI-assisted coding, throwing away generated code more easily than code he had written himself. Once the changes became smaller, he started adding tests and refactoring selectively. The player is being open sourced so others can create similar book experiences. ## Key ideas ### An AI companion can give readers context while they are inside a book [00:15](https://www.youtube.com/watch?v=Kcka7rzcxLk&t=15s) Gandecki began after reading a book about Donald Trump's re-election and struggling to follow all the people mentioned. He built an AI companion that identified people on the page, found images, and gave a summary tied to the current context. The first version looked poor, but it helped him understand what he was reading. That small tool became the basis for a larger reading experience rather than a general chatbot outside the book. ### The reading experience combines narration, animation, music, and scene context [00:59](https://www.youtube.com/watch?v=Kcka7rzcxLk&t=59s) The Snow Queen demo shows a scene in which the sorcerer's apprentices fly away with a mirror that distorts reality. The system tells the story, displays changing visuals, and adds music as the mirror crashes and shatters. Gandecki then shows an English version built for the conference using 1984. His point is that the book can become an experience with media that follows the action, rather than a page with an unrelated AI assistant beside it. ### Voice interaction needs an explicit speaking gesture and a fast response [02:03](https://www.youtube.com/watch?v=Kcka7rzcxLk&t=123s) Readers can send a voice note to the book and ask what is happening in the current scene. Gandecki criticizes common voice assistants for delays, interruptions, and starting at the wrong time. His design has the reader hold a control while speaking and release it when finished. He says the system responds in about 100 milliseconds. The reader can then scroll ahead and ask what happened since the previous question, with the system summarizing the intervening section. ### Embeddings let readers search for remembered scenes without knowing the wording [03:37](https://www.youtube.com/watch?v=Kcka7rzcxLk&t=217s) Exact search fails when a reader remembers an event rather than a phrase or page. Gandecki uses the example of finding the scene where Winston met O'Brien in 1984. Embedding-based search can locate the relevant passage from a natural-language description. The reader can inspect that scene and return to the previous reading position. The system can also research a question about the book by reading everything up to the point the reader has reached, which helps avoid spoilers. ### AI-assisted coding works best as repeated waves of disposable prototypes [04:27](https://www.youtube.com/watch?v=Kcka7rzcxLk&t=267s) Gandecki started with vanilla JavaScript and code produced through rapid AI-assisted experimentation. He argues that planning everything up front is often wasteful because it is faster to describe an idea to the AI, generate something, see its weaknesses, and try again. Throwing away AI-written code feels easier than discarding code he wrote himself. He describes the process as waves of change. As the waves become smaller, conventional engineering practices such as tests and refactoring become more useful. ### Refactoring should account for pain and usefulness, not code quality alone [05:31](https://www.youtube.com/watch?v=Kcka7rzcxLk&t=331s) Gandecki warns against automatically cleaning up the worst-looking code. He gives an example of JavaScript for OpenAI audio processing that has no types and is confusing, but rarely needs to change. He leaves it alone because it is not causing pain. His decision rule considers how bad the code is, how painful it is to live with, and how easy it would be to change. A badly written but isolated and harmless section may be a poor refactoring target. ### The AI should disappear behind human-reviewed creative work [06:26](https://www.youtube.com/watch?v=Kcka7rzcxLk&t=386s) Gandecki rejects experiences that expose AI as a chatbot, image generator, or unreliable voice assistant. In his system, AI creates an initial draft, then people handle the rest. He says humans still need to judge whether generated music is good, whether graphics look right, and whether an avatar matches the person described in the book. The product combines ordinary building blocks, including search, visuals, music, and book context, into an experience where the underlying AI is hidden. ### Structured book metadata makes generated media usable in the player [08:25](https://www.youtube.com/watch?v=Kcka7rzcxLk&t=505s) The production process combines language models for scene analysis and character detection with an overall music theme for each book. For Sherlock Holmes, Gandecki gives Victorian London and noir music as examples, with a sad tone. Scene analysis and mood detection drive music generation. The system also produces structured XML metadata that maps text to characters and other elements, so the player can display the right avatars and media without a person manually annotating every part of the book. He says the player is being open sourced. ## Notable quotes - "You basically don't know what you don't know and if you start to especially right now the time it takes to plan everything up front is often wasted because you can much quicker just tell your thinking to the AI and generate something that works and then you see oh, that's actually not that great." (04:49) - "But throwing away code written by AI actually feels great." (05:07) - "So, you want to make the AI disappear." (07:00) - "Today we are open sourcing the player. So, anyone can create the Netflix style experiences for books." (09:07) ## Tools & references mentioned - BookGenius - The Snow Queen - 1984 - Sherlock Holmes - Donald Trump - Winston - O'Brien - Siri - OpenAI - Google ## Who should watch - You are building a reading, learning, or media product and want to add context without forcing users into a generic chat window. - You want practical guidance on when to move from fast AI-generated prototypes to tests and refactoring. - You are exploring semantic search, generated audio, or structured metadata for long-form content. ## Related talks - [Imagination Engineering: "Live in the future and then build what's missing."](https://aietalks.com/talks/imagination-engineering-live-in-the-future-and-then-build-whats-missing) (Eve Bouffard, Y Combinator, 16:04) - [The Rise of the AI Software Engineer](https://aietalks.com/talks/the-rise-of-the-ai-software-engineer) (Jesse Han, Morph Labs, 03:48) - [The AI Evolution](https://aietalks.com/talks/the-ai-evolution) (Mario Rodriguez, GitHub, 19:32) - [Beyond the Prototype: Using AI to Write High-Quality Code](https://aietalks.com/talks/beyond-the-prototype-using-ai-to-write-high-quality-code) (Josh Albrecht, Imbue, 17:59) - [Using AI to Build an Infinite Game](https://aietalks.com/talks/using-ai-to-build-an-infinite-game) (Jeff Schomay, 11:08)