Recommendation systems may become a larger consumer application of LLMs than search because feeds drive engagement across large consumer products.
2
YouTube adapted Gemini with Semantic IDs that turn videos into tokens in a domain-specific language, allowing the model to reason over watch sequences and video concepts.
3
The model produced useful recommendations for difficult cases, but YouTube had to reduce serving costs by more than 95% and use smaller checkpoints to meet its scale requirements.
Summary
Devansh Tandon describes YouTube's effort to adapt Gemini into a large recommender model, or LRM. The system represents every video with a Semantic ID derived from its title, description, transcript, audio, and video frames. These IDs give YouTube videos a language-like structure that Gemini can learn alongside English. Continued pre-training teaches the model to connect text with videos and predict missing items in user watch sequences. YouTube then uses prompts containing user context and watch history to generate candidate recommendations. The approach works especially well for difficult cases and fresh, less-known content, but the model is expensive to serve at YouTube's scale. The team reduced TPU serving costs by more than 95%, and also used offline inference tables and smaller Gemini checkpoints. Tandon expects future systems to let users steer recommendations in natural language, explain recommendations, and eventually combine recommendation with generated content.
Recommendations may have more consumer impact than LLM search
Tandon argues that recommendations are an underhyped LLM application because users interact with them constantly without seeing the underlying system. YouTube recommendations appear on Home, Watch Next, Shorts, and parts of Search, and they drive a large majority of watch time. The general problem takes a user and their context as input, then returns videos. That context can include demographics, location, the last 100 videos watched, engagement depth, comments, and subscriptions. Tandon says the same reasoning applies to other consumer applications with a billion or more daily users, where feeds may affect more engagement than a search interface.
YouTube built a Gemini-based large recommender model
YouTube started asking how its recommendation stack could be rebuilt on top of Gemini. The resulting system is called LRM, for large recommender model. The team begins with a Gemini checkpoint, teaches it information about YouTube, and creates a unified YouTube-specific checkpoint. That checkpoint can then be aligned for tasks such as retrieval and ranking. YouTube has launched the model in production for parts of retrieval and is experimenting with it for ranking. Tandon presents this as an adaptation of a general-purpose model into smaller custom versions for different recommendation surfaces.
Semantic IDs give videos a language-like representation
An LLM needs tokens, so YouTube needed a compact representation for billions of videos. Semantic ID extracts features from a video's title, description, transcript, audio, and video frames, combines them into a multidimensional embedding, and quantizes the result with RQ-VAE. Tandon describes the resulting IDs as the atomic units of a new language for YouTube videos. Related videos can share prefix tokens, such as videos about sports and volleyball, while still receiving unique identifiers. This replaces hash-based tokenization with tokens that carry semantic structure. YouTube uses Semantic ID in production, and the work was described in the paper "Better Generalization with Semantic IDs: A Case Study in Ranking for Recommendations."
Continued pre-training teaches Gemini to connect English with videos
The team adapts Gemini in two broad stages. First, it links text and Semantic IDs. A video with a Semantic ID can be paired with its title, creator, or topics, and the model learns to connect those descriptions with the video token. Second, it trains on user watch paths. A prompt can contain a sequence of watched videos with some items masked, and the model learns to predict the missing items from engagement patterns. After these tasks, the checkpoint can reason across English and YouTube's video language. For example, it can infer that a fourth video interests technology fans because it concerns AI, based on the Semantic ID representation.
Generative retrieval finds unusual connections in user history
For generative retrieval, YouTube constructs a prompt containing user demographics, device and location context, the current video, and past watch history. Gemini then decodes recommended videos as Semantic IDs. In an example involving an Olympics highlight, an older production system returned more men's track races. LRM found related women's races by combining the user's demographic information with their previous viewing. Tandon says the approach is especially useful for users with limited known history and for difficult recommendation tasks. The model has been experimented with and launched in several YouTube locations, including retrieval use cases.
LRM learns quickly and uses training data efficiently, but its serving cost was too high for billions of users. YouTube reduced TPU serving costs by more than 95% before launching the approach in production. The team also removed personalized information from the prompt and ran the same model offline to create a table of candidate videos for each watched video. For the head of YouTube's corpus, the system can perform inference offline and serve recommendations through a simple lookup. This avoids the cost of running a personalized generative model for every request while retaining some of the model's differentiated recommendations.
Video recommendation has harder freshness and scale constraints than language modeling
Tandon contrasts a language model's vocabulary with YouTube's corpus. An English model may have roughly 100,000 dictionary words, with about 1,000 added each year. YouTube has 20 billion videos, millions of new uploads every day, and a strong need to understand fresh content. If a major artist releases a video, recommending it within minutes or hours matters. LRM therefore needs continuous pre-training on the order of days and hours, rather than the three-to-six-month intervals Tandon associates with classical Gemini pre-training. YouTube also has to use smaller and more efficient checkpoints, such as Gemini Flash and models below it, because larger models cannot meet its latency and user-scale requirements.
The recipe is a domain language, bilingual adaptation, and prompts
Tandon reduces the project to three steps that other recommendation teams could adapt. First, represent the application's content with rich features, create embeddings, and quantize them into atomic tokens. This creates a domain-specific language. Second, adapt an LLM with training tasks that connect natural language to those tokens and let the model reason across both. The result is a bilingual model that speaks English and the domain language. Third, construct personalized prompts from user demographics, activity, and actions, then train task-specific or surface-specific models. The final system generates recommendations through an LLM rather than relying only on traditional retrieval and ranking components.
Future systems may let users steer recommendations directly
Tandon says current LLM-based recommendations mostly improve a feed invisibly. He expects systems to let users express goals in natural language, steer the recommendations toward those goals, and receive explanations for why candidates were selected. In that setting, search and recommendation begin to overlap. He also expects recommendation to combine with generated content. A system might recommend a personalized version of existing content, and later generate content specifically for one user. He describes that direction as further away than the current work, while saying progress in AI could bring it sooner than expected.
"The biggest limitation was that the serving costs are too high, especially for the scale that YouTube operates at with billions of users."10:03
Who should watch
You are building recommendations or feeds for a large consumer product and want to understand where a foundation model can fit into retrieval and ranking.
You need to represent fast-changing domain content as tokens that an LLM can reason over alongside natural language.
You are weighing generative retrieval against serving cost, latency, freshness, and the need for offline candidate generation.
