Semantic item IDs use content and behavior together to reduce cold-start problems and improve recommendations for new items.
2
LLMs can create labels and synthetic data for search and recommendation systems, while smaller distilled models handle production filtering.
3
Unified models can replace separate ranking systems when their inputs and tasks can be aligned, reducing operational cost without giving up task performance.
Summary
Eugene Yan presents three ways to combine language modeling with recommendation and search systems: semantic IDs, LLM-augmented data, and unified models. Semantic IDs add content information to item representations, which helps with cold-start items and sparse interaction histories. Examples from Kuaishou, Indeed, Spotify, Netflix, and Etsy show how these ideas work in production systems. Yan describes Indeed's use of GPT-4 to label poor job recommendations, followed by distillation into a lightweight real-time classifier. Spotify used LLMs to generate natural-language query recommendations that helped users discover podcasts and audiobooks. Netflix unified several ranking tasks in one contextual ranker, while Etsy combined query, product, user, and quality signals in a shared embedding system. Yan is also clear about the limits: a single model may need to become two or three unified models when improvements for one task hurt another.
Semantic IDs give new items useful content information before they have much behavior
Hash-based item IDs do not encode an item's content, so a new item starts with no learned history. This creates cold-start and sparsity problems, especially for long-tail items with only a few interactions. Yan describes Kuaishou's trainable multimodal semantic IDs, which combine visual, video-description, and audio embeddings with behavioral signals. The system uses ResNet for visual features, BERT for descriptions, and VGGish for audio, then clusters concatenated content embeddings into trainable cluster IDs. The model learns to map content space into behavioral space. Yan says the semantic IDs improved cold-start coverage by 3.6% and cold-start velocity by an unspecified reported amount, while also outperforming regular hash-based IDs on clicks and likes.
LLM-generated labels can turn sparse negative feedback into a useful production filter
Indeed had a problem with poor job recommendations sent by email. Users rarely gave explicit negative feedback, while a lack of action was ambiguous. The team first used human experts to label user and job pairs, then tested language models. Mistral and Llama 2 performed poorly, while GPT-4 reached about 90% precision and recall but took 32 seconds and cost too much. GPT-3.5 was cheaper but incorrectly removed too many good recommendations. Fine-tuning GPT-3.5 produced the desired precision at about one quarter of GPT-4's cost and latency, but it was still too slow for online filtering. Indeed distilled those labels into a lightweight classifier that reached 0.86 AUROC and was fast enough for real-time use.
Recommendation quality can improve user outcomes even when the system sends fewer recommendations
Indeed's lightweight classifier reduced bad recommendations by about 20%. The team had expected that showing fewer recommendations might reduce applications, but application rate increased by 4% and unsubscribe rate fell by 5%. Yan uses this example to argue that recommendation volume is not enough. Users had lost trust in the email recommendations, and unsubscribing was difficult to reverse. Better filtering improved the recommendations that remained. The case also shows why an LLM does not need to run in the online path. A larger model can provide training labels, while a smaller model handles the latency requirements of production.
LLMs can extend existing search-query generation methods instead of replacing them
Spotify needed to help users discover podcasts and audiobooks after building a service known mainly for songs and artists. The company generated candidate queries from catalog titles, playlist titles, and search logs using conventional methods. It then used an LLM to generate natural-language queries and ranked them alongside immediate search results. This produced query recommendations that could inform users about new content categories without a separate banner. Yan says Spotify saw a 9% increase in exploratory queries. His recommendation is to keep conventional techniques that already work and use an LLM where it adds coverage, such as tail queries and new categories.
A unified ranker can match specialized models while reducing duplicated systems
Search, ads, and different recommendation surfaces often have separate models and engineering pipelines. Yan describes Netflix's Unified Contextual Ranker, or Unicorn, which handles search, pre-query recommendations, similar-item recommendations, and similar-video recommendations. The model takes shared inputs including user ID, item ID, search query when present, country, and task. Netflix also imputes missing inputs. For an item-to-item recommendation with no search query, it can use the current item's title to find similar items. Yan says the unified model matched or exceeded the specialized models on multiple tasks. The benefit is a common foundation for future changes, although the work may still need to be split across two or three aligned unified models.
Shared query and product embeddings can combine relevance, personalization, and product quality
Etsy wanted better results for both very specific and broad searches, including queries such as "mother's day gift" that may not appear literally in product titles. Its unified embedding and retrieval system uses T5 for text and product-description embeddings, query-product logs for query embeddings, and shared representations for text tokens, product categories, and user location. User preferences come from prior queries and purchases. Etsy also adds a quality vector containing signals such as ratings, freshness, and conversion rate. Yan says the system increased conversion across the site by 2.6% and increased search purchases by more than 5%. The example shows how retrieval can account for user preferences and item quality alongside lexical relevance.
"Semantic IDs not only outperform regular hash-based IDs on clicks and likes, but they were able to increase cold-start coverage by 3.6%."04:21
Who should watch
You build recommendation or search systems and are dealing with cold-start items, sparse feedback, or long-tail queries.
Your team uses separate models and pipelines for different recommendation surfaces and wants to assess whether shared inputs could support a unified ranker.
You need to use LLMs for labeling or data generation but still meet the cost and latency requirements of an online system.
