Instacart uses LLMs with conversion data and taxonomy context to improve query understanding, especially for long-tail queries.
2
LLM-generated rewrites help users get results when retailer catalogs do not match the exact query they entered.
3
Search pages can generate complementary and substitute shopping content, but the generation needs Instacart-specific behavior data, ranking, and evaluation.
Summary
Instacart's search system has to handle routine grocery restocking alongside new product discovery. Vinesh Gudla and Tejaswi Tenneti describe how LLMs help with long-tail queries, category classification, query rewrites, and discovery content. A general-purpose model often produced sensible answers that did not match Instacart users. For example, it interpreted "protein" as foods such as chicken and tofu, while users often meant protein bars or shakes. Instacart improved the results by adding top-converting categories, query annotations, and subsequent-search data to the prompts. The team also precomputes most LLM outputs from search logs, caches them, and serves them through a feature store rather than calling a model during a live search. The talk is candid about the remaining engineering work: ranking generated content, aligning it with revenue, preventing hallucinations, and building automated evaluation. The same approach supports substitute products and complementary content on the results page.
Grocery search has to find planned items and support discovery
Customers often search for many items in one shopping trip, with most queries covering products they have bought before. Search therefore needs to retrieve a known product quickly while also helping people discover something new. Discovery benefits customers, advertisers, and Instacart because it can expose new products and encourage larger baskets. The team compares this with walking through a pasta aisle, where a shopper may notice another pasta brand, sauce, or related ingredient. Customers told Instacart that search helped them find the original product but left them with no useful next step for related items.
Broad and rare queries expose the limits of engagement-trained models
Broad queries such as "snacks" map to many products, but products that have not been shown cannot collect the engagement data needed to rank higher. Very specific queries such as "unsweetened plantbased yogurt" happen less often, so the system also lacks enough data to train on them. Instacart had improved recall, but precision remained difficult, especially for long-tail queries. The same lack of data affected product discovery, where traditional methods struggled to infer useful related items.
LLM category classification works better with Instacart conversion context
Instacart maps queries to multiple labels in a taxonomy with about 10,000 categories, of which about 6,000 are commonly used. Earlier fastText-based neural models and an NPMI fallback worked well for head and torso queries but had low tail coverage. A direct LLM approach produced plausible categories but misunderstood user behavior. For "protein," it suggested chicken, tofu, and other protein foods, while Instacart users often wanted protein shakes, bars, or supplements. The team added the top converting categories for each query, generated candidates, ranked them, and validated them downstream. For "Werner soda," the LLM identified ginger ale more precisely than the previous model. Tail-query precision improved by 18 percentage points and recall by 70 percentage points.
Query rewrites recover results across uneven retailer catalogs
Retailers have catalogs of different sizes, so the same query may return results for one retailer and nothing for another. A rewrite can broaden the request. For example, changing "1% milk" to "milk" gives the customer products to consider rather than an empty result. Instacart used an LLM to produce substitute, broad, and synonymous rewrites. For "avocado oil," examples included olive oil as a substitute, healthy cooking oil as a broader rewrite, and avocado extract as a synonym. The team saw offline improvements from better models and a large reduction in searches with no results online.
Precomputation makes LLM search features practical at serving time
Instacart has a very large head and torso of repeated queries followed by a long tail. The team precomputes LLM outputs for head and torso queries in batch, caches them, and serves them with little latency impact. Existing models handle the long tail, while the team is testing a distilled Llama model for the very long tail. The same serving pattern is used for discovery content: historical search logs are processed in batch, with query content metadata and possible carousel products stored in a feature store. Online search then performs a quick lookup instead of waiting for an LLM call.
Discovery content gives shoppers a next action after the first result
Search results can be useful for the exact query while offering no help with the shopper's next intent. Instacart used LLMs to generate substitute results when exact products were unavailable and complementary content when exact results existed. A "swordfish" search could suggest other seafood or meaty fish such as tilapia. A "sushi" search could add Asian cooking ingredients or Japanese drinks at the bottom of the page. The team reports improvements in engagement and revenue per search, but the first prompts produced common-sense answers that did not match what users wanted. Adding top-converting categories, query annotations, and subsequent-query data made the content more relevant.
Generated content needs its own ranking and evaluation system
The discovery system had to avoid duplicating products already shown and keep generation aligned with Instacart's domain knowledge. The prompt needed to distinguish "dishes" as cookware from "Thanksgiving dishes" as a food-related query. The team also had to align generation with revenue, improve content ranking, and check that outputs did not hallucinate or violate product requirements. Traditional PCTR and PCBR models did not work well for this content, so the team used strategies such as diversity-based ranking. They used an LLM as a judge, and describe evaluation as more difficult and more important than they expected.
A unified query-understanding model could prevent inconsistent decisions
Instacart's query-understanding stack contains separate models for normalization, tagging, classification, and categories. The speakers argue that consolidating these functions into a small or large language model could make results more consistent. They give "humm" as an example: a brand tagger correctly recognized a kombucha brand, but a spell corrector changed the query to "hummus," producing confusing results. A unified model could also pass richer intent context downstream, such as the fact that a query is looking for ingredients for a recipe. In the question period, they add that context matters at every stage, such as preserving "women's perfumes" rather than passing only "perfumes" to retrieval.
"Search has a dual role. It needs to both support quick and efficient find the product they're looking for and also enable this new product discovery."01:28
Who should watch
You are building LLM features for e-commerce search and need to combine general model knowledge with user or marketplace data.
Your search stack has separate query normalization, tagging, classification, and rewrite models that create inconsistent outputs.
You need to serve generated recommendations at low latency and are deciding what to precompute, how to rank it, and how to evaluate it.
