LinkedIn built 360Brew as one foundation model for ranking and personalization across multiple surfaces and tasks.
2
The team trains a 150-billion-parameter model, then distills it through smaller models because the large model cannot meet online recommendation latency and throughput requirements.
3
Gradual distillation, pruning, mixed-precision quantization, and sparse attention reduced latency by 7x while increasing per-GPU throughput by 30x.
Summary
Hamed Firooz and Maziar Sanjabi describe LinkedIn's effort to build one foundation model for ranking and recommendation instead of maintaining separate models for each task. Their approach converts member profiles, histories, and interactions into prompts, then trains a model through continued pretraining, fine-tuning, instruction fine-tuning, and alignment. The largest version, Blue XL, has 150 billion parameters. It improves cold-start behavior and can perform competitively on tasks outside its training domains, but it is too expensive to serve directly. The team therefore distills it through intermediate models, applies gradual pruning, uses mixed precision, and sparsifies attention. These steps produced a 7x latency reduction and a 30x increase in throughput per GPU across several releases. The speakers are direct about the engineering cost: automation is needed to run experiments, and context selection must balance user freshness, positive and negative examples, and KV-cache constraints.
A single foundation model can replace many separately trained ranking systems
Firooz says recommendation systems are embedded in feeds, job search, shopping, and other everyday products, but they are usually trained for individual tasks. That makes them disjoint, limits the architectures teams can use, and forces organizations to roll out models one at a time. LinkedIn's mission with 360Brew is to build one large language model with a holistic understanding of a member's journey and use it across personalization tasks. The intended model also supports new surfaces without starting a full data-collection and deployment process for each one. The same model can be prompted with a user, an entity, and a task, then asked to judge relevance.
Zero-shot prompting, in-context examples, and instructions target new and cold-start cases
The team wanted 360Brew to work on a new problem or surface without building a new ranking model from scratch. Firooz describes a zero-shot interface where a developer specifies the task, the entity, and the user, then asks for their relevance. In-context learning is aimed at cold-start users. A small number of examples, or a description of what the user may like, can give the model enough information to make recommendations. The system is also intended to accept direct member instructions, such as niche interests or topics the member wants to explore, and use those instructions to find relevant information.
Promptification turns LinkedIn's user data into a language-model input
Sanjabi calls the data preparation step the "magic of promptification." The team takes member history, profile information, and past interactions, then converts them into a prompt. The prompt contains an instruction, member-profile details, and examples of information previously shown to the member. It ends with a question about what the user will do with a new item or piece of information. This format lets the model generalize across instructions rather than treating every recommendation task as a separate model. An off-the-shelf LLM can do some of the work, but the speakers say it needs training to perform well.
The largest model is a teacher for smaller production models
The model-development pipeline begins with an open-source model, followed by upcycling to control model size, throughput, and quality. LinkedIn applies continuous pretraining, fine-tuning, instruction fine-tuning, and alignment before producing Blue XL, a 150-billion-parameter model. That model delivers the best quality in their setup, but recommendation systems need high throughput and low latency, so it cannot directly serve online traffic. The team distills it into smaller models, down to a 3-billion-parameter model that can be productionized. Their experiments show that training a small model from scratch is weaker. The recipe is to go big first and then go small.
Data, model size, and context length each improve quality up to a limit
Sanjabi describes three ways to improve the model. More logged user behavior improves performance, with useful history extending across periods such as six months or a year. Increasing the model size also improves quality in their mixture-of-experts experiment, including a comparison from 7B to 8x22B. Context length matters because it controls how much of a user's history reaches the model. Performance rises as more history is provided, then falls at longer lengths. The team attributes that decline to poor generalization of the model used in the experiment at long context lengths, rather than to the extra context becoming inherently less informative.
360Brew helps most when users have little interaction history
The cold-start comparison measures the gap between 360Brew and production models for users with different amounts of interaction history. The gap grows as the number of interactions decreases, including the range below five interactions and the range below 100 interactions. Firooz attributes the improvement to the world knowledge the model brings into the recommendation system. The model also generalizes to four tasks that were completely outside its training domains. On those tasks, it can match or beat production models trained specifically for each task. The speakers connect this result to faster rollout of new LinkedIn features and surfaces.
Serving requires gradual compression rather than one aggressive reduction
LinkedIn recommendation systems can have tens of thousands of queries per second and require latency around 400 to 500 milliseconds or less. Firooz says the team uses several efficiency levers, while keeping the large teacher model in the training path. Distillation proceeds step by step, for example from 150B to 8B, then 3B, then 1B, with each smaller model distilled from the previous one. Pruning follows the same gradual pattern. The team removes transformer heads or MLP capacity in stages, distills again, and repeats. Aggressive pruning at the beginning can reduce model quality by up to 1 percent, while gradual pruning can avoid measurable information loss in their experiment.
Mixed precision and sparse attention reduce serving cost
The serving stack uses FP8 for activations and model parameters, but the language-model head remains in FP32. Firooz says using lower precision throughout causes the output numbers to collapse and harms calibration, which makes it harder to distinguish recommended items. The team also sparsifies attention because every item does not need to attend to every other item. When scoring many candidate items, the candidates can attend to historical user and profile information without attending to one another. This lets the system score up to 500 items in a query segment while avoiding unnecessary attention work. Across four or five releases, the combined changes cut latency by 7x and raised per-GPU throughput by 30x.
History selection must balance freshness, relevance, class balance, and caching
For long user histories, LinkedIn tested retrieval of the most relevant historical items and chronological ordering with time decay. Chronological order was good enough for most applications because recommendation systems favor fresh activity. The team also has to decide how many positive and negative examples to place in the prompt and how much information to include. Serving adds another constraint: a smarter history arrangement can interfere with KV caching. Sanjabi says making the context selection smarter than chronological order is therefore an engineering and serving-design problem, rather than an obvious improvement.
"If you put everything together, we can have a significant reduction in the latency."15:00
Who should watch
You are building ranking or recommendation models for several products and want to understand the case for a shared foundation model.
Your team has a large language model with good offline quality but needs to reduce its serving latency and GPU cost.
You need to handle cold-start users, new recommendation surfaces, long user histories, or experiments across distillation and quantization.
