# Building LinkedIn's GenAI Platform

Xiaofeng Wang, LinkedIn | AI Engineer Summit 2025 | 17:53

Source: https://www.youtube.com/watch?v=n9rjuBuShko
Channel: AI Engineer (https://www.youtube.com/@aiDotEngineer). Summarised by AIE Talks.
Page: https://aietalks.com/talks/building-linkedins-genai-platform
Published: 2025-04-16
Tags: agent-skills, agents, mcp, memory, observability

## TL;DR
- LinkedIn's GenAI platform grew from a small orchestration library into a system covering prompts, memory, skills, agent execution, and observability.
- Agent systems blur the boundary between model optimization and application serving, so platform teams need to connect AI engineers with product engineers.
- LinkedIn prioritizes software engineering ability and hiring potential, then builds a diverse team whose members can learn the missing skills from one another.

## Summary
Xiaofeng Wang describes how LinkedIn's GenAI platform developed alongside its products. The team began with a model gateway and Python notebooks for prompt engineering, then added a Python SDK, prompt versioning, conversational memory, distributed agent orchestration, a centralized skill registry, experiential memory, and detailed observability. LinkedIn's first multi-agent system, LinkedIn Hiring Assistant, pushed the platform toward handling retries, traffic shifts, tool invocation, and autonomous behavior. Wang argues that agent systems combine models, retrieval, and external tools in ways that blur the old separation between model optimization and serving. A platform helps engineers use this ecosystem through one interface and gives the company a place to apply governance. His advice is to start with an immediate product need, grow the platform incrementally, choose Python where possible, reuse scalable infrastructure, and focus closely on developer experience. For hiring, he favors strong software engineering, potential, critical thinking, and team diversity over a single perfect candidate.

## Key ideas
### LinkedIn's first GenAI products exposed the limits of a simple model gateway
[01:04](https://www.youtube.com/watch?v=n9rjuBuShko&t=64s)
LinkedIn launched Collaborative Articles in 2023 with a straightforward prompt-in, text-out experience built around GPT-4. The early platform work included a gateway for centralized model access and Python notebooks for prompt engineering. The product used Java in the online path and Python in the backend. Wang says the team did not consider this a platform yet. They soon found that the simple setup could not inject enough of LinkedIn's data into the product experience, which led them toward a second generation of GenAI applications.

### The second generation added orchestration, prompt versioning, and conversational memory
[02:19](https://www.youtube.com/watch?v=n9rjuBuShko&t=139s)
LinkedIn's next generation included a copilot experience that used a member's profile, a job description, and retrieval-augmented generation to produce a personalized job-fit recommendation. The team built a Python SDK on top of a popular orchestration framework, with integrations into LinkedIn's large-scale infrastructure. They also unified the technology stack because moving Python prompts into Java was costly and error-prone. A prompt source of truth gave developers version control and structure for meta-prompts. Conversational memory tracked model interactions and retrieved content so those details could be injected into the final response.

### Multi-agent products forced the platform to handle distributed execution and reusable skills
[04:33](https://www.youtube.com/watch?v=n9rjuBuShko&t=273s)
LinkedIn's first real multi-agent system was LinkedIn Hiring Assistant, which helps recruiters with work such as posting jobs, evaluating candidates, and reaching out to them. The platform expanded its Python SDK into a distributed agent orchestration layer that could manage agent execution, retries, and traffic shifts. Wang calls skills or APIs a central part of agent design because agents need to take actions. LinkedIn built a centralized skill registry where developers could publish APIs. The registry supports skill discovery and invocation, making it easier for applications to call APIs for specific tasks.

### Agent memory needs to capture experience across several time horizons
[05:36](https://www.youtube.com/watch?v=n9rjuBuShko&t=336s)
LinkedIn extended conversational memory into experiential memory. This storage extracts, analyzes, and infers knowledge from interactions between agents and users. The team organizes memory into working memory, long-term memory, and collective memories. Wang says these layers help an agent remain aware of surrounding content. This goes beyond retaining the current conversation. It gives the platform a structure for storing information learned from interactions and making that information available to later agent behavior.

### Observability must support replay because autonomous agents are hard to predict
[06:47](https://www.youtube.com/watch?v=n9rjuBuShko&t=407s)
Agents can decide which API to call and which language model to use, so their behavior is harder to predict than a fixed request pipeline. LinkedIn built an in-house observability solution on top of OpenTelemetry to capture telemetry at a very low level of detail. The team can use that data to replay agent calls. It also added an analytics layer to guide later optimization. Wang places observability alongside orchestration, prompt engineering, skills, content, and memory as one of the platform's main layers.

### A unified platform hides ecosystem complexity while enforcing governance
[07:36](https://www.youtube.com/watch?v=n9rjuBuShko&t=456s)
LinkedIn's GenAI ecosystem also includes teams working on model fine-tuning, open-source models, responsible AI, and machine learning infrastructure. Wang describes the platform as a unified interface over these separate parts. In the SDK, a developer can switch from an OpenAI model to an internal model by changing one parameter, while still handling prompt engineering separately. Centralization also gives LinkedIn a place to enforce best practices and governance. Developers can access the wider ecosystem without understanding every underlying component.

### Agent systems connect model optimization and product serving
[09:39](https://www.youtube.com/watch?v=n9rjuBuShko&t=579s)
Wang argues that GenAI systems differ from traditional AI systems because the boundary between model optimization and model serving has disappeared. In older systems, AI engineers and product engineers could work in separate stacks and codebases. Agent systems combine multiple interacting components, such as models, retrievers, and external tools. Wang uses Berkeley AI Research's definition of a compound AI system and says the platform must bridge the skills of AI engineers and product engineers. This shared system creates a new need for company-wide tooling and development practices.

### The team hires for software engineering strength and grows capability through diversity
[11:46](https://www.youtube.com/watch?v=n9rjuBuShko&t=706s)
Wang's ideal platform engineer can build infrastructure integrations, design developer-facing interfaces, understand AI and data science, learn new techniques, and stay hands-on. Since finding every trait in one person is difficult, LinkedIn makes tradeoffs. The team prioritizes strong software engineering over AI expertise and hires for potential rather than degrees or past experience. It combines full-stack engineers, data scientists, AI engineers, data engineers, fresh graduates, and people from startups. Collaboration helps members acquire missing skills. Wang also stresses critical thinking because tools and approaches may become outdated within six months or a year.

### The platform should grow from immediate needs and fit existing developer workflows
[14:43](https://www.youtube.com/watch?v=n9rjuBuShko&t=883s)
Wang recommends starting with a small solution to an immediate need instead of building a complete platform first. LinkedIn began with a simple Python library for orchestration and expanded it as new requirements appeared. The company also reused its messaging infrastructure as a memory layer, which Wang describes as cost-efficient and scalable. Developer experience determines adoption. Platform technology should fit existing workflows so developers can become productive without changing how they work more than necessary.

## Notable quotes
- "We started to build this platform piece by piece and then this platform started to emerge." (09:39)
- "The ideal candidate in this team is a strong software engineer who can build infrastructure integration." (11:46)
- "We usually prioritize the stronger software engineering skills over the AI expertise." (12:35)
- "Instead of trying to build this full-fledged platform at the beginning, try to solve immediate need." (14:43)

## Tools & references mentioned
- LinkedIn
- Collaborative Articles
- GPT-4
- LinkedIn Hiring Assistant
- Berkeley AI Research
- OpenTelemetry
- OpenAI

## Who should watch
- You are starting a GenAI platform team inside a company with an established application stack and need to decide what to build first.
- Your agents need shared tools, memory, model access, retries, and observability rather than isolated application code.
- You are hiring platform engineers and want guidance on trading off AI experience, software engineering ability, and team composition.

## Related talks

- [Building the Platform for Agent Coordination](https://aietalks.com/talks/building-the-platform-for-agent-coordination) (Tom Moor, Linear, 19:43)
- [Platforms for Humans and Machines: Engineering for the Age of Agents](https://aietalks.com/talks/platforms-for-humans-and-machines-engineering-for-the-age-of-agents) (Juan Herreros Elorza, Banking Circle, 21:15)
- [Why Agent Engineering](https://aietalks.com/talks/why-agent-engineering) (Shawn Wang, Latent.Space, 11:45)
- [AI Platform Engineering](https://aietalks.com/talks/ai-platform-engineering) (Patrick Debois, 28:18)
- [Hiring & Building an AI Engineering Team](https://aietalks.com/talks/hiring-building-an-ai-engineering-team) (Dr. Bryan Bischof, Hex, 29:07)