LinkedIn's recursive self-improvement loop evaluates agent traces, diagnoses root causes, proposes prompt edits with GEPA, and validates changes on held-out data.
2
The evaluation system needed human review because the judge was better at recognizing strong outputs than distinguishing weaker ones, and its rubric sometimes confused plausible answers with complete ones.
3
The team deployed an initial GEPA-optimized change to production, while keeping human review in the loop and improving the evaluation and reflection layers before increasing automation.
Summary
Saloni Gandhi and Ziwei Guo describe LinkedIn's recursive self-improvement system for a user-understanding agent. The agent turns member data such as profiles, search history, preferences, and location into structured career signals with confidence scores, citations, and reasoning. The improvement loop collects traces, scores outputs with an evaluation judge, groups failures, diagnoses root causes, proposes prompt changes with GEPA, validates candidates, and checks the winner on held-out data before making it the new baseline. The talk is candid about the hard part: the loop can only improve an agent when its evaluators are trustworthy. Human reviews exposed weak score boundaries, incomplete rubric definitions, formatting penalties, and reflection agents that sometimes blamed the wrong component. LinkedIn improved its rubrics and reflection process, deployed an initial optimized change to production, and is working toward reusable infrastructure that other agent teams can adopt.
Recursive self-improvement is a loop around production traces and evaluation
Ziwei Guo defines recursive self-improvement as building an AI system that learns from issues found in production traces. For LinkedIn's career agent, the process starts with agent outputs and logged trajectories. An evaluation engine, such as an AI judge, checks those traces and reports where the system performs well or poorly. The team feeds that feedback into another iteration. Guo stresses that evaluation must be objective and aligned with user feedback because it determines which bottleneck the system works on next. The loop is intended to keep improving the agent from observed failures rather than relying only on manual development work.
The first optimization target is the harness around the base model
The system includes planning and logic for orchestration, context and memory for session state, tool interfaces for APIs and data sets, and artifact storage for outputs and generation logs. Evaluation logic measures these components. The base language model powers the harness, but optimizing that model is outside the current scope. LinkedIn's progression starts with prompt updates, then moves toward structured context, automatic workflow design, harness code, and eventually optimizer code. The team is currently focused on prompt optimization, while the broader design allows future changes to reach deeper parts of the agent system.
The improvement pipeline separates diagnosis, editing, and validation
The seven-step process begins by running the agent and saving samples and traces. An evaluation judge scores the outputs. The reflection stage then investigates whether a failure comes from the agent, model, prompt, tool, harness, judge, or rubric. Based on that diagnosis, the system proposes an edit. Candidate edits are evaluated with the same judge and rubrics, and promising prompts are selected. The updated prompt is finally tested on a held-out set against the old system. A successful change becomes the new baseline, after which the process can run again on a weekly, biweekly, or monthly cadence.
The user-understanding agent turns member evidence into structured career signals
The target agent receives a member ID and fetches information such as search history, profile data, location, and preferences. It synthesizes those signals into a user-understanding output. One field might contain preferred job titles, with examples such as network manager or account manager. The output also includes a confidence level and citation data that explains why the agent selected a title. A frequent title in saved searches can support the conclusion. The agent can also provide reasoning about preference anchors, primary titles, saved searches, and adjacent signals. This structure gives the evaluation system several dimensions to inspect.
Root-cause reflection clusters failures before proposing fixes
The reflection process first clusters failing examples by evaluation dimension. A second judge diagnoses each row and assigns a root-cause tag, such as prompt, agent, harness, code, judge, or rubric. Similar tags are then merged because different descriptions can point to the same problem, such as missing profile data or profile data that fails to show the member's browsing history. The system ranks canonical issues by frequency and produces a scorecard for human action. It also stores the diagnosed causes to guide later reflection runs. This turns many individual failures into a smaller set of recurring problems.
Evaluator quality can limit the value of automatic optimization
Saloni Gandhi says the team learned that evaluator quality matters as much as optimization quality. In a review of about 180 to 200 human judgments, the judge aligned with humans about 55% of the time for score-one outputs, 57% for score-two outputs, and 80% for score-three outputs. The judge was better at recognizing strong answers than separating weaker answers. It also sometimes accepted a plausible job title while missing an important signal, confused domains, functions, and skills, penalized harmless formatting, or made inconsistent inferences. The team revised title normalization, formatting rules, evidence handling, and score boundaries. Human alignment increased from 57% to 64%, while mean error decreased from 96 to 74.
A convincing explanation can still identify the wrong cause
The L2 reflection agent diagnoses why an evaluation failed, but human reviewers found root-cause attribution harder than failure detection. Agreement was higher when the agent blamed the underlying model and lower for other causes, especially rubric problems. The original taxonomy did not explicitly allow the L1 judge to be wrong, so the reflection agent sometimes blamed the rubric instead. It also sometimes repeated the L1 rationale without checking it against the actual output. Gandhi warns that a reflection system can produce persuasive explanations while misidentifying the underlying problem. Prompt changes based on that diagnosis could modify the wrong part of the system.
Scaling recursive improvement requires shared evaluation and trust infrastructure
LinkedIn's broader plan moves from individual agent implementations toward reusable infrastructure. The proposed layers are a base agent, reliable evaluation, agentic reflection, and the recursive improvement loop. Each layer depends on the previous one, so the team treats evaluation infrastructure as a prerequisite for scaling. The infrastructure would standardize judges and metrics calibrated with human feedback, collect traces, detect recurring failures, generate and test candidate updates, and run held-out evaluations and regression tests. The goal is to let teams adopt recursive improvement without building every component themselves. The system should measure downstream agent quality and engineering productivity, rather than simply counting automated prompt edits.
"The ultimate goal isn't to maximize the number of automated prompt edits. It's to produce measurable improvements in agent quality while maintaining some sort of confidence that those changes are genuinely beneficial."Saloni Gandhi33:20
Who should watch
You are building an agent that produces structured outputs from user or business data and need a repeatable way to learn from production failures.
Your team is considering prompt optimization or automated agent improvement, but you do not yet know whether your evaluator and root-cause analysis can be trusted.
You maintain several agents and want to understand what shared evaluation, reflection, validation, and regression infrastructure would be needed before scaling automation.