AI agents need to learn from production experience without losing capabilities that already work.
2
Production logs and feedback must be converted into replayable learning environments before they can support verifiable improvement.
3
A useful continual learning loop routes each repair to the right layer and checks new gains against regression tests from past environments.
Summary
Soheil Feizi presents continual learning as a way for AI agents to improve from real use without forgetting prior capabilities. He separates the agent into model, harness, and memory layers, then argues that the hard problems are obtaining useful feedback and deciding how to act on it. A production log is only an observation. To make it useful for learning, teams need to turn it into a replayable environment with simulated users, tools, and evaluators. Feizi calls this verifiable continual learning when each update is tested before and after, measured for improvement, and checked against prior environments. He gives four principles: replayability, holisticness, lifelongness, and efficiency. The talk compares fine-tuning, prompt search, trace-to-harness repair, memory updates, and skill distillation. RELAI's proposed loop performs root-cause analysis, routes changes to the appropriate layer, and produces a reviewable agent update.
Continual learning changes agents through experience without forgetting
Feizi defines continual learning as letting agents learn from experience by acting, receiving feedback, and improving without forgetting. An agent interacts with diverse users, complex tools, and different data policies. Updates can happen at the model layer by changing model weights, at the harness layer through prompts, skills, tools, code, and workflows, or in session and persistent memory. The central problems are finding out whether an agent did well and deciding what to change after receiving feedback.
Production logs need to become executable learning environments
Benchmarks provide tasks, evaluators, grades, and sometimes behavioral feedback during development. Production usually provides session logs instead, often without explicit feedback. Automatic models, code, or the agent itself can critique logs at scale, while human experts provide lower-volume domain feedback. Feizi says log plus feedback is still not testable. The system must infer a replayable environment with tool behavior, mock or real data, synthetic users, and evaluators that define success. Once executable, candidate agents can be rerun against the same scenario.
The smallest durable repair should go to the right agent layer
Feizi describes three update layers. Model updates use supervised fine-tuning or reinforcement-learning post-training, but they are expensive because they change weights. Harness updates can alter prompts, skills, tools, or code around the model. Memory updates store facts and learned skills so the agent does not repeat failures. He argues that a good learning engine should not focus on one layer exclusively. It should find the smallest durable change at the layer that explains the failure.
Existing update methods trade testability against speed
Model methods such as SFT, DPO, GRPO, RLV R, and LoRA generally need benchmarks and explicit evaluators, so logs must first be converted into replayable environments. Trace-to-harness methods can ask a coding agent to inspect a log and improve the agent, but the change is wipe-based and may be ineffective or cause hidden regressions. Prompt search methods such as GEPPO mutate prompts and retain candidates that score well, but they also require benchmarks and evaluators. Memory methods such as LETA and MemZero work directly from logs and feedback and are cheap, but are usually unverified.
Verifiable continual learning tests every fix and protects prior behavior
Feizi defines verifiable continual learning as improving an agent from its own experience while proving that each fix helps and does not break what already worked. The process has three steps: turn a failure into an executable test, measure the score before and after the update, and run regression tests from prior cases. This changes an informal repair into an update that can be evaluated and reviewed.
Replayability, holisticness, lifelongness, and efficiency define the framework
Replayability turns a single failure into a rerunnable test. Holisticness recognizes that one failure can have several causes and repairs across memory, prompts, tools, workflows, or the model. Lifelongness requires a new fix to improve the latest case without breaking past environments, with regression included inside optimization rather than added afterward. Efficiency matters because memory edits are cheaper than harness changes, while model-weight updates can be expensive, and the whole loop must run often.
RELAI's loop turns signals into reviewable agent updates
RELAI's proposed loop accepts logs, feedback, instructions, or prompts and lifts them into replayable learning environments. It then performs root-cause analysis, routes a repair to memory, model, or harness, and uses regression-aware optimization. The output is a reviewable version update that explains what changed and why the change improves the agent without creating regression. Feizi says the setup can be added with a learning harness and two commands, one to create environments and another to optimize the agent.
A support-agent benchmark tests improvements against regression traps
Feizi describes a fictional tool-using support agent benchmark with reproducible test beds, a single source of truth for policies, deterministic evaluators, and regression traps. A command about a rude and adversarial caller can generate a learning environment containing personas, intents, mock or real tools, and evaluators. In the example, the agent initially scores 78 percent, with two evaluators showing low results. After optimization, Feizi reports an average improvement of 10 percent in one loop and a score increase from 87 percent to 97 percent. He also gives a production example about keeping fast eligible refunds without extending generosity beyond refund thresholds.
"Production logs are not learning environments. We need to transform them into replayable learning environments to simulate and evaluate the agent on the same patterns and scenarios."21:54
Who should watch
You are running an agent in production and have logs or user feedback, but no reliable way to replay failures before changing the system.
Your team is deciding whether a failure belongs in model training, prompts and tools, or memory, and needs a way to compare those repairs.
You already optimize against recent cases and need regression checks that protect older agent capabilities.
