# The Self-Improving OSS Agent Stack Marc Klingen, Langfuse | AI Engineer | 17:07 Source: https://www.youtube.com/watch?v=TeErpYBUIeM Channel: AI Engineer (https://www.youtube.com/@aiDotEngineer). Summarised by AIE Talks. Page: https://aietalks.com/talks/the-self-improving-oss-agent-stack Published: 2026-10-06 Tags: continual-learning, evals, human-in-the-loop, observability ## TL;DR - Agent improvement is moving from a manual loop of reading traces and updating evals to AI proposing changes against datasets and evaluation criteria. - Humans should set goals, review dataset and evaluator changes, and decide which failure patterns matter, while AI handles repetitive analysis and implementation work. - Teams should connect production traces and user feedback to offline datasets, experiments, and evals, while keeping ownership of the trace data for long-term use. ## Summary Marc Klingen describes an agent improvement loop that connects online tracing and monitoring with offline datasets, experiments, and evaluations. Teams collect production traces, identify failures, reproduce them in datasets, propose changes, and back-test new agent versions before deployment. AI can now help with much of this work, especially proposing implementation fixes and maintaining datasets and eval criteria. Klingen argues that people still need to set the direction, decide which failures matter, and review changes that redefine the target. In his demo, a coding agent examines Langfuse's changelog writer, finds that it leaks internal jargon, updates the datasets and evaluators, and compares a new implementation with the existing version. User edits, approvals, and change requests provide feedback without requiring developers to label everything manually. As agents process more historical traces, Klingen says teams need a scalable data layer that they own and can retain without aggressive sampling. ## Key ideas ### Agent development is becoming a loop between production data and offline evaluation [01:42](https://www.youtube.com/watch?v=TeErpYBUIeM&t=102s) Klingen describes a reference stack with an online side and an offline side. Online tracing and monitoring show how users interact with an agent in production. Offline datasets, experiments, and evals let teams reproduce issues and test changes before deployment. The loop sends improved versions back to production, where new user behavior creates more data. He names observability tools such as Datadog alongside ML operations tools such as MLflow and Weights & Biases, then explains that Langfuse was built to connect these two sides. Without that connection, datasets drift away from production or teams monitor production without benchmarking changes offline. ### Better models make higher-level improvement loops possible [03:16](https://www.youtube.com/watch?v=TeErpYBUIeM&t=196s) The talk lays out layers of loops, starting with the model's next-token decisions and moving toward high-level reasoning about what an agent should fix. Klingen says simple coding-agent workflows were difficult when Langfuse began in 2023. A GitHub issue to pull request system could then handle only single-page applications because multi-file edits were too difficult. He says later model improvements made higher-level loops possible, including agents that reason about failure patterns, propose fixes, and check whether those fixes work. ### People still need to define the data and decide which failures matter [04:06](https://www.youtube.com/watch?v=TeErpYBUIeM&t=246s) Humans currently need to review which examples enter an agent's dataset and how the agent will be evaluated. They also need to inspect proposed fixes because an agent can overfit to a dataset or to an accidental failure definition. A production trace may reveal a strange use case that does not matter for the product's actual purpose. Klingen's recommendation is to keep people involved in setting the goals and boundaries, while allowing AI to handle lower-level work against those boundaries. If AI is given complete control, he warns that the system can create more slop by continually changing what counts as success. ### AI can propose implementation fixes and increasingly maintain evals [05:01](https://www.youtube.com/watch?v=TeErpYBUIeM&t=301s) Klingen says the most common use of AI in improvement loops is proposing fixes to the agent implementation. Given production traces, datasets, and evaluation criteria, an agent can try different models, context arrangements, or other implementation changes and hill-climb against the existing target. Teams are also starting to use AI to keep datasets aligned with real user behavior and to propose new evaluators from recurring error patterns. Examples include checking that a support agent answers in the user's language or that a response is concise. Humans still review these changes because they define the boundary against which later automation operates. ### The target of an agent changes as teams learn what the work requires [07:00](https://www.youtube.com/watch?v=TeErpYBUIeM&t=420s) Klingen uses customer support to explain why an agent's target is initially vague. A team may begin with a broad goal such as automating customer support, but production error cases reveal the specific behaviors the system must handle. The team gradually assembles a more precise map of the work while the system is running. People should remain involved in setting the direction and changing the goals, datasets, and evaluators. AI can then automate the repetitive work below that level. This division lets teams respond to newly discovered error classes without manually inspecting every trace. ### A partly automated loop can reduce time while increasing coverage [08:10](https://www.youtube.com/watch?v=TeErpYBUIeM&t=490s) Klingen contrasts a fully manual workflow, which demands substantial weekly effort, with a fully automated workflow that may produce many tokens without meaningful progress. He prefers a middle position where people stay involved at the important decision points and AI handles tedious analysis. Manual work is limited by a team's time, patience, and ability to inspect large amounts of data. An AI system can examine more error cases, but its output still needs direction. The intended result is less human time spent on repetitive work and more thorough testing of possible improvements. ### Implicit user actions can provide feedback without manual labeling [10:04](https://www.youtube.com/watch?v=TeErpYBUIeM&t=604s) Production systems can generate useful signals through ordinary user and team actions. In customer support, a customer may say that an answer is wrong, or an internal agent may accept or edit a proposed message before sending it. Those approvals, edits, and change requests can enter the improvement loop as feedback. Klingen says this gives the system a stream of signals without requiring the agent developer to label every example by hand. The loop can combine these signals with traces and run lower-level improvement steps automatically. ### The changelog demo turns leaked jargon into an eval and a code change [10:24](https://www.youtube.com/watch?v=TeErpYBUIeM&t=624s) Langfuse's demo uses a changelog writer that turns merged pull requests into documentation and public release updates. A human can approve the generated change or request edits through a GitHub pull request, and those actions become feedback for the agent. The coding agent finds that the writer is factually accurate but has low clarity and leaks internal jargon such as implementation terms that customers do not use. It proposes changes to the dataset and evaluators for jargon and customer-facing language, then changes the agent's context and skill. The new version is back-tested against the old one before the change is merged. ### Long-lived trace data becomes part of the agent's improvement system [15:58](https://www.youtube.com/watch?v=TeErpYBUIeM&t=958s) Klingen says the improvement loop creates demand for a scalable data system because agents can read and query large amounts of historical data. Langfuse was historically more write-intensive because it ingested traces and evals. As agents consume that data for analysis, the workload becomes more read-intensive. He argues that teams should own the data layer so old traces remain available as useful context instead of being trapped in a commercial system with short retention or aggressive sampling. The storage system needs to be scalable and inexpensive enough to retain the data for a long time. ## Notable quotes - "You want to be involved in setting the goals and how you make changes to data sets and evaluators." (07:49) - "You want to be in the loop at the most important steps of setting the direction of where you want to go, but you take yourself out of everything that's tedious." (08:57) - "Thereby you have an in-stream of signal that your loop can act on." (10:05) - "You want to really own the data layer because now the traces of, for example, a year ago are interesting context." (16:21) ## Tools & references mentioned - Langfuse - GitHub Copilot - Devin - Cursor - Datadog - MLflow - Weights & Biases - Codex - GitHub ## Who should watch - You are building an agent and still inspect production traces, update datasets, and write evals by hand each week. - Your product generates useful feedback through edits, approvals, or change requests, but that feedback is not yet connected to agent improvement. - You need to retain and query historical traces for automated analysis, and are deciding whether to own the underlying data layer. ## Editor's note Marc Klingen shows a changelog writer that leaks internal jargon, then updates its dataset, evaluators, and implementation before back-testing the new version. Kitaru records each model call and tool result from a real agent run, then replays the code with the same inputs and responses without touching real systems. This lets a team rerun a failed run after changing the prompt, model, or code. Written by the AIE Talks editors (the Kitaru team), not by the speaker. ## Related talks - [Break It 'Til You Make It: Building the Self-Improving Stack for AI Agents](https://aietalks.com/talks/break-it-til-you-make-it-building-the-self-improving-stack-for-ai-agents) (Aparna Dhinakaran, Arize AI, 14:25) - [The Agentic AI Engineer](https://aietalks.com/talks/the-agentic-ai-engineer) (Benedikt Sanftl & Burak, Mutagent, 34:50) - [Improving Agents is a Data Mining Problem](https://aietalks.com/talks/improving-agents-is-a-data-mining-problem) (Vivek Trivedy, LangChain, 20:02) - [From Vibes to Production: Evaluating and Shipping AI Agents That Work 101](https://aietalks.com/talks/from-vibes-to-production-evaluating-and-shipping-ai-agents-that-work-101) (Laurie Voss, Arize AI, 1:51:27) - [Agents Building Agents](https://aietalks.com/talks/agents-building-agents) (Alfonso Graziano, Nearform, 30:14)