Self-improvement loops work best when the system has a clear, reliable target function, such as whether a classification matches its ground-truth label.
2
A paper-classification loop improved accuracy from 68% to 83% on validation data, with the largest gain coming from its first prompt update.
3
Domain experts need to define concrete quality criteria, review examples, and identify failure modes before an automated loop can improve an application efficiently.
Summary
Annabell Schäfer argues that self-improvement loops are only as useful as the target function they optimize. Coding provides an unusually clear signal because code either compiles or does not. Other applications, including healthcare, compliance, and chatbots, need more careful definitions of what good means. Schäfer demonstrates this with an arXiv paper-classification task. A simple prompt, a ground-truth dataset, and an optimizer produced a 15% improvement over the baseline, with the first iteration adding ten percentage points. The loop found confused label pairs, added classification rules, and included examples of common mistakes. Schäfer is honest about the limits of the experiment: author-selected paper labels allow some creative freedom, so the ground truth is not perfectly objective. For less deterministic applications, she recommends replacing vague scores such as helpfulness or correctness with specific checks and failure categories. Domain experts should create examples, explain decisions, review production data, and help build evaluators that provide a strong signal.
Coding created an unusually simple model for automatic improvement
Schäfer says the current enthusiasm for loops comes partly from coding, where an agent can optimize against a clear target: whether the code compiles. Compilation does not prove that the implementation is good or that every feature matches the user's intent, but it confirms that something working was produced. In healthcare, compliance, medicine, and chatbot applications, the target is much less definite. A prompt given to an agent is always incomplete, and the destination may change as the team learns more about the domain. A binary test such as "does the code compile?" cannot capture these applications on its own.
A useful target function needs a measurable connection to the real task
Langfuse looked for a task with an especially clear target function and chose single-label classification. Each arXiv paper had a ground-truth label assigned by its author, and the classifier had to choose from a fixed list such as order, complaint, or inquiry in the example. The score was simple: compare the predicted label with the true label and calculate accuracy. Schäfer uses this experiment to study what an auto-optimization loop can learn when the feedback is genuinely binary. She also shows that even this apparently clean target has limits because authors have some freedom when choosing paper categories.
The experiment separated fitting, validation, and testing
The classification setup used 200 items for fitting, 100 for validation, and 300 for testing. The agent used a simple prompt and GPT-5 nano, chosen to see how a small, inexpensive model performed in the loop. An optimizer running through Claude Opus 4.8 proposed prompt changes, with access to a prompting guide and a task description. The test set was kept out of the loop until the end. This separation let the team check whether a prompt update generalized beyond the examples used to find errors, rather than rewarding a change that only memorized the fitting data.
The loop accepted prompt changes only when validation improved
The loop first measured baseline accuracy on the fitting and validation sets. It then examined fitting errors, including frequent categories and commonly confused label pairs. The optimizer proposed a prompt update aimed at the largest error category. The update was kept only if it also improved the validation score. The process stopped after 15 runs or when it reached 92% accuracy. At that point, the system ran once on the untouched test set. Schäfer also describes an escape from endless iteration: validation and explicit stopping criteria prevent the agent from continuing to spend tokens after progress has stopped.
The simple loop improved accuracy, but the labels were not perfectly objective
The baseline ended at 68% accuracy. By the third or fourth iteration, validation accuracy reached 83%, a 15% improvement, and then largely plateaued around 80%. The result generalized to 80.2% on the 300-item test set. Looking more closely, Schäfer found that paper authors had creative freedom in choosing labels. A paper could appear to fit one description while its author selected another category, and that reasoning was not recorded in the dataset. The experiment therefore had a strong measurable signal, but its ground truth still contained human judgment that the classifier could not fully recover.
The first update worked because the error signal was detailed and countable
The largest gain came in the first iteration, when accuracy rose from 68% to 78%. The optimizer counted 64 errors, identified dominant patterns, found the most common label confusion, and formed a hypothesis about which prompt change would address it. The revised prompt added a general classification approach, guidance for separating similar classes, rules for choosing a specific label over a broad one, and examples of frequently misclassified paper pairs. Schäfer expected the system might add descriptions to the labels, but it chose rules and examples instead. The important property was that the feedback showed exactly what was wrong on a right-or-wrong basis.
Vague evaluator scores provide weak feedback for automatic improvement
Binary classification feedback worked well in the experiment, but most applications do not have a deterministic yes-or-no evaluator. Even the same language-model judge may produce different results across runs. Schäfer says broad evaluators such as correctness, helpfulness, and hallucination are often weak signals when used for auto-improvement, especially when they return an unexplained number on a scale from zero to one, five, or ten. Each score needs a defined meaning and clear criteria for when it should be assigned. Without that definition, the score can change with context and remain inconsistent across evaluations.
Domain experts can turn quality judgments into concrete checks
Schäfer recommends defining what good means for the specific application. Instead of asking whether an answer is generally correct, a team might check whether it is supported by the retrieved knowledge base. A brand-voice evaluator could check whether the company name is spelled correctly or was accidentally translated. Another evaluator could classify which of five known failure modes occurred. These checks give the loop a stronger signal because they connect directly to decisions that matter in the domain. The examples and failure categories should come from reviewing real data and asking domain experts why one output is acceptable while another is not.
Human review has to continue after the first evaluator is built
Schäfer says experts should help create examples, review sample runs, explain decisions, identify failure modes, and define quality criteria. Once the application reaches production, humans should continue reviewing its data rather than relying only on coding agents. The scope of use can change, users can try new things, and the failure modes can shift. Production review helps the team find problems before customers do and add examples that represent future use. Validation protects against overfitting, while loop instructions and stopping conditions give the system a way to stop instead of running for hours without progress.
"The teams who are investing heavily here in the middle, so making sure they capture what they actually want to work, the target function, and build this out and make sure they have good evaluators that are evaluating this are the ones who manage to continuously upgrade and improve their application over time."02:02
Who should watch
You are building an agent that rewrites its own prompts and need to know whether your evaluator gives it useful feedback.
Your application deals with healthcare, compliance, retrieval, brand voice, or another domain where a general helpfulness score is too vague.
You want a practical way to involve domain experts in dataset creation, failure analysis, validation, and production review.
