Long-horizon agents need memory when the answer falls outside the context window, but a memory harness adds cost without improving results when the whole task already fits.
2
A ranked decisions ledger recalled information better than no memory, vector RAG, or gating the harness based on whether memory seemed necessary across 68 Xbench questions.
3
The correct memory alone does not guarantee a correct answer, and bad recall can waste tokens and send an agent in the wrong direction.
Summary
Stefania Druga describes memory as a write, manage, read control loop around a model. Her harness uses small research agents with no durable memory, a core trace shown on every turn, a recall block, and an archival block for information across sessions. She compares no recall, vector RAG, a ranked decisions ledger, and an oracle that supplies the correct memory. On a literature review task where all papers fit in context, memory produced the same performance at higher cost. On long-horizon Xbench questions, where the answer appeared at step 124 and the question arrived at step 500, ranked recall performed best across 68 questions. The oracle still fell short because the model could ignore or misuse the supplied memory. The ranked policy also worked across models and benchmarks while using fewer tokens. Druga argues that recall policy deserves its own evaluation metric, alongside decisions about what to store, rank, and retain across sessions.
Druga describes context rot as a model contradicting itself, repeating work, or drifting away from the original question because earlier information has fallen out of active context. She connects the problem to a trend toward longer tasks and fewer model releases. If those trends continue, systems will need to manage long-running work without relying on a new model release for every problem. Her concern is practical: an agent that forgets what it already did can spend time repeating steps, while an agent that loses the question can produce work that no longer answers it.
Local models make controlled memory experiments possible
Druga says companies are reducing AI spending by using more local models, better routing, caching, cleaner context, and better visibility into usage. She argues that local models are becoming useful for agent tasks and tool use. Her experiments run on an M3 Ultra with 96 gigabytes of memory and 28-core CPUs, using Qwen 27B quantized at 4 and DeepSeek V4 Flash. The machine is in Tokyo and she controls it from her phone. It became hot during continuous evaluations, so she placed fans around it.
Druga asks viewers to think of memory as a write, manage, read loop rather than as a database attached to a model. Her harness starts with small research agents that have no durable memory, so memory comes from the harness. A core of traces is always shown to the agent. A recall block tests different retrieval policies, while an archival block keeps information across sessions. This design lets her study how memory is written, managed, selected, and carried forward instead of treating storage as the whole memory problem.
The model stays fixed while Druga changes the recall block. The baseline uses no memory. The next condition uses vector RAG to retrieve information by similarity. The decisions ledger records what was decided on each turn and prioritizes those decisions. The final condition is an oracle that gives the harness the ground-truth memory needed for each loop. This ladder separates the question of whether memory exists from the question of which memory the system chooses to retrieve.
Memory adds cost when the whole task fits in context
Druga tests the harness on a literature review containing papers with large scientific claims, including a Nature paper that claimed the discovery of 742,000 promising materials and was later retracted. The retraction is much smaller in the corpus than the original headlines and citations. Since all the papers and relevant information fit inside the context window, memory does not improve performance. The system gets the same result with and without memory, while the harness adds cost.
Ranked recall helps when the answer is far outside context
In an Xbench long-horizon task, the answer appears at step 124, while the question is asked at step 500. The relevant information is outside the model's context window, so the agent must use the harness to retrieve it. Druga compares the recall policies and finds that ranked recall produces the right answer more often than no memory. Across 68 questions, the ranked decisions ledger performs best, beating a policy that first gates the harness on whether memory seems necessary.
The oracle supplies the right information for a loop, but it does not force the model to use that information. The model can retrieve something else, ignore the supplied memory, or become confused by it. That is why the oracle does not reach maximum performance. Druga's ablations test arbitrary examples, the wrong step, and the most recent step. The ranked recall policy remains the best-performing condition, and the result holds for Qwen 27B, DeepSeek V4 Flash, and the Spider V2 benchmark.
Druga says ranked recall is not only more accurate, it also costs less. Bad memory uses more tokens and can send the agent in the wrong direction. A structural policy for recall can therefore reduce both token use and budget. She asks engineers to evaluate recall as its own metric, including what kinds of memories are stored, how they are ranked, how the recall function is designed, and what survives across repeated runs and multiple sessions.
Local execution gives full control at a real speed cost
Running the experiments locally lets Druga control the data, full traces of computation, and evaluations. She describes this as a form of sovereignty, but is clear about the cost. DeepSeek V4 Flash only runs serially in her setup because it does not support batch querying, so evaluations take a long time. She continued them on the Tokyo machine and even on the flight to the talk. For her, that slower setup is useful because every part of the memory pipeline can be inspected and changed.
"You can think of memory as a write, manage, read loop, so it's not just the database store, it's actually this control loop around the model."03:41
Who should watch
You are building agents that run for hundreds of steps and need information from earlier parts of a task.
Your current memory system retrieves similar text but does not track decisions or measure whether recall improves the answer.
You want to run memory experiments locally and need to understand the tradeoff between control, token cost, and evaluation speed.
