Your AI Agent Keeps Making the Same Mistakes

Sparsh Jain, CentralAgent AI14:58 · Nov 2026 · 1,218 views
Thumbnail for Your AI Agent Keeps Making the Same Mistakes Watch on YouTube
TL;DR
  1. 1

    Adding more context eventually increases cost and time while reducing accuracy and performance.

  2. 2

    Agents need experiential memory that records cases, strategies, skills, updates, inactive judgments, and forgotten information.

  3. 3

    A self-evolving memory system uses formation, updating, retrieval, observability, evaluation, and looping to improve across every teammate's work.

Summary

Sparsh Jain argues that an agent with more stored information does not necessarily become better. At enterprise scale, dumping all available context into a prompt can increase cost, effort, and latency while lowering accuracy. His team rebuilt memory around experience and judgment. Their system separates short-term memory from fact-based and experiential long-term memory. Experiential memory stores detailed cases, reusable strategies, and generalized skills. It also updates old notes, deactivates notes that no longer apply, retains inactive notes to preserve the path to a judgment, and forgets information that no longer matters. The system completes this lifecycle through formation, updating, and retrieval. Jain then describes verifier models, observability, evaluation loops, and dreaming cycles that let the system improve after sessions. Because one agent can learn from every teammate, its accumulated experience can grow faster than any individual person's experience.

Key ideas
00:30

More context stops helping once the memory base becomes large

A senior executive told Sparsh Jain that the agent kept repeating a mistake after a human corrected it, including in later sessions with another team. The first response was to provide the agent with all available context and memory. Jain says this worked at the start, when the context base was small. At enterprise scale, the context became too large. Supplying every fact increased cost, effort, and overall time, while reducing accuracy and performance. His conclusion is that more context does not equal better intelligence after a certain point. The system needs a way to turn experience into better judgment instead of repeatedly passing along every piece of information.

02:15

Experiential memory adds judgment to ordinary facts and session history

Jain says humans improve because they learn from experience, make mistakes, and course-correct. He describes agent memory as short-term working memory plus long-term memory. Long-term memory contains fact-based memory and experiential memory. The experiential layer stores abstracted learnings from work with the agent. Jain divides it into case-based, strategy-based, and skill-based memory. Case-based memory contains the detailed situation and lesson. Strategy-based memory records what worked in a situation and what to use in a similar one. Skill-based memory generalizes a lesson so the agent can reuse it across cases. Jain says adding this layer produced a 10x performance gain for his team.

04:18

Memory must change over time as systems and environments drift

Jain says building memory infrastructure exposed a larger problem: the systems, data, and surrounding environment change quickly. An agent that works today may fail a month or a year later. After many experiments, his team treated memory as something that must evolve rather than something built once. The lifecycle has three parts: formation, updating, and retrieval. Updating adds notes from new facts or experiences, changes older notes, and deactivates notes that are no longer useful. The system therefore keeps adjusting its judgments as the world and its own work change.

05:29

Inactive notes preserve the path that led to a current judgment

Jain says retaining only the final judgment can cause an agent to return to an older mistake. His system keeps inactive notes instead of deleting every superseded conclusion. A surface-level research task can use active notes. Deeper research can inspect active and inactive notes, which preserve the temporal journey that led to the current judgment. This gives the agent access to earlier reasoning when it needs to understand how a conclusion changed. The memory lifecycle also includes forgetting. Jain uses Solomon Shereshevsky, a person described as having near-perfect recall, to explain why total retention can be harmful. When every thought receives equal priority, even simple decisions become overwhelming.

08:37

Retrieval limits the performance of everything built behind it

Jain says a system's maximum performance cannot exceed the performance of its retrieval. He illustrates this with an agent asked to pull directions to a home when the user owns homes in San Francisco and New York. If retrieval returns the New York home when the user meant San Francisco, the rest of the memory and knowledge system cannot fix the result. Formation and updating may have worked correctly, but the system received the wrong information at the point of use. Jain therefore treats retrieval as part of the memory lifecycle, alongside formation and updating, rather than as a minor implementation detail.

08:17

A verifier and evaluation loop can make memory updates less manual

Jain says his team initially built experiential and evolving memory systems, but the process remained manual. He argues that newer models are capable of acting as verifiers, reducing the need for humans to review every change. Around the memory layer, his team adds observability. They combine it with looping and evaluation so that each session or day can produce learnings, improve the system, test changes independently, and then apply them to the real system. Jain describes this as a move away from writing only prompts and toward writing loops that make the infrastructure more autonomous.

10:46

Dreaming cycles clean and reorder memory between periods of work

Jain compares the system's cleanup process with sleep. He says people remove unimportant material, bring priorities forward, and rearrange what remains while resting. His agent infrastructure has looping and evaluation cycles that clean information that no longer matters, update what should be prioritized, and rearrange accumulated context. The goal is for the agent to perform better when it works again. This process addresses the problem created by continuous accumulation: even useful memories can make the system harder to use when they are not periodically reviewed and reorganized.

11:31

Shared agent experience can compound across a whole team

Jain says an agent working with him gains his experience, while an agent working with another teammate gains that teammate's experience too. With ten team members, the agent can accumulate experience from all ten people rather than from one person's work alone. He connects this shared memory to teams made from humans and AI agents. He predicts that more autonomous companies will have agents collaborating with each other and retaining what they learn. His claim is that performance can compound because the agent learns from every team member and across different domains and systems.

"This experiential layer can be doubled down into three different categories: the case-based, strategy-based, and the skill-based."03:11
Who should watch
  • You are building an agent that receives more context and memory but still repeats corrections or old mistakes.
  • Your system needs to update judgments as data, tools, and operating conditions change.
  • You want one agent to learn from work done by several engineers and need a design for memory cleanup, evaluation, and retrieval.