# Lessons from Studying Every Memory System

Shlok Khemani, Independent | AI Engineer World's Fair 2026 | 19:31

Source: https://www.youtube.com/watch?v=5ZGyKWjQDr0
Channel: AI Engineer (https://www.youtube.com/@aiDotEngineer). Summarised by AIE Talks.
Page: https://aietalks.com/talks/lessons-from-studying-every-memory-system
Published: 2026-08-12
Tags: context-engineering, memory, product-strategy

## TL;DR
- Consumer memory systems have moved from user-managed fact lists toward running profiles combined with tools that retrieve past conversations.
- ChatGPT and Claude make different compute trade-offs: ChatGPT uses a larger profile updated less often, while Claude uses a smaller profile updated daily.
- The hardest memory problem is connecting information across products and sources such as email, calendars, and photos, then noticing and resolving conflicts.

## Summary
Shlok Khemani examines how consumer AI memory has changed by reverse engineering ChatGPT, Claude, Gemini, and other products. ChatGPT began with a visible list of extracted facts, then moved to a dense running profile updated in the background. Claude started with retrieval tools over past conversations and later added a smaller, visible profile that updates daily. Khemani argues that memory has no standard architecture. Teams choose among profiles, retrieval tools, files, and other systems based partly on compute costs. A profile costs compute to update and costs again whenever it enters a context window. He also describes running profiles as a form of continual learning outside model weights. His strongest criticism concerns context across products. AI systems maintain separate memories, rarely reason over email or calendars, and may fail to notice contradictions even when evidence exists. The talk is a practical account of current design trade-offs and a pointed criticism of how personal AI products handle context.

## Key ideas
### Consumer memory began as a visible list of extracted facts
[02:09](https://www.youtube.com/watch?v=5ZGyKWjQDr0&t=129s)
ChatGPT's first serious memory system arrived in February 2024. A user could say, "remember that I'm vegetarian," and the model would extract a fact, store it in a memory list, and add that list to every new conversation. Users could inspect and delete entries in settings. Khemani considers this a reasonable first implementation, but says it made users responsible for managing memory during ordinary conversations. It also preserved facts after they stopped being true. He gives the example of a memory saying he was going to Bengaluru even though he was in San Francisco, with the old fact still entering his context.

### A running profile removes manual work but can preserve false assumptions
[04:08](https://www.youtube.com/watch?v=5ZGyKWjQDr0&t=248s)
In April 2025, ChatGPT added what Khemani calls a running profile. Every few days, the system reviews recent conversations, extracts what it considers important, and updates a profile that enters every new conversation. The profile packs a lot of information into dense keyword-like clues, relying on the model to infer connections from them. Khemani says his profile had 16 sections and was almost 4,000 tokens long. This moved memory management into the background, but it did not solve staleness. Conversations about choosing between Thailand and Turkey led the profile to claim that he had travelled to both, even though he went only to Thailand.

### Claude initially treated memory as on-demand retrieval
[07:23](https://www.youtube.com/watch?v=5ZGyKWjQDr0&t=443s)
Claude's first memory system, released in August 2025, took a different approach from ChatGPT. It gave the model two tools instead of a user profile or fact list. One searched previous conversations by keyword or topic, and the other searched by time period, allowing queries such as what the user discussed last week. Each conversation began without user context, and the model retrieved information only when it decided that retrieval was needed. Khemani published a post calling this architecture the opposite of ChatGPT's approach. On the same day, Claude released a second version with a running profile.

### The two products converged while keeping different profile designs
[08:03](https://www.youtube.com/watch?v=5ZGyKWjQDr0&t=483s)
Claude's second version added a running profile that users could view in settings. It was about 1,000 tokens, used complete sentences instead of ChatGPT's dense clues, and updated every 24 hours. Users could request edits, manage earlier edits, and delete information that was no longer true. ChatGPT later added a tool for retrieving summarized context from past conversations, made a generated summary of its profile visible in some form, and allowed requested edits. It also deprecated the original fact list. Khemani's conclusion is that both systems now combine a profile with retrieval tools, but their profile size, update schedule, visibility, and editing behavior still differ.

### Memory has no standard architecture and belongs inside the product
[11:14](https://www.youtube.com/watch?v=5ZGyKWjQDr0&t=674s)
Khemani rejects the assumption that retrieval-augmented generation is the default answer for memory. He expected systems to chunk conversations, create embeddings, store them in a vector database, and run semantic search. ChatGPT and Claude evolved with other approaches. Gemini uses a running profile with creation and update timing logs. Agent systems such as Claude Code, OpenClaw, and Hermes use combinations of markdown files, heartbeat processes, knowledge bases, and skills. Because memory develops with the rest of a product, Khemani says serious teams should build it alongside their product rather than outsource it or treat it as an afterthought.

### Memory design is a compute budget decision
[12:30](https://www.youtube.com/watch?v=5ZGyKWjQDr0&t=750s)
A running profile has a maintenance cost and a serving cost. Maintenance depends on how often the profile is updated and how much compute each update uses. Serving costs rise when the profile is longer because it enters every conversation's context window. Khemani describes ChatGPT as choosing a larger profile, about 4,000 tokens, updated every few days. Claude uses about 1,000 tokens and updates every 24 hours. The first approach spends more on every conversation and less on updates, while the second spends less on serving and more on frequent updates. The design depends on how much compute the product can assign to memory.

### Running profiles already form a continual learning loop outside model weights
[13:47](https://www.youtube.com/watch?v=5ZGyKWjQDr0&t=827s)
Khemani describes a running profile as a continual learning process. The profile enters each conversation, the conversation adds new information, and a later synthesis step folds that information back into the profile. The loop repeats, but it happens outside the model's weights. He leaves open whether individuals will eventually get self-learning models whose weights update from personal data. That raises questions about what data would start the process, how the data would be generated, and who would pay for the training. He cites Guan's essay "Guardian Angels" as a detailed discussion of what a one-model-per-person future might look like.

### Memory cannot resolve facts that the system cannot bring into context
[15:39](https://www.youtube.com/watch?v=5ZGyKWjQDr0&t=939s)
Even a sophisticated memory architecture is limited by the information it can gather about a person. Khemani returns to the Thailand and Turkey mistake. The actual decision happened in a conversation with his partner, which ChatGPT could not hear, while flight and hotel bookings in his email provided evidence for Thailand. Even if ChatGPT had email access, he says it did not reason over that source or update its profile from it. The problem is not simply that the system reached the wrong conclusion. It did not notice that the information conflicted or become curious about resolving the gap. Khemani calls this a product problem.

### Personal AI products keep separate memories and force users to rebuild context
[17:34](https://www.youtube.com/watch?v=5ZGyKWjQDr0&t=1054s)
Khemani's personal stack includes chatbots, assistants, specialist applications, agents, and hardware devices. Each product builds its own memory, and those memories are not shared. When something in his life changes, he must update each system separately. The products also fail to reason over rich sources such as email, calendars, and photos. This leaves him rebuilding context from scratch across tools. He says the experience does not feel like personal AI in 2026, although the field itself is only about three years old and still has many unresolved design problems.

## Notable quotes
- "The point being that there is no one way to do memory." (11:14)
- "You have to really think about how much compute you want to put into memory." (13:38)
- "It's just that products today are not designed to help us with this." (17:09)
- "So for me, none of this feels like 2026." (18:08)

## Tools & references mentioned
- ChatGPT
- Claude
- Gemini
- Poke
- GPT-4
- Lance Martin
- Claude Code
- OpenClaw
- Hermes
- Guan
- Guardian Angels

## Who should watch
- You are designing memory for a consumer AI product and need to choose between profiles, retrieval tools, or file-based systems.
- Your team is treating memory as a generic RAG feature and wants concrete examples of how major products made different choices.
- You are building a personal assistant that needs to use information from email, calendars, conversations, and other applications instead of keeping isolated memories.