# RAG is dead, right?? Kuba Rogut, Turbopuffer | AI Engineer Europe 2026 | 11:13 Source: https://www.youtube.com/watch?v=UM6sFg_jdlE Channel: AI Engineer (https://www.youtube.com/@aiDotEngineer). Summarised by AIE Talks. Page: https://aietalks.com/talks/rag-is-dead-right Published: 2026-06-09 Tags: caching, embeddings, rag, search ## TL;DR - RAG includes vector search, full-text search, grep, glob, regex, and filters, rather than only a single vector database call. - Agentic retrieval lets an agent search repeatedly and choose the retrieval tool it needs as it works through a task. - Indexing is cached compute: an upfront embedding cost can reduce repeated search work, tokens, time, and money at runtime. ## Summary Kuba Rogut argues that the claim that RAG is dead comes from defining RAG too narrowly. A retrieval system can combine semantic search with full-text search, grep, glob, regex, and filters. An agent can call these tools repeatedly, inspect the results, and search again when it still lacks the context needed for a task. He uses Cursor as an example of upfront codebase indexing followed by lightweight semantic searches, and contrasts it with Claude Code's per-session grep-based discovery. Cursor reported gains in answer accuracy, code retention, and dissatisfied requests, even though semantic search does not run for every query. Rogut describes embeddings as cached compute. Indexing costs more at the start, while repeated queries can become cheaper and faster. He closes with Jeff Dean's point that large context windows still need staged retrieval to find the right smaller set of tokens. ## Key ideas ### RAG includes many retrieval methods, not only vector search [01:39](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=99s) Rogut separates retrieval augmented generation from the narrow definition of a single vector search call. Retrieval can use vector search, full-text search such as BM25, grep, glob, regex, and basic filters. The generation step passes the selected material into an LLM. He says agentic search gives an agent tools that let it progressively find and reason over context. The agent can grep files, read one, decide that it still lacks what it needs, and search again until it can continue with the task. ### Cursor pays an upfront indexing cost to make code search cheaper later [03:15](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=195s) When a new codebase or branch opens in Cursor, the system parses, chunks, and embeds the code so it is available for semantic search. Cursor uses Merkle trees to compare codebases opened by people on the same team. If two codebases are similar, it copies existing data and only re-chunks and re-embeds changed files. Rogut presents this as an upfront cost that supports lightweight retrieval calls at runtime. The agent can ask a question such as how metadata filtering works and receive targeted results without rediscovering the codebase from scratch. ### Cursor's semantic search measurements show gains even when the tool is not always used [04:44](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=284s) Rogut cites Cursor's internal context benchmark, where semantic search produced roughly a 12.5% or 13.5% average increase in answer accuracy across models. He also cites an almost 24% increase for an earlier Composer model. In an online A/B test, Cursor reported a 2.6% increase in code retention in large codebases and a 2.2% decrease in dissatisfied user requests. Rogut explains that these online numbers look smaller because semantic search does not benefit every query, so the tool is not used for every request. ### Claude Code and Cursor make different retrieval cost tradeoffs [06:10](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=370s) Rogut says Claude Code does not use vector search. Boris Cherny described early versions that used a local vector database but did not find that approach useful for Claude Code. Rogut compares Claude Code's per-session discovery with Cursor's indexed approach. Claude Code may grep, read, assess, and repeat each time an agent needs to understand something. Across many agents, developers, and days, the same discovery work can happen repeatedly and consume tokens. Cursor pays to parse and embed once, then uses a lightweight query at runtime. ### Embeddings are cached compute whose value depends on repeated use [06:40](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=400s) Rogut describes embeddings and semantic search as cached compute. In his example, an agent using per-session discovery repeatedly spends tokens to learn how metadata filtering works. The indexed approach performs parsing and embedding upfront, then retrieves a smaller result set when the agent asks its question. He says this can save tokens, time, and money, and make the agent faster. The tradeoff is deliberate: a team chooses whether the upfront indexing cost is worthwhile based on how often the same code or information will be queried. ### Agentic retrieval searches iteratively instead of making one vector database call [08:38](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=518s) Rogut says sophisticated customers are moving away from the simple pattern of running vector search once and placing the result in the context window. Their agents make many calls, reason through several steps, and use semantic search, full-text search, or other methods as needed. They fetch only what the particular task requires. Retrieval therefore becomes an iterative process in which the agent searches, learns from the result, and searches again to understand more. Rogut connects this pattern to products that were difficult to build with one-shot retrieval. ### Stage retrieval narrows huge stores to the tokens an agent actually needs [09:44](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=584s) Rogut cites Jeff Dean's point about very large context windows. Even if a model could accept a trillion tokens, Dean said it would still need a lightweight mechanism to narrow them down into smaller useful sets. Rogut quotes the idea as, "you don't need a trillion at once, you need the right million." He applies this to Turbopuffer customers with trillions of tokens stored in the system. The retrieval problem is selecting the right 100,000, 10,000, or million tokens to pass into the context window. ## Notable quotes - "Retrieval is not just vector search." (01:59) - "It's really giving the agents a set of tools to progressively and iteratively find and reason over context." (02:49) - "Embeddings and semantic search are kind of cached compute." (06:42) - "You don't need a trillion at once, you need the right million." (10:09) ## Tools & references mentioned - Turbopuffer - Cursor - Claude Code - Codex - Composer 2 - Boris Cherny - Jeff Dean - Gemini - BM25 ## Who should watch - You are deciding whether a code or knowledge search system should index data ahead of time or discover it separately for each agent session. - Your current RAG pipeline makes one vector search call and you want examples of how agents can combine search tools across several steps. - You work with large context windows or large corpora and need a way to narrow retrieved material before sending it to a model.