# RAG and the MongoDB Document Model Ben Flast, MongoDB | AI Engineer World's Fair 2024 | 13:13 Source: https://www.youtube.com/watch?v=2Ey275TX4ZU Channel: AI Engineer (https://www.youtube.com/@aiDotEngineer). Summarised by AIE Talks. Page: https://aietalks.com/talks/rag-and-the-mongodb-document-model Published: 2025-02-08 Tags: caching, embeddings, rag, search ## TL;DR - RAG adds private, current application data to an LLM prompt so the model can answer with relevant context. - MongoDB stores vectors alongside application documents, allowing vector search and transactional data to work from the same document model. - Search Nodes let teams scale vector indexes separately from the transactional database resources that store application data. ## Summary Ben Flast explains how MongoDB's document model changes the way teams can build retrieval augmented generation systems. He starts with the basic RAG flow: embed a user's question, search for similar data, and send the retrieved documents with the question to an LLM. MongoDB extends this by storing embeddings directly inside JSON-like documents alongside application data. Atlas Vector Search uses HNSW indexes and an aggregation stage for approximate nearest-neighbor search, with controls for dimensions, similarity, candidate count, limits, and filtering. Flast also describes Search Nodes, which separate vector-search scaling from the resources used for transactional workloads. MongoDB integrations with LangChain, LlamaIndex, Microsoft Semantic Kernel, AWS Bedrock, and Haystack support features such as vector stores, semantic caching, and chat history. His examples show how one database can hold user data, meetings, conversation history, and vectors for agent applications. ## Key ideas ### RAG supplies the private and current context an LLM lacks [01:02](https://www.youtube.com/watch?v=2Ey275TX4ZU&t=62s) Flast begins with a simple limitation: an LLM connected to nothing cannot know something like the balance in your bank account. Models also have a training cutoff and do not contain a company's private data. RAG adds application data at prompt time. That data might include company information, product details, or order history already stored in the application database. In the usual flow, the user's question goes to an embedding model, the resulting vector is searched against a vector database, and the retrieved documents are sent to the LLM with the original question. The model then returns an answer to the user. ### MongoDB documents keep application objects together [03:12](https://www.youtube.com/watch?v=2Ey275TX4ZU&t=192s) MongoDB stores JSON-like documents in BSON, so an application can keep an object such as a customer or contact directly in the database. Flast contrasts this with relational designs where an application may assemble an object from several tables. A document can contain names, profiles, and other JSON data in one structure. He describes documents as able to model different forms of data, including JSON, tabular, key-value, geospatial, and graph data. The result, in his view, can make systems more efficient and productive to build, while MongoDB can scale horizontally through sharding. ### Embeddings can live beside the data they describe [05:00](https://www.youtube.com/watch?v=2Ey275TX4ZU&t=300s) Atlas Vector Search adds HNSW indexes for approximate nearest-neighbor search over MongoDB data. Flast's example document contains fields such as a symbol, quarter, and content, with an additional embedding field containing the vectorization of the whole document or a selected part of it. The embedding can also refer to data stored elsewhere and mapped back to the document. Vector fields can have up to 4,096 dimensions. An index definition specifies the vector path, dimensions, and similarity function. MongoDB keeps the index synchronized as documents change. ### Vector search exposes controls for relevance and cost [06:29](https://www.youtube.com/watch?v=2Ey275TX4ZU&t=389s) Flast describes MongoDB's dollar Vector Search aggregation stage. A query supplies the query vector, the document path containing the indexed vector, a limit for the number of returned results, and a number of candidates. The candidate setting controls how many entry points into the HNSW graph are used, which lets developers tune result accuracy. A filter can also be applied as a pre-filter while the graph is traversed, so the search considers documents relevant to the specific query. ### Search Nodes separate vector workloads from transactions [07:19](https://www.youtube.com/watch?v=2Ey275TX4ZU&t=439s) A transactional MongoDB deployment has a primary and two secondaries for durability and availability, but vector search can have a different resource profile. Search Nodes let teams place vector indexes on a separate node type and scale those nodes independently from the infrastructure holding transactional data. Flast says this lets a workload receive the resources it needs without scaling both parts together. Atlas keeps a unified interface and document model while allowing separate scaling decisions for search and database operations. ### MongoDB integrations add memory and retrieval features to RAG [08:13](https://www.youtube.com/watch?v=2Ey275TX4ZU&t=493s) Flast names integrations with LlamaIndex, LangChain, Microsoft Semantic Kernel, AWS Bedrock, and Haystack. These integrations provide more than a basic vector store. With semantic caching, a retrieved and augmented prompt can be compared semantically with cached requests. A cache hit returns the stored answer without another LLM call. With LangChain chat message history, an application stores prior messages in MongoDB and adds that history to later prompts, alongside vector search, to maintain continuity in a conversation. ### A single operational database can support agent data [10:45](https://www.youtube.com/watch?v=2Ey275TX4ZU&t=645s) Flast describes an AI teammate company that listens to meetings, tracks activity, retrieves information, and prompts a user with material needed for tasks such as writing an email or scheduling a project. MongoDB stores the company's vector data, user data, meeting data, and chat history. Some of that information is not a typical semantic-search corpus. It benefits from being held in a transactional database that also has vector search. Flast connects this model to agents that need more ways to store and interact with application data. ### Atlas includes deployment and security options around search [11:59](https://www.youtube.com/watch?v=2Ey275TX4ZU&t=719s) Flast closes by describing Atlas features around the database and search workload. He mentions security controls, privacy, uptime, and automation for application performance. Atlas can be deployed in more than 100 regions across major cloud providers. Search Nodes are included in those deployment choices, allowing teams to adjust where and how vector-search resources run. He also points to a forever-free Atlas tier where Vector Search is available. ## Notable quotes - "Without context there's only so much you can do with the LLM, and so that's where RAG comes in." (01:42) - "What you do is you take your embeddings, you add them directly into the documents that you're already storing in your database." (05:22) - "This allows you to really tune the amount of resources that you bring to bear to perfectly serve your workload." (07:31) - "You can use caching to reduce the amount of calls that are being sent to your large language model." (09:43) ## Tools & references mentioned - MongoDB Atlas - Atlas Vector Search - LangChain - LlamaIndex - Microsoft Semantic Kernel - AWS Bedrock - Haystack - HNSW - Search Nodes ## Who should watch - You are building a RAG or agent application and want vectors, user records, conversations, and other operational data in one database. - Your current design separates application objects across relational tables and you want to compare it with MongoDB's document model. - You need vector-search capacity to scale independently from the transactional database that stores the rest of your application. ## Related talks - [RAG is dead, right??](https://aietalks.com/talks/rag-is-dead-right) (Kuba Rogut, Turbopuffer, 11:13) - [When Vectors Break Down: Graph-Based RAG for Dense Enterprise Knowledge](https://aietalks.com/talks/when-vectors-break-down-graph-based-rag-for-dense-enterprise-knowledge) (Sam Julien, Writer, 15:47) - [Agentic GraphRAG: Simplifying Retrieval Across Structured and Unstructured Data](https://aietalks.com/talks/agentic-graphrag-simplifying-retrieval-across-structured-and-unstructured-data) (Zach Blumenfeld, Neo4j, 15:25) - [RAG in 2025: State of the Art and the Road Forward](https://aietalks.com/talks/rag-in-2025-state-of-the-art-and-the-road-forward) (Tengyu Ma, MongoDB, 18:48) - [Agentic GraphRAG: AI's Logical Edge](https://aietalks.com/talks/agentic-graphrag-ais-logical-edge) (Stephen Chin, Neo4j, 15:27)