RAG remains difficult because document formats, chunking, embeddings, search methods, users, and questions vary between projects.
2
OpenRAG combines Docling for document processing, OpenSearch for hybrid retrieval, and LangFlow for orchestration and agentic search.
3
The stack provides an opinionated baseline that can run with local models and can be customised through LangFlow, APIs, and MCP.
Summary
Phil Nash argues that RAG has not become a solved problem just because context windows have grown. Real systems still have to handle difficult PDFs, changing embedding models, different chunking strategies, search methods, document sets, and user expectations. OpenRAG is IBM's open-source baseline for this work. It combines Docling for parsing and chunking, OpenSearch for keyword and vector retrieval, and LangFlow for visual orchestration and agents. The system supports hybrid search, filters, multiple embedding models, local or remote models, and agentic retrieval, where an agent chooses which searches to perform. Nash demonstrates document uploads, knowledge filters, cloud connectors, model settings, and the editable LangFlow implementation. He also shows how to add guardrails, calculators, and other tools. OpenRAG does not claim to solve every RAG problem. Its value is giving teams a working stack that they can evaluate and adapt to their own data.
RAG systems remain difficult because every project has different data and retrieval needs
Nash rejects the idea that larger context windows have made RAG irrelevant. A business with less than a million tokens of information might put everything into a prompt, but paying for that input on every question is another matter. Even the usual RAG recipe has many difficult parts: extracting text, choosing chunks, embedding them, searching, and selecting context. PDFs are troublesome, embedding models change, and teams can add summaries, chunk expansion, reranking, or query rewriting. The documents, users, questions, interaction patterns, and expectations differ from one system to another, so Nash argues that a high-quality baseline is useful.
OpenRAG combines three open-source projects into an extensible baseline
OpenRAG brings together Docling for document processing, OpenSearch for search indexing, and LangFlow for visual orchestration and agents. Nash presents the stack as opinionated because it makes default choices, while leaving room to change the parts that matter for a particular application. The goal is to avoid rebuilding every common RAG component from scratch. OpenRAG can use remote model providers or locally hosted models, and the whole system can run offline. That includes Docling, which Nash says can operate in air-gapped situations without external services.
Docling handles document structure instead of treating every file as plain text
Docling supports HTML, Markdown, Word documents, slides, spreadsheets, audio, video, and PDFs. Its standard PDF pipeline uses focused models for tasks such as text, table, and image extraction, layout analysis, and OCR. A separate vision-language model pipeline uses the Granite Docling 258 million parameter vision model to process a PDF in one step. Docling produces an intermediate Docling Document with structure represented in an XML-like format called DocTags. Those DocTags can become Markdown, HTML, or JSON. Its chunker uses the document hierarchy to create chunks that preserve how the source is organised.
OpenSearch provides hybrid retrieval, filters, and options for changing embedding models
OpenRAG does not prescribe one embedding provider. It supports external providers such as OpenAI and What's Next AI, along with Ollama for local embeddings. OpenSearch stores the embedded chunks and supports vector search, keyword search, filtering, and aggregation. OpenRAG uses hybrid vector and keyword search by default, with filters for more targeted retrieval. It can also search across multiple embedding models, although Nash notes that this slows vector search. The default setup uses the JVector KNN plugin, whose live indexing and disk-based architecture mean the entire index does not have to fit in memory.
Agentic retrieval lets the model decide which searches to run
Traditional RAG embeds a user query, retrieves the nearest chunks, and passes the top results to a language model. OpenRAG instead gives the query to an agent with instructions and tools. The agent decides what searches to perform, can run as many searches as needed, and decides how to use the results. Nash demonstrates this with a question about OpenRAG. The answer is already present in the agent's prompt, so no knowledge search is needed, but the agent still checks the current date. When asked about LangFlow, it searches the documentation and produces an answer with follow-up suggestions.
OpenRAG's interface exposes documents, chunks, filters, connectors, and model settings
The knowledge section lets users upload an individual file or a whole folder, inspect documents and chunks, and create filters based on stored data. Those filters can restrict a chat interaction to particular documents. Google Drive, SharePoint, and OneDrive connectors let users sync document directories after configuring Google OAuth. Settings expose cloud and local model providers, system instructions, embedding models, chunk size, and chunk overlap. Docling options include table structure capture, OCR, and picture descriptions. Nash notes that adding more models to the ingestion pipeline makes processing slower.
LangFlow makes the retrieval agent editable instead of locking users into the defaults
The 'edit in LangFlow' control opens the implementation of the chat and generation flow. The agent receives chat input through a prompt template that can include knowledge filters. Its tools include a URL ingestor exposed through an MCP server, a calculator, and OpenSearch's multi-model embedding component. Nash adds guardrails by placing a guardrail component between chat input and the prompt, then parsing the result and routing failures to a chat error message. Because the flow is editable, teams can add their own components and models. OpenRAG also exposes an API and an MCP server so other agents can use it.
OpenRAG is a baseline whose usefulness still depends on the data and users
Nash answers the question of whether RAG is solved by pointing back to the application: the answer depends on the data and the users. OpenRAG provides a baseline made from Docling, OpenSearch, and LangFlow, while leaving room to tune the stack for a particular dataset. He identifies version 0.4.0 as the current release in the talk and invites people to try it, contribute to the project, or work on the component projects. The front end is a Next.js application, while the rest is a Python application.
