Vector retrieval can return the wrong answer when documents use highly concentrated, repeated language or when naive chunking combines related facts.
2
Writer combined context-aware splitting, a specialized text-to-graph model, JSON stored in a Lucene-based search engine, knowledge graphs, and Fusion-in-Decoder.
3
Writer reported higher accuracy and faster responses than seven vector search systems on Amazon's RobustQA dataset, while graph relationships enabled multihop answers and exposed retrieval sources.
Summary
Sam Julien explains why Writer moved beyond standard vector retrieval for enterprise RAG. Large companies often have terabytes of documents with repeated terms, similar product specifications, and facts spread across pages. Nearest-neighbor search can select the wrong passage or lose the relationship between facts. Writer's approach converts context-aware chunks into graph nodes and edges with a specialized model, stores the resulting data as JSON in a Lucene-based search engine, and combines graph retrieval with Fusion-in-Decoder. Julien presents this as a historical account rather than a production blueprint. The team changed its design when graph databases created scaling, cost, and query challenges. He reports that the resulting system performed best on Amazon's RobustQA benchmark against seven vector search systems, with the fastest response time in that comparison. The graph also supports multihop questions, source display, subqueries, and answers involving complex document layouts.
Enterprise documents become difficult for vectors when their language is dense and repetitive
Julien says enterprise data is often measured in terabytes and uses specialized language with little variation between documents. A mobile phone company might have thousands of documents that all mention megapixels, cameras, and battery life. When a user asks the system to compare two phone models, vector retrieval can return many plausible passages without preserving enough structure to tell them apart. This is different from a broad collection where documents discuss clearly unrelated subjects. Writer's retrieval work focused on these concentrated patterns because they made ordinary similarity search unreliable for enterprise questions.
Naive chunking can separate or confuse facts that humans connect easily
Writer first tried regular search, then chunking documents, embedding the chunks, and using nearest-neighbor search. Julien gives an example about Apple's founding timeline: a human can identify that the Macintosh was created in 1984, but naive chunking places that fact near the Lisa's introduction in 1983. The vector search can then associate the wrong year with the Macintosh. Julien's point is that matching a chunk by similarity does not guarantee that the answer preserves the right relationship between facts. Context-aware splitting became part of Writer's later design.
A graph preserves relationships while retrieval selects evidence
Writer moved to graph-based RAG because a graph could preserve relationships in the source text and provide more context to the model. The system queried a graph structure to retrieve relevant documents and then generated an answer. Julien says this could be combined with full-text search and similarity search rather than used alone. The graph approach was especially useful for concentrated enterprise data, where many passages contain the same terms. The team's decision came from trying to solve customer problems rather than following the retrieval technique that was most popular at the time.
Writer replaced difficult graph infrastructure with components that matched its team's expertise
The team ran into four practical issues with graph databases at the time: converting data into a structured graph was costly at scale, database growth exposed expertise and cost limits, Cypher struggled with the required similarity matching, and language models handled text-based queries better than complex graph structures. Writer responded by building a specialized model that mapped data into graph nodes and edges and could run on CPUs or smaller GPUs. It also used context-aware splitting and chunking to preserve semantic relationships. Instead of keeping the graph in a graph database, Writer converted the graph structure to JSON and stored it in a Lucene-based search engine.
Julien revisits the original RAG paper, which describes a retriever and a generator without the prompt-and-context pattern that later became common. He describes Fusion-in-Decoder as an alternate path built from that architecture. Passages are processed independently in the encoder for linear scaling, then combined in the decoder for evidence aggregation. Writer applied knowledge graphs to Fusion-in-Decoder so the system could understand relationships among retrieved passages. Julien says the graph-based version improved efficiency and reduced cost, and Writer built its own implementation because it develops its own models. The team later published a white paper with its findings.
The reported benchmark result came from combining several retrieval techniques
Writer compared its retrieval system, which combined a knowledge graph with Fusion-in-Decoder, against seven vector search systems using Amazon's RobustQA dataset. Julien says Writer achieved the best accuracy and the fastest response time in that comparison. The talk does not present the benchmark as a single magic component. He describes the system as an assembly of techniques chosen through research and repeated problem solving. The product description gives the reported accuracy as 86.31 percent, while the talk itself focuses on the comparison's accuracy and response-time ranking.
Graph structure adds product capabilities that vector retrieval struggles to provide
The graph lets Writer expose snippets, subqueries, and sources that show how an answer was retrieved. Those details can appear in the product and in the API for developers. The system can also answer multihop questions by reasoning across multiple documents and topics. Julien says the approach helps when an answer is split across pages, or when a similar term does not exactly match the user's wording. His practical advice is to use whatever architecture preserves useful relationships, including a graph database, a search engine, or another design that stores graph-derived data.
"The main takeaway being as you're seeing in several of these talks like the first talk about hybrid search there are many different ways that you can get the benefits of knowledge graphs in rag."02:28
Who should watch
You are building RAG for product documentation, financial records, technical specifications, or another corpus where many documents repeat the same terms.
Your vector system retrieves plausible passages but loses the relationship between facts, mixes versions, or cannot answer questions spanning multiple documents.
You are deciding whether to add a graph database and want an example of using graph-derived data with a search engine instead.
