Your RAG is Tripping, Here's the Real Reason Why

Benjamin, EyeLevel.ai06:13 · Feb 2025 · 803 views
Thumbnail for Your RAG is Tripping, Here's the Real Reason Why Watch on YouTube
TL;DR
  1. 1

    RAG errors usually come from poor ingestion and retrieval, especially when enterprise documents contain tables, figures, and scattered context.

  2. 2

    EyeLevel.ai preserves document context by creating semantic objects with original text, generated metadata, and separate search and completion versions.

  3. 3

    EyeLevel.ai avoids vector databases and uses multi-field search plus fine-tuned models to improve retrieval accuracy.

Summary

Benjamin argues that many RAG hallucinations originate in the retrieval system rather than in the LLM or prompt. Enterprise documents create problems because text can be extracted incorrectly, context can disappear during chunking, and visual information can be ignored. EyeLevel.ai addresses these issues during ingestion. Its pipeline uses a fine-tuned vision model to locate text, tables, and images, then sends them through multimodal processing. The resulting semantic objects retain the original chunk, generated metadata, and rewritten versions for search and completion. Queries are rewritten into a compatible format, searched across the whole object, and re-ranked with a fine-tuned LLM. Benjamin says Air France and Dartmouth have reported accuracy above 95%, while a recent study of the platform reached 98% on complicated real-world documents. The talk is a product explanation, but it gives a clear account of why ordinary chunking and vector search can fail on enterprise material.

Key ideas
00:58

RAG errors often come from ingestion and retrieval quality

Benjamin says RAG applications can have error or hallucination rates as high as 35% when the knowledge base contains complicated enterprise documents. He places the source of these errors in RAG itself, especially the quality and relevance of retrieved content, rather than in the LLM or the prompt. He groups the content problems into three types: text that is extracted badly, information around a chunk that disappears during chunking, and visual elements that are never extracted. He says these ingestion problems are common, while advanced techniques to address them can take hundreds of hours to implement.

02:04

Enterprise documents need visual and multimodal ingestion

For Air France, the knowledge base contains hundreds of thousands of documents with tables, figures, and text scattered across pages. EyeLevel.ai starts by running a vision model fine-tuned with millions of documents to identify where images, tables, and text appear. Dedicated multimodal pipelines then extract the visual and written information. This approach treats a document as more than a stream of OCR text. Benjamin presents it as a response to the way enterprise documents distribute meaning across layouts, figures, and written passages.

02:49

Chunking can remove the context needed to answer a question

Benjamin compares chunking a document with asking questions about a book while receiving random paragraphs. The extracted paragraph may be relevant, but the system can lose the section name, the document identity, and other surrounding information. EyeLevel.ai creates semantic objects to preserve that context. Each object contains the original chunk text and automatically generated metadata about the material around it. The system also rewrites the content into two formats, one intended for search and another intended for answer completion.

04:37

Semantic objects preserve more information than vectorized OCR

Benjamin shows an Air France figure and contrasts two ingestion outcomes. OCR followed by vectorization and storage in a vector database leaves substantial information behind. EyeLevel.ai's pipeline produces a richer representation that includes both a search version and a completion version of the text. The object also retains the original text and generated metadata. The point is to make the searchable representation carry the document context and structure that a flattened text extraction would lose.

04:59

EyeLevel.ai searches across several views of each object

When a search query arrives, EyeLevel.ai rewrites it into a format compatible with its semantic objects. It then searches the original text, generated metadata, and search version together. A fine-tuned LLM re-ranks the results to improve their order and relevance. Benjamin says more than nine fine-tuned models are involved across ingestion and search. This design gives the retrieval system several representations of the same source instead of relying on one vectorized chunk.

02:27

The platform deliberately avoids vector databases

Benjamin says EyeLevel.ai does not use vector databases and argues that they may not be the best solution for many RAG applications. Its alternative is a multi-field search over semantic objects. The object combines source text, contextual metadata, and rewritten text, allowing retrieval to use more than a single embedding representation. His argument is tied to the document problems he describes, especially lost layout, missing context, and visual content that never reaches the index.

02:05

The product claims high accuracy on difficult documents

Benjamin says Air France and Dartmouth report that their RAG applications respond correctly more than 95% of the time. He also cites a recent study in which the platform achieved 98% accuracy against complicated real-world documents and outperformed some popular market solutions by as much as 120%. These figures are presented as the result of the ingestion and retrieval pipeline. He says users can build enterprise production applications in minutes rather than months.

"The source of these errors is rarely the LLMs or the prompts. Instead, it's typically RAG itself, or more specifically, the quality and relevance of retrieved content."00:58
Who should watch
  • You are building a RAG system over PDFs or other enterprise documents with tables, figures, and complex layouts.
  • Your retrieval results contain relevant-looking text but lose the section, document, or visual context needed for a correct answer.
  • You want to compare vector search with multi-field retrieval and understand where a commercial RAG platform puts its engineering effort.