# Forget RAG Pipelines: Build Production-Ready Agents in 15 Minutes

Nina Lopatina & Rajiv Shah, Contextual AI | AI Engineer World's Fair 2025 | 1:15:43

Source: https://www.youtube.com/watch?v=lArgRvBV3tQ
Channel: AI Engineer (https://www.youtube.com/@aiDotEngineer). Summarised by AIE Talks.
Page: https://aietalks.com/talks/forget-rag-pipelines-build-production-ready-agents-in-15-minutes
Published: 2025-06-27
Tags: evals, mcp, rag, structured-outputs

## TL;DR
- RAG can be consumed as a managed service instead of being assembled and maintained from separate extraction, storage, retrieval, reranking, and generation systems.
- Accurate document parsing, hybrid retrieval, reranking, grounded generation, citations, and claim-level checks work together to reduce unsupported answers.
- Teams can evaluate agents with natural-language unit tests, customize their settings through an API or UI, and connect the resulting agent to clients such as Claude Desktop through MCP.

## Summary
Rajiv Shah and Nina Lopatina walk through Contextual AI's managed RAG platform, from workspace setup and document ingestion to agent creation, evaluation, and MCP integration. Their demo uses NVIDIA financial reports and spurious-correlation documents to test table reasoning, cross-document retrieval, citations, and the system's ability to repeat unusual source claims with their caveats. Shah explains the platform's document understanding, chunking, hybrid retrieval, instruction-following reranker, grounded language model, and claim-level grounding checks. Lopatina shows how natural-language unit tests can score an agent on numerical accuracy, causation, evidence, limitations, synthesis, and relevance. The workshop also covers modular APIs, JavaScript support, VPC deployment, consumption pricing, and a basic MCP server for Claude Desktop or Cursor. The speakers are candid about unfinished areas, including complex document extraction, structured-data queries, scale, permissions, government cloud deployment, and frequently updated content.

## Key ideas
### RAG can be managed like other infrastructure services
[01:52](https://www.youtube.com/watch?v=lArgRvBV3tQ&t=112s)
Rajiv Shah argues that teams should be able to consume RAG the way they consume language models, embeddings, or vector databases. Contextual provides a no-code path for business users, configurable orchestration for developers, and modular components for teams that already have a pipeline. Developers can adjust querying and generation, while existing systems can use only a parser or reranker. Shah says assembling BM25, reranking, vector storage, and other models is enjoyable once, then becomes a maintenance burden as document volume grows.

### The workshop starts with document ingestion and an agent
[06:56](https://www.youtube.com/watch?v=lArgRvBV3tQ&t=416s)
Nina Lopatina sets up a workspace at app.contextual.ai, creates one API key, and loads NVIDIA quarterly reports plus spurious-correlation reports into a data store. The platform exposes document metadata and lets users inspect how files were parsed. For tables, the inspection view contains raw text and a rendered preview, preserving both the table structure and its contents. Lopatina then creates an agent with a system prompt and asks questions through the UI and API, showing that the same agent can be integrated programmatically.

### Accurate parsing matters most when documents contain tables and figures
[31:03](https://www.youtube.com/watch?v=lArgRvBV3tQ&t=1863s)
Shah says simple PDFs are easy to use in demos, while complex tables, image-only pages, charts, and multimodal documents create downstream problems. A shifted table value or an invented detail from a vision model can damage retrieval and generation later. Contextual's parsing flow adds metadata, performs layout analysis, applies OCR when needed, captions images, uses a table extraction mode, and preserves section structure. It produces a Markdown or JSON-style representation before chunking the document and attaching bounding boxes for source attribution.

### Query processing combines reformulation, hybrid retrieval, filters, and reranking
[35:16](https://www.youtube.com/watch?v=lArgRvBV3tQ&t=2116s)
The query path can translate a request, use conversation history, expand abbreviations, or decompose a long question into smaller subqueries. Each reformulated query can pass through semantic and lexical search, including BM25. A data-store filter can remove irrelevant document groups when the query identifies a company or subject. Shah describes a typical flow that retrieves roughly 100 chunks and then uses an instruction-following reranker to select a smaller set, with the default output described as 15 results. Developers can give the reranker instructions, such as preferring recent documents.

### Grounded generation and claim checks address different failure points
[38:01](https://www.youtube.com/watch?v=lArgRvBV3tQ&t=2281s)
Shah separates hallucination control into retrieval quality, model grounding, and post-generation checks. Contextual uses a language model trained specifically to respect supplied context rather than answer from general knowledge. The model can distinguish factual content from commentary and can be configured to return only facts. A grounding check decomposes each response into claims, compares them with the retrieved documents, and highlights unsupported claims in the UI. Source attribution also links answers to document images and locations, so users can inspect where a number came from.

### The demo tests quantitative reasoning and resistance to conventional answers
[20:54](https://www.youtube.com/watch?v=lArgRvBV3tQ&t=1254s)
Lopatina asks for NVIDIA's annual revenue by fiscal year using quarterly data, and the agent adds the quarters while showing source references from separate files. It also identifies when data-center revenue overtook gaming revenue, locating the crossover in Q1 fiscal year 2023. With spurious-correlation reports, the agent returns the supplied correlation coefficient and the document's warning about data dredging. When asked to infer causation, it rejects that conclusion. When the requested relationship is not supported across the documents, it says the information is unavailable.

### Natural-language unit tests turn response quality into specific checks
[41:14](https://www.youtube.com/watch?v=lArgRvBV3tQ&t=2474s)
The evaluation workflow uses Contextual's LMUnit model as a judge. Users provide prompts and generated responses, then define tests in ordinary language. Lopatina's examples check numerical extraction, the distinction between correlation and causation, multi-document calculations, acknowledgment of limitations, evidence for quantitative claims, and unnecessary extra information. Scores run from one to five. One NVIDIA revenue response received 2.2 for avoiding unnecessary information, which points to a possible system-prompt change. The speakers recommend running a batch rather than relying on one query and using category plots to find areas that need work.

### MCP makes the RAG agent available inside other clients
[48:09](https://www.youtube.com/watch?v=lArgRvBV3tQ&t=2889s)
Shah demonstrates a basic Contextual MCP server that lets Claude Desktop or Cursor call a RAG agent when a question matches its subject. The setup uses a repository, a server.py file, Contextual APIs, and a descriptive tool docstring that tells the client what the agent handles. The client configuration points to the local server. Shah shows that separate servers can support technical and financial queries. He also mentions that the same APIs and SDKs can support custom applications without using Contextual's UI.

## Notable quotes
- "We can treat Rag just like any other managed service." (01:52)
- "If we don't do a great job at extraction, if we don't do a good job at retrieval, well, then we're not going to give good information to that generator model." (29:45)
- "We want it to instead stay grounded, respect the context that we're giving." (15:58)
- "So we can run this live at time be able to get those groundness scores show you that with the response as well." (40:31)
- "The idea here is to go in that same space where there's a lot of these other companies doing this parsing thing and making sure that we have a module that you can just stick in and replace and kind of use with ours." (1:10:47)

## Tools & references mentioned
- Contextual AI
- Hugging Face
- NVIDIA
- Qualcomm
- Claude Desktop
- Cursor
- MCP
- BM25
- OpenAI
- Gemini
- Snowflake
- LMUnit
- Ragas
- Firecrawl
- PDFPlumber
- BERT

## Who should watch
- You are maintaining a RAG system assembled from separate parsers, vector stores, retrievers, rerankers, and language models, and want to see how a managed alternative is configured.
- Your documents contain financial tables, charts, images, or unusual source claims, and you need to test parsing, citations, quantitative answers, and unsupported claims.
- You are building an application in JavaScript or TypeScript and want a modular RAG API or an MCP connection for Claude Desktop or Cursor.

## Related talks

- [Building Production-Ready RAG Applications](https://aietalks.com/talks/building-production-ready-rag-applications) (Jerry Liu, LlamaIndex, 18:35)
- [RAG Agents in Prod: 10 Lessons We Learned](https://aietalks.com/talks/rag-agents-in-prod-10-lessons-we-learned) (Douwe Kiela, Contextual AI, 16:56)
- [RAG for VPs of AI](https://aietalks.com/talks/rag-for-vps-of-ai) (Jerry Liu, LlamaIndex, 26:51)
- [RAG at scale: production-ready GenAI apps with Azure AI Search](https://aietalks.com/talks/rag-at-scale-production-ready-genai-apps-with-azure-ai-search) (Pablo, Microsoft, 21:53)
- [OpenRAG: An open-source stack for RAG](https://aietalks.com/talks/openrag-an-open-source-stack-for-rag) (Phil Nash, IBM, 15:52)