# Beyond RAG: Build a Relational Context Engine from Scratch

Peter Werry, Unblocked | AI Engineer | 38:19

Source: https://www.youtube.com/watch?v=spxoDAfh5ig
Channel: AI Engineer (https://www.youtube.com/@aiDotEngineer). Summarised by AIE Talks.
Page: https://aietalks.com/talks/beyond-rag-build-a-relational-context-engine-from-scratch
Published: 2026-10-11
Tags: context-engineering, guardrails, rag

## TL;DR
- RAG based on semantic similarity is only half of retrieval because engineering questions often include dates, states, authors, and relationships.
- A reliable relational context engine needs schema discovery, identity resolution, query planning, validation, and retries.
- The same query engine can power both a structured query interface and a chat agent that reasons over the returned data.

## Summary
Peter Werry argues that vector retrieval cannot answer many practical engineering questions on its own. A request such as "PRs I merged last week" requires the system to understand an author, a merge state, and a time range. His workshop builds the structured half of retrieval by having a model write database queries instead of answering directly. The implementation discovers a schema from sampled documents, maps human names to system identities, supplies the current date during query planning, and restricts generated queries with procedural validation. Validation is deny by default, with allow lists for operators and fields. Errors from validation or execution go back to the model for another attempt. The workshop also adds full text search when exact matching fails, then uses the same engine behind a chat agent. Werry is clear that the validation layer needs more work before an internal deployment.

## Key ideas
### Semantic retrieval cannot answer questions with time and relationship constraints
[00:52](https://www.youtube.com/watch?v=spxoDAfh5ig&t=52s)
Werry defines RAG for the workshop as semantic similarity search or vector retrieval, then says it is only half of retrieval. Engineering context is spread across pull requests, review comments, tickets, documents, wikis, and chat. Those records also have authors, dates, states, labels, and relationships. A request such as "PRs I merged last week" may retrieve semantically similar pull requests, including one merged three years ago. The answer requires the author, merge state, and temporal condition to be checked as structured data.

### The model should write a database query instead of answering the question
[06:21](https://www.youtube.com/watch?v=spxoDAfh5ig&t=381s)
The engine treats the language model as a query writer. It receives a natural-language request and produces a query that accesses the database. Werry says query generation alone is insufficient for reliable operation. The model needs to know the schema and relationships, resolve names such as Peter to a GitHub login, and produce output that can be checked before execution. The workshop builds these pieces in six stacked pull requests, starting with a stubbed query playground and gradually making it capable of running queries.

### Schema discovery can infer a usable structure from sampled documents
[10:19](https://www.youtube.com/watch?v=spxoDAfh5ig&t=619s)
When the source schema is unavailable, the engine samples 300 documents and merges their nodes into a single tree. It skips nulls because missing fields cannot constrain a type. Enum extraction uses procedural code rather than relying entirely on a model, which Werry describes as more deterministic. The resulting schema describes the GitHub pull request data and creates indexes. That schema is then passed to the model when it synthesizes a query. Werry also says this approach is useful for exploring arbitrary databases, although Unblocked uses static schemas when it needs correctness to be tightly controlled.

### Identity resolution maps human names to system identifiers
[12:28](https://www.youtube.com/watch?v=spxoDAfh5ig&t=748s)
A person's name is not always the same as their account name. Werry uses his own example: asking for Peter's pull requests will fail if the database stores a different GitHub login. The engine therefore resolves names before it builds the query. In the workshop, simple fuzzy matching based on Levenshtein distance is enough to connect the natural-language name to the stored login. Werry says an LLM could synthesize these mappings dynamically, but the demonstration uses straightforward procedural logic.

### Query planning needs schema context, forced extraction, and the current date
[14:44](https://www.youtube.com/watch?v=spxoDAfh5ig&t=884s)
The query planning step passes the discovered schema to the model and forces a tool call for name extraction and query synthesis. This prevents the model from skipping the identity step. The prompt also includes the current date and time so the model can generate temporal conditions such as "last week." In the demonstration, the engine turns "open PRs by Peter" into a query that maps Peter to the GitHub login "puery." The model generates the query, while the engine executes it against the data.

### Validation must allow only safe operators and restore hidden tenant metadata
[16:33](https://www.youtube.com/watch?v=spxoDAfh5ig&t=993s)
Generated queries are untrusted. Werry warns that clauses such as "where" or "function" could allow JavaScript to run on the server, creating a prompt injection risk. The engine therefore uses an allow list of permitted operators and defaults to denial. Hidden fields can also expose internal metadata. For multi-tenant data, Werry says the system should remove any model-generated tenant field and inject the trusted tenant metadata itself. He recommends procedural code for these checks rather than asking an LLM to perform them.

### Retries turn validation and execution errors into feedback for the model
[20:00](https://www.youtube.com/watch?v=spxoDAfh5ig&t=1200s)
The plan can fail validation because it contains a forbidden operator, or it can fail during execution because the query combines incompatible clauses. The engine sends the error back to the model and allows another attempt up to a configured maximum. In the demonstration, the model eventually removes a disallowed regular expression, but its exact-match replacement cannot find "authentication." Adding full text search fixes that case. The final query combines full text search with author and state filters.

### A chat agent can reason over the same query engine
[22:58](https://www.youtube.com/watch?v=spxoDAfh5ig&t=1378s)
The query playground and chat tab use the same backing engine. The query interface generates and runs structured queries directly. The chat agent adds reasoning over the returned data. Werry demonstrates a question about whether the team worked on authentication-related tasks last week. The agent generates a query, retrieves the data, and reasons about the result. In the recap, he lists schema-aware prompting, dynamic schema discovery, identity resolution, validation, and error retries as reusable implementation ideas.

## Notable quotes
- "RAG is only half of retrieval and today we're going to build the other half which is turning natural language into real database queries." (00:52)
- "The model doesn't actually answer the question in this paradigm. It writes the query that accesses the database." (06:21)
- "You want to create basically an allow list of allowed operators and potentially an allow list of allowed fields." (16:57)
- "The error loop is actually one of the most important things." (24:24)

## Tools & references mentioned
- Unblocked
- RAG
- GitHub
- Linear
- Jira
- Notion
- Confluence
- Slack
- Microsoft Teams
- MCP
- Levenshtein distance

## Who should watch
- You are building retrieval over engineering data and semantic similarity keeps returning records with the right meaning but the wrong date, author, or state.
- Your application lets a model generate database queries and you need a concrete pattern for validation, identity mapping, and retries.
- You want one structured retrieval engine that can support both direct query interfaces and a conversational agent.

## Editor's note

Peter Werry is clear that the validation layer needs more work before an internal deployment. Kitaru can record the full agent run, including model calls and tool results, then replay it after validation code changes. The replay uses the recorded inputs and responses, so the team can check whether a previously failing query now behaves as intended without touching real systems.

Written by the AIE Talks editors (the Kitaru team), not by the speaker.

## Related talks

- [Beyond RAG: A Relational Context Engine That Cuts Token Burn](https://aietalks.com/talks/beyond-rag-a-relational-context-engine-that-cuts-token-burn) (Peter Werry, Unblocked, 19:44)
- [Agentic GraphRAG: Simplifying Retrieval Across Structured and Unstructured Data](https://aietalks.com/talks/agentic-graphrag-simplifying-retrieval-across-structured-and-unstructured-data) (Zach Blumenfeld, Neo4j, 15:25)
- [RAG Needs a Map: Using GraphRAG to Retrieve Connected Context](https://aietalks.com/talks/rag-needs-a-map-using-graphrag-to-retrieve-connected-context) (Nyah Macklin, Neo4j, 52:59)
- [GraphRAG: The Marriage of Knowledge Graphs and RAG](https://aietalks.com/talks/graphrag-the-marriage-of-knowledge-graphs-and-rag) (Emil Eifrem, Neo4j, 19:15)
- [Context Engineering: Connecting the Dots with Graphs](https://aietalks.com/talks/context-engineering-connecting-the-dots-with-graphs) (Stephen Chin, Neo4j, 26:50)