# AI on Your Lakehouse: Context Comes in Shapes, Not Queries Zach Blumenfeld, Neo4j | AI Engineer World's Fair 2026 | 1:59:10 Source: https://www.youtube.com/watch?v=kRkcNOsRyYg Channel: AI Engineer (https://www.youtube.com/@aiDotEngineer). Summarised by AIE Talks. Page: https://aietalks.com/talks/ai-on-your-lakehouse-context-comes-in-shapes-not-queries Published: 2026-07-23 Tags: agents, context-engineering, rag, search ## TL;DR - Agents can access warehouse tables and document stores while still missing the context needed to connect the right pieces. - A metadata graph can guide Text2SQL through table relationships without copying the warehouse data into Neo4j. - Trees, full-text search, and community detection give agents different views of document structure, themes, and connections. ## Summary Zach Blumenfeld presents a hands-on workshop for giving agents better context over lakehouse data. The example combines BigQuery tables with manuals, bulletins, recalls, and work-order records from a fictional auto repair chain. Blumenfeld builds three graph shapes. A connection shape captures databases, tables, columns, and join paths as a metadata graph, so an agent can write more accurate SQL while the data remains in the warehouse. A containment tree gives documents and sections a navigable outline, with links between related documents. A theme shape uses Leiden community detection to find groups of highly interconnected documents. These shapes support both narrow technician questions and broader questions about missing documentation, repeated repair problems, and mismatches between field work and documented procedures. The workshop uses Neo4j, NeoCarta, MCP, Claude Code, Cypher, and graph data science skills. Blumenfeld is clear that graphs do not replace vector or full-text search. They add structure that similarity search cannot provide on its own. ## Key ideas ### Lakehouse access does not guarantee the agent has the right context [02:02](https://www.youtube.com/watch?v=kRkcNOsRyYg&t=122s) Blumenfeld separates the lakehouse into structured warehouse data and unstructured documents. Text2SQL and vector search can retrieve pieces from both sides, but an agent may still miss how those pieces fit together. In the AutoFix example, the data includes work orders, vehicles, diagnostic codes, procedures, parts, manuals, safety bulletins, and recalls. The difficult questions are often broader than finding a repair for one vehicle. The organization may want to know what documentation is missing, which material is not being used, or which repair patterns repeat across the whole dataset. Blumenfeld says semantic search is especially weak at questions that require proving a negative or understanding relationships across many tables and documents. ### The workshop uses three graph shapes for different kinds of questions [06:19](https://www.youtube.com/watch?v=kRkcNOsRyYg&t=379s) Blumenfeld introduces a connection shape, a table-of-contents shape, and a themes shape. The connection shape provides a semantic layer over warehouse data. The table of contents is a containment tree for document libraries, folders, files, and sections, with additional links between documents. Themes surface global patterns and unknown groupings in linked documents. The first shape is used on structured data, while the tree and themes focus on unstructured documents. He describes the patterns as portable beyond BigQuery, with directions for Databricks and Snowflake. Together, the shapes let an agent move between schema understanding, document navigation, search, and broad pattern analysis. ### A metadata graph can guide SQL without moving warehouse records [21:47](https://www.youtube.com/watch?v=kRkcNOsRyYg&t=1307s) The sample BigQuery schema contains work orders, vehicles, diagnostic codes, procedures, work-order parts, and parts. NeoCarta reads metadata from BigQuery and creates a graph of the database, schema, tables, columns, representative values, and reference join paths. The graph does not copy the underlying warehouse records. Instead, the agent reads the metadata graph through an MCP server and uses it to build a Text2SQL query. In the example question, 'Which vehicle received part IC 2042?', the agent uses the recorded join paths to connect the relevant tables. Blumenfeld says this approach avoids the synchronization, scale, and security issues that can arise when production data is copied into a graph. ### A deterministic containment tree gives documents an outline and link structure [45:25](https://www.youtube.com/watch?v=kRkcNOsRyYg&t=2725s) The document graph models a technical library, its folders, documents, and nested sections. It also records section order and links from one document or section to another. Each node receives a hierarchical URI, such as a technical library followed by a manuals folder and a document name. An agent can use the URI to retrieve a subtree, choose a traversal depth, or follow links from a particular document. Blumenfeld contrasts this with heavier GraphRAG pipelines that use an LLM for entity extraction. The workshop's loader uses the existing document structure and named links, so it is deterministic, faster, and idempotent. It works best when document names, headings, and links already carry useful meaning. ### Full-text search adds lexical grounding to document navigation [1:06:23](https://www.youtube.com/watch?v=kRkcNOsRyYg&t=3983s) The workshop adds a Lucene-style full-text index over document and section nodes. Folders are excluded because they do not contain the searchable text. The search query first retrieves matching nodes and scores them, then can filter results by a hierarchical URI. That lets an agent search only under a particular recall, manual, or other subtree. Blumenfeld also uses semantic expansion, where the model expands a term such as 'engine shuttering' to related language such as 'misfire' or 'rough idle' before searching. He describes search as an augmentation to the outline, since an agent might search first and then traverse links, or navigate first and use search to locate relevant content. ### Leiden community detection surfaces themes from document links [1:22:24](https://www.youtube.com/watch?v=kRkcNOsRyYg&t=4944s) The themes shape starts with the links between documents. The workshop projects that graph into memory, collapses section-level links to the document level, and runs Leiden through Neo4j's Graph Data Science library. Leiden assigns documents to groups whose members are more interconnected with each other than with the rest of the graph. The result can reveal natural clusters around subjects such as braking, rotors, pads, hydraulic lines, or vehicle control systems. The output includes theme identifiers, cohesion information, shared link targets, and highly connected documents. The workshop does not ask an LLM to name the groups. The names and source documents come directly from the data, which makes repeated runs stable while limiting usefulness when the source labels are poor. ### Agents can combine the shapes for technician and estate-level analysis [1:43:22](https://www.youtube.com/watch?v=kRkcNOsRyYg&t=6202s) In the final demonstrations, the agent combines document search, outline traversal, the connection graph, and warehouse queries. For a technician asking about a vehicle identification number and diagnostic code, it finds the relevant document, follows cross-links to procedures, then queries work-order history to identify a replacement part. For a broader question, it compares documented procedures with field problems and finds diagnostic codes that lack documentation or cases where the documented repair does not match the observed work. A theme-based question groups bulletin and recall patterns with affected cars and work orders. Blumenfeld says these examples can run with the predefined tools, while the Neo4j CLI lets the agent inspect the graph schema and write custom Cypher when a question crosses the shapes. ### The graph shapes add structure rather than replacing search [1:05:47](https://www.youtube.com/watch?v=kRkcNOsRyYg&t=3947s) Blumenfeld rejects the idea that the graph replaces semantic search. He expects hybrid systems to use full-text search or vector retrieval alongside graph navigation. Search is useful for grounding a term and finding passages. Trees help an agent understand what is present and follow document structure. Themes support broad questions about clusters and coverage. Connection graphs guide joins across large warehouse schemas. The workshop keeps the structured and unstructured graphs separate, with the agent extracting a code, part, or other value from one side and using it to query the other side in real time. Blumenfeld says explicit links between the two could make that mapping more deterministic, but he leaves them out to keep the starting implementation simple and model agnostic. ## Notable quotes - "And really where graph can come in to help with these shapes is not only in these single questions that someone might have about how I have a broken part that I need to replace and how do I repair this vehicle, but oftentimes it's going to be on these estate level questions." (04:46) - "What we're doing is we're using the graph as a semantic layer." (27:02) - "And the benefits of having a deterministic load like this is number one, it's going to be item potent." (50:41) - "So, I don't think it's a replacement, but as we'll see later in the course when we get to some of the estate level questions, there's certain things that semantic search is not as great at answering that really require document navigation to understand how everything connects together." (1:05:47) - "So, the shapes that I are actually defined inside of our specs." (1:53:56) ## Tools & references mentioned - Neo4j - NeoCarta - BigQuery - Databricks - Snowflake - Graph Academy - Claude Code - Anthropic - MCP - Cypher - Neo4j CLI - Graph Data Science - Leiden - Lucene - PageIndex - Microsoft GraphRAG - Obsidian ## Who should watch - You are building an agent over a warehouse with many tables and need help selecting the right join paths. - Your documents have folders, sections, and cross-links that vector search does not expose clearly. - You want to test graph-based context patterns without moving all production records into a graph database.