Why Your Agents Need Decision Traces, Not Just Documents

Zach Blumenfeld, Neo4j20:12 · May 2026 · 13K views
Thumbnail for Why Your Agents Need Decision Traces, Not Just Documents Watch on YouTube
TL;DR
  1. 1

    A knowledge base helps an agent answer questions, while a context graph also gives it past decisions, precedents, and reasons for acting.

  2. 2

    Decision traces can be searched through both text similarity and graph structure, allowing similar past cases to surface even when documents use different words.

  3. 3

    Neo4j's open-source tools provide a scaffold for context graph applications, memory APIs, data connectors, ontology generation, and integrations with several agent frameworks.

Summary

Zach Blumenfeld explains context graphs as a way to give agents information for making decisions, rather than only information for answering questions. His financial analyst example combines customer data, transactions, and policies with past decision traces, precedents, and the reasons behind earlier outcomes. The demo uses hybrid retrieval: vector search finds semantically similar traces, while graph embeddings find traces with similar structures and relationships. Blumenfeld then shows the `uvx create-context-graph` scaffold, which generates a frontend, backend, graph data, demo records, and an MCP server for a chosen domain and framework. The underlying `neo4j-agent-memory` package combines short-term conversation history, long-term entities, and reasoning traces. It also extracts entities through spaCy, GLiNER, and an LLM fallback, followed by merging and deduplication. Blumenfeld is open about unfinished parts, including automatic writing of new decision traces and timestamp links between reasoning steps.

Key ideas
01:35

Context graphs give agents the information needed to choose an action

Blumenfeld contrasts a normal knowledge base with a context graph. A knowledge base can help a financial analyst agent answer questions about a customer, transactions, policies, risk factors, or a risk score. A context graph adds past decision traces, precedents, and information about why earlier decisions were made. That lets the agent recommend whether to accept or reject a request and explain the decision. In his model, systems of record contain facts, entities, and current state, while context graphs also carry precedents, causal chains, and expected outcomes.

03:04

A context graph connects entities, events, policies, and human reasoning

The graph model includes entities and events such as decisions, transactions, and approvals. It also stores context from policies and reasoning produced by AI systems or recorded by employees and other people who made earlier decisions. These parts connect the current case to the reasoning behind previous cases. Blumenfeld describes this as a way to give an agent subject-matter expertise through stored context. The demo data copies information that might come from a CRM, a support system, and other sources. The application uses Claude in the agent runtime, OpenAI embeddings, Neo4j, vectors, and a Next.js frontend.

06:14

Precedent retrieval combines semantic similarity with graph structure

In the demo, the agent first retrieves context about a customer's profile and then pulls decision traces. It uses hybrid search for precedents. A vector index can find textually related concepts such as a fraud rejection. Graph structure adds another signal by showing how earlier decision traces were connected and produced. Blumenfeld explains graph embeddings as vectors created from connected graph nodes. Similar decision traces can then be found by vector similarity even when the same information would be difficult to retrieve from ordinary documents.

08:51

The scaffold creates a working context graph application from one command

Blumenfeld introduces `uvx create-context-graph`, a command that creates a full application scaffold. The generated project includes a backend, frontend, graph setup, demo data, and the pieces needed to run an agent. The command accepts an application name, domain, framework, and demo-data option. The resulting application can query the graph with Cypher, show a schema visualization, retrieve traces, and answer questions over the loaded data. Blumenfeld presents it as a fast starting point for engineers who want to try context graphs without assembling every part themselves.

11:31

The scaffold supports preset domains, custom ontologies, and external data

The project includes 22 built-in domains, including healthcare and financial services. A developer can also describe a custom domain, after which the tool generates an ontology or graph schema to organize the data. Connectors can import information from GitHub, Notion, Jira, and Slack instead of relying only on fixture data. The application includes graph-native queries, decision-trace retrieval, an MCP server, and multi-turn conversation. Blumenfeld lists integrations with Pydantic AI, OpenAI, LangGraph, Crew, Strands, Google ADK, and other agent frameworks.

13:17

Agent memory has short-term context, long-term entities, and reasoning traces

The `neo4j-agent-memory` package underneath the project provides three forms of memory. Short-term memory contains conversation history and session context. Long-term memory contains entities extracted from conversations and resolved when they recur. Reasoning memory contains the decision traces held in the context graph. The package is designed to move useful information from short-term conversations into durable entities and linked traces. In the diagram Blumenfeld describes, conversations connect to extracted entities, and those entities connect to reasoning traces.

13:57

Entity extraction uses staged recognition followed by cleanup

For raw text, the memory package runs entity extraction through several stages. It starts with spaCy, uses GLiNER, and falls back to a large language model for more advanced cases. A separate process handles merging, deduplication, and enrichment. This helps resolve repeated mentions into useful long-term entities rather than leaving every mention as a separate record. Developers can also use structured data, their own language-model extraction process, or other named-entity recognition tools. The package includes predefined entity and relationship types for some domains to guide extraction and mapping.

16:27

Some parts of automatic decision-trace creation are still unfinished

During questions, Blumenfeld says timestamps can be added to reasoning steps and linked with relationships such as 'caused' or 'next', although he is unsure whether the current system supports that fully. In the first demo, the agent can be prompted to store decisions, but it does not appear to save them automatically without that instruction. The create-context-graph project is also still working on how new decision traces should be written. Blumenfeld suggests that future versions may consider signals such as sentiment or a quality score for decisions.

"And so, if you have system, right? Where you've had past decisions and tickets that have been Now up just like you would with vector search for another type of text, and that might be very very hard to pull out if you just had it inside of documents."07:45
Who should watch
  • You are building a RAG agent that can retrieve facts but needs to explain or repeat decisions made in earlier cases.
  • Your application has connected data from systems such as CRM, support, Slack, or Jira, and ordinary document retrieval loses those relationships.
  • You want a quick Neo4j-based prototype and need memory, entity extraction, ontology generation, or an agent-framework integration.