Structured LLM memory helps models understand and modify repositories when a plain context window gives shallow or broken results.
2
GraphRAG for Code supports repository-wide questions, Python-to-Rust translation, documentation, and multi-file feature changes in large codebases.
3
Benchmark QED evaluates retrieval systems across local and global questions, while LazyGraphRAG showed strong results against vector RAG across different context sizes.
Summary
Jonathan Larson explains how Microsoft Research uses graph-structured representations to give LLMs repository-level understanding. In a small terminal game, regular RAG describes only generic components, while GraphRAG for Code identifies the player, obstacles, background, and controls. The same approach translated a Python game into working Rust and helped a GitHub Copilot coding agent add jumping to the 100,000-line Doom codebase by planning changes across multiple files. Larson then introduces Benchmark QED, an open-source evaluation toolkit with components for query generation, LLM-based evaluation, and dataset summarization or sampling. Its questions span local and global retrieval, as well as data-driven and persona-driven tasks. He also presents LazyGraphRAG results against vector RAG at 8K, 120K, and one million token context sizes, reporting that LazyGraphRAG won most comparisons and cost a tenth as much as the one-million-token context setup in their test. He closes with planned product integrations in Azure Local and Microsoft Discovery.
Graph structure gives LLMs a usable memory of a codebase
Larson says structured LLM memory is an enabler for effective AI applications, and he pairs it with agents for stronger results. His first example applies GraphRAG to coding and enterprise productivity. The system builds a representation of a repository so the model can answer questions that require understanding the whole codebase. He distinguishes global queries, which span the repository, from local queries, which concern individual files or smaller sections. The rest of the demonstration tests whether that structure helps with code translation and feature development, rather than only question answering.
GraphRAG for Code produces repository-level explanations
Larson demonstrates a terminal game implemented in about 200 lines across seven files. A language model had never seen the code, and simply placing all of it in the context window led to a generic answer that only called it a game with separate components. GraphRAG for Code described the actual behavior: a player jumps vertically, obstacles move horizontally, there is a static background, and the space bar triggers jumping. Larson presents this as a global repository query, because the answer depends on combining information from the full set of files.
Graph structure makes multi-file code translation practical
The team asked a model to translate the same Python game directly into Rust. With the source files placed into the model and no GraphRAG structure, the model generated Rust code that failed to compile and did not work out of the box. GraphRAG for Code generated a set of Rust files from its structured view of the repository. Larson says the resulting game ran natively in Rust. The example is small, but it tests whether the system can preserve relationships among several source files during a translation rather than producing isolated code fragments.
Repository-wide understanding extends to a 100,000-line codebase
The next test used the roughly 30-year-old Doom codebase, with about 100,000 lines across 231 files. Larson expected language models to know Doom because the code is widely available in training data, but tests showed that they did not know the repository's specific details well enough to make meaningful modifications. GraphRAG for Code generated high-level documentation for modules spanning many files, such as the game's sound system. Users could then drill down from those global descriptions into individual files through local queries.
GraphRAG helps coding agents plan changes across files
The team asked whether repository understanding could support feature development. They connected the GitHub Copilot coding agent to GraphRAG for Code and created a Doom issue to add jumping for the player. Larson describes this as a difficult multi-file change, because an agent may edit one file correctly while breaking related files elsewhere. In the demonstration, the agent used GraphRAG for Code to make a holistic, top-down plan and changed many files. Larson says the result worked out of the box, unlike the other agents they tried on the task.
Benchmark QED evaluates retrieval across local and global questions
Benchmark QED is an open-source toolkit for measuring local and global quality in systems such as GraphRAG. Its AutoQ component generates questions for a target dataset. AutoE evaluates the answers with an LLM judge, and AutoD summarizes and samples datasets. AutoQ combines two axes: local versus global questions, and data-driven versus activity-driven questions. A local question about junior doctors striking in South Korea contains specific retrieval clues. A global question about public-health initiatives for underserved communities requires understanding the dataset as a whole.
LazyGraphRAG performed well against vector RAG at several context sizes
Larson describes comparisons between LazyGraphRAG and vector RAG using 8K, 120K, and one-million-token context windows. On the charts, LazyGraphRAG won 92%, 90%, and 91% of the data-local comparisons against the corresponding vector RAG setups. He says the system maintained dominant performance across local and global questions, even though he expected vector retrieval to have an advantage on local questions. The longer context windows did not produce the expected improvement on global questions. In their test, LazyGraphRAG also cost a tenth as much as the one-million-token context-window setup.
LazyGraphRAG is being incorporated into Microsoft products
Larson says LazyGraphRAG is being lined up for inclusion in Azure Local and Microsoft Discovery. Microsoft Discovery uses graph-based scientific co-reasoning, moving from hypothesis to experiment, learning, and knowledge. In the demonstration, the copilot generates reasoning over scientific knowledge represented in graphs. Larson says GraphRAG and LazyGraphRAG power those graphs and the answers produced from them. He closes by returning to the idea that structured LLM memory gives agents more power when they operate over code or scientific knowledge.
"It features a player character that can jump vertically, obstacles that move horizontally across the screen, and a static background layer."03:30
Who should watch
You are building code assistants that need to understand relationships across a repository instead of retrieving isolated files.
Your agent makes partial or conflicting edits when a feature requires changes in several files.
You need an evaluation set that tests local and global retrieval, or you want to compare vector RAG with graph-based methods at different context sizes.
