# How We Mapped a 35-Million-Line Mainframe in 4 Days

Stephen Chin, Neo4j & Neeraj Deshmukh, Blitzy | AI Engineer NYC 2026 | 19:44

Source: https://www.youtube.com/watch?v=oWqsBuhRRZ4
Channel: AI Engineer (https://www.youtube.com/@aiDotEngineer). Summarised by AIE Talks.
Page: https://aietalks.com/talks/how-we-mapped-a-35-million-line-mainframe-in-4-days
Published: 2026-11-06
Tags: agents, coding-agents, context-engineering, multi-agent, testing

## TL;DR
- AI can translate a COBOL formula into Java while missing the batch jobs, data layouts, account updates, and other places that depend on it.
- Blitzy maps an entire software estate as a knowledge graph, then gives each agent the precise code, relationships, constraints, and evidence needed for its task.
- Parity testing compares old and modern systems at every stage, including values, persisted state, transaction output, errors, and rounding behavior.

## Summary
Stephen Chin and Neeraj Deshmukh explain why modernizing mainframe software is a system-understanding problem. A direct COBOL-to-Java conversion may reproduce an interest formula while missing the jobs, copybooks, data layouts, account updates, and downstream outputs connected to it. Larger context windows do not solve this because models lose accuracy as context fills, and ordinary retrieval methods do not preserve the system's relationships. Blitzy turns the codebase into a graph whose edges retain repository, revision, symbol, and source-span evidence. Agents receive task-specific context from that graph, while a human approves the plan and decisions at each important stage. Code generation is paired with continuous builds, unit tests, runtime checks, data comparisons, and end-to-end tests. The examples include a one-cent rounding difference, a tier-one bank conversion completed in 18 days, and reverse engineering a 35-million-line mainframe in four days. The same graph can later support new features, compliance work, and fraud use cases.

## Key ideas
### Mainframe behavior is spread across the whole system
[01:24](https://www.youtube.com/watch?v=oWqsBuhRRZ4&t=84s)
Neeraj Deshmukh uses an AWS card demo to show an interest calculation. A batch job calls the interest program, which uses copybooks and data layouts before producing VSAM data. The result depends on inputs, the job that runs it, account updates, and transaction output. Converting only the formula from COBOL to Java misses record precision, account updates, transaction output, the batch contract, and 14 other places where the logic is used. The problem is that the AI lacks the system context needed to preserve the result, not that it cannot write the new code.

### More context does not preserve the relationships in a codebase
[03:09](https://www.youtube.com/watch?v=oWqsBuhRRZ4&t=189s)
Stephen Chin says vector databases provide similarity search, while relational databases offer a larger repository through SQL queries. A graph provides multi-hop context and keeps the connections between pieces of information. He says models advertised with million-token context windows may have 200,000 tokens or less of usable context, with accuracy falling as the window fills. Neeraj adds that the effective context is about 20 to 30 percent. The practical answer is to give a model the precise facts and relationships needed for its current task.

### The knowledge graph keeps evidence attached to code relationships
[06:03](https://www.youtube.com/watch?v=oWqsBuhRRZ4&t=363s)
The graph links programs to jobs, copybooks, outputs, and other code elements. Each edge keeps its identity, including where it came from in the repository, which revision it belongs to, and which symbols it touches. The system also stores evidence about the file and source span used to extract the relationship. The graph updates on every merge, so obsolete information can be invalidated. This gives agents a current map of how a change fits into the wider software estate.

### Agents receive a task packet instead of the entire repository
[07:08](https://www.youtube.com/watch?v=oWqsBuhRRZ4&t=428s)
Each Blitzy agent receives a task description plus the relevant graph location, source slice, constraints, available evidence, and unanswered questions. Neeraj calls this infinite code context because the agent can reach the right information at the right time rather than trying to remember the entire estate. Blitzy first reverse engineers the software, business logic, workflows, and dependencies. User requirements then become an agent action plan. A human reviews and approves that plan before agents generate code.

### Human approval and testing remain part of the generation process
[07:53](https://www.youtube.com/watch?v=oWqsBuhRRZ4&t=473s)
The action plan defines what will change and what must stay the same. A domain expert reviews it before work begins, and people retain ownership of the decisions that matter. Agents then generate code with validation built into each step. The result is a tested pull request and a project guide documenting what happened and how it was checked. Because the system also has build instructions and can observe runtime behavior, it can work with more than static source files.

### Large agent teams depend on shared context and repeated tests
[09:34](https://www.youtube.com/watch?v=oWqsBuhRRZ4&t=574s)
Blitzy coordinates thousands of agents around one plan. Each agent can access the effective context, so information is not lost as work moves between tasks. Testing happens throughout code generation, and Neeraj says the system spends more compute on testing than on code generation. The controls include system-level context from the graph, continuous validation, security and compliance guardrails, and a human in the lead. The aim is to preserve precision, rounding, balances, transaction records, failures, and existing rules.

### Parity testing checks behavior beyond successful outputs
[11:35](https://www.youtube.com/watch?v=oWqsBuhRRZ4&t=695s)
The old and modern systems are tested with the same inputs and compared across returned values, persisted state, transaction output, and error outcomes. The comparison includes bad situations, not only the happy path, and differences are expected to be absent. Tests run at several layers: generated code must compile, unit tests must pass, runtime integration must fit the existing framework, writes must preserve data, and the complete workflow must match. One Fortune 100 insurer produced output byte for byte identical to its mainframe baseline.

### A one-cent difference can expose a failed modernization
[13:28](https://www.youtube.com/watch?v=oWqsBuhRRZ4&t=808s)
The interest example exposes a language difference that a normal code review could miss. Java's behavior includes rounding up in this calculation, while COBOL behaves differently. The converted program compiles and produces results that look reasonable, but one cent differs from the mainframe result. Without a parity test, the error would remain hidden. Neeraj points out that a small discrepancy can matter across a million transactions.

### The mapped system can support later business changes
[16:37](https://www.youtube.com/watch?v=oWqsBuhRRZ4&t=997s)
After modernization, the same knowledge layer, governance rules, and expanded test suite can support changes such as adding a promotional interest rate. The speakers also describe uses beyond code conversion, including new product development, feature work, security remediation, audit evidence, compliance, tracking business-rule changes, and fraud-related systems. Stephen says the graph can be extended into an enterprise layer that captures business rules and processes, allowing teams to work with the wider business rather than only source code.

## Notable quotes
- Neeraj Deshmukh: "The failure here is not that AI can't write the code. It's that AI doesn't know the 14 other places where that logic, that code is being used." (02:58)
- Stephen Chin: "What a graph gives you is this multi hop accurate information and context for your model." (04:28)
- Neeraj Deshmukh: "Nothing happens without a human in the lead making decisions at every key point." (08:19)
- Neeraj Deshmukh: "We actually spend more computes on testing than we do on code gen." (09:57)
- Neeraj Deshmukh: "If you didn't have that parity test, you wouldn't have caught this." (14:10)

## Tools & references mentioned
- Neo4j
- Blitzy
- AWS card demo
- COBOL
- Java
- VSAM
- PostgreSQL
- TypeScript
- Python
- MCP server
- Nodes conference
- Blu Age sandbox
- DORA compliance

## Who should watch
- You maintain COBOL or other legacy systems where business behavior is spread across jobs, data layouts, stored procedures, and undocumented dependencies.
- You are evaluating agentic code migration and need a process for context selection, human review, testing, and behavioral parity.
- You work in a bank, insurer, or other regulated company that needs evidence for changes and cannot accept a migration based only on generated source code.

## Editor's note

Neeraj Deshmukh shows that a converted program can compile and look reasonable while a one-cent rounding difference remains hidden. Kitaru records a real agent run, including every model call and tool result, then replays it with the same inputs and responses after a change. That gives teams a repeatable run for checking whether a fix preserves the behavior that failed before.

Written by the AIE Talks editors (the Kitaru team), not by the speaker.

## Related talks

- [AI Cut Our Research Time From 7 Days to 15 Minutes](https://aietalks.com/talks/ai-cut-our-research-time-from-7-days-to-15-minutes) (Gareth Yoder, Vanguard, 15:39)
- [AI Coding Agents Are Breaking Big Codebases](https://aietalks.com/talks/ai-coding-agents-are-breaking-big-codebases) (Dan Adler, Sourcegraph, 12:23)
- [How Anthropic Merges 30,000 Code Changes a Day](https://aietalks.com/talks/how-anthropic-merges-30-000-code-changes-a-day) (Manish Kapur, Sonar & Addy Osmani, Anthropic, 18:53)
- [Benchmarking Coding Agents on New vs Legacy Codebases](https://aietalks.com/talks/benchmarking-coding-agents-on-new-vs-legacy-codebases) (Denys Linkov, Wisedocs, 18:08)
- [We Kept Rebuilding Our AI Agent. Here's What Survived](https://aietalks.com/talks/we-kept-rebuilding-our-ai-agent-heres-what-survived) (Fuad Ali, Arize AI, 20:06)
