# Teaching AI to Find Real Vulnerabilities David Brumley, Bugcrowd | AI Engineer World's Fair 2026 | 27:17 Source: https://www.youtube.com/watch?v=ZFxh7sqbUZo Channel: AI Engineer (https://www.youtube.com/@aiDotEngineer). Summarised by AIE Talks. Page: https://aietalks.com/talks/teaching-ai-to-find-real-vulnerabilities Published: 2026-08-01 Tags: benchmarks, evals, reinforcement-learning, security ## TL;DR - AI hacking tasks should climb from finding a bug and triggering a crash to memory read and write primitives and full code execution. - Security benchmarks need deterministic graders that verify the specific vulnerability an exploit triggers, because language models tend to claim success and exploit the easiest bug. - On 41 real V8 vulnerabilities, advanced models were much better at full sandbox escapes than at merely triggering crashes, which made crash-based benchmarks poor measures of hacking ability. ## Summary David Brumley explains how to build reinforcement learning environments for cybersecurity. He starts with the way people learn to hack: practice on tasks that become harder, while acquiring skills that move from locating a bug to triggering it, controlling memory, and executing code. The same ladder can guide model training. The hard part is measurement. A model should not grade its own success, and a benchmark with several vulnerabilities can reward a model for repeatedly finding the easiest one. Brumley's audit task asks for proofs of all discovered vulnerabilities, then uses deterministic checks and stack backtraces to identify distinct bugs and calculate precision and recall. He applies this approach to 41 real vulnerabilities in V8, Chrome's JavaScript engine. Crash rates were high across models, while full sandbox escapes separated stronger systems from weaker ones. The talk argues for real vulnerable software, reproducible sandboxes, honest graders, and tests that reveal whether an exploit works. ## Key ideas ### People learn hacking through a ladder of increasingly difficult tasks [00:12](https://www.youtube.com/watch?v=ZFxh7sqbUZo&t=12s) Brumley describes picoCTF, a contest played by about a million high school students each year. Richard Zhu, who used the name fluorescence, learned by searching for information, reading write-ups, and practicing on progressively harder problems. Brumley recruited him to Carnegie Mellon, and Zhu later became a Pwn2Own winner who was the first to hack a Tesla, receiving $375,000 and a new Tesla. Brumley says models can be taught in the same way. Tasks should vary in target difficulty, from toy programs and CTF problems to hardened targets, while the required skills progress from locating a bug to triggering a crash, gaining arbitrary memory reads and writes, and achieving code execution. ### A crash proves a bug exists, but it does not prove that a model can hack the target [05:59](https://www.youtube.com/watch?v=ZFxh7sqbUZo&t=359s) Brumley separates vulnerability discovery from weaponization. A simple program may reward an input that causes a crash, but a real exploit must take control of the program. His ladder can ask whether a model can trigger a vulnerability, gain arbitrary read and write access, hijack control flow, launch an unwanted external program, or create a reverse shell. He says existing benchmarks such as CyberGym and AIxCC often stop at the crash stage. That makes them poor tests of high-value exploitation, since crashing a program does not let an attacker steal data or escape a sandbox. ### A language model should not be trusted to judge whether its own exploit worked [06:57](https://www.youtube.com/watch?v=ZFxh7sqbUZo&t=417s) A reinforcement learning environment needs a vulnerable application, a sandboxed and reproducible runtime, an orchestrator, tools for reading and writing in the container, and a grading oracle. Brumley rejects an LLM as the oracle in this setting because models will say they succeeded at hacking even when they did not. Instead, the environment runs the model's submitted input and checks a concrete result, such as whether the program crashed or whether it launched a forbidden process. The model is asked to exploit the vulnerability rather than merely name one, so the submitted exploit becomes evidence that can be tested. ### Benchmarks with several bugs reward models for repeating the easiest vulnerability [10:30](https://www.youtube.com/watch?v=ZFxh7sqbUZo&t=630s) Real programs often contain more than one vulnerability, which breaks the assumption behind many existing evaluations. If a grader checks only for a crash, a model can keep submitting the easiest crashing input. If the benchmark tells the model the vulnerable function or gives a backtrace, the model no longer has to reason about the program. Brumley cites DARPA's Cyber Grand Challenge, where 50% of its hand-curated challenges contained unknown vulnerabilities, and AIxCC, where 18 discovered bugs were unintended. His conclusion is that experts cannot reliably construct realistic tasks with exactly one bug. ### The audit task scores all discovered vulnerabilities with deterministic checks [13:43](https://www.youtube.com/watch?v=ZFxh7sqbUZo&t=823s) Brumley's audit task asks the model to find all vulnerabilities it can discover and submit a proof for each one. The grader runs every proof, checks that it triggers a real fault, and uses stack backtraces to determine whether different submissions represent different bugs. This allows the benchmark to incorporate vulnerabilities the creators did not know about. Precision measures the fraction of submitted findings that are real vulnerabilities, while recall measures the fraction of known vulnerabilities that the model found. The combination rewards finding new bugs without allowing the model to spam unsupported claims. ### V8 makes a useful test because a real exploit must escape its sandbox [17:35](https://www.youtube.com/watch?v=ZFxh7sqbUZo&t=1055s) The team chose V8, the JavaScript and WebAssembly engine used by Chrome, because attacker-controlled JavaScript can reach it through a malicious website. V8 also runs in Edge, Node.js, and Cloudflare Edge Workers. A crash inside V8's sandbox is expected to be contained and is not enough. A valuable exploit must cross the sandbox boundary, which often requires chaining vulnerabilities. Brumley describes a 16-capability ladder that moves from triggering a vulnerable line and crashing an in-sandbox object to gaining in-sandbox read and write primitives, finding a second bug, escaping the sandbox, and achieving arbitrary code execution. ### Crash success did not distinguish models, while full code execution did [21:10](https://www.youtube.com/watch?v=ZFxh7sqbUZo&t=1270s) The experiment covered 41 V8 vulnerabilities that the team manually verified as exploitable. GPT and Mythos triggered vulnerabilities 39 times out of 41, or about 95%, while lower-powered models such as Gemini, Kimi, Minimax, and GLM still succeeded about half the time. Those results would make many models look capable if crashing were the definition of hacking. At the full arbitrary-code-execution stage, Mythos succeeded on 30 of 41 examples, or 73%, and GPT succeeded 68% of the time. Gemini and Kimi succeeded 0% of the time. Brumley says Mythos also produced exploit paths that were not public, including a V8 path experts had thought would be too difficult in practice. ### New vulnerabilities help test whether models are reasoning instead of memorizing [24:51](https://www.youtube.com/watch?v=ZFxh7sqbUZo&t=1491s) Brumley's team mines open-source software for unique proofs of vulnerability and turns them into reinforcement learning environments. These are zero days that the models have not seen, which gives the team a way to test whether success came from memorized public exploits. The environments are reproducible Docker images with an MCP interface, and the team provides them through exploitbench.ai. The team can provide up to 10,000 reinforcement learning environments per month to companies it works with. Brumley says some Mythos transcripts were withheld because of an NDA and because the model created weaponized exploits that were not public. ## Notable quotes - "In other words, hacking is really a ladder." (04:37) - "The LLMs will always say they were successful hacking." (07:31) - "You can't just say, "Well, we're going to hand curate an environment with just one vulnerability."" (13:27) - "Crashing a program is different than hacking it." (17:56) - "The key TLDR for this is don't define the task by a single bug, let the program define the task." (16:43) ## Tools & references mentioned - Carnegie Mellon University - Bugcrowd - picoCTF - Pwn2Own - Richard Zhu - George Hotz - Sang Heon Lee - Anthropic - OpenAI - CyberGym - AIxCC - Cyber Grand Challenge - DARPA - V8 - Chrome - Microsoft Edge - Node.js - Cloudflare Edge Workers - Mythos - exploitbench.ai - CVE-2023-670T - CVE-2024-767965 - CVE-2024-0519 ## Who should watch - You are building an evaluation or reinforcement learning setup for cybersecurity models and need graders that verify real exploits. - Your benchmark has multiple vulnerabilities, or you worry that models are repeatedly solving the easiest bug. - You want to compare model ability on real exploitation rather than treating a program crash as a complete hack.