Version-Controlled Search: The Web as It Was

Ryan Ahern, Exa19:13 · Oct 2026 · 4,473 views
Thumbnail for Version-Controlled Search: The Web as It Was Watch on YouTube
TL;DR
  1. 1

    Agents perform many detailed searches for each task, so search quality, token cost, and latency affect the whole agent trajectory.

  2. 2

    Exa Snapshot lets users retrieve a URL or run a search using only web information available on or before a chosen date.

  3. 3

    Point-in-time search lets teams backtest web-derived signals and agent features while checking whether predictions generalize instead of overfitting.

Summary

Ryan Ahern introduces Exa Snapshot, a version-controlled search system built from Exa's repeated web crawls. Snapshot can return a URL as it appeared on or before a chosen date, or reproduce search results using only information available at that time. Ahern connects this to financial modeling. Teams can use historical web data to build signals such as software prices, sentiment, company information, or adverse media, then test whether those signals could have predicted later events. The same method can evaluate agentic features that search the web during a prediction task. Ahern's example is acquisition prediction: a live search would reveal an announced deal and contaminate the test, while a dated search can restrict the agent to earlier information. Snapshot also lets teams test many examples, use holdouts, and detect features that fit the training set without generalizing. The talk is practical, although its claims about search quality are presented from Exa's perspective.

Key ideas
01:16

Agents are becoming the layer through which people search

Ahern says many users now ask ChatGPT, Claude, or another agent to search instead of typing directly into Google. Agents can issue many searches, describe their needs in detail, and keep exploring instead of stopping after the first few results. That changes the requirements for search infrastructure. Exa provides crawling, indexing, embeddings, and retrieval for systems such as Cursor and Gamma. Ahern also describes financial users who need web information that is not available in structured datasets, including market information, counterparties, and emerging startups.

04:13

Search quality has to be judged by the whole agent task

Ahern gives Exa's three priorities as quality, cost, and latency. Better retrieval can reduce the number of tool calls and lower the chance that irrelevant information sends an agent in the wrong direction. Cost includes the total inference cost of completing a task, not just the price of one search. Exa's dynamic highlights select useful snippets across search results rather than returning full pages. Ahern says this can provide higher-quality context with 95% fewer tokens than full-page text. He also describes Exa's infrastructure as supporting both low-latency searches and long-running deep research.

06:52

Snapshot preserves the web at earlier points in time

Exa Snapshot contains more than 400 billion snapshots of web pages that can be searched by date. Exa already checks crawled pages for freshness, which lets it retain versions of URLs from those checks. One use case starts with a known URL and asks for its version on or before a date. The more general use case is version search: run a search as if it had happened in the past and receive only results that would have been available then. This gives teams a historical view of both page content and search results.

08:43

Historical search prevents future information from contaminating a test

Ahern uses reporting about OpenAI's annualized revenue as an example. An earlier search might have surfaced a roughly $70 billion figure, while later reporting discussed a figure closer to $50 billion after accounting for revenue passed through cloud providers. With version search, a researcher can ask what information existed before the later correction became widely known. The researcher can then test whether an SEC filing, an alternative report, or another source could have exposed the discrepancy earlier. A live search would include the later reporting and make that historical test unreliable.

11:45

Web-derived signals can feed ordinary predictive models

The simplest workflow takes unstructured historical web information and turns it into structured signals for an existing model. Examples include software prices, sentiment, and terms and conditions. Ahern points out that these values may change or disappear, so a dated source is needed to know what the signal actually was at the time. Teams can use the extracted signals to enrich models that already rely on other data. Snapshot gives them a way to construct those signals from information that was available at the relevant date.

12:58

Agents can add researched features to established models

A second workflow adds one or two agentic features to an existing model. Ahern gives adverse media screening as an example. An agent can search across many sources and condense the results into a structured feature. Equity research could use a similar process to collect estimated revenue, headcount, or other company attributes. Snapshot lets teams ask what the feature would have returned one month or four months earlier, then compare that result with later outcomes and with the team's existing process.

15:02

Snapshot can control web access inside a larger predictive agent

Ahern describes a more advanced setup in which an agent manages a prediction process, accesses internal knowledge and data feeds, and uses web search as one of its tools. Without date controls, that agent could find the announcement of an acquisition and predict it with perfect hindsight. Version-controlled search restricts the tool to information available before the event. This removes one obvious form of leakage without requiring a private corpus that attempts to recreate the whole web.

16:27

Historical evaluations expose features that overfit

In a workshop example, one feature asked whether an investor had publicly called for a sale, while another measured whether a share price had fallen at least 60 percent from its five-year high. A feature based on the company being named Dominion illustrated overfitting. An agent trained on live data might learn an accidental property of the examples. With Snapshot, teams can run evaluations over historical cases, use holdout data, and change the agent or feature descriptions before testing again. Ahern argues that live production testing would take months and would make this kind of diagnosis much harder.

"You can now go back and be like, "Okay, before everyone figured it out, before the market and before some financial analyst published the think piece, what information existed on the internet?""09:28
Who should watch
  • You are building an agent that needs web search and want to understand how repeated searches affect task cost and result quality.
  • You are creating financial or risk models that use information from changing web pages and need to test signals at their historical point in time.
  • Your agent or model performs well on live data, but you need to check for future-information leakage and features that overfit.