The Search Engine for the Agentic Web

Will Bryk, Exa17:49 · Sept 2026 · 5,847 views
Thumbnail for The Search Engine for the Agentic Web Watch on YouTube
TL;DR
  1. 1

    Machine-issued web searches are expected to exceed human searches in 2026, then grow to roughly a thousand times more in the following years.

  2. 2

    Exa was built around the idea that language models need retrieval because even very large models are tiny compared with the internet.

  3. 3

    Search for agents needs to handle complex information requests, return precise results, run quickly, and control the tokens passed to downstream models.

Summary

Will Bryk explains why Exa became a search company for AI systems. He expects machine searches to pass human searches in 2026 and then grow far beyond them. Traditional search works well for short queries, but it often recommends pages instead of answering requests such as finding every person in Singapore who works on AI search. Exa began with a thought experiment: compare a complex query with every document using a language model. That would produce near-perfect retrieval, but at roughly $10 million per search, so the company spent years reducing the cost with embeddings, keywords, and other systems. Bryk describes Exa's current tools for complex queries, fast retrieval, token extraction, structured output, and private data. He argues that agents need access to a much larger information system because models cannot contain everything about the world.

Key ideas
00:32

Machine searches are expected to pass human searches in 2026

Bryk opens with a chart showing web searches per day over the past 30 years. Human searches dominated until the 2020s, but he says AI systems are expected to exceed humans in 2026. In the following years, he expects AI systems to search roughly a thousand times more than people. Exa already serves more than 5,000 companies and over 400,000 developers, including coding agents such as Cursor, go-to-market systems such as HubSpot, and financial agents that need current data.

02:02

The web is too large and messy for people to understand directly

Bryk describes the web as roughly a trillion pages containing blog posts, company sites, images, tweets, and other material. People need tools that filter and combine this information because it cannot fit in anyone's head. He calls the failure to provide trustworthy access to this information 'misinformed anarchy.' Without better information, he says, people can be manipulated and make poor decisions as AI systems become more powerful.

03:10

Traditional search recommends pages instead of answering complex requests

Bryk uses the query 'shirts without stripes' to show how a mainstream search engine can return shirts with stripes. He says this happens because the engine behaves partly like a recommendation system instead of a database that returns exactly what the user asked for. Queries such as finding everyone in Singapore who works on AI search and listing their writing expose the same limitation. An agent needs a result set that matches the conditions, rather than a collection of links containing some of the words.

06:05

Perfect retrieval would compare every query with every document

The thought experiment behind Exa is to take a complex query and a document, then ask a language model whether they match. Bryk says a model such as GPT-3 can do that well. Running the comparison across a trillion documents for every search would create a near-perfect search engine, but it would cost about $10 million per query. The engineering problem became reducing that cost by billions or trillions of times.

06:51

Embeddings make complex retrieval possible at practical cost

Traditional keyword search is efficient and handles simple queries. For more complex requests, Bryk says search needs neural networks, especially embeddings. Instead of running a neural network over every document for every query, a system can preprocess documents into embeddings that preserve some of the model's intelligence. Exa combines embeddings with keywords because each approach has limitations. This lets it handle requests such as finding shirts without stripes.

08:25

Exa found its business when people asked for an API

Exa launched a consumer search engine under the name Metaphor in 2022. Two weeks later, ChatGPT changed the market. The company then received a Twitter message asking for programmatic access to its search engine. Bryk initially said no because there was no API, but more requests followed, including one from his downstairs roommate. The team realized that AI systems would always need retrieval because even a huge model is tiny compared with the internet. That turned the search engine into a search API for AI systems.

09:28

Agents need search that is complex, fast, and economical with tokens

People tend to search with short queries such as 'SpaceX news.' Bryk says agents instead make much heavier use of search and may ask for lists of companies, people, articles, or other records. Exa supports complex queries that may take minutes, alongside a 200-millisecond endpoint for systems such as voice agents. It can also extract only the most important tokens from retrieved documents, reducing the amount of text passed to a language model. Structured output lets an application request fields such as a person's college, graduation year, or most-cited paper.

13:38

Exa is building access to private data alongside the public web

Bryk says agents care about whether information is useful, not whether it comes from the public web or a private source. Exa's newer system lets data providers make their information available to developers through the platform. Providers can decide what their data is worth, while developers choose the sources they want. His example combines public web information with Similarweb data to answer a question about companies and monthly website visitors.

14:42

Perfect information would feel like a year of research in one second

Bryk describes Exa's long-term goal as a system where any information query works, regardless of complexity. His comparison is spending an entire year researching people, companies, or news and receiving the result in a second. He expects AI agents to create an enormous number of searches because assistants and software products will ground each interaction in retrieval. He connects this goal to public understanding, including the coming 2028 presidential election.

"If we take a query, a complex query, and a document, and we run GPT-3 over it, and we say, does this match? It'll do a really good job of saying, does it match?"06:08
Who should watch
  • You are building an AI agent that needs current web information, company lists, research results, or structured records.
  • You need to choose between keyword search, embeddings, fast retrieval, and deeper searches for an agent workflow.
  • You are interested in how search infrastructure changes when machines issue far more queries than people.