Neural search represents documents as embeddings that capture meaning, allowing queries such as "shirts without stripes" to return semantically appropriate results.
2
AI agents need search that accepts long, contextual requests and returns large amounts of precise information, rather than search designed for humans clicking a few links.
3
An effective agent should choose between neural search, keyword search, and other retrieval methods based on the user's request.
Summary
Will Bryk explains why Exa was built around neural web search and why AI agents need a different retrieval system from traditional human-focused search. Keyword indexes work well for simple queries, but they struggle with meaning, negation, long context, and requests for comprehensive results. Exa instead embeds documents and queries, then compares those representations to find semantically related information. Bryk argues that an AI agent may need several retrieval methods in one task. In his example, a neural search finds engineers in San Francisco interested in information retrieval, then a keyword search finds each person's GitHub page. The live coding section also shows filters for result count, dates, and domains, plus Exa's research endpoint, which runs searches and language-model calls in the background to produce a report or structured result. The talk is an argument for giving agents finer control over how they search the web.
Traditional search began with keyword matching and authority ranking
Bryk starts with Google in 1998, where a query such as "Australia" finds documents containing that word. Google then ranked results partly with PageRank, using the structure of links on the web to estimate authority. He presents this as a simplified view of traditional retrieval: create an inverted index that maps words such as "brown" to documents containing them, then run a keyword comparison when a query arrives. This approach made sense for short human queries, but it does not capture every aspect of what a person means.
Embeddings can represent document meaning instead of only its words
Bryk describes Exa's central idea as turning each document into an embedding, which is a list of numbers produced by a transformer. The representation can capture the document's meaning, its ideas, and even how people refer to it on the web. At search time, the system embeds the query and compares it with document embeddings. His example is "shirts without stripes": keyword search may return shirts with stripes because it does not understand the word "without", while neural search can retrieve results that match the intended meaning.
Language models still need search because the web is larger and newer than their weights
After ChatGPT launched, Bryk's team questioned whether search still had a role. He says the answer is yes because a language model cannot store the whole web in its weights, and the web keeps changing. A model such as GPT-4 may not know obscure personal websites or current information. Combining an LLM with Exa lets the model search for those sources and use the returned material in its answer. Search supplies information that cannot be reliably recalled from model parameters alone.
AI agents need search designed for complex, high-volume requests
Bryk says traditional engines were built for humans who type short queries, read a few links, and care about the page interface. Agents can process far more information and can send long, detailed requests. They may ask for startups with several properties, search with multiple paragraphs of user context, or request every company in a category. Bryk argues that agents need precise control over the requested result, support for large contextual queries, and the ability to return hundreds or thousands of relevant items instead of only a small set of links.
Semantic retrieval expands the kinds of questions a web search system can answer
Bryk maps search onto a larger space of possible queries. Older systems handled basic keyword requests such as Stripe pricing or a GitHub page. AI systems introduced requests such as explaining a concept, debugging code, or finding people in San Francisco who know assembly. He also describes more structured requests, such as finding every article that argues one position rather than another and filtering by author. His goal is for the web to behave more like a database that an agent can query in many ways.
Agents should combine neural and keyword search within one task
In the live demo, Bryk creates an agent called Mark that turns information into Markdown. For a query about personal sites of engineers in San Francisco who like information retrieval, he chooses neural search because the request depends on semantic meaning. For a GitHub lookup, he switches to keyword search because it resembles a conventional search. He then combines both: neural search finds people, the agent extracts their names, and keyword search finds their GitHub pages. The retrieval method changes with each subtask.
Search APIs need controls because agents decide how to retrieve information
The Exa dashboard exposes controls for result count, date ranges, domain restrictions, and the choice between neural and keyword search. Bryk says these controls are useful because an AI agent may call the search API directly and choose the settings for a particular request. In his example, the agent first gathers a list of relevant engineers, then performs another kind of search for each person. Exa can return 10 results in the demo, with larger result counts available on an enterprise plan.
Exa's research endpoint runs retrieval and model calls behind one request
Bryk closes by describing a research endpoint that Exa had launched that day. It performs multiple searches and language-model calls in the background to produce a report or structured output for the user's request. He compares it to a deep-research API. This extends the same design from the demo: an application gives the system a broad information task, and the system performs the retrieval steps needed to assemble the requested result.
"The fundamental idea was just like the bitter lesson: what if we could train transformers to output embeddings for documents?"07:06
Who should watch
You are building an agent that needs web retrieval for long, semantic, or multi-step requests.
Your current RAG system uses one retrieval method and struggles when a task mixes broad discovery with exact lookups.
You want a concrete example of how an agent can switch between neural search and keyword search across subtasks.
