Plain file traversal works well for small, local, text-based codebases because code has structure and references that guide an agent.
2
Large company document collections need a hybrid harness that combines semantic and keyword search with listing, metadata filtering, grep, and reading.
3
Complex document search depends on parsing, screenshots, permissions, freshness, and a pipeline that keeps ingested data synchronized.
Summary
George He compares the local file traversal used by Claude Code with pre-indexed search for company data. Code is small, local, text-based, and full of import and test references, so an agent can explore it without embeddings. Company data has a different shape. It may contain millions of PDFs, slides, scanned documents, and schematics, with permissions and changing content that make local traversal expensive and difficult. He recommends giving the agent a harness that lets it choose its path. Hybrid search provides a rough direction, while listing, metadata filtering, grep, and file reading help the agent inspect and ground the answer. Complex documents need structural parsing and page screenshots. He discusses LlamaParse, LiteParse, OCR quality, multi-tenancy, freshness, and layered storage. In the demo, an agent searches 135 Alphabet financial filings and builds a cash-flow table from multiple source files.
Claude Code can skip embeddings because code gives agents useful local structure
George He explains that Claude Code experimented with a vector database and did not choose it. Code is usually small, already on disk, semi-structured, and written in a text format. Imports, tests, and file hierarchy leave breadcrumbs that tell an agent where to look next. Keeping a vector index synchronized would add maintenance work as the code changes. Local globbing and file traversal are therefore enough for many coding tasks. The same choice does not automatically apply to company documents, where the data is larger, less standardized, and often stored remotely.
The right search approach depends on the size and shape of the corpus
For roughly 100 to 1,000 files, George says an agent may be able to use local file search without spending too many tokens on traversal. At thousands or millions of files, repeatedly reading files and carrying file names through the model becomes too expensive. The system needs a rough direction before the agent starts reading. George describes this as a compass rather than an exact answer. Larger context windows do not remove the cost of sending large amounts of text or binary data through the model, so the search design has to account for ingestion, latency, and token use.
A harness should let the agent combine retrieval with direct file inspection
George recommends giving the agent tools instead of forcing one retrieval method. Semantic or hybrid search can narrow a large corpus to a useful group of files. The agent can then list files, filter by metadata, grep for specific terms, and read the relevant sections. This combination gives the agent a way to expand or narrow its search after seeing the first results. It also helps ground the final answer in source material. Remote files can be inspected through these tools without downloading the entire corpus into the agent's context.
Hybrid search gives the agent a better first direction
George recommends hybrid search for enterprise use cases. Keyword search can find exact terms that matter to developers and agents, while vector search captures semantic similarity. Combining and weighting both approaches can improve retrieval over either method alone. He also mentions re-ranking an initial set of results with an LLM or another deeper process. This first retrieval step does not replace reading the source. It gives the agent a smaller area to inspect before it uses file listing, metadata filters, grep, and reading to establish what the documents actually contain.
Complex documents need structural parsing and page images
Charts, scanned pages, tables, PowerPoints, and schematics do not fit a simple stream of text. George says the parsing flow should preserve structure and spatial relationships, often by converting content into markdown or another format an agent can use. The system should also render and store page screenshots. A screenshot can give the agent a human-compatible view of a diagram or layout that text extraction misses. LlamaParse handles this kind of document parsing, and LlamaIndex has open-sourced LiteParse for structural content extraction.
Parsing quality affects what the language model can recover later
George asks teams to measure whether their OCR and parsing systems preserve tables, layout, and the actual content of a document. He names Textract and Tesseract as examples of OCR tools whose output still needs checking. If a table loses its representation or a page loses its spatial relationships, the language model has less information to work with during semantic reading. Parsing can appear acceptable at first and then fail when the agent needs to understand where information appeared on the page. Screenshots and structured output can cover different parts of this problem.
Production search has to keep permissions and fresh data synchronized
A production system must handle multi-tenancy, permission metadata, and data freshness. George uses Google Drive and SharePoint as examples of sources whose access rules need to be imported without making the whole system slow. Re-parsing and re-indexing everything whenever permissions or source data change is expensive. He breaks synchronization into ingestion, parsing, and indexing. Ingestion brings documents into a synchronized system. Parsing converts them into a standard representation such as markdown or text. Indexing makes the processed data available to the retrieval and traversal tools.
The demo uses hybrid retrieval and remote file tools to build a financial table
George demos a corpus of 135 Alphabet financial filings in HTML and PDF form, with metadata such as year and company. A hybrid query for cash-flow statements returns text chunks and corresponding page screenshots. The agent can then grep a specific file, inspect an offset, and read a selected range without downloading the full remote file. George asks the agent to produce a cash-flow table from 2021 to 2025. The agent searches, reviews screenshots and context, reads several files, and returns a summary grounded in multiple source documents.
"The five tools that we'll talk through today in enabling kind of document search or complex document search are what are the tools that we're going to use?"08:57
Who should watch
You are deciding whether an agent should traverse files directly or use embeddings and a pre-indexed search system.
Your document corpus contains PDFs, slides, scans, tables, or schematics that plain text chunking does not represent well.
You are building enterprise retrieval and need to account for permissions, multi-tenancy, freshness, and remote file access.
