Scientific discovery requires systems that can connect facts across distant fields rather than answer simple questions from nearby text.
2
RAG often needs reasoning before retrieval, including analysis of the question and a structured representation of the document collection.
3
A useful benchmark must hide the known discovery from the model, measure links and hypotheses at several levels, and test whether the system stays grounded in older literature.
Summary
Hubert Misztela argues that scientific discovery is a harder information-retrieval problem than ordinary question answering. He uses the discovery behind RNA interference to show why: three results from different biology subdomains, with different terminology, took about eight years to connect. A standard RAG pipeline may fail because it embeds a convoluted question and retrieves chunks without first working out what the question means. Misztela proposes reasoning before retrieval, through techniques such as routing, graph-based representations, and relevance analysis. He also describes a benchmark design that removes papers published after the discovery, so the model cannot simply recall the answer. Success can mean finding known facts, connecting less obvious facts, generating new hypotheses, or explaining the mechanism behind a discovery. His experiments suggest that reasoning over both the question and the document collection can produce more grounded results, although brute-force checking remains useful.
A Nobel Prize discovery came from connecting results across biology subdomains
Misztela begins with experiments on petunia flowers in the early 1990s. Scientists tried to overexpress a gene to produce a stronger flower color, but the color unexpectedly flipped. Biologists encountered related results in three different subdomains, each using different names and vocabulary. The connection between those phenomena was resolved only after roughly eight years, leading to a Nobel Prize discovery in medicine. Misztela uses this example to define the challenge for AI: the relevant evidence may exist in the literature, but it is distributed across fields that do not describe the same process in the same way.
Standard RAG can retrieve poorly when the question itself is semantically complex
A naive RAG system processes a question and previously processed documents, represents them in an embedding space, retrieves similar chunks, and generates a response. Misztela points out that this pipeline assumes the model can understand a convoluted question well enough before retrieval begins. If an LLM struggles to answer a complex question directly, there is little reason to expect it to formulate the right retrieval from that question and then produce the right answer. Techniques such as reranking, document indexing, and hybrid retrieval address parts of the pipeline, but they do not remove the need to interpret the question first.
Reasoning before retrieval can operate on the question and the evidence collection
Misztela describes reasoning before retrieval as work that makes the question or the data easier to retrieve from. Routing is one example. It analyzes the question before sending it to an embedding search. Graph RAG and graph readers provide another example by representing knowledge from documents as graphs rather than relying only on text chunks. These steps help when the question has more complicated semantics. The same pattern appears in his chemistry work, where molecules are represented in a latent space and that space is searched or optimized for a purpose. He sees a common structure in reasoning, embedding, retrieval, and further reasoning.
The number of concepts and chunks in a question can signal the need for reasoning
Misztela frames retrieval complexity using relationships familiar from databases. A one-to-one question asks for one concept or one piece of information. A one-to-many question may require several pieces. The difficulty increases when several concepts are hidden across several chunks in the document collection. In that case, the system needs to parse the question before retrieval so it knows what to search for and how the concepts relate. He connects this to the needle-in-a-haystack family of tests, including cases with multiple needles. The point is to evaluate the relationship between a question and its answer, not only whether one exact passage was found.
Different forms of reasoning can be assigned to tools that specialize in them
For Misztela, reasoning means processing information in a logical way. He names several forms: aggregation and arithmetic, causal reasoning, algorithmic reasoning, probabilistic reasoning such as Bayesian inference, and structured or compositional reasoning. LLMs may help with these tasks, but they do not have to perform every part themselves. A causal-inference library can handle causal analysis. Python can handle algorithmic work, with a model generating code through a tool. This division matters for scientific systems because the model can coordinate specialized operations instead of being expected to carry out every kind of reasoning in natural language.
A scientific-discovery benchmark must prevent the model from recalling the answer
The proposed evaluation starts with a scientific question and a defined reasoning and retrieval type. The system then uses only papers published before the discovery. This knowledge cutoff prevents the LLM from using information learned from later descriptions of the result. Misztela describes the setup as recreating the situation before the discovery happened. The broader goal is a system that can apply the same process to many scientific problems, rather than a RAG system built to answer many questions over one fixed dataset. Groundedness is therefore part of the evaluation, alongside computational efficiency and relevance.
Success should be measured from finding known links to explaining new mechanisms
Misztela proposes several levels of success. The first is finding hypotheses or relationships already supported by the dataset. The next is finding less obvious links between facts from separate biology subdomains. Another is exhaustive retrieval of the relevant facts. A harder level is forming a new hypothesis from the evidence. The highest level would explain how the relationship works, not only identify that the facts are connected. He notes that humans knew what was happening in the motivating discovery before they knew how it happened. That distinction makes explanation a harder target than simply recovering the discovered relationship.
Reasoning over the question and corpus improved grounded retrieval in the experiments
The experiments moved beyond naive embedding distance. Misztela describes using Jenks natural breaks to select clusters of retrieved embeddings rather than choosing an arbitrary number of chunks. Strict prompting improved groundedness after an initial system used knowledge from after the discovery. A relevance classifier that passed every chunk through an LLM added little because its output was redundant with embedding distances. A more sophisticated prompt that assessed how relevant each paper was to advancing the hypothesis performed better. It found literature links related to DNA and one link related to RNA, which was the direction associated with the target discovery. Misztela concludes that reasoning before retrieval moved the results closer to the grounded answer.
"The ideal situation, or like the high level of success, is finding not only what is related between those different facts in the literature, but also explaining how that happens."16:20
Who should watch
You are building a RAG system for research questions that combine evidence from different fields or document collections.
Your evaluations allow a model to recall a discovery from its training data, and you need a benchmark that recreates the pre-discovery information state.
You want practical ways to decide when to add query reasoning, graph representations, classifiers, or external reasoning tools to retrieval.
