Biology produces experimental datasets that are too large for scientists to inspect manually, creating a need for agents that can analyze data through measurable steps.
2
Spatial biology provides a useful starting point because its workflows produce matrices and images that can be checked with deterministic graders.
3
Biology benchmarks need human verification and carefully designed tasks because scientific ground truth is often ambiguous and model refusals create a separate biosecurity evaluation problem.
Summary
Kenny Workman describes LatchBio's effort to build agents and benchmarks for scientific biology workflows. Modern single-cell and spatial experiments can produce terabytes of data, but the scientific work involves more than writing code. An agent must process measurements, analyze data, interpret results, and connect them to an experimental question. LatchBio started with spatial biology because its workflows produce structured data and have verifiable intermediate states. The team built Spatial Bench around data nodes, task prompts, configurations, and deterministic Python graders. Workman explains why this approach still needs scientists: biological tasks contain open choices about normalization, thresholds, gene categories, and other analysis details. LatchBio therefore uses human verification and longer tasks that simulate paper result sections or industry drug decisions. The company is extending its benchmarks across omics, drug discovery, and biosecurity, where both unsafe assistance and inappropriate refusals need to be measured.
Biology experiments already produce more data than a scientist can inspect
Workman begins with the growth of experimental data from single-cell biology, spatial biology, and proteomics. Single-cell experiments can produce two to six terabytes per run, while spatial runs can produce seven terabytes. A single experiment can therefore exceed what a scientist can safely store on a consumer laptop. The data comes from measuring RNA, proteins, or their location in tissue, and the throughput is still increasing. Workman uses this scale to motivate agents that can work through large datasets instead of relying on a person to inspect every result manually.
Experimental biology can provide a verifiable substrate for agent training
Workman frames biology data analysis as an executable scaffold for training and evaluating agents. A modern experiment usually involves choosing a biological model, generating data, processing it, interpreting the results against prior literature, and making a claim. The process searches for a signal in a large amount of noise. He compares data analysis in biology with code, where a task can have a checkable result even when the broader activity is complex. Biology analysis could provide similar intermediate checks, although Workman says the claim needs more investigation rather than being accepted without qualification.
LatchBio moved from biotech data infrastructure toward biology agents
LatchBio began as a data tool vendor for biotech and pharma, storing, transforming, and filing data from large experiments. Over time, the company shifted toward software for the manufacturers of experimental kits, helping scientists analyze the resulting data. Agents began using that infrastructure as tools, although biological operations can take days or weeks rather than the short tool calls common in coding examples. LatchBio then adapted coding models into products with chat interfaces, dashboards, and external compute. A scientist could ask an agent to compare malignant and non-malignant tissue and identify genes that are overexpressed.
Frontier models still miss the scientific reasoning between code and biology
Workman says frontier models cannot yet be trusted to do real biological work. They can know biology and write code, but that combination does not automatically produce scientific insight from real-world data. Biology also requires data analysis and domain reasoning. LatchBio chose spatial biology as a starting point because it combines rich measurements with visible phenomena, such as development in a mouse embryo. The team had to understand how spatial data was captured and analyzed before it could build useful agents. Workman expects scientific agent products to develop patterns similar to coding agents, with systems that dispatch and coordinate work.
Spatial biology turns tissue measurements into structured analysis tasks
Workman explains sequencing-based spatial analysis as one example. A slide contains beads with DNA attached to them. A tissue section, such as a piece of a tumor, is placed over the slide, allowing RNA to be captured with geometric information about where it came from. The output is a large matrix of numbers alongside a high-content image. Reaching a scientific conclusion requires several processing steps, and those steps vary with the technology, tissue, and disease context. Since the field does not have strong agreement about every step, LatchBio needed a way to measure whether an agent was doing useful scientific work.
Spatial Bench breaks difficult biological work into checkable components
LatchBio built Spatial Bench because existing evaluations focused mainly on question answering or failed to represent the relevant experiment types. The benchmark had 146 problems spanning different spatial biology kits and analysis tasks. Since end-to-end grading was too sparse while models were still weak, the team divided workflows into manageable parts. An evaluation could contain data nodes such as a numerical matrix or image, a prompt describing a scientific goal, grader configuration, and a deterministic Python grading function. Workman says good tests must have a checkable success condition, durable ground truth across valid analysis paths, and conclusions that require interaction with data rather than memorized facts.
Human scientists reveal ambiguity that automated benchmark design misses
After reviewing model trajectories from several releases, LatchBio found that its assumptions about biological tasks were often wrong. Without a canonical answer, having scientists grade one another's work became the best available proxy. A prompt might ask an agent to divide genes into biological categories, score cells, find neighboring cells within an appropriate radius, and calculate a correlation. Each instruction leaves choices about gene definitions, normalization, radius, and count extraction. Human review exposed these open choices as flaws in the tasks. It also showed that numerical quality-control thresholds often came from arbitrary convention. Designing an evaluation forces those assumptions into the open.
Long-horizon biology tasks need intermediate rubrics and contextual judgment
LatchBio built longer tasks to represent real biological work, where several experiment types are combined and every step is interpreted against the study design, prior literature, and the original goal. Some tasks simulate a paper's result section or an industry's decision about whether to continue a drug program. One example asks an agent to use tumor and metastatic biopsies, combining genetic and mRNA information, to find the part of the original tumor that seeded metastatic growth. Workman says no current models get this benchmark task right. Because a final verifiable reward is too sparse, the team is testing rubrics around invariant intermediate points in the analysis. These rubric scores correlate only loosely with final outcomes so far.
Biosecurity evaluations must measure both unsafe answers and refusals
LatchBio is extending its evaluation work into biosecurity. Workman describes routine scientific tasks alongside red-team tasks that appear harmless but contain dangerous intent. A request to clone a gene into bacteria might claim to involve GFP, while the real target could be a toxin or something that helps bootstrap a virus. The team found that routine tasks were used much more often than red-team tasks, which complicates evaluation. Refusing ordinary biology questions is itself a problem because models may refuse basic questions about mitochondria. LatchBio is combining these tests with preprints, evaluations, and agent trajectories as it develops a broader biosecurity program.
"Single cell experiments can yield two to six terabytes per run. Spatial runs can yield seven terabytes of run."01:29
Who should watch
You are building agents for scientific workflows and need benchmark tasks that test data analysis rather than biology trivia.
Your biology evaluation has a plausible final answer, but scientists disagree about the analysis path, thresholds, or task wording.
You work on model safety and need to evaluate ordinary biology assistance alongside requests that conceal dangerous biological intent.
