Single-cell measurements could support models of cells, tissues, and organs, with possible uses in rejuvenation and drug development.
2
RNA sequencing is the most available single-cell modality, but it captures noisy snapshots and differs across labs and machines.
3
Flow matching models currently appear more effective than transformer-based models because they model the distribution of single-cell data instead of compressing it into a latent vector.
Summary
Akram Baharlouei explains foundation models for single-cell biology from the perspective of a machine learning engineer without a biology background. He starts with cellular reprogramming and the Yamanaka factors, then connects single-cell measurement to longer-term goals such as virtual cells and faster drug development. RNA sequencing dominates current model training because it is easier to measure at scale, but the data is noisy, heterogeneous, sparse in biological dimensions, and affected by technical differences between labs. Transformer models such as SCGPT treat genes as tokens and cells as sentences, then use masking and prediction objectives. Baharlouei says these models often lose information in their latent representations, and simple linear models can match or beat them. He presents flow matching as a better fit because it predicts the distribution of the data. His conclusion is that progress depends on better measurement quality as well as larger datasets and models.
Cellular reprogramming gives single-cell modeling a medical goal
Baharlouei begins with Shinya Yamanaka's discovery of four transcription factors in 2006. When overexpressed in an old skin cell, they could reprogram it into an embryonic-stem-cell-like state. He connects this work to regenerative medicine, where a cell might be changed into another type, and to partial reprogramming, where its age could be changed without changing its type. He says the hope is to restore youthful function and eventually develop treatments, possibly including an mRNA medicine. He also mentions OSK as a first reprogramming medicine expected to be tested in humans in 2026.
Single-cell data could support models of larger biological systems
The reason to study individual cells is to build a unified model of a cell and then extend that model to tissues, organs, and eventually the human body. Baharlouei points to the Human Cell Atlas as an effort to map cells throughout the human body. He groups ideas such as the virtual cell, virtual tissue, virtual human, and digital twin around the same long-term aim: modeling living systems to improve understanding, treatment, and drug development. He also compares biology with computing. While computing has benefited from Moore's law, drug development remains slow, expensive, and prone to failure, with a pipeline that can take up to 10 years and cost billions.
Single-cell biology uses several measurement modalities
Different single-cell modalities measure different parts of cell biology. Genomics measures DNA, which is mostly similar across cells. RNA sequencing measures gene expression at a given time, producing a matrix of genes and their counts for each cell. Proteomics measures proteins, which perform many functions in the body, but it is difficult to run at high throughput. Morphology captures cell shape and structure through imaging and can provide spatial information about where a cell sits inside tissue. Baharlouei says RNA sequencing is currently used most often for foundation-model training because it is easier to measure and scale.
RNA sequencing is abundant, but it is a noisy snapshot of a changing system
Single-cell RNA datasets can contain tens of millions of cells, and Baharlouei says he has heard of datasets with 500 million or even one billion cells. Scale does not solve the measurement problem. Two identical cells may produce different readings because cells change through growth, RNA copying, and division, with some processes happening in bursts. Current sequencing captures a snapshot from this moving process rather than the whole movie. The measurements also contain biological noise and heterogeneity. Technical factors matter as well, since different laboratories and machines can produce different data. Better foundation models therefore need improved measurements across more biological dimensions, not only larger RNA datasets.
Transformer models treat cells like sentences made from gene tokens
Models such as SCGPT and Informer apply a language-model pattern to single-cell RNA data. They treat a cell as a sentence and genes as tokens. In a BERT-style training process, the model masks some genes and learns to predict their counts from the other genes, using bidirectional attention to learn relationships between them. The resulting representation can support tasks such as cell-type prediction and perturbation-response modeling. Baharlouei describes the common architecture as compressing single-cell data into a latent vector, then decoding it for generation or classifying it for other tasks.
Compression can erase information and leave large models behind simple baselines
Baharlouei argues that compressing single-cell data into a latent vector loses a lot of information. In the benchmarks he mentions, simple linear models can perform as well as, or sometimes better than, complex models trained with substantial compute. He cites two NeurIPS papers from the previous year: one benchmarking multimodal imaging and RNA-sequencing data, and another studying perturbation-response modeling. His conclusion from these comparisons is uncomfortable for expensive foundation-model training. Single-cell models do not yet perform as strongly as comparable models in language or imaging domains.
Flow matching models the data distribution directly
Baharlouei introduces flow matching models as another approach. These models start from Gaussian noise and learn to match the distribution of the data. He presents PrimeFlow, an available arXiv model, as an example that appears to perform better than transformer and autoregressive approaches. In his visual comparison, PrimeFlow's predictions follow the ground-truth distribution, while an autoencoder-based model such as CPA appears to predict the mean. The distinction matters because matching a distribution can preserve variation that is lost when a model compresses the data and predicts a single central outcome.
Progress requires better measurements as well as larger models
Baharlouei's final view is that flow matching currently works better for single-cell data, but scaling remains necessary if models are expected to generalize to unseen data. He calls for much larger datasets and models, while also stressing data quality and the way measurements are collected. Simply increasing the number of RNA-sequencing cells would not make the data more realistic. Other modalities need technical progress, and measurements should better reflect the behavior of real organisms. His closing references include SCGENE, scope, perturbation work from NeurIPS, and PrimeFlow.
"The more we can model this living organisms, the better we are in understanding our body and how we can treat medicine, we can develop drugs."03:58
Who should watch
You are building machine learning systems for biology and need a clear account of what single-cell measurements contain before choosing a model architecture.
You are considering RNA-sequencing foundation models and want to understand why scale alone does not remove noise, heterogeneity, or lab-to-lab variation.
You work on perturbation prediction, cell-type classification, or biological simulation and want to compare latent-vector transformers with distribution-matching approaches.
