From Tokens to Cells: Foundation Models for Single-Cell Biology
summary
Single-cell datasets scale to tens to hundreds of millions of cells but are noisy, batchy and inherently heterogeneous, so masked-transformer models that treat genes as tokens often compress away critical signal and sometimes lose to simple linear baselines. Flow-matching generative models like PrimeFlow flow-matching outperforms transformers by matching distributions instead of predicting means. Concrete wins: benchmarking shows distributional models improve perturbation and generative fidelity; concrete gaps: compressing loses critical info and measurement quality, not just scale, limits usefulness.
AI-written summary · about
chapters (8)
- 0:00Single-cell biology intro
- 1:14Why single-cell biology matters in medicine
- 3:46Measuring cells across modalities
- 8:45Data heterogeneity challenges
- 10:30Transformer models for single-cell
- 13:02Benchmarking foundation models
- 14:09Flow matching vs transformers
- 15:14Key takeaways and future
related talks
23mFLUX, Open Research, and the Future of Visual AI
33mFrontiers of Foundation Models for Time Series
31mFoundational Models for Time Series Forecasting: are we there yet?
26mRethinking the Transformer: Toward Native Multimodal Architectures
21mFoundational Models for Time Series Forecasting: are we there yet?
01h 00mReducing the Dimension of Language: A Spectral Perspective on Transformers
33mTrillion-Token Pretraining
34mTransformers End to End Experience training...