Geneformer

Geneformer is a context-aware transformer pretrained on ~30 million single-cell transcriptomes (Genecorpus-30M), using a novel rank-value encoding: genes are ranked by expression within each cell and the model sees only their order, sidestepping dropout and batch artifacts. Published in Nature (Theodoris et al., 2023), it demonstrated that transfer learning enables predictions in network biology — dosage sensitivity, chromatin dynamics, disease genes — and can discover candidate therapeutic targets (e.g., for cardiomyopathy) via in-silico perturbation experiments that were later validated experimentally.

The V1 model (10M params, 6 layers) has grown into the V2 family (104M and 316M variants) trained on 100M+ cells. Publicly available via Hugging Face, Geneformer is one of the two canonical single-cell foundation models (alongside scGPT) and the standard baseline for cellular transfer-learning research.