Geneformer
The first widely used transformer trained on millions of single cells, able to predict which genes matter in a disease.
Nature 2023: pretrained on ~30M cells (later 95M) using rank-value gene encoding; in silico perturbation identified therapeutic targets in cardiomyopathy and has been applied to tumours.
Generic schematic · not to scale · placeholder for the ai computation front
Flagged finding · Neural network
How it works
Transformer over rank-ordered gene expression with masked learning.
Strengths
- Transfer learning with little labelled data
Limitations
- Rank encoding loses magnitude
- Modest gains over linear baselines on some tasks
Since
2023
Latest papers
topLatest papers · live from Europe PMC
Query for this technology: (TITLE:"Geneformer" OR ABSTRACT:"Geneformer") AND (cancer OR tumor OR tumour OR oncology OR carcinoma OR lymphoma OR leukemia OR leukaemia OR myeloma OR sarcoma OR melanoma OR glioma). Results are unfiltered search hits about Geneformer, not a curated reading list.