A monthly-updated benchmark for AI answers to oncology questions with citation accuracy
Test the large language models doctors and patients are already using against a continually refreshed set of cancer questions, scoring not just correct answers but whether the sources they cite are real and support the claim.
Clinicians and patients use general-purpose language models for oncology questions; evaluations are static, quickly outdated and rarely check citations. The proposal is a living benchmark: new questions each month drawn from recent practice changes, expert-graded answers, and scoring of citation validity and support, with public leaderboards and per-cancer breakdowns, run by an independent academic consortium.
- Knowledge reaches practice too slowly · Knowledge diffusion is slow: it takes years for a proven result to change what most patients receive, and no one can keep up with the literature.
- AI that is built but not validated or deployed · Thousands of cancer AI models are published; a handful are in clinical use, and fewer have shown they help patients.
- Misinformation and unproven therapies · Patients are sold unproven treatments and frightened away from proven ones.
Pages like this
not linked directly; found by shared links- IdeaA public benchmark and audit of chatbot answers to cancer questions
Shares Misinformation and unproven therapies, AI that is built but not validated or deployed.
- IdeaFund expert editors for the cancer pages of Wikipedia and Wikidata
Shares Misinformation and unproven therapies, Knowledge reaches practice too slowly.
- IdeaA verified information layer that labels and links cancer content across platforms
Shares Misinformation and unproven therapies, Knowledge reaches practice too slowly.
- IdeaFund oncologists to maintain cancer articles on Wikipedia in many languages
Shares Misinformation and unproven therapies, Knowledge reaches practice too slowly.
- IdeaLiving plain-language evidence summaries for every common cancer question in 20 languages
Shares Misinformation and unproven therapies, Knowledge reaches practice too slowly.
- IdeaA plain-language explainer for every guideline recommendation, in every major language
Shares Misinformation and unproven therapies, Knowledge reaches practice too slowly.
- IdeaA public API serving the current standard of care for any cancer, stage and biomarker
Shares AI that is built but not validated or deployed, Knowledge reaches practice too slowly.
- IdeaA randomised trial of AI-generated treatment recommendations versus tumour boards
Shares AI that is built but not validated or deployed, Knowledge reaches practice too slowly.