Red-team programmes that attack cancer AI before patients do
Pay independent experts to try to break cancer AI tools with unusual images, rare cases, bad scans and data shifts, and publish what breaks them.
Robustness of medical AI to artefacts, rare presentations, adversarial inputs and distribution shift is poorly characterised. The proposal funds standing red teams (imaging physicists, pathologists, security researchers) that stress-test cleared and pre-clearance cancer AI with curated adversarial and edge-case corpora, publish failure modes in a common taxonomy, and feed results to the registry and developers, as is done for cybersecurity and increasingly for general-purpose AI.
- AI that is built but not validated or deployed · Thousands of cancer AI models are published; a handful are in clinical use, and fewer have shown they help patients.
Pages like this
not linked directly; found by shared links- IdeaA neutral public evaluator for cancer AI, on the model of NIST
Shares Sequestered, prospectively collected benchmark datasets that no one can train on, AI that is built but not validated or deployed.
- IdeaExternal validation at five or more sites in two countries before clearance
Shares Sequestered, prospectively collected benchmark datasets that no one can train on, AI that is built but not validated or deployed.