ideasIdea
A mandatory silent (shadow) trial before any cancer AI goes live
Before an AI tool is allowed to influence care at a hospital, it would run invisibly alongside clinicians for months so its real-world performance at that site is known first.
Shadow deployment (the model runs on live data but its outputs are hidden and compared with clinicians and outcomes) is standard practice at a few pioneering centres but not required. The proposal makes a pre-specified silent trial (minimum case numbers, pre-declared performance thresholds, subgroup analysis, comparison with local clinicians) a condition of go-live at each site, reported to the registry, as the AI equivalent of laboratory method verification before a new assay is used.
Hypothesis
Site-level silent trials will identify locally unacceptable performance in a meaningful share of deployments that passed regulatory clearance, preventing harm and building trust at sites where the tool passes.
Rationale
Clinical laboratories must verify every new assay locally before use, because performance depends on local conditions; AI has the same dependence and no equivalent rule.
What would test it
Require silent trials for all AI deployments in one hospital network for two years; report the proportion failing local thresholds and the reasons.
Maturity
early clinical
Who has to act
clinic
Cost to try
Small (under $1M)
Years to first evidence
1
Bottlenecks it attacks
- AI that is built but not validated or deployed · Thousands of cancer AI models are published; a handful are in clinical use, and fewer have shown they help patients.