Vinay Prasad

It turns out the goal of cancer screening is NOT to find biopsy proven cancer. That is a poor surrogate for what you want to find. Side note: they run the problem of: the biopsies that would have been taken if their algorithm existed do not exist. Will return to this...

Cancer screening can find one of several things 1 something that is not cancer 2 a cancer that isn't going to bother you in your natural life (harmless) 3 a cancer that was going to harm you, but we found it, and can cut it out, and now it isn't going to harm u (curable) AND...

4. A cancer that has already spread and is going to harm you even though we found it (spread-already) We want a cancer screening test that finds more curable cancer. We don't want to find more 'not cancers' (#1), but here is the tricky bit

we also do not want to find more harmless cancers (#2), AND we don't want to find more spread-already cancers (#4). We want to find more curable cancers, but less benign lesions, harmless cancers, and spread already/ damage done cancers!

THERE AREN'T ANY.... No one knows. So if you unleash AI on a problem and prove you are better at finding biopsy proven cancer you have no idea if you are changing the ratio of harmless to curable to spread-already And without knowing that you don't know you are helping

You may paradoxically be making it worse! BTW, as I was reading this I see more evidence that @cragcrest is awesome...https://twitter.com/cragcrest/status/1212736228640120833?s=20 …

Now back to this idea of biopsies that don't exist. The other big problem with AI of diagnostic imaging is retrospective validation does not account for the fact that prospective deployment may change the way data is collected

There may be biopsies that AI would have encouraged that do not exist, and we don't know the results of tests that were not done. Anyway, back to my bigger point. Cancer screening is the LAST thing you should ask AI to do FIRST

Whatever gains we think (10-15% RRR on cause specific mortality in cochrane meta-a) are contingent on therapies at the time (bad, and new drugs erode screening gains), and the specific modalities used.

When you change the rules around how the study is interpreted, you cannot be sure that the net result is better EVER if you find more cancer and EVEN if you find less non-cancer. Because you don't know: harmless from curable from spread-already ratios in what you find

And you will not know that unless you pony up and conduct a 15 year multicenter RCT. There are so many better diagnostic tests with short term mortality outcomes that AI should be applied to FIRST. Don't make it harder than you need too.

See also last season of Silicon Valley.

In response to several comments that see this as not improving outcomes but labor saving: There is no way on earth that if you deploy this prospectively you will only lower #1, and keep 2-4 perfectly identical That is an artifact of retrospective studies