Then for each of the other 5 tested hypotheses, anywhere from 20-40% of the labs found them significant. That's...a lot of variability. And this was a high-N dataset! (n=108 people)
Show this thread
Glad that people did this study, but the results are kinda horrifying. 70 fMRI teams analyzed the SAME DATA for 9 pre-determined hypotheses. At best, 85% of teams agreed one hypothesis was significant, and 94% agreed that three more were not.https://www.biorxiv.org/content/10.1101/843193v1 …
-
-
-
So in regular science, where each lab has a different set of fMRI data from a different group of 15 people and analyzes their own pet hypotheses....the concordance will be far worse. The implications seem grim.
Show this thread -
Would love to see an equivalent analysis for something like 2p calcium imaging. Would concordance be better or worse than for fMRI?
Show this thread -
(Of course, synaptic physiologists would get similar results in cross-lab analyses, but that's because (a) our data have the best SNR, (b) it's 1D, and (c) we are gods. cc
@Meaghan_Creed)Show this thread
End of conversation
New conversation -
-
-
Not as bad as it sounds. Frequentist (artificially binarized) results exaggerate team differences. Higher concordance in unthresholded results.
-
You're aware that what people report in fMRI and other studies is, exactly, frequentist results, right? I.e. it's the title or it isn't. It's kind of a big problem that more than half of the pre-set hypotheses on a *large dataset that was identical across groups* were discordant.
End of conversation
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.