Conversation

The key figure here is harrowing. For p<0.05, the false positive rate is always >33% if prior odds are 1:10!

Quote Tweet

New Nature Human Behavior paper: 72 of us make the case to redefine statistical significance from .05 to .005. osf.io/mky9j

Looks interesting, but can you explain (or link to) what “prior odds” are? So far I don't understand the chart.

Our initial estimate for Pr(H1)/Pr(H0), based on our own beliefs, scientific consensus, etc. en.wikipedia.org/wiki/Prior_pro

So if I believe very much in my hypothesis (1:10 or more) than with >33% probability I will have a statistically significant (0.05) result?

Replying to

It’s the other way: with >33% you will see a p<0.05 result where there is actually none.

Replying to

Right, where there is actually none, important detail. But, yes, that's what I meant.