If I follow those threads, I get a paper with an interaction effect close to 0.05. Interaction effects with such p-values do not replicate.
On the hot new IAT study: A 1 standard deviation fall in IAT would close 6% of the math grade gap between Italian immigrants & natives. Objectively graded scores predicted 80% of the math grade gap. Pygmalion in the classroom may not be a myth, but it's not most of story.https://twitter.com/petesivey/status/1073399348283621376 …
-
-
-
Noteworthy! Still, p-values overrated compared to magnitudes, and these IAT magnitudes arepic.twitter.com/eCzmIRbN8h
-
I agree with that. My point is just that in addition to the magnitudes being tiny, interaction effects do not replicate at these magnitudes / p-values. So no reason to think that even the tiny effect is true.
-
I just investigated this in a study using real data. At p < 0.05, 97% of main effects replicated vs 7-20% of interaction effects. http://rpubs.com/Jonatan/interactions …
-
I know I should Google these questions, but I'll just ask the expert: Does this fail for Bonferroni-type reasons? I.e., is the interaction akin to a multiple comparison? And where's a good theoretical argument on this?
-
They fail because interactions require much larger sample size to have decent power. (Theory described here: https://andrewgelman.com/2018/03/15/need-16-times-sample-size-estimate-interaction-estimate-main-effect/ …) With low power, most of the significant effects we happen to see, are due to randomness.
- 1 more reply
New conversation -
-
-
To be fair, one author emphasizes that the main contribution is about effect of the "revealing" treatment: https://twitter.com/michelacarlana/status/1073593431304400896 … Also that the policy implications are unclear:https://twitter.com/michelacarlana/status/1073123867953192961 …
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.