A great future metascience project (dissertation for someone) would be to investigate what proportion of significant moderators from META-ANALYSES were ever subsequently tested with a direct well-powered study, and what proportion of those do/don't replicate.
-
It's been done before using previous and later metas. Moderators did generally not replicate was the conclusion. I think it was Schmidt and Hunter who did this or at least mentioned it in their book. Tldr have low priors for moderators because interactions are rare.
Sort of discussed here, but not citing the study I'm thinking of (quoting 3rd edition). But this study is about right. …https://systematicreviewsjournal.biomedcentral.com/articles/10.1186/2046-4053-2-107 …pic.twitter.com/o4BdHIsfNS
