Jon de Quidt

What about selection on observables strategy using matching estimators. Can you produce a density plot for that method as well? I suspect more room for p-hacking there.

Anyone else surprised by RCT pattern? Looks worse than in Brodeur et al (2016), even if still better than DiD and IV. RCT stds for reporting IMO improved in past 5 years. Puzzling?

Could it be publication bias in top places rather than p-hacking? E.g. if null result RCTs which might have landed in AEJ or JDE are now in EDCC or WBER and so out of the sample?

wouldn't that just increase the mass left of the threshold *uniformly*?

I'm not sure. I think (?) it will lead to a spike to the right of the threshold and continuous declines afterwards. Would be consistent with the pattern we see? FWIW, I also don't think pub bias has increased *that* much. Really intrigued with what's going on...

What is the theory for “less manipulation” in RDs?

My guess: RDD's are usually about specific events. So, you can't fuss with the timing in order to make it look different. Either there is a discontinuity at the time/period/point in question or there is not. You can fuss around with controls, but not with the var of interest.

That's the same in RCTs.

Right, the only rhs variables, outside the principle var(s) of interest, in an randomized control trial and the controls. So the method itself does not present additional bits to fuss with. But that still leaves us with an unexplained difference in distributions across methods.

Assuming that the test for difference in distributions is significant, but these graphics pass my eyeball test. What I am wondering aloud, without evidence I admit, is perhaps the differences in subject matter create more opportunity for fussing with controls in RCTs than RDDs.