Emil O W Kirkegaard

Makes you wonder if maybe their measures of hormones are just poor?

The preprint studies by our lab include positive controls (e.g. effects of hormones on general sexual desire, weak within-woman effects of hormones on appearance) and the assays were done by commercial labs. So, unlikely to be that.

Also worth bearing in mind that the sample sizes in our replications are at least double that of the original studies and in some cases an order of magnitude larger. Also, surprisingly few of the original studies actually measured hormones....

Unsurprisingly, the results we replicate are the ones with measured hormones and decent Ns (e.g. results reported by Puts and colleagues and Roney and colleagues).

I have no issues with the claims being wrong. I was just wondering if you have done a simple test-retest (within a few hours) study on some subsample to establish reliability? Some time ago, I looked for a such study in these hormone studies, never found it.

They are mostly longitudinal analyses, with samples taken at the time of behavioral testing. Test-retest of homrone levels is good (lowest for cortisol, as you’d expect from the lit)

I’m a bit surprised you couldn’t find test retest data for hormone measures though. There are lots of studies published on it (eg one looking at how they change over the day, cycle, following different types of tasks, including control tasks, etc).

Care to link to a few? What's the re-test correlations like?