I have been asked many times if we submitted the Social Sciences Replication Project to Science and Nature first. We did, and it did not make it through editorial review. But, I did not perceive any resistance to the findings by the Editors. In fact, ->https://www.nature.com/articles/s41562-018-0399-z …
-
-
Respectfully, thanks for the peek behind the curtain...but identifiability aside, I'd be SUPER uncomfortable as a reviewer if any of the comments I wrote to an editor and authors was blasted out to 15k+ twitter followers (plus retweets, etc). Without consent, please don't share.
-
As long as anonymity is intact, there's no harm in sharing like this. There's no repercussion for the reviewer. Could be an incentive! It is now my goal to review a Nosek paper with a comment so incendiary that he Tweets it. List me for future review recs! You'll be soooo sorry!
-
It is a fine goal. But, you'll also probably have to stage asking me to tweet it. It only occurred to me to do so in response to Q about "why rejected". If we are launching a "nasty review comments" twitter campaign, I'd have a monster thread! :)
-
So was that review at Science or Nature, where you said it didn't pass editorial review? Or was that a reviewer at NHB? lest anyone impute hostile motives to me, I'm big a fan of this paper,
@BrianNosek and@TheNewStats . -
This is a small part of one review from the original submission to Science. And, I will clarify in return that Will is completely within bounds of reason, collegiality, and good taste to raise the objection!
-
Right back at you! I wouldn't have raised objections if I thought it would be anything less than collegial. And the arguments I've made today are more 'testing the waters'/devil's advocate than anything else. I'm sure an intrepid twitterer can find me posting reviews i hated
End of conversation
New conversation -
-
-
Interesting to see the reviewer question motives (of both original and current paper), even though in the new paper every effort is made to be transparent and reproducible.
-
Hmm, I see the reviewer as having the misconception that the authors (of original and current) are monolithically aligned with their respective results; as opposed to all working towards scientific truth, with the separate individual results being stochastic data points.
-
The funny thing to me is that at this point a replication where the effect size did not go down would almost be so novel that it would be noteworthy and exciting in itself...
End of conversation
New conversation -
-
-
I've received very similar comments in reviews. Goes to show that replication definitely is not mainstream yet in the big world outside our bubble.
-
I'm submitting my first pure replication paper soon, and am expecting the worst in peer review. I'm sure I'll receive similar reviewer comments. It'll sting. I'll move on.
-
My experiences with rep papers have been mostly positive. Of the 5 grad class rep projects completed so far, 4 are pub'd and we received a very positive R&R (minor revisions) on the 5th. So, 5/5 (for class project rep's). Most in the first place we submitted. So there is hope.
-
What journals were those?
-
Of the 5 I mentioned: Journal of Experimental Social Psychology x 2, Scientific Data x 2, and Collabra (revision about to be submitted). First rep study I did with
@eplebel was pub'd in Psych Science. -
here's a curated list of seven of
@LorneJCampbell 's articles reporting replications (powered by@curatescience): http://curatescience.org/author-page.html …pic.twitter.com/voP66L3U0D
End of conversation
New conversation -
-
-
It's not of interest to the scientific community to see multiple replications with much larger, more reliable, sample sizes? Who made him/her the spokesperson for the scientific community? My scientific community craves high-quality replications. It’s about truth, not reputation.
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
-
-
There's a whole sociology of science PhD dissertation topic in that one excerpt! (and the fact that these journals both listed RPP in their top breakthrough projects of the year, what, two years ago?)
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
-
-
This is just brutal. I’m really sad that people *still* can’t see the value in high power replication. Sigh. PS Congrats on another great paper and thanks for the continued outstanding work to improve science.
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
-
-
Someone is unaware of the time-reversal heuristic.
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
-
-
Honestly, that's a poor response. The reviewer is constructing a ridiculous straw man, and making a counter argument based on a childish view of the "scientific community" and speculations about the authors' motivations. Makes me think the reviewer is simply incompetent.
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.