A tension here: academia is increasingly awful as a career. That drives away many of the best researchers. Reforms that add to the suffering risk making a bad situation even worse.
Currently, reviewing papers is unpaid scut-work. Not surprising not everyone does it well.
-
Show this thread
-

If peer review has reputational risk—it will be public knowledge if you recommend accepting a deliberately bad paper—many people may decline the job. (Though, successful detection of a hoax should look good on your CV!)
Publishers might have to pay reviewers…2 replies 3 retweets 37 likesShow this thread -

… and the total number of papers published might drop precipitously if reviewers were more reluctant to recommend publication.
That would be good. Everyone agrees there’s WAY too much stuff published under the current system. A 90% reduction would be great.4 replies 3 retweets 57 likesShow this thread -

I’d suggest that every PhD student be required to perform at least one attempt at Sokaling as a graduation requirement.
Learning what should count as unacceptably bad research is a critical part of learning how to do it well. And of spotting the difference in the lit.3 replies 12 retweets 65 likesShow this thread -

Initially, everyone would go for low-hanging fruit in Sokaling attempts: the easiest ways to get nonsense past reviewers.
Reviewers would quickly catch on to the simplest tricks… then subtler errors.
And I hope this would lead to a virtuous upward spiral of quality.2 replies 3 retweets 45 likesShow this thread -

Three questions:
𝔸) Would this make academia more adversarial, and thereby even more awful?
𝔹) Can a system develop that is adequately resistant to gaming (Goodhart’s Law)?
ℂ) Is it realistic to imagine something like this could actually happen?6 replies 6 retweets 54 likesShow this thread -

I’d love to hear your opinions about 𝔸 & 𝔹 !
As for ℂ — the crisis in academia is now obvious to all. Things cannot go on as they are. Unusual action becomes possible in extremis.
Recent dramatic process reforms in social psychology are startling, and inspiring.9 replies 5 retweets 42 likesShow this thread -
Replying to @Meaningness
I left psych because of how dogmatic and unfounded it was circa 2008. For instance the work of Carol Gilligan and Howard Gardner was utterly unsubstantiated and yet it persevered in texts, papers and quizzes.
1 reply 0 retweets 0 likes -
Replying to @ratman720 @Meaningness
I believe funding is the key rather than reputation. Reputation in academia amounts to no more than an apology tour, and has no reach compared with the promotional tours from satisfying media positional bias
1 reply 0 retweets 0 likes -
Replying to @ratman720 @Meaningness
Your best bet is to attack the problem from funding perspectives. Fund less first of its kind and more replication efforts. Sokaling journals is great but will have limited impact.
1 reply 0 retweets 0 likes
Plausible… do you have thoughts about how to reform funding decisions?
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.