Making Sokaling a routine part of the academic process would go a long way towards fixing it, I think. A few dozen Sokalings are a good start to raise awareness, but if—say—5% of all submitted papers were Sokals, reviewers and editors would become much more careful.
-
-

I’d love to hear your opinions about 𝔸 & 𝔹 !
As for ℂ — the crisis in academia is now obvious to all. Things cannot go on as they are. Unusual action becomes possible in extremis.
Recent dramatic process reforms in social psychology are startling, and inspiring.Show this threadThanks. Twitter will use this to make your timeline better. UndoUndo
-
-
-
a) Yes. I also worry it would make it more conservative - it's hard enough to get niche papers about weird ideas published already (speaking as someone who is trying) b) Probably, but not easily. c) No. It requires unilateral coordination from people it would hurt.
-
Maybe that was too harsh. You might be able to do something with the idea on a sort of alt-peer-review basis - build a community of people who are actively working on improving their research together and use it as a sort of "certification" process by submitting preprints to it.
- 9 more replies
New conversation -
-
-
Good/important thread. Specifically regarding 15A, you might find this result interesting -https://twitter.com/davidmanheim/status/1047768360501596161 …
-
Thank you! Will read when I get a chance
End of conversation
New conversation -
-
-
𝔹 seems to asks for a game robust against gaming, so I’m skeptical. Low-cost Sokalling defenses not reflecting academic integrity are numerous and include ID, credential, and publication history checks.
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
-
-
A) Sadly yes. Unless we get some other way of assessing performance than "publish or perish" the core problem is there. B) Not sure. C) No. It could it science was only done in one country, but would be impossible to coordinate internationally, which is what really matters.
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
-
-
𝔸: Sokaling to date has been interdisciplinary. Perhaps intradepartmental politics could benefit from waging open war with the outgroup.
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.
How would this work in practice? A paper would have to be registered as a Sokaling before first submission, along with an explanation of what the author thinks is wrong with it. A cryptographic time-locked database could ensure honesty about this.
Fraud—invented facts—are a different problem from nonsense. In science, publishing a paper based on false, made-up data would not generally count. Either you use real data and give a bogus interpretation, or you describe a worthless data-collection process (with fake data).
This proposal is not nice. Unfortunately, it is too late for nice. Many-to-most academic fields run on a go-along-to-get-along basis, and now have large negative net value as a result.
Some fields should simply end. I suggested that for nutrition:
A tension here: academia is increasingly awful as a career. That drives away many of the best researchers. Reforms that add to the suffering risk making a bad situation even worse.
Currently, reviewing papers is unpaid scut-work. Not surprising not everyone does it well.
… and the total number of papers published might drop precipitously if reviewers were more reluctant to recommend publication.
That would be good. Everyone agrees there’s WAY too much stuff published under the current system. A 90% reduction would be great.
I’d suggest that every PhD student be required to perform at least one attempt at Sokaling as a graduation requirement.
Learning what should count as unacceptably bad research is a critical part of learning how to do it well. And of spotting the difference in the lit.
Initially, everyone would go for low-hanging fruit in Sokaling attempts: the easiest ways to get nonsense past reviewers.
Reviewers would quickly catch on to the simplest tricks… then subtler errors.
And I hope this would lead to a virtuous upward spiral of quality.
Three questions:
𝔸) Would this make academia more adversarial, and thereby even more awful?
𝔹) Can a system develop that is adequately resistant to gaming (Goodhart’s Law)?
ℂ) Is it realistic to imagine something like this could actually happen?