How would this work in practice? A paper would have to be registered as a Sokaling before first submission, along with an explanation of what the author thinks is wrong with it. A cryptographic time-locked database could ensure honesty about this.
-
-
This reminds me of the blind injection of fake gravitational wave signals in LIGO. For them, a small cabal in the collaboration does the dirty work. In my subfield, big shot groups would have to divert energy from competing with each other to do this.
-
I haven’t heard about that (with LIGO)! Can you say more?
- 2 more replies
New conversation -
-
-
this ought to be the default position in all kinds of research, especially research where methods produce, at best, correlative results.https://twitter.com/ProfFeynman/status/1047530013929553920 …
-
I think adversarial "red team" actors will be partially effective but also divisive and prone to politicization for all the usual reasons, mostly because there's money and career status at stake. A shift in norms to make it more acceptable to say "I don't know" is best
End of conversation
New conversation -
-
-
I think it would divert energy into Sokal-chasing. It would become a sophisticated game, the way securing funding has become a sophisticated bureaucratic and interpersonal game. To the detriment of time+energy spent actually doing science.
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
-
-
W.r.t. (A) there is definitely an implementation of this that makes academia more adversarial, and more miserable, especially in the short term given the scope of the problem.
-
In a way it’s like a low-level continuous Denial of Service (DoS) against academia. Most large enough technology companies experience such low-level attacks and can eventually automatically detect and prevent them. Something similar could possibly be developed here.
- 1 more reply
New conversation -
-
-
I think that what you're referring to as a crisis has been the case with academia for 2k years (have you read academic publications, even post sci-rev, say, 17th, 18th, 19th c.?) and is mostly a case of mismatched expectations.
-
Academics, even respectful ones, used to regularly publish proofs of God's existence, described non-existing animals, and in 19th-20th cs wrote a lot on race and eugenics. Academia was never "fixed." It just doesn't work the way you think it's supposed to.
- 5 more replies
New conversation -
-
-
I'd debate to what degree ℂ is truly "obvious to all". There are still papers coming out arguing there is no crisis, and surveys finding large (albeit minority) portions of various scientific communities not seeing problems. Ideology also means many *want* the fields this way >>
-
I saw plenty of left social psychologists acting like this hoax was "no big deal" and various other forms of downplaying, almost entirely predictable by the partisan alignment of the social psychologist. I fear ideological polarization makes the problem not (currently) solvable.
- 1 more reply
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.
Fraud—invented facts—are a different problem from nonsense. In science, publishing a paper based on false, made-up data would not generally count. Either you use real data and give a bogus interpretation, or you describe a worthless data-collection process (with fake data).
This proposal is not nice. Unfortunately, it is too late for nice. Many-to-most academic fields run on a go-along-to-get-along basis, and now have large negative net value as a result.
Some fields should simply end. I suggested that for nutrition:
A tension here: academia is increasingly awful as a career. That drives away many of the best researchers. Reforms that add to the suffering risk making a bad situation even worse.
Currently, reviewing papers is unpaid scut-work. Not surprising not everyone does it well.
… and the total number of papers published might drop precipitously if reviewers were more reluctant to recommend publication.
That would be good. Everyone agrees there’s WAY too much stuff published under the current system. A 90% reduction would be great.
I’d suggest that every PhD student be required to perform at least one attempt at Sokaling as a graduation requirement.
Learning what should count as unacceptably bad research is a critical part of learning how to do it well. And of spotting the difference in the lit.
Initially, everyone would go for low-hanging fruit in Sokaling attempts: the easiest ways to get nonsense past reviewers.
Reviewers would quickly catch on to the simplest tricks… then subtler errors.
And I hope this would lead to a virtuous upward spiral of quality.
Three questions:
𝔸) Would this make academia more adversarial, and thereby even more awful?
𝔹) Can a system develop that is adequately resistant to gaming (Goodhart’s Law)?
ℂ) Is it realistic to imagine something like this could actually happen?
I’d love to hear your opinions about 𝔸 & 𝔹 !
As for ℂ — the crisis in academia is now obvious to all. Things cannot go on as they are. Unusual action becomes possible in extremis.
Recent dramatic process reforms in social psychology are startling, and inspiring.