I suspect that peer review *actually causes* rather than mitigates many of the “troubling trends” recently identified by @zacharylipton and Jacob Steinhardt: https://arxiv.org/abs/1807.03341
Show this thread
-
I frequently serve as an area chair and I manage a small research group, so overall I see a lot of reviews of both my group’s work and others’ work
Show this thread
It’s very common for reviewers to read empirical papers and complain that there is no “theory”. But they don’t ask for theory to address any specific question. I think they are just looking for an easy reason to reject—-they skim and don’t see scary equations.
-
-
This is easily addressed by adding useless mathiness. Reviewers generally don’t call it out for being useless. It passes the “I skimmed and saw a scary equation or pretentious theorem name” test
Show this thread -
Similarly, reviewers often read a submission about a new method hat performs well and say to reject it because there is no explanation of why it performs well
Show this thread -
If you do add an explanation, no matter how implausible or unsupported by evidence, that’s usually enough to placate reviewers
Show this thread -
Reviewers often see papers that use empirical observations to understand how a system works, and respond with complaints that there is no new algorithm. This is easy to address by throwing a practically irrelevant new method into the paper.
Show this thread -
Reviewers seem to hate “science” papers, but it’s possible to sneak science in the door if add some token amount of new method engineering
Show this thread -
(This last one is a bit less consistent than the others. I’ve seen a few science papers get high review scores without having to sell out... but they often get rejected from a few conferences before being lucky enough to get reviewers who get it)
Show this thread -
Some of the other troubling trends would probably happen without peer review, but I see reviewers basically asking to add mathiness, spurious explanations, and spurious novelty all the time
Show this thread
End of conversation
New conversation -
-
-
Btw, the opposite also works: Write a theory paper. Surely there will be a reviewer who says that the lack of experiments is troubling. Sadly, it is rather rare for reviewers taking the time to understand the contributions.
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.