Ian Goodfellow

I suspect that peer review *actually causes* rather than mitigates many of the “troubling trends” recently identified by @zacharylipton and Jacob Steinhardt: https://arxiv.org/abs/1807.03341 

I frequently serve as an area chair and I manage a small research group, so overall I see a lot of reviews of both my group’s work and others’ work

It’s very common for reviewers to read empirical papers and complain that there is no “theory”. But they don’t ask for theory to address any specific question. I think they are just looking for an easy reason to reject—-they skim and don’t see scary equations.

This is easily addressed by adding useless mathiness. Reviewers generally don’t call it out for being useless. It passes the “I skimmed and saw a scary equation or pretentious theorem name” test

Similarly, reviewers often read a submission about a new method hat performs well and say to reject it because there is no explanation of why it performs well

If you do add an explanation, no matter how implausible or unsupported by evidence, that’s usually enough to placate reviewers

Reviewers often see papers that use empirical observations to understand how a system works, and respond with complaints that there is no new algorithm. This is easy to address by throwing a practically irrelevant new method into the paper.