Dan Luu

Half of the papers I read that purport to be about production systems are actually about fatally flawed prototypes that never really worked.

I don't think people will speak up without anonymity. And maybe not even then.

I once had someone make the case I could be fired because of comments similar to the ones I've made here.

IMO we're v. sensitive to this because we incentivize publishing bogus papers and produce more than our share.

well, I don't think industry authors are making the overall bogus paper problem any worse, on average

Ok, fair enough, but I think that we should know better. "We" often claim 2x gain when real system is a no-op.

Many of the bogus academic papers I read just dont build out something where you could measure the real effect

or they just have no idea what the baseline should be. IMO that's a lot more excusable than what "we" do.