SCIENCE GANG FIGHT I've pretty closely read both papers from a conf talk on empirical engineering, and now I gotta read the rebuttal super closely, toohttps://twitter.com/vlfilkov/status/1196714347403202560 …
-
Show this thread
-
Replying to @hillelogram
Are these folks really trying to defend their paper? It's one of the most bogus papers I've ever read! Additionally, the "igon value" problems make it clear that the authors don't understand the thing they're allegedly studying.
2 replies 0 retweets 3 likes -
Replying to @danluu
The 2017 CACM paper fixes some of the bigger flaws, like counting V8 as Typescript, but is still very questionable. Have you read the replication? https://dl.acm.org/citation.cfm?id=3340571 … It's really good (and the target of the refutation).
1 reply 0 retweets 1 like -
Replying to @hillelogram
Yeah, it's better, but IMO the methodology is fundamentally bogus. But even if the methodology could work, the errors in the paper are bizarre. Confusing memory safety and implicit coercion, statements like the screenshot, etc. None of the 4 authors noticed or knew better?pic.twitter.com/MZdX6rpPF6
3 replies 0 retweets 1 like -
Replying to @danluu @hillelogram
When I reviewed empirical PL papers, almost all had methodological problems or studied something meaningless, but this was the only paper that convinced me that the authors were less knowledgable on the subject than typical HN commenters.
1 reply 0 retweets 4 likes -
-
Replying to @hillelogram
Ok, I read the rebuttal in detail and it's as bizarre as the original paper. Just for example, consider the screenshotted quote. What could "We were the first to correct our work immediately after noticing this very issue" even mean? This makes no sense.pic.twitter.com/NMxO10unGq
2 replies 0 retweets 3 likes -
Replying to @danluu @hillelogram
So they recognized the issues "very soon", but according to the date from the footnote, this was well after I wrote a blog post that got > 1M hits pointing out the issues. But it's technically true they were "the first to correct our work" since I didn't submit a corrected paper!
1 reply 0 retweets 2 likes -
Replying to @danluu @hillelogram
(their footnote dates their correction to sometime in 2015, my blog post was was November 2014). Many of their defenses rely on sleight of hand to make technically correct but actually totally bogus statements. It's like reading a defense from the "power poses" authors.
1 reply 0 retweets 2 likes -
Replying to @danluu @hillelogram
Or the defenses of the early ego depletion and priming studies. But unlike in the social sciences, very few people in "software engineering" (the field they're publishing in) will call out this kind nonsense.
1 reply 0 retweets 3 likes
IMO, this is unfortunate since, based on what's studied, the field ought to be one of the most practical fields for working programmers, but top conferences are full of nonsense (the original paper was published in FSE, which is tied with ICSE as a top conference in the field).
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.