SCIENCE GANG FIGHT I've pretty closely read both papers from a conf talk on empirical engineering, and now I gotta read the rebuttal super closely, toohttps://twitter.com/vlfilkov/status/1196714347403202560 …
-
Show this thread
-
Replying to @hillelogram
Are these folks really trying to defend their paper? It's one of the most bogus papers I've ever read! Additionally, the "igon value" problems make it clear that the authors don't understand the thing they're allegedly studying.
2 replies 0 retweets 3 likes -
Replying to @danluu
The 2017 CACM paper fixes some of the bigger flaws, like counting V8 as Typescript, but is still very questionable. Have you read the replication? https://dl.acm.org/citation.cfm?id=3340571 … It's really good (and the target of the refutation).
1 reply 0 retweets 1 like -
Replying to @hillelogram
Yeah, it's better, but IMO the methodology is fundamentally bogus. But even if the methodology could work, the errors in the paper are bizarre. Confusing memory safety and implicit coercion, statements like the screenshot, etc. None of the 4 authors noticed or knew better?pic.twitter.com/MZdX6rpPF6
3 replies 0 retweets 1 like -
Replying to @danluu @hillelogram
When I reviewed empirical PL papers, almost all had methodological problems or studied something meaningless, but this was the only paper that convinced me that the authors were less knowledgable on the subject than typical HN commenters.
1 reply 0 retweets 4 likes -
-
Replying to @hillelogram
Ok, I read the rebuttal in detail and it's as bizarre as the original paper. Just for example, consider the screenshotted quote. What could "We were the first to correct our work immediately after noticing this very issue" even mean? This makes no sense.pic.twitter.com/NMxO10unGq
2 replies 0 retweets 3 likes -
Replying to @danluu @hillelogram
Is it conceivable this could be a language issue, e.g. there may be an idiom in the authors' native language(s) with a similar logical structure where it means something more like "it was our first priority to..."? (Yes that's not incredibly logical, but many idioms aren't.)
1 reply 0 retweets 1 like -
Replying to @glaebhoerl @hillelogram
Possibly, but then I don't think it makes sense for them to call the criticism a "
#miss" for that reason. I suspect they meant something like "we fixed this before Emery Berger's paper was published", but that would still be months after I published my criticism.1 reply 0 retweets 1 like -
And even if they were first, their paper is full of results like.... a table that lists bitcoin, litecoin, and qBittorrent as TypeScript projects. They clearly published without basic checks on correctness at any level. Even if they fixed that, they still published it!
1 reply 0 retweets 1 like
Since people often criticize papers for having a single error, writing off the entire paper for a single sloppy mistake, I'd like to make it clear that the paper we're discussing is full of sloppy mistakes like this. You could teach a class off of the mistakes in the paper.
-
-
Replying to @danluu @hillelogram
Alright, I don't have any dog in the broader hunt here, just that specific sentence vaguely reminded me of some of the "un-englishisms" I've encountered while proofreading papers myself :)
0 replies 0 retweets 0 likesThanks. Twitter will use this to make your timeline better. UndoUndo
-
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.