Half of the papers I read that purport to be about production systems are actually about fatally flawed prototypes that never really worked.
-
-
Replying to @johnregehr @danluu and
What about catalog of reasonable baseline algorithms and tasks?
1 reply 0 retweets 0 likes -
Replying to @cjordansquire @johnregehr and
It might be way subject specific, though, so it'd take a series of posts/community wiki/SO post.
1 reply 0 retweets 0 likes -
agree with
@johnregehr that this seems v. hard. Past example tasks (e.g., SPEC) got heavily gamed...1 reply 0 retweets 1 like -
Replying to @danluu @cjordansquire and
For most papers I care about Idk how you'd even create a standard let alone a non-gamable one.
1 reply 0 retweets 2 likes -
Replying to @danluu @cjordansquire and
Many papers claim something like "We build A, 2x better than B", where both are proprietary.
1 reply 0 retweets 3 likes -
Replying to @danluu @cjordansquire and
AFAICT there's no way to verify the claim other than "know a guy who isn't invested in lying"
1 reply 0 retweets 0 likes
And, unfortunately, I don't really see how that can change with proprietary systems.
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.