Many of the bogus academic papers I read just dont build out something where you could measure the real effect
-
It might be way subject specific, though, so it'd take a series of posts/community wiki/SO post.
agree with @johnregehr that this seems v. hard. Past example tasks (e.g., SPEC) got heavily gamed...
-
-
For most papers I care about Idk how you'd even create a standard let alone a non-gamable one.
-
Many papers claim something like "We build A, 2x better than B", where both are proprietary.
-
AFAICT there's no way to verify the claim other than "know a guy who isn't invested in lying"
-
And, unfortunately, I don't really see how that can change with proprietary systems.
End of conversation
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.