It should comes as no surprise, then, that the first author of this paper (@GaborMelis) is a former Kaggle contest champion 
https://www.kaggle.com/melisgl/competitions …
This is why I like Kaggle, or in general, competitions with a hidden test set, a reproducibility constraint, and a money incentive. It forces you to be honest with yourself and your peers. Academic incentives encourage intellectual dishonesty.https://twitter.com/egrefen/status/928897300675661824 …
-
-
-
Should *come*. I really need to proof-read more.
End of conversation
New conversation -
-
-
true. but current major drawback of competitions is that they have an agenda of getting better number in the eval metric which doesnt inspire novelty... You'd often see huge ensembles of models and difference in LB at extreme decimal places. 97.108 == 97.109 imo
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
-
-
Interesting. In a talk,
@antgoldbloom said something along the lines of how academics often do terrible at Kaggle competitions (at least initially)Thanks. Twitter will use this to make your timeline better. UndoUndo
-
-
-
Right, because as we’ve seen repeatedly in the last decade, monetary incentives NEVER produce dishonesty
-
I'm pretty sure the benefits only apply when all three conditions are in place. We could call it a .... _virtuous triangle_.
End of conversation
New conversation -
-
-
couldn't agree more
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
-
-
Agreed, contests are nice that way. OTOH, they are surprisingly hard to run right. Often evaluation variance is too high, there is a data set leak, or the dataset is broken in some other way. Plus, there are no "good idea papers" just bad scores.
-
Hard to run, but when hundreds of participants give it a shot, any bug in the data/eval becomes apparent after a few weeks. Self-correcting system. Also, if you have a good idea that scores poorly, you can always write about it. The score forces you to be honest about its merits
End of conversation
New conversation -
-
-
A big problem with Kaggle is the precariousness of the model from a workforce point of view. You end up with companies having thousands working for them for peanuts
Agree with you about its efficiency compare to Academia. Hope we will find a middle path.Thanks. Twitter will use this to make your timeline better. UndoUndo
-
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.