In machine learning, models that generalize are those with significant constraints -- regularization, limited memorization capacity, architecture encoding abstract assumptions about the data. The purpose of models is to generalize, not to optimize
-
-
Show this thread
-
In research, extensive hyperparameter tuning & architecture search results in ovefitting to a task (to the distribution of a specific dataset, even with no information leaks from the test set). But the goal of research is to generalize, not to optimize. To create new knowledge.
Show this thread -
The trap of compute is the temptation to substitute research for optimization
Show this thread
End of conversation
New conversation -
-
-
Very true. La nécessité est la mère de l'invention. But there is also the (somewhat paradoxical) observation that doing good research is (partially) motivated toward bringing funding support for improving one's computing facilities ...
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.