Having access to "infinite compute" is absolutely not a necessary factor for doing good research. In more ways than one, the opposite is truehttps://twitter.com/benhamner/status/966823205842243584 …
Show this thread
In machine learning, models that generalize are those with significant constraints -- regularization, limited memorization capacity, architecture encoding abstract assumptions about the data. The purpose of models is to generalize, not to optimize
-
-
In research, extensive hyperparameter tuning & architecture search results in ovefitting to a task (to the distribution of a specific dataset, even with no information leaks from the test set). But the goal of research is to generalize, not to optimize. To create new knowledge.
Show this thread -
The trap of compute is the temptation to substitute research for optimization
Show this thread
End of conversation
New conversation -
-
-
I wish it were more to explain...
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.