Having access to "infinite compute" is absolutely not a necessary factor for doing good research. In more ways than one, the opposite is truehttps://twitter.com/benhamner/status/966823205842243584 …
Show this thread
-
In research, extensive hyperparameter tuning & architecture search results in ovefitting to a task (to the distribution of a specific dataset, even with no information leaks from the test set). But the goal of research is to generalize, not to optimize. To create new knowledge.
Show this thread
The trap of compute is the temptation to substitute research for optimization
-
-
I argue what you're describing is the trap of competitions based on well known datasets. When working on research with commercial applications in problem spaces with unknown solvability, more compute is better imho. Trade compute time for engineering time.
-
In this particular case limiting compute allowed for much faster iteration, thus saving engineering time, but it surely depends on the problem.
End of conversation
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.