Many people sharing this essay arguing that "computational scale beats clever new ideas". It takes for granted backprop, better activation functions, better learning methods, conv nets, better regularization techniques, etc etc. In other words, it seems to ignore the clever ideashttps://twitter.com/gdb/status/1106329741785653248 …
-
-
A better essay collecting some (correct!) examples in the same general direction is this paper by Banko & Brill: http://www.aclweb.org/anthology/P01-1005 … See eg this great graph, showing performance as a function of training data size. In this example: more data >> smarter algorithmpic.twitter.com/zwEmQSGMcP
Show this thread -
There is, IMO, a good paper to be written following this up, carefully understanding the relationship between scale and clever ideas.
Show this thread
End of conversation
New conversation -
-
-
sutton wasn't arguing a dichotomy. his argument was that scale of search and learning beats cleverness (human knowledge models). reinforcement learning is the process of acquiring better knowledge models of a problem. cleverness scales linearly with computational power.
- 1 more reply
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.