Many people sharing this essay arguing that "computational scale beats clever new ideas". It takes for granted backprop, better activation functions, better learning methods, conv nets, better regularization techniques, etc etc. In other words, it seems to ignore the clever ideashttps://twitter.com/gdb/status/1106329741785653248 …
-
-
Nice followup from
@gdb, making the point that it's a false dichotomy (scale versus cleverness):https://twitter.com/gdb/status/1106386231338762240 …Show this thread -
A better essay collecting some (correct!) examples in the same general direction is this paper by Banko & Brill: http://www.aclweb.org/anthology/P01-1005 … See eg this great graph, showing performance as a function of training data size. In this example: more data >> smarter algorithmpic.twitter.com/zwEmQSGMcP
Show this thread -
There is, IMO, a good paper to be written following this up, carefully understanding the relationship between scale and clever ideas.
Show this thread
End of conversation
New conversation -
-
-
GANs are a meta method... Levering pawns is not. The former is universal, the latter is specific for the problem. AlphaZero may not have needed this levering pawn prior.
-
But did levering pawns get discovered by AlphaZero? I would argue that it did absent of any human cleverness.
- 1 more reply
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.