One interesting thing about the ARC competition is that it serves to highlight how people who use deep learning often have little idea of what deep learning actually does, and when they should be using it or not
Show this thread
-
Differentiability & minibatch SGD are the strengths of DL: besides making the learning practically tractable, the smoothness & continuity of the function & the incrementality of its fitting work great to learn to approximate latent manifold. But its strengths are also its limits
Show this thread
The whole setup breaks down when you are no longer doing pattern recognition -- when you no longer have a latent manifold (any kind of discrete problem) or no longer have a dense sampling of it. Or when your manifold changes over time.
-
-
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
-
-
I'm not sure what a discrete problem that someone would try DL on looks like. Do you have some bad examples?
-
NLP is essentially a discrete problem. We see the quality of GPT-2 despite huge efforts.
- Show replies
New conversation -
-
-
The game of go and chess are highly discrete. One slightly wrong move and the game is lost. Still AlphaZero does very well. Even move proposals from a ResNet (without search) beats very good amateur go players.
-
These games involve a mixture of pattern recognition (what a player would call 'intuition') and explicit reasoning. It's not all or nothing. The better you are at pattern recognition, the less you need to rely on reasoning, and inversely.
- Show replies
New conversation -
-
-
> "data manifold changes over time" Would this help mitigate?https://twitter.com/semanticbeeng/status/1247873043810189313 …
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.