if nobody bites on question 3, it will make my next medium post a lot easier to write :)https://twitter.com/GaryMarcus/status/1209640096900812800 …
-
-
Replying to @GaryMarcus
3. Anything that doesn't operate over a vector space is not deep learning.
1 reply 0 retweets 2 likes -
Replying to @EliSennesh @GaryMarcus
3. RCN +CHMMs (Vicarious), Bayesian Program Learning (Lake), most probabilistic programming, pure spectral / PCA meths 4 extracting features, most evol. programming/GAs, certain “hybrid systems” like recent Tenenbaum... whatever doesn’t rely on backprop as core engine of learning
2 replies 1 retweet 6 likes -
Deep learning is the use of backprop/auto-diff. to allow gradient based learning of free parameters in large networks of “parametrized functional modules” which may include functions or compositions thereof supporting attention+variables, and may include non-learned modules also.
1 reply 0 retweets 3 likes -
When most or all free parameters in the system are being learned by backprop-enabled gradient descent, that’s deep learning.
1 reply 0 retweets 2 likes -
Your definitions are good but I prefer clusters induced by definitions to be context dependent. For example, I can have a giant network where each function is the dot product & each edge a weight. It would be differentiable but it can be rewritten as a single linear transform.
1 reply 0 retweets 1 like -
Replying to @sir_deenicus @AdamMarblestone and
Convnets, Simple RNNs & even MLPs can be deep yet need not be coupled to an attention mechanism Regarding vector spaces: lots fit here (linear models trivially). You can have recursive probabilistic programs that are effectively over a vector space & deep in any meaningful sense
1 reply 0 retweets 1 like -
Replying to @sir_deenicus @AdamMarblestone and
My point is your definitions aren't wrong but more so that I don't like hard clusters. I'd like to be allowed to group Model View Update for UI programming with GANs (if talking about coalgebraic) or deep reinforcement learning with shortest path depending on my current context.
2 replies 0 retweets 2 likes
Sure! And I’m not sure definitions are very important either way. But since Gary wants one...
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.