What's deep learning? The "common usage" definition as of 2019 would be "chains of differentiable parametric layers trained end-to-end with backprop". But this definition seems overly restrictive to me. It describes *how we do DL today*, not *what it is*.
-
-
A single Dense layer is not DL. But a Dense stack is.l DL. K-means is not DL. But stacking k-means feature extractors is DL. When in 2011-12 I was doing stacked matrix factorization over matrices of pairwise mutual information of locations in video data, that was deep learning.
Show this thread -
Programs typically written by human engineers are not DL. Parametrizing such programs to learn a few constants automatically is still not DL. You need to be doing representation learning with a chain of feature extractors.
Show this thread -
By definition, deep learning is a gradual, incremental way to extract representations from data. In its modern incarnation, it's even at least C1 continuous (more typically C inf). That last part isn't essential, but *incrementality* is intrinsic to DL.
Show this thread -
So DL is a fundamentally different beast from symbol manipulation and regular programming, which is fundamentally discrete, flow-centric, and doesn't usually involve intermediate data representations. You could do symbol manipulation with DL, but it involves lots of extra steps.
Show this thread -
These are two entirely different takes on data manipulation. Deep learning isn't just end-to-end gradient descent, but not every program is deep learning either. In fact, deep learning models only represents a tiny, tiny slice of program space. It can't hurt to look beyond it.
Show this thread
End of conversation
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.