Usual reminder: when I've been saying for the past 5+ years that deep learning is interpolative, I don't mean it does linear interpolation in the original encoding space (which would be useless). It does interpolation on a low-dimensional manifold embedded in the encoding space.
-
-
To do actual extrapolation from a few samples, you need a discrete model (like a computer program), not just a parametric curve. A simple example you can play with to understand the difference is list sorting: try to solve it either via program synthesis or via a DL model.
-
You can use 2-3 ex to synthesis a discrete program that will work with any possible list of ints, in any range, of any length. With DL, you'll need to first make the problem interpolative: train on millions of lists, then can only generalize to lists of similar length / ranges
- Show replies
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.