Usual reminder: when I've been saying for the past 5+ years that deep learning is interpolative, I don't mean it does linear interpolation in the original encoding space (which would be useless). It does interpolation on a low-dimensional manifold embedded in the encoding space.
-
-
You can use 2-3 ex to synthesis a discrete program that will work with any possible list of ints, in any range, of any length. With DL, you'll need to first make the problem interpolative: train on millions of lists, then can only generalize to lists of similar length / ranges
-
So how can we deal with the extrapolation problems and solve the causation, and not the correlation? Is there any algorithms for solving these problems or we have only program synthesis?
End of conversation
New conversation -
-
-
Extremely insightful thread. One of the first research problems I encountered turned out to be much easier to solve after sorting. During a discussion with my advisor, my naive suggestion to make it an "end-to-end ml solution" was struck down by this simple question:
-
"How do you think Neural nets can learn how to sort?" I also wrote about it here:https://twitter.com/parthsh_/status/1411895057305653255?s=20 …
End of conversation
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.