You can add location information to your Tweets, such as your city or precise location, from the web and via third-party applications. You always have the option to delete your Tweet location history. Learn more
Usual reminder: when I've been saying for the past 5+ years that deep learning is interpolative, I don't mean it does linear interpolation in the original encoding space (which would be useless). It does interpolation on a low-dimensional manifold embedded in the encoding space.
Anyway, check out chapter 5 of my book for details on how DL models manage to do it, and why this is conducive to (local) generalizationhttps://www.manning.com/books/deep-learning-with-python-second-edition?a_aid=keras&a_bid=76564dff …
Manifold interpolation is like asking: what's the mid-point between a fork and a spoon (it terms of their function)? Which you can approximate if you've got a database of utensils related by pairwise functional distance
Taking a bunch of numerical features (e.g. images) describing spoons and forks and interpolating linearly between them would be entirely meaningless...
It's funny but most applied mathematicians would not call interpolation generalization.
The term "generalization" means something completely different in math compared to ML.
I’ve wondered a bit / gone down rabbit holes in the recent past about the connections between language embeddings and topology, but it would be fascinating to see if there are task / data independent forms that emerge in the low-dim manifold where interpolation takes place.
Makes sense. Thinkinging in topology always seems to help me think about ML. Makes me think about how many languages evolved with similar grammars, showing somewhat of the innate structure of language in our brains.
What do you think of "Extrapolation"? How can deep learning solve extrapolation problems?
No. The only way to extrapolate statistically is to represent the problem in such a way that it is interpolating instead. This requires some form of prior knowledge or constraint
Interesting take. Isn't that what humans do as well? Creativity is constrained by whatever we've already seen before, and is somehow an interpolation of already seen phenomena.
Yeah and the weird thing is that when humans making predictions we can tell the evidences which are, no surprises, hundreds of training examples we’ve seen before …
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.
