Gary Marcus

exactly what i have been arguing, for two decades (eg The Algebraic Mind) and in my 2018 critical appraisal of deep learning cc @stanfordnlp https://twitter.com/weihua916/status/1025137101040508931 …

And of course the way you extrapolate is to change the representation so that extrapolation becomes interpolation. "No extrapolation without representation"

Representation: that's what deep learning is all about.

Learned convolutional filters are certainly a representational advance. Word embeddings certainly capture some important regularities. What else would you consider to be interesting learned representations?