Gary Marcus

Deep learning more and more looks like a scientific revolution in the sense of Thomas Kuhn. As exciting as seeing engineering successes, is to experience first hand how a research field goes through a “paradigm shift”.

i stand by these claims. Importantly though the term classical is not mine, and interested readers should read the original source, The Algebraic Mind.

I'm happy that you do; for one thing, your work helps to make the point that it is not obvious that neutral networks can learn such things, and this helps explain the relevance of some of our results. But I disagree that NNs are fundamentally unable to learn rules, variables, 1/n

tree structure, or compositionality, at least under reasonable definitions. Not reasonable: picking your favorite symbolic system, checking whether a NN learns exactly that, and interpreting failure as evidence that the whole class cannot be learned.

I feel like this raises as many concerns as it addresses... shouldn’t a properly compositional system be able to do quite a large range of tasks? So for any given task, why would an apparently compositional net fail?

We looked at a simple arithmetic task with addition, substraction and brackets. Simple, but an infinite domain and clearly compositional. Networks approximate answers within training range almost perfectly, generalize quite well, with errors increasing with length of expressions.

Isn’t that pretty much as I anticipated? Cc @rgalhama