Gary Marcus

Yes, please read the book. But also compare the argument you just made - “it’s already been tried, and failed” — to what might have been applied to deep learning in 2010.

I haven't read Gary's book yet, but I doubt he's advocating 'endowing machines with common sense' (his words) 'in the obvious way was tried for 50 years' (Felix's words)

I'm inclined to agree that a focus on 'common sense and deep understanding' is the right approach. But I suspect it can be done with deep learning and the right architecture

of course, a lot rests on what you mean by right architecture, and whether you include symbol-manipulation (which we argue is a key component) in that scope. you and i probably ultimately agree on the value of *hybrid* architecture.

I think we can agree that the brain has specialized components that work in concert to achieve human cognition. We can furthermore agree that each of these components is built of specialized neural networks. Hybrid architectures are a stepping stone towards more general kinds.

if you have specialized components that operate under different principles, you have hybrid systems.

Can you more formally define what these hybrid systems you have in mind are? I sense there's a lot of overlap between what DL researchers consider as architecture choices or other inductive biases.

In lieu of @GaryMarcus definition, I would define a hybrid system as that having components that are both grown (i.e. trained like a neural network) and human-engineered (i.e. tree search). I argue that the latter can also be grown and thus it is a stepping stone.