Gary Marcus

...But they have existed for a very long time: since the early 90s for Q1 and since the early 80s for Q2. Now that the DL machinery works, and that so many people are working on both Qs, we have a shot at making real progress.

I guess the remaining question for your position are: GQ1. Will DL be part of the solution (you said yes) GQ2. Do you agree with "vectors, not symbols; diff functions, not hard logic" GQ3. If not, how do you propose we make reasoning compatible with DL?

GQ1. DL is part of the solution GQ2. "vectors, not symbols” is false dichotomy diff functions: yes, in part Operations a la logic, we do need (contra your view) GQ3. Outputs of deep learning may serve as input to reasoning; symbolic techniques needed for some inferences.

Most humans don't actually do much that resemble your answer to GQ3, except a small number of humans using pen and paper, and only in the last couple of millennia. Right now, we need to get machines to the level of a house cat. Never mind symbolic mathematics and formal logic.

I don’t argue that people can’t use formalisms that involve derivatives just because most people can’t explicitly explain what a derivative is. I think you are confusing formal, conscious use of a certain kind of machinery with what brain does unconsciously.

If you see the reasoning engine as *separate* (and qualitatively different) from the deep learning system that provides it with inputs, then we disagree. Unless this reasoning "system" is a pen and paper to do math/logic. And much of human intelligence functions without it.

That may distill a second point of disagreement; I see nothing wrong with (for some purposes) having separate systems for (eg) image classification vs reasoning. Certainly it is possible in principle to engineer systems that way; what’s your objection? Efficiencies of training?

I wonder if we are mixing different methodological questions here. If I need to build a robust and reliable system today, I would mix deep learning, probabilistic programming, and symbolic methods. 1/