Gary Marcus

I don’t argue that people can’t use formalisms that involve derivatives just because most people can’t explicitly explain what a derivative is. I think you are confusing formal, conscious use of a certain kind of machinery with what brain does unconsciously.

If you see the reasoning engine as *separate* (and qualitatively different) from the deep learning system that provides it with inputs, then we disagree. Unless this reasoning "system" is a pen and paper to do math/logic. And much of human intelligence functions without it.

That may distill a second point of disagreement; I see nothing wrong with (for some purposes) having separate systems for (eg) image classification vs reasoning. Certainly it is possible in principle to engineer systems that way; what’s your objection? Efficiencies of training?

I wonder if we are mixing different methodological questions here. If I need to build a robust and reliable system today, I would mix deep learning, probabilistic programming, and symbolic methods. 1/

But at the same time, I think it is critically important to see if we can push DL methods to provide a unified solution to perception, reasoning, and action (with robustness and safety). 2/

There are recent formulations of perception, reasoning and action as a process of inference that the ML/RL community is ignoring. I wonder why that is, especially as they provide rather simple answers to many questions you describe as open and difficult.

Can you post a pointer? I'm not sure what you are referring to. There is quite a lot of work in ML on viewing reinforcement learning as inference.

I’m referring to Prof. Karl Friston’s work on Active Inference. Among many other findings, it dispenses with reward functions and describes action as a result of inference on learned sequences of observations. It’s a rather simple principle with enormous explanatory power.