Gary Marcus

(Thread) On the heels of the recent @GaryMarcus & @ylecun Twitter "debate", here's one of my gripes. I find myself constantly frustrated in any deep learning vs. symbolic debate because symbol pushers tend to claim ownership over capacities like "reasoning".

In the worst instances reasoning *capacities* and symbolic *mechanisms* are entirely conflated. In less pernicious instances reasoning capacities are presumed to require symbolic methods.

Symbolic methods do indeed have properties that would be useful for reasoning, including a natural propensity for compositionality. But these useful properties are not "owned" by symbolic methods -- they are properties that learning-based methods can have too!

And let's not forget the decades of history showing why symbolic methods are ultimately insufficient, with the symbol binding problem being near the top of the heap.

Moreover symbolic methods aren't even adequate cognitive models for human reasoning! See the long history of work by Johnson-Laird and others who demonstrate the insufficiency of a logical calculus for explaining human reasoning.

IMO, one of the most promising directions involves instilling learning-based methods (i.e., DL) with properties useful for reasoning. E.g., meta-learning for symbol binding (see our previous work on one-shot learning), or non-local inductive biases for better relational reasoning