Gary Marcus

Open letter to @ylecun: I have been explicit that I believe that symbol-manipulation is part of the solution to AGI; Hinton has ridiculed that idea. Where do you fit in? With me? W Hinton? If in between, where? The field would benefit from a clear statement of your view.https://twitter.com/tabithagold/status/1070736319901519876 …

I have expressed my position on this many, many times (including in the recent book "Architects of Intelligence"). But you still seem misunderstand it every time and to insist that we disagree, when we don't actually disagree that much. I'm tired of wasting my time.... 1/2.

... but here we go again 1. Whatever we do, DL is part of the solution 2. Hence reasoning will need to be compatible with DL 3. That means using vectors instead of symbols, and differentiable functions instead of logic. 4. This will require new architectural concepts. 2/2

...continued 5. The argument that "DL sucks" is simply wrong. DL is gradient-based optimization of multi-module system. That is not going away. 6. Supervised and reinforcement learning as they exist today are insufficient. .....

... 7. Something like self-supervised learning is necessary. Now.....

The real questions are Q1. Exactly how do we get DL systems to learn to reason? Q2. How do use self-supervised learning to get machines to learn abstract representations of the world (call them symbols if you wish, but really patterns of activity of neural nets, aka vectors)?

Now, whether we actually agree or disagree depends entirely on the details of the answers to these Qs. Hence the pointlessness of the discussion and the necessity to work on answers. I've listed these Qs as the most important ones in AI in all my talks of the last 5 years....

...But they have existed for a very long time: since the early 90s for Q1 and since the early 80s for Q2. Now that the DL machinery works, and that so many people are working on both Qs, we have a shot at making real progress.