Gary Marcus

It’s not all or nothing. There are just degrees of understanding. These large language models do have some surprising level of understanding: they can figure out entity types or do phrase translations. But they also lack common sense and make silly coherence mistakes.

Which makes the concept of “understanding” not very interesting and overused. Better to refer back to tasks and benchmarks and not extrapolate from that as @tdietterich suggested in his recent medium piece.

If that's a direct quote it seems wrong to me, but I also think the power of prediction is only starting to reveal itself (as models get more stuff to predict about). I think it's remarkable the knowledge that prediction can impart.

This speaks to a level of general befuddlement that even leading AI/ML folks have gotten into about what current deep learning-based algorithms are actually doing. Of course, it's OK for experts to be confused, but the press keeps amplifying this as if DL is somehow intelligent.

To me the hype surrounding the current wave of #AI is in it self a phenomenon a lot more interesting than any of the actual developments in the field. Frequent antropomorphisations of these relatively primitive statistical engines and media buying it all is simply perplexing.

This is probably due to the fact that ML researchers have to ridiculously oversell their papers if they are to get published, let alone read and cited. The field is just so saturated that people have to pass off incremental improvements as if they're worthy of a Nobel prize.