Gary Marcus

astonished that anyone knowledgeable could claim that neural nets are (obviously) an “abstraction of neural processing” when we don’t yet know how brains work. if you don’t know how Y works you can’t really speak with certainty about whether X is an abstraction of Y. Period.https://twitter.com/tyrell_turing/status/1200072223299657728 …

Oy. “Knowing how Y works” means having a theory of Y that explains something and hasn’t been contradicted by evidence, yet. “X is an abstraction of Y” just means a theory of Y based on a simpler theory X. How can you ever “know how Y works” without making abstractions?

symbol-manipulation has not been rejected by evidence yet, but most readers of this thread are "sure" the brain doesn't do it (despite abundant evidence from linguistics and psychology that suggests symbol-manipulation is part of what brain do)

But you probably can do symbol-manipulation with ANNs, why not. You'd just need a larger network and some fun architectures. It doesn't negate the fact that the brain is made of networks, and ANNs are mathematical abstractions that capture most interesting facts about networks.

would urge you to read my lengthy discussion of this in the Algebraic Mind, chapters 2 and 3

I'd love to, at some point!! But also, I'm 100% sure the discussion you're having with @tyrell_turing is based on a tragic misunderstanding :) You both refer to "ANNs", but a) it's almost guaranteed that GPT has nothing to do with how language is represented in the brain, and ...