"My favorite Gary Marcus quote", via Geoffrey Hinton:) http://www.cs.toronto.edu/~hinton/marcusquote.html …
-
I am not nearly knowledgeable enough to comment on this. But I remember the argument being that while machine translation & language seem to be tasks better suited for symbolic AI, feature representations from DL without an explicit understanding of tasks still do well on them
machine translation and even Superglue are poor proxies for genuine comprehension the fact that large datasets have correlations with things doesn’t mean models like BERT can comprehend a narrative, medical report, or news article
-
-
but thanks to
@sshkhr16 for clarifying what hinton’s argument might be, even though i am not convinced by it. - 3 more replies
New conversation -
-
-
And when you train BERT/GPT kind of large language models on billions of sentences with equal capacity models, they rarely see sentences/phrases which are out of training data, leading to supposedly better generalization (memorization) :D
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
-
-
Can you explain what you mean by a model to comprehend?
-
Douglas Hofstadter wrote an essay about Google Translate that explains ithttps://www.theatlantic.com/technology/archive/2018/01/the-shallowness-of-google-translate/551570/ …
End of conversation
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.