While the transformer is an important contribution to ML (and the implied idea of maintaining identity via attention imho relevant to AGI), GPT-2 is a text prediction tool, not an AGI architecture. The quality of a chatbot built on GPT-2 is unrelated to how close we are to AGI.
-
-
Replying to @Plinz @rutumulkar
that’s retrenchment and revisionist history. few months ago people were perfectly happy to talk about gpt-2 as an understanding tool; it’s in the original paper, the title of the Bet paper, sutskekers quote to the New Yorker, etc. i have posted about this before.
2 replies 0 retweets 1 like -
Replying to @GaryMarcus @rutumulkar
I don't find this claim in the GPT-2 paper, and have not met any serious researcher who claimed that GPT-2 is able to understand text in any real sense. The NLU community may use the word "understanding" more broadly than it should, but is very aware of the limitations.
3 replies 1 retweet 1 like -
Replying to @Plinz @rutumulkar
Did anyone in deep learning community call Sutskever out for equating understanding and prediction in the New Yorker? for pitching GPT-2 in orig. blog as having "rudimentary reading comprehension? Or BERT paper for having "understanding" in title? AFAIK, no.
1 reply 0 retweets 1 like -
On the one hand, “understanding” and pure NLP papers shouldn’t be in same sentence/article. On the other, lack thereof in pure NLP has ~0 to do with how close we are to AGI. In right grounded lifelong causal environment, prediction may (or may not) be key driver of understanding.
1 reply 0 retweets 1 like -
That is a ridiculous statement. NLP -> Information Extraction -> Knowledge graphs -> causal reasoning. It is all connected. maybe we have different definitions of NLP?
1 reply 0 retweets 1 like -
That reasonable opinions and definitions could differ on such a subject is quite expected and ok to me.
2 replies 0 retweets 0 likes -
hmm. i think AGI without NLU is logically possible but extremely unlikely, and that real progress towards NLU would show up on the kind of dynamic understanding benchmark i am trying to develop
1 reply 0 retweets 0 likes -
Is the language “grounded” in your benchmark? Or just text->text? I don’t see getting far in a pure text universe.
1 reply 0 retweets 0 likes -
as i noted yesterday
@Wolfram_Alpha occasionally does fine in such circumstances i think a strong an innate kernel plus wikipedia could take the right algorithm a long way, much further than any current system1 reply 0 retweets 0 likes
Sounds interesting!
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.