So bottom line is - no - we are not even close to AGI right now. Even simulated conversations like GPT-2 are crappy.https://twitter.com/GaryMarcus/status/1194072212841517056 …
-
While the transformer is an important contribution to ML (and the implied idea of maintaining identity via attention imho relevant to AGI), GPT-2 is a text prediction tool, not an AGI architecture. The quality of a chatbot built on GPT-2 is unrelated to how close we are to AGI.
that’s retrenchment and revisionist history. few months ago people were perfectly happy to talk about gpt-2 as an understanding tool; it’s in the original paper, the title of the Bet paper, sutskekers quote to the New Yorker, etc. i have posted about this before.
-
-
I don't find this claim in the GPT-2 paper, and have not met any serious researcher who claimed that GPT-2 is able to understand text in any real sense. The NLU community may use the word "understanding" more broadly than it should, but is very aware of the limitations.
-
Did anyone in deep learning community call Sutskever out for equating understanding and prediction in the New Yorker? for pitching GPT-2 in orig. blog as having "rudimentary reading comprehension? Or BERT paper for having "understanding" in title? AFAIK, no.
- 7 more replies
New conversation -
-
-
BERT paper, sorry for typo
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.