Sutskever to @NewYorker: "If you train a system which predicts the next word well enough, then it ought to understand"
ML fans when I show GPT-2 has no idea what happens as events unfold over time: GPT-2 is just system for predicting next words; not fair to ask it to understand.
-
Show this thread
-
Replying to @GaryMarcus @NewYorker
It’s not all or nothing. There are just degrees of understanding. These large language models do have some surprising level of understanding: they can figure out entity types or do phrase translations. But they also lack common sense and make silly coherence mistakes.
1 reply 0 retweets 2 likes -
Which makes the concept of “understanding” not very interesting and overused. Better to refer back to tasks and benchmarks and not extrapolate from that as
@tdietterich suggested in his recent medium piece.3 replies 0 retweets 2 likes
i feel though that the deep learning community (eg sutskever) happily uses the term understanding when it suits them, back down when facts are explored.
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.