completely agree: remarkable ≠ semantically coherent. system has no idea that a unicorn has one horn, blithely asserts it has four, etc. simultaneously close and not close to Searle’s room, not at all close to genuine understanding.https://twitter.com/Grady_Booch/status/1096841495519232000 …
-
They report the n-gram overlap with the training set, so it's at least not overfitting in the trivial sense. To be clear, I'm happy with just syntactic coherence, and agree it won't learn "real" semantics. But I view that as a problem with next step prediction rather than DL.
but you can get syntactic coherence with n-grams, if you are indifferent to semantics.
-
-
As a non-linguist, my jargon was probably wrong. I meant that these examples show tracking of novel entities over long time horizons, but are bad at inferring their real-world properties. N-grams can't even do the former, and many argued that neither could neural networks.
-
large enough ngrams could do same, and GPT-2 still wanders off. not saying it isn’t better than ngrams, but i think the spirit and profile of what can and can’t be done is similar.
- 3 more replies
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.