completely agree: remarkable ≠ semantically coherent. system has no idea that a unicorn has one horn, blithely asserts it has four, etc. simultaneously close and not close to Searle’s room, not at all close to genuine understanding.https://twitter.com/Grady_Booch/status/1096841495519232000 …
-
N-gram models of the size needed to capture this range of structure would overfit to an absurd degree. At what point does "local conference" make the jump to global?
What makes you think there isn’t overfitting to a high degree? The pastiche-y nature of the output seems like that to me. Some of the coherence violations are actually local (unicorns can’t have four horns), and I don’t see this approach as on a track to genuine coherence.
-
-
They report the n-gram overlap with the training set, so it's at least not overfitting in the trivial sense. To be clear, I'm happy with just syntactic coherence, and agree it won't learn "real" semantics. But I view that as a problem with next step prediction rather than DL.
-
but you can get syntactic coherence with n-grams, if you are indifferent to semantics.
- 5 more replies
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.