looking for pretrained language models that i can readily test a draft world-state change benchmark on.
GPT is scoring essentially zero.
what’s out there that is better that I can try?
@ylecun @AntoineBordes @jaseweston still would like to test your recurrent entity networks
-
when we read or hear a story or just look around, we build an internal cognitive model of what’s going on; we update that model as we learn more. eg we track who has done what to whom, when, where, & why language neural nets like GPT-2 don’t see http://rebooting.ai 4
this paper by Facebook does this in a very limited domain, using different techniques, not just stats about words in context but also memory, which is essential: https://arxiv.org/abs/1612.03969 Peter Norvig’s dissertation on story understanding tried to address this within classical AI
-
-
Language doesn't exist in a vacuum. It's based on having experiences in and knowledge of the real world.
-
language is just sounds that attached onto preexisting mental representations. Granted that we learn complex sequences of sounds in their own right but without the grounding of each sound to mental representations we have a "mindless" chat bot.
- 1 more reply
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.