looking for pretrained language models that i can readily test a draft world-state change benchmark on.
GPT is scoring essentially zero.
what’s out there that is better that I can try?
@ylecun @AntoineBordes @jaseweston still would like to test your recurrent entity networks
-
For someone not deeply involved with NLP, what is a world state change benchmark about? Or is it a more general "changing the language but keeping the underlying laws" idea, for zero or few shot learning?
when we read or hear a story or just look around, we build an internal cognitive model of what’s going on; we update that model as we learn more. eg we track who has done what to whom, when, where, & why language neural nets like GPT-2 don’t see http://rebooting.ai 4
-
-
this paper by Facebook does this in a very limited domain, using different techniques, not just stats about words in context but also memory, which is essential: https://arxiv.org/abs/1612.03969 Peter Norvig’s dissertation on story understanding tried to address this within classical AI
- 3 more replies
New conversation -
-
-
To be fair, none of the statistical language models are trained on tracking or understanding tasks. They are trained to be statistical language models. Is someone claiming otherwise?
-
1. yes Hinton has dismissed my claim "talent of feature detectors.. doesn't translate into understanding novel sentences, in which each sentence has its own unique meaning" on basis of "success" of statistical systems (eg Google Translate)
End of conversation
New conversation -
-
-
complex combinatorial responses to internal and external stimuli are observed during episodic memories, these responses form the basis of all internal representations and all potential thought. If you don't model the combinatorial responses, you aren't making AGI.
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.