NTM, DNC, Neural ALU, and many other types of external memory networks all represent small steps to address this concern (several did so prior the Lake article). Obviously there are still limitations, but its an active part of DL research.
“when generalization requires systematic compositional skills, RNNs fail spectacularly.” https://cims.nyu.edu/~brenden/LakeBaroni2018ICML.pdf … extending results discussed in Marcus 2001 The Algebraic Mind., reported 1998 cog psych See also related results by Bengio lab: https://arxiv.org/abs/1810.08272 Cc @Zergylordhttps://twitter.com/PeterJungX/status/1066399221467791363 …
-
-
- 2 more replies
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.