The key advantage of deep learning is its reliance on global optimization -- it learns a hierarchy of features jointly, which solves the fundamental problem of information loss. That's also one of its main weaknesses: it makes DL extremely inefficient due a lack of modularity.
-
-
This Tweet is unavailable.
-
This Tweet is unavailable.
- Show replies
-
-
-
But many models do exploit modularity, where they train components separately, and then composing without propagating gradients everywhere. E.g. the deepmind RL agent in the grid cells paper
-
Or anything using word embeddings, or any embeddings.
End of conversation
New conversation -
-
-
I would argue that modularizing this would impose human perceived structure to the problem, thereby introducing the very bias it has successfully avoided so far.
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.