Two recent algorithms, World Models by @hardmaru and Schmidhuber (2018) and Curiosity by @pathak2206 et al. (2018), have approx. equal performance when a learned module (RNN and embedding in ICM, respectively) is instead left as a fixed randomly initialized module.
@_brohrer_
-
-
This Tweet is unavailable.
-
It was also the main idea behind Rosenblatt's original Perceptrons. He is not given enough credit for that, imo.
- Show replies
-
-
-
Here they remove the last FC layer of many conv networks architectures and substitute it w/ a random projection and achieve similar results to the FC onehttps://arxiv.org/abs/1801.04540
-
If you remove a trained layer, add a random one, and then train a new layer on top, what you're looking at is just whether the random layer is destructive or not (with enough units, it's likely overcomplete, so not destructive). This is very different and not surprising
End of conversation
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.