nicely done - great explanation! What do you think of identity initialization for RNNs?
A clear explanation of why orthogonal initializations are good for your deep nets, by @Smerity:http://smerity.com/articles/2016/orthogonal_init.html …
-
-
-
Identity is an orthogonal matrix so +1 :) Scaled identity matrix seems slightly more odd - encourages die off.
- Show replies
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.