That's what we get for doing non-convex stuff and focusing on perf: init RNN with identity matrix ~ new Hinton paperhttps://twitter.com/graphific/status/624184955107700736 …
-
@syhw@adnothing also tried it on more different kinds of tasks / datasets.
@ogrisel @syhw @adnothing In my experience it's not clear if identity init is any better than orthogonal init in RNNs.
-
-
@fchollet@syhw@adnothing have studied the impact of the recurrent non-linearity (ReLU vs tanh)?Thanks. Twitter will use this to make your timeline better. UndoUndo
-
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.