There's also the fact that gradient decent gets really weird when it comes to RNNs
-
@fchollet i haven't had much luck with anything besides clipped RMSProp with momentum. Adam, adadelta, etc never seem to quite get it.
@kastnerkyle At least in this example, replacing rmsprop with adam (default params) resulted in x2 convg. speed : ) https://github.com/fchollet/keras/blob/master/examples/imdb_lstm.py …
-
-
@fchollet@kastnerkyle Pretty sure adam's strength is in learning embeddings - seen it work super well for that.Thanks. Twitter will use this to make your timeline better. UndoUndo
-
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.