To be learnable, a problem has to be stable. Therefore in a stacked ML architecture, the lower levels should evolve at slower scale.
-
@fchollet neat. Citation? Experiences?
@stephenroller Personal exp. Try it: take a deep MLP, and test 1) constant learning rates, 2) higher LR for higher levels, 3) the reverse
-
-
@fchollet do you write your own or use a package for quickly creating NNs? -
@stephenroller both. I use Torch7 for CNNs, Theano for low-dim problems, and currently numpy+pypy to write my own stuff (large sparse space) - Show replies
New conversation -
-
-
@fchollet@stephenroller For different lr per layer: RMSProp , AdaGrad, and AdaDelta do this to get better scores (sometimes) :)Thanks. Twitter will use this to make your timeline better. UndoUndo
-
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.