Snapshot ensembles - a new take on an old idea... I'd like to see a comparison with Polyak averaging.http://openreview.net/forum?id=BJYwwY9ll …
-
how do you do it? taking the average of all steps, only the last k steps, ...?
at inference time, use a slow exponential average of the weights taken after every previous batch. Easy to accumulate in TF/Theano
-
-
do you have a reference ?
-
https://arxiv.org/pdf/1206.5533.pdf … Acceleration of stochastic approximation by averaging. SIAM J. Control and Optimization
End of conversation
New conversation -
-
-
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
-
-
I think what you should be doing is putting some of these references and comments on the OpenReview page!
-
Good point! Just did!
End of conversation
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.