My summary of interesting optimization papers from ICLR 2019: part I https://medium.com/@yaroslavvb/iclr-optimization-papers-i-fluctuation-dissipation-relations-for-sgd-a638ad9964cc …, part II https://medium.com/@yaroslavvb/iclr-optimization-papers-ii-44f03b98dc5f?postPublishedType=repub …, part IIIhttps://medium.com/@yaroslavvb/iclr-optimization-papers-iii-1e1edc050ba6 …
We had a paper on signSGD showing that sending just the sign of gradients is sufficient to obtain near state of art performance. Theoretical analysis showing it converges and also provide a rate https://arxiv.org/abs/1810.05291
-
-
This is pretty cool. Thanks for sharing.
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
-
-
Oh cool, I went over original signSGD in my ICML post, but these look like useful results for distributed setting
-
And thanks to Strom’s older work (2015), and with some later proofs from Stich this year, one can show that you don’t even need to send about 99% of those signs
End of conversation
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.