2/4: Do lottery tickets contain generic inductive biases or are they overfit to the particular dataset and optimizer used to find them? Encouragingly, we found that lottery tickets generalize across related, but distinct datasets and across optimizers: https://arxiv.org/abs/1906.02773?fbclid=IwAR0exCzOb_rchjZBJVCWgSaTuJaXRrsE9AQcK4RpfXdEKEpGuUmxw7jcn7w …
-
-
Show this thread
-
3/4: Is the lottery ticket phenomenon a general property of DNNs or merely an artifact of supervised image classification? We show that the lottery ticket phenomenon is a general property which is present in both
#reinforcementlearning and#NLP: https://arxiv.org/abs/1906.02768?fbclid=IwAR2prqiChhwXj8_CqxyMcV1ePqVzFvojYgsuieRmP94xE8Ewa5kAkI_T1_U …Show this thread -
4/4: Can we begin to explain lottery tickets theoretically? We introduce a new theoretical framework on the formation of lottery tickets to help researchers advance toward a better understanding of lucky initializations: https://arxiv.org/pdf/1905.13405.pdf?fbclid=IwAR0MKAue_0k1d-DugT8UBAvRM_fjKBodtQsSqNel1gGF8EVmV14aChCCBMA …
Show this thread
End of conversation
New conversation -
-
-
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
-
-
Thanks Ari Morcos & Yuandong Tian
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.