Further experimentation done by the @fastdotai community on how to combine the new optimizers that have just come out. This is why I love being a part of it. Sota results on limited epoch Imagewoof training which correlates directly with Imagenet performance. Great work!https://twitter.com/federicolois/status/1166919084052754432 …
-
I've got an architecture that relies on self attention that I've been wanting to try for a while. It's such an interesting method.
Yeah it’s hugely powerful in my experience. The only thing you have to be careful about is numerical stability