Also consider the symmetric version (pixel i effects pixel j as much as pixel j affects i) implemented here: https://github.com/mgrankin/SimpleSelfAttention/blob/master/Imagenette%20Simple%20Symmetric%20Self%20Attention.ipynb …
I was really curious to see what self-attention would do for image recognition. Looks promising! And this simple self-attention implementation looks great! I'm going to try it in my own work soon.https://twitter.com/jeremyphoward/status/1132380386875412480 …
-
-
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.