It's interesting to observe how the spread of ideas actually works in practice. I've seen DeOldify cited many places, yet the key insights that to me seem like low hanging fruit don't actually get traction. I suppose not having a paper might be part of it but it's still odd.
-
I am always amused to see papers in 2020 that have very basic training loops that take forever to converge. What is your exp with self attention?
4/ I can't get into too many details here but the main challenge in getting more out of self-attention is keeping things stable. But it's worth it.
-
-
Thanks Jason, have you tried axial attention?
-
I hadn't even heard of it before but looking at the abstract it already sounds interesting. Thanks for this!
End of conversation
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.