It's interesting to observe how the spread of ideas actually works in practice. I've seen DeOldify cited many places, yet the key insights that to me seem like low hanging fruit don't actually get traction. I suppose not having a paper might be part of it but it's still odd.
-
-
Replying to @citnaj
I am always amused to see papers in 2020 that have very basic training loops that take forever to converge. What is your exp with self attention?
4 replies 0 retweets 1 like -
Replying to @thecapeador
1/ Well I can tell you this much: I've had several instances where I've accidentally disabled self attention, and I'm alerted to it because it makes such a visual difference. Namely- large details lose consistency (like stretches of forest or skin) and small details are missed.
1 reply 1 retweet 7 likes -
Replying to @citnaj
have you tried replacing the convnets by a full self attention model, like the ones on the Axial Attention papers?
1 reply 0 retweets 0 likes
Replying to @thecapeador
I haven't yet but it sounds like quite a reasonable idea.
2:38 PM - 11 Sep 2020
0 replies
0 retweets
0 likes
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.