Still vastly underused in image to image to this day and I don't understand why: Unets and self-attention. I keep seeing papers come out that have obvious problems that could be solved using these two simple things!
-
That seems like what transfer learning helps doing, but I'm guessing it's something different? My lack of knowledge of self-attention probably doesn't help to wrap my head around how they could help. One more thing to check out, thanks!
Yeah it's transfer learning- with the added bonus of a thoughtful design of extracting features from key slices of the vision model and processing them to utilize that transfer learning more effectively. Self-attention should be considered a key component much like fc and conv.
