Still vastly underused in image to image to this day and I don't understand why: Unets and self-attention. I keep seeing papers come out that have obvious problems that could be solved using these two simple things!
-
Care to elaborate?
3/ If you start from scratch, you're really putting your network at a needless disadvantage. Stand on the shoulders of giants!
-
-
That seems like what transfer learning helps doing, but I'm guessing it's something different? My lack of knowledge of self-attention probably doesn't help to wrap my head around how they could help. One more thing to check out, thanks!
-
Yeah it's transfer learning- with the added bonus of a thoughtful design of extracting features from key slices of the vision model and processing them to utilize that transfer learning more effectively. Self-attention should be considered a key component much like fc and conv.
End of conversation
New conversation -
-
-
I'm curious to see what U-Nets with EfficientNet backbone can do. I was also reading the DeepLabv3 paper, it seems that its dilated convolutions are wonderful to preserve fine grained details. It looks like U-Nets win doing blob-like segmentation while DeepLabv3 wins on details.
-
1/ The only EfficientNets I've had some success with are B3 and B4- but still not as good as ResNet. You get all excited about using a backbone with a big increase in accuracy, but it seems in practice that these nets are highly optimized in one single direction- visual rec.
End of conversation
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.