1/ Higher Top 1 accuracy in image classification actually isn't a great indicator that a new vision model is going to work out well in practice in an image to image task (like DeOldify). I've learned this the hard way after getting excited about a new shiny model many times!
-
Show this thread
-
2/ EfficientNets come to mind- I just have not been seeing the benefits relative to the classification performance. But this really applies generally, even within a set of architectures that have been trained the same way (FaceBook's wsl models, BiTM, etc).
1 reply 0 retweets 8 likesShow this thread -
3/ In those cases, you just don't really know until you actually try. You may be surprised, I can tell you that much! So don't just pick the one with the highest Top-1 accuracy in ImageNet...
5 replies 1 retweet 11 likesShow this thread -
Replying to @citnaj
This. One of my most popular tweets was an offhand remark on how to eek out an extra 0.1% on ImageNet, which I regarded as useless b/c ‘interesting problems’ are not Imagenet, but the tweet *exploded* in popularity, which I found puzzling. Then I realized: they were all Kagglers
1 reply 0 retweets 2 likes -
Replying to @drscotthawley
Oh well that makes perfect sense! Do you have a favorite architecture for your interesting work?
2 replies 0 retweets 1 like -
Replying to @citnaj
I will say: skip connections + 1cycle learning rate schedule = at least a factor of 10 for lowering loss / decreasing training time. :-)
2 replies 0 retweets 1 like
Yes agreed on those being crucial. Generally there's so few things that seem to -actually- matter in practice. Training regime on the pretrained backbone (going beyond ImageNet 1K, namely) would be another I'd say I feel confident o being important.
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.