1/ Higher Top 1 accuracy in image classification actually isn't a great indicator that a new vision model is going to work out well in practice in an image to image task (like DeOldify). I've learned this the hard way after getting excited about a new shiny model many times!
-
-
This is really interesting question about metrics that capture good downstream performance. I guess one good proxy is to look at tasks that use image model backbones (eg object detection), but would be interested to hear more ideas!
-
(Same as your experience: with exception of EfficientDet, most top object detection models on test-dev seem to stick to using resnet/resnext rather than other architectures)
- Show replies
New conversation -
-
-
Thanks! I'd be quite curious to see how things evolve for you. Metrics are really important, but also quite tricky to determine as tasks diversify
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.