Gary Marcus

you are kidding right?

@abhijitysharma i would urge you to try this out and to read carefully Alcorn et als new arXiv paper, to understand the strengths and weaknesses of data augmentation.

That paper doesn't study data augmentation at all AFAICT. They're using a pre-trained imagenet baseline, which explicitly avoids the kind of data augmentation necessary to recognize these synthetic 3d images in unusual poses.

I wasn’t saying it did, I was saying that one could learn something by trying to use data augmentation as a candidate solution. Simple tricks like translation won’t work, and even 3-d/6d rotation may not work and may be prohibitively cost/not viable.

OK. I don't think this paper helps show the "strengths and weaknesses of data augmentation". It's already been seen that data augmentation and convnets can give good pose invariance. You can help it along a bit using stuff like Group Equivariant Convolutional Networks

The Alcorn paper simply shows that you can't expect to use different poses at inference time than you had in your data or used in data augmentation at training time, unless you force appropriate symmetry in your architecture

Or, put differently, it highlights how fragile deep learning is when tested outside of distribution, and shows how (their word) “naive” DNN’s understanding of objects is.

Yes it does show that, for some values of "outside of distribution". That's why things like thoughtful architecture and loss function selection and data augmentation choices are important.