Gary Marcus

Modern deep CNNs are unable to generalize over tiny image transformations such as translation, scale, contrary to widespread belief: https://arxiv.org/abs/1805.12177  @filippie509 @GaryMarcus

No surprise there. CNNs are only slightly positionally invariant due to the small convolution. Invariance in deep learning must be obtained from the use of huge numbers of training samples. There are always gaps that adversarial techniques can easily uncover.

And what's wrong with having a large number of training samples ?

For many use cases, it is impractical to get enough training examples to make up for the fact that CNNs cannot learn anything deep about the images. Better neural network designs exist.

I am not convinced, honestly. Invariance is over-rated. And I say this as somebody who was a fan of planar homologies and weird shit like that. I consider it better to train the invariance by a well-thought out regime of training data, including probably synthetic data.

Invariance isn’t just over-rated, worse: enforcing invariance in a design can actually be harmful to general task performance. Avoid if possible. So I think I agree with you there, and raise you.