Gary Marcus

Modern deep CNNs are unable to generalize over tiny image transformations such as translation, scale, contrary to widespread belief: https://arxiv.org/abs/1805.12177  @filippie509 @GaryMarcus

No surprise there. CNNs are only slightly positionally invariant due to the small convolution. Invariance in deep learning must be obtained from the use of huge numbers of training samples. There are always gaps that adversarial techniques can easily uncover.

And what's wrong with having a large number of training samples ?

For many use cases, it is impractical to get enough training examples to make up for the fact that CNNs cannot learn anything deep about the images. Better neural network designs exist.

I am not convinced, honestly. Invariance is over-rated. And I say this as somebody who was a fan of planar homologies and weird shit like that. I consider it better to train the invariance by a well-thought out regime of training data, including probably synthetic data.

Invariance isn’t just over-rated, worse: enforcing invariance in a design can actually be harmful to general task performance. Avoid if possible. So I think I agree with you there, and raise you.

Come on guys. Invariance is an absolute must. There can be no intelligence without it. The brain does not use lots of training samples to achieve it. The brain uses timing as a glue. Transformations of an object do not break invariance because the brain can predict future states.

But invariance should be learned, not imposed as a prior. That is the point. Different signals have different invariances. Translation is just one of many...