Pedro Domingos

The race in AI should be toward models with fewer and fewer parameters for a given performance, not the opposite.

Maybe it's unavoidable, see the law of robustness :-).

Except smoothness is not required for generalization.

For sure there is a huge mismatch there. But I still wonder what is the correct notion of generalization... clearly the textbook statistical learning one is not appropriate.

There’s nothing wrong with accuracy on unseen data. How it relates to accuracy on training data is another matter.

Yes that's exactly what I meant: nothing wrong with "generalization" (aka accuracy on unseen data) but the "textbook statistical learning" (aka assuming same distribution for training and test) might not be the right lens for deep learning.