I recently challenged Hinton and @YLecun to stand by the idea expressed in these words from their 2015 Nature paper, by making a public bet. Neither would. I stand by my assertion that is precisely this that holds back deep learning, except brave work by people like @egrefen.https://twitter.com/MLWave/status/1065310317935304704 …
-
I mean, a lot of DL research involves trying to produce models which generalise better by encoding priors about the domain, or inductive biases, into architectures and learning methods. From ConvNets to Capsules, there are plenty of examples of this...
Convolution is a great innate prior — defined in advance to work over all instances of a variable.
