For the record, RNNs are Turing-complete and can implement any function: https://stats.stackexchange.com/questions/220907/meaning-and-proof-of-rnn-can-approximate-any-algorithm …https://twitter.com/fchollet/status/1010278944782708737 …
-
I think it's helpful to emphasize that NN models learn a small subset of all programs because of optimization (not representation) issues.
It's also profoundly misleading to draw an artificial distinction between "representation" & "optimization". The choice of an appropriate search space & encoding/structure for that space is the most important part of any search process. DL does not have an optimization problem.
-
-
I think there are opportunities/successes in finding both better hypothesis classes (eg. Resnets) and optimization/search algos (batchnorm).
-
Both are important and intimately tied to each other as you say.
End of conversation
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.