Gary Marcus

advocates of #machinelearning, I am told that you all know that (current) #ML is limited. fair enough. but which limits are you willing to *publicly* acknowledge?https://twitter.com/NotSimplicio/status/1173373706674085888 …

What even is "ML"? I think it's trivial to point to a specific model is rattle off its limitations. Well-defined class of models: basically the same. General toolkit and way of approaching a problem: more difficult. ML is more the latter, no?

so what are the well-defined classes of models for which you personally publicly acknowledge limitations @roydanroy? (and which limitations).

For starters, linear regression cannot learn nonlinear relationships.

I'm a little uncomfortable saying what a Resnet-150 trained by SGD cannot do. That's a research question. Surely, it cannot do something that cannot be done with 150 steps of approximate real computation. Of course, that includes an infinite number of things.

If you add memory you get a Turing machine, and then the question is, how do you learn it? I'm a little uncomfortable saying what SGD can and cannot do. It's not a well defined model class anymore.

can SGD do the things that Davis and I say it can't?