Gary Marcus

There are many examples of this in the DL community, including work by me and colleagues on e.g. program induction https://arxiv.org/abs/1506.02516  and synthesis https://www.jair.org/index.php/jair/article/view/11172 … (aka neurosymbolic ML). Many great papers by other DL/ML peeps on these topics.

Don't get me wrong: it's great that people are pointing these things out and testing the limits of DL. But it's good to form a balanced view. Things aren't as black and white as "omg neural nets can do everything" vs "omg here's one failure mode let's ditch it all"

Agree w @egrefen here, too, but find that no matter how hard I advocate for hybrid models people always think I am arguing against all of ML... I think ML will play a huge role in AGI, but only w proper biases, including representations of operations over variables.

Properly understood, #MachineLearning is the study of the relationship of biases to what can be learned with each (and how to implement them computationally). It's bias, all the way down.

and yet innateness is a dirty word in ML. amazing.

Gary, are you really arguing that anything you can't get your ML algorithm to do, you can just build in and declare it to be "innate". It is totally ad hoc science to rely on innateness to solve your machine learning problems.

Where are the principles that tell us what should be innate vs learned?