Gary Marcus

Depends what you mean by bootstrap. Cognitive systems have multiscale learning: we start with the tools selected by evolutionary learning, and supplement them with social and interactive learning. Where to start the bootstrap? In which environment epoch?

Conversational AI systems where reasoning and 'common sense' are by-products need 1) grammar, how words are ordered and marked for recognition 2) semantics, the meaning of phrases in a language 3) discourse pragmatics #deeplearning systems so not provide this !

AI has always been in the habit of searching for its car keys under the streetlamp. We need to embrace what Dennett called "the whole iguana".

Agreed. I guess one problem is it is so dang hard to make measurable progress on the bootstrap problem. It's hard to write a proposal convincing anyone your ideas will work.

Since I mostly flag up our disagreements here, I should say that I totally agree with you on this point

ML question 2: Can you physically build a dataset an arbitrary NN architecture can't fit?

Reminds me a bit of Moore's law where we went from thousands to billions of transistors but we dont generally do things 10^6 times faster or better, and are running into physical limits. When will we run into training limits for deep nets? 100bn, 1tn params?

Word.