Jason Antic

1/ Clickbait. The "wall" is: "Not every area has reached the limit of scaling, but in most places, we're getting to a point where we really need to think in terms of optimization, in terms of cost benefit, and we really need to look at how we get most out of the compute we have."https://twitter.com/GaryMarcus/status/1202366200811884545 …

2/ I'm a big believer in optimization and intelligent use of limited resources, so I'm in agreement! I have to embrace this really, given my resource limited position. But "Hit the Wall" in context means a lot less than I think this headline conveys.

3/ As far as this apparent broader lack of civility between deep learning advocates and their discontents: I see a whole lot of arguing over nothing. I think what Jerome Pesenti says in the article is a pretty normal sentiment among most researchers and practitioners:

4/ "Deep learning and current AI, if you are really honest, has a lot of limitations. We are very very far from human intelligence, and there are some criticisms that are valid: It can propagate human biases, it’s not easy to explain, it doesn't have common sense, it’s more on

5/ the level of pattern matching than robust semantic understanding. But we’re making progress in addressing some of these, and the field is still progressing pretty fast. You can apply deep learning to mathematics, to understanding proteins,

6/ there are so many things you can do with it." Yes. Exactly. It's a tool. The vast majority of us using it aren't parading it around calling it one step from AGI. We know better esp after dealing with it the realities of daily trial/error/nudging of hyperparameters, etc.

7/ Those who are overselling deep learning seem to be those who write books/articles that need views, or do PR, or are trying to sell snake oil. Yes that's bad. But creating drama out of thin air is just a waste of time and emotion (lots of Twitter fights over this lately!).