First, what is by now a chestnut "deep learning is great engineering but no science." This is (a) wrong, and (b) contains a fundamental misunderstanding, common in computer science, about what engineering is.
-
-
এই থ্রেডটি দেখান
-
(a) Sure, no theoretical consensus on why deep learning methods work. But the important part was: Before they started working, it seemed clear that they shouldn't. Far too many parameters, not enough control of overfitting. Poor Mr L2 regularization can do only so much.
এই থ্রেডটি দেখান -
We are still grappling with how to revise our conceptual understanding of ML.
When important experimental results are not explained by theory, that is a major conceptual advance. A mystery rather than a solution, but finding new mysteries is important.এই থ্রেডটি দেখান -
(b) "It's just engineering". Too many computer science use this phrase to refer to software development tasks that, even when difficult, apply only standard ideas. Thus if you a researcher, engineering is uninteresting, something to minimize
এই থ্রেডটি দেখান -
The CS "just engineering" viewpoint is a disservice to the term "engineering". In fact, engineering is a subject of research. Turns out, there are entire university buildings full of people called "Professors of Engineering", and they claim to do research. Who knew?
এই থ্রেডটি দেখান -
Engineering is the application of science and math to problems in technology. Some engineering is simple applications following existing paths, that's not research. Other engineering is conceptually new --> that is research.
এই থ্রেডটি দেখান -
Some of computer science research is pure mathematics. That's cool. The majority of CS research, though, is engineering: developing new ways to apply theoretical and scientific methods to solve technology challenges.
এই থ্রেডটি দেখান -
I forgot about (a) "no new ideas". Lots of the "deep learning tricks" are pretty interesting conceptually: e.g., attention, fancier versions of dropout, learning rate adaptation, deep generative models.
এই থ্রেডটি দেখান -
I'd better go, because my wife is about to kill me. Tell you what, if this thread gets more than 100 likes, I'll respond to points about research at Google.
এই থ্রেডটি দেখান
কথা-বার্তা শেষ
নতুন কথা-বার্তা -
-
-
Hey, Charles? You're over 120 likes already. So you're on the hook, dude.
-
Yeah me and my big mouth. What did I get myself in to?
কথা-বার্তা শেষ
নতুন কথা-বার্তা -
-
-
Thanks. I think it is most controversial how he disparages all research of industry AI labs, pretty much ignoring actual advances of DeepMind and the like. Some of these labs *are* like Bell labs for the AI field. Is this just the old arrogance of university academia again?
-
This critique of industry labs per se raised some objections in myself as well. However, the problem of graduate student descent is real and especially Google Brain/Deepmind published a good bunch of papers falling into the category of https://arxiv.org/abs/1807.03341
কথা-বার্তা শেষ
নতুন কথা-বার্তা -
-
-
I’m comfortable accepting that ML is engineering more than it is science, but it nevertheless can and does provide new tools for scientific discovery. Many people say “ML is an empirical science” which is a statement I am much less comfortable with.
-
Why would one perform experiments in a non empirical science? What does empiric mean to you then, theory-less?
কথা-বার্তা শেষ
নতুন কথা-বার্তা -
-
-
Also, I used retweet rather than like, but I still want to be counted
-
Because of your retweet I liked it. So it counted
কথা-বার্তা শেষ
নতুন কথা-বার্তা -
-
-
ML is statistics, so science, but you may mean ït is not mimicking intelligence. It is, for instance, not linguistics, but can be used with linguistics.
-
You don't consider "tf * idf", where software works with training and test set to classify, science? Is Claude Shannons information theory science? Sparck Jones' "tf * idf"is equivalent to it. She did part of the work of behemoth Bell Labs with her own team.
কথা-বার্তা শেষ
নতুন কথা-বার্তা -
-
-
There is no problem in engineering and it is very useful but we may not know how humans acquire language by doing more optimization
-
How language acquisition happens is more important and principled question than How humans acquire it
-
Yes but perhaps both cannot be separated perhaps as language is the only thing humans have over other mammals.
কথা-বার্তা শেষ
নতুন কথা-বার্তা -
লোড হতে বেশ কিছুক্ষণ সময় নিচ্ছে।
টুইটার তার ক্ষমতার বাইরে চলে গেছে বা কোনো সাময়িক সমস্যার সম্মুখীন হয়েছে আবার চেষ্টা করুন বা আরও তথ্যের জন্য টুইটারের স্থিতি দেখুন।
s:


(post 1 of "until my beer gets cold")