- make it possible - make it work - make it efficient - make it dependable and invisible - move on to the next layer and never think about it again
Show this thread
-
Do you really think that in DL we are at the make it efficient stage? It seems to me that we are still exploring what neural architectures make the most sense for certain domains (and that’s not exactly an issue of efficiency).
We're still exploring the space of problems we can solve, but I think that was mostly 2014-2016 (while 2011-2013 was "wow, it works"). DL is now a well-established solution for a certain scope of problems, but still a very inefficient one (data-inefficient, power-inefficient...)
-
-
Constantly retraining from scratch very similar models, taking days at a time (sometimes weeks or months) seems particularly wasteful. At a lower-level, most models seem to use way more ops/params than they need, and backprop + SGD itself is fundamentally inefficient
-
Nods. And that I think is consistent with my view that we haven’t necessarily figured out the optimal architecture...we grok their general shape, but they are not necessarily efficient,
- Show replies
New conversation -
-
-
That makes sense.
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.