Just how many people are working in ML research? The sheer pace of new results/capabilities is mind-boggling. I’ve never seen anything like it in any field. A lot of it seems like incremental knob twiddling, but the median results of the twiddling seem non-trivial.
Conversation
Replying to
Must be far from the Pareto. Apparently any near-random tweak to frameworks and training protocols will yield some improvement. So the bottleneck is GPU time it seems.
4
17
Replying to
there are very few research areas where we’ve thrown this much money at a narrow space for this long, and we’re coming up on the ~10yrs it takes for a breakthrough
11
"Incremental knob twiddling" really sounds like "neural networks exists since 1960s, it's been only incremental knob twiddling ever since".
No.
What you see is an industrialization of AI.
3


