Just how many people are working in ML research? The sheer pace of new results/capabilities is mind-boggling. I’ve never seen anything like it in any field. A lot of it seems like incremental knob twiddling, but the median results of the twiddling seem non-trivial.
Conversation
Must be far from the Pareto. Apparently any near-random tweak to frameworks and training protocols will yield some improvement. So the bottleneck is GPU time it seems.
Replying to
1
2
Show replies
Replying to
I don’t think so. People seem desperate for GPU time/budget. I think if GPU supply webtv10x and dropped 90% in cost, a flood of new waiting researchers would jump in
2
Replying to
GPU VRAM at the moment, but even that's being scaled drastically in the past few weeks (depending on what models you're referring specifically to)
it feels fast because we're proliferating industrial cloud computing tasks down to high-desktop tier computers -> boom of users
4
Replying to
Bear in mind that the underlying ecosystem of libraries and services have improved exponentially, to the point where any software engineer can jump in, load a model, build an API around it.
Huge funnel at the top.
1




