The race in AI should be toward models with fewer and fewer parameters for a given performance, not the opposite.
-
While there are a few scientists training impractically large models, there are infinitely more engineers using distillation, pruning, quantization, and architecture optimization to get the same result with smaller and deployable models.
It’s more than a few scientists. It’s a major trend in AI, spearheaded by large groups at several top AI labs.
-
-
It’s not a good trend. AI research becomes a monopoly game for people who can have access to 1,000s of GPUs and data storage that can hold the Common Crawl…
-
So it's sort of "monopolistic" AI.
Keskustelun loppu
Uusi keskustelu -
-
-
It's "a few" because (1) only "a few" have easy access to the required computing resources. (2) distributing large models on multiple GPUs and training them on many multi-GPU nodes requires a lot of infrastructure engineering. That's only available in a few industry labs.
-
You’re confusing the head of the power law with the whole distribution. Everyone and their cat is trying to learn larger and larger models to the extent they can. Also, by Moore’s law etc. the requirements keep getting easier to meet.
Keskustelun loppu
Uusi keskustelu -
Lataaminen näyttää kestävän hetken.
Twitter saattaa olla ruuhkautunut tai ongelma on muuten hetkellinen. Yritä uudelleen tai käy Twitterin tilasivulla saadaksesi lisätietoja.