We won’t find intelligence at the bottom of gradient descent.
-
To paraphrase Ulam, using a term like "gradient-free optimization" is like referring to the bulk of zoology as the study of trunk-free animals. GD is fast per step, but extremely inefficient at extracting information from examples. (Nearest-neighbor is better at that!)
What you're doing is like arguing a billion years ago that bacteria were the apex of evolution. (They search for food by concentration gradient descent - do you have a better option?) You'll doubtless be proved right in the short term, but (very) wrong in the long term.
-
-
Even GD's speed advantage is predicated on the unfair advantage of GPUs, and less than it seems given how overparameterized models have to be to overcome local optima - which nonconvex optimizers don't. (E.g., https://homes.cs.washington.edu/~pedrod/papers/iclr18.pdf …)
Kiitos. Käytämme tätä aikajanasi parantamiseen. KumoaKumoa
-
Lataaminen näyttää kestävän hetken.
Twitter saattaa olla ruuhkautunut tai ongelma on muuten hetkellinen. Yritä uudelleen tai käy Twitterin tilasivulla saadaksesi lisätietoja.