Is "learning from humans" the new "layer-wise pre-training"?
-
Aren’t we (humans) actually pre-trained? Our DNA was optimized by evolution and we get that pre-training for free initializing our params
In DNA terms, the "source code" for the brain is a few megabytes. That means we contain virtually no hard-coded high-level representations.
-
-
True, instead of hard-coding high-level representations, evolution "decided" it was better to pre-wire us with something more powerful 1/3
-
An optimizer allowing the model to independently learn high-level representations. Or maybe the optimizer IS the model or vice-versa 2/3
- Show replies
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.