Well, many researchers understand this and are actively trying to build systems which has these inductive biases.https://twitter.com/GaryMarcus/status/1200646264150323200 …
-
Honestly, I'd be very surprised if any of the ten + 1 primitives you mention make controversial inductive biases (state of literature, 2019). Now are these primitives learnable from data? Perhaps some, but possibly at the cost of abysmal sample-complexity and insane compute.
i have never seen any of them learned robustly.
ironically the great contribution of @ylecun (arch-anti-nativist) was to figure out how to innately represent translation invariance in a neural network.
way too inefficient (as you say) to learn it from a fully-connected system
-
-
"way too inefficient (as you say) to learn it from a fully-connected system" It's like training a fully-connected network on ImageNet, hoping that convolutional weight-sharing will somehow emerge in a reasonable amount of time; I'd be (pleasantly) surprised if
@ylecun disagrees. -
Nevertheless, we have somehow evolved from single-celled organisms (without much capacity to represent objects), and we're here after billions of years of "compute". The question now is whether we need that kind of compute to have machines as intelligent as us.
- 1 more reply
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.