right: the modem machine learning take on innateness is to pretrain a model on a massive corpus for a specific language. but is that the right *kind* of innateness? certainly it’s not what humans are born with.https://twitter.com/martingoodson/status/1193188416189583360 …
-
When you design the architecture of an ANN you are instilling in it innate inductive biases that make it good at learning particular types of tasks. This may not be the form of innateness you would like to see, fine, but innate capabilities are not wholly ignored in DL.
