François CholletVerified account

The reasons we have so little innate visual knowledge are interesting. Basically, any visual knowledge involves many bits of information, and has to be encoded via hardwiring connections in the visual cortex (or before). This is an extremely low bandwidth process.

Because it's so slow, it's only applicable for information that is stable over hundreds of millions of years. Very little of the visual world is stable over that time frame (e.g. the visual difference between male & female faces cannot be hardcoded because it changes too quickly)

Further, there needs to be strong evolutionary pressure associated with this information over this extremely long time horizon. Very little of the visual world involves life and death questions. But snakes and spiders must have been a major threat to our evolutionary ancestors.

Also, note that this is why the take "evolutionary innate knowledge is the human equivalent of pretraining in neural networks, see, humans are not data-efficient after all" is so incredibly braindead and ignorant

Humans come into the world with a lot of priors, but they are very specifically scoped, and they're very much unlike pretraining knowledge in neural networks. Most of them are metalearning priors. Babies don't come with pretrained ImageNet weights.

Crucially, our prior knowledge was not evolved in the past 500k years. It is very ancient and shared by many of our distant cousins (pretty much 100% shared by great apes in particular). It isn't what makes us special.