François CholletVerified account

I have often wondered the same thing. Any thoughts on why?

Catastrophic Forgetting?

I have lots of memories before 5. But definitely not all of it. Most learning comes subtly. Memories tend to mark the learning that comes with surprise.

Calibration of experience means small outliers over long periods are also large outliers for short memories. "Experience replay" eventually downweights the early trauma in the context of new info, so that inputs are within expectation, and outputs are properly moderated.

That's the epitome of "internalization".

Because brain optimizes for energy efficiency? Once it discovers sufficient information about world for survival and pleasure, it will turn off unnecessary energy-intensive learning parts.

its not weird at all. The brain is still not developed then