Definitely an element of this. Takes more work every passing year to get same level of fog reduction so I guess you start losing the battle at some point.
On the plus side, for some people, age makes it easier to empty the mind.
From the first lines of the Yoga Sutra (in some translations):
"Yoga is the cessation of compulsive functions of mind."