to 
Because there are/have been cultures without sexism. There have never been cultures without something like prestige, AFAIK.
-
Let me try to be clear about your argument. I presume we agree on skills and accomplishments differing across people and correlating over time. You define prestige as a mapping of both skills and non skill advantages into a single dimension and reject that aspect?
Yup. Prestige is perceived accomplishments; old latin for illusion. Heavily related to social hierarchy and class. Adding a secondary unmeasurable quality to ones accomplishments that is great than the merit of the work.