Being repulsive to men is simultaneously a holistic evaluation of a bunch of evolved actors who were under strong selective pressure to be good at making overall health judgements. IOW, no medical test is as good at assessing overall health as that.
You're now repeating your assumption. Average women are attractive; high floor, low ceiling in attractiveness because all women have fairly similar capacities for fertility. All else equal being far from the female average on a trait is *unattractive*