I doubt you've ever actually been ugly. When you were young, in your so-called "ugly duckling" phase, you would still have had the cuteness of a child. Sometimes it requires an outsider's perspective to identify things -- to, so to speak, see the forest for the trees.
I'm not saying you have an easy life, because I know nothing about you other than how you look. But I will say that you have more opportunities as a result of being attractive than you would if you weren't.