Well to paraphrase Dr. Chase from "House"-That's what attraction means, that society likes looking at you. If society doesn't like looking at you, then that means you're not attractive.
So to the OP. We are the "society" man. Each and everyone of us makes up that entity you described which makes it seem like society is some dictator faraway telling us what to do. There are people who absolutely adore obese people but they're heavily outnumbered by people who don't. That majority makes up the so called "society." There's nothing wrong with finding certain types of people unattractive (well at least for you, maybe not for that person you don't find attractive but that's a lesson in life). Why do we find certain people unattractive? Well, that there is a doctoral dissertation in the works.
Oh yeah, attraction goes beyond than just looks but I guess that part isn't relevant for this topic.
