Of course different people want the world to be in their image. Some images are more equitable then others however and having a framework that doesn't overly restrict other peoples visions allows them to attempt to change the world or their personal lives in the direction they want. Only through the free exchange of ideas can we affect such change.
Not providing people with enough power to be affect such change just leads to rebellion anyway, so for stabilities sake freedom is quite important.
The other point about freedom is that a society that is sufficiently free, allows people to actually live out the style of life that they want. The more control the more hypocracy as people just live their lives in secret. Of course one can be too free and then it just defaults to the most vicious people. Read Somalia for example. It is a balancing act.
Furthermore, there are plenty of freedoms I support that I don't partake of. For example, the right to smoke tobacco, gay marriage, the right to go to the church of your choice etc. So its certainly not just about selfishness.
