I could be wrong, but I have always suspected that some people became priests (or nuns) because they were internally conflicted about something; and I suspect that until recently a lot of men might become priests because they were conflicted by their own sexuality. I know it is entirely possible for someone to be celibate for long periods of time (potentially their entire life) I think it would probably be much more difficult if you didn’t acknowledge the desires within yourself that you were trying to deny.
Basically, most men are willing to admit that they’re attracted to young girls at an age that would get them arrested if they followed through with their most basic impulses; and since they know this they made decisions in their life that has (mostly) put them in a position where temptation is unlikely to arise. On the other hand, if you had someone who was more attracted to young girls than most men who didn’t admit to this they might (foolishly) choose to be a junior high teacher and become the coach of the girls volleyball/basketball team; if he did this, you’re basically putting a candle on an unstable raft that is floating on a lake of gasoline and expecting nothing bad to happen.