Silent Hill (in particular 2). Psychological scariness is way scarier than the "jump out and scare me" tactics which the RE series usually relies on. The radio in Silent Hill is one of the best devices used to create the sense of F.U.D.
That said, my biggest "boo" moment would probably be in FEAR. (Spoliers ahead)
I can't remember which part it is exactly, but you approach the top of a ladder and as it turns you around to climb down it, you see Alma approaching the ladder. 2 seconds later as you get to the bottom and climb off, you see your dead friend approaching you at the bottom. Even my friend watching the game at the time jumped at that part.