I didn't really mind the ladder and plank puzzles. They were a bit of a break from the zombie and human slaying. There's definitely a need for pacing in these sorts of games, but I agree with you that some of the slower segments really do drag on a bit.
In addition to that, the AI really isn't as spectacular as it looked pre-release and the whole "Ellie running around in circles in front of a guard without his noticing at all" thing is ridiculous and immersion-breaking.
Other than those gripes, it's an excellent game. It's 7th in the gen for me, and my favourite PS3 exclusive. As you say, the real strength is in the characters and story, but the restricted supplies lend a great deal of urgency to the game, which you tend not to feel in action games that aren't Demon's Souls.
Perhaps, like Demon's Souls, it could have done more to discourage death (to echo Yahtzee's sentiment), but with the amount I died in that game I would probably have got frustrated rather quickly.
(Former) Lead Moderator and (Eternal) VGC Detective 
