There is no market for 17-30+ year olds who want a to carry around a dedicated handheld gaming device (aka: a modern "Game Boy").
I think that's all there is to it.
Everyone's just ignoring the elephant in the room.
Times have changed. The modern smartphone between texting, web browsing, social media, apps, video, music, and $1-$2 games eats up all the free time, and even the larger smartphones are far more pocket friendly than a Vita or 3DS.
There is a market for gaming handhelds but its primarily going to be 6-15 year old kids (ie: kids who are too young to have a smartphone) ... and Sony is never going to beat Nintendo on that demographic. Even for Nintendo this demographic is shrinking, as the iPad is also chewing up this market. The iPad mini is probably going to wound the handheld market even further this fall.
The "hardcore" gamers who think they decide the market and are making a big fuss about Vita on message boards are a tiny, tiny niche of the market. I think people just need to accept the truth. Times change, technology has changed.
The gaming handheld is a 1980s/1990s/early 2000s concept, but in the modern world it's becoming an outdated concept. If people want to play Call of Duty that's what their PS3/360/Wii U and a 50-inch TV is for. On the go, they don't need that, anymore than a person going to McDonalds or a burger joint is expecting to be able to order a three-course steak dinner.