It's all software. All, all software.

The small low resolution screen, limited processing power, and small storage space of handheld gaming systems has traditional meant that games were developed on a similar scale of games developed for game consoles that were released (roughly) 10 years before the handheld system. On top of this games tend to be on the "low budget" side of things, and gain from better tools and knowledge. When all is factored in, handheld games can often be made for (roughly) 10% of what a mid-budget console game on a current generation console costs to develop which means they can be successful while being sold at a lower price even with lower sales.

Sony's strategy of producing a more expensive handheld that "matches" the home console experience results in games needing to be developed at a higher budget, sold at a higher price, and needing higher sales to be successful. Right now, developers are skimping on the games to keep the budget resonable, fewer games are being developed in general, and (combined with the high price of the system) it creates a perfect storm for the system.

SONY is perfect.

I wouldn't call all handhelds casual by default. Nintendo tried to make the 3DS seem hardcore to no avail, and Sony tried TOO hard to make the Vita hardcore. One of those systems needs to hit the middle ground to really tap into the handheld market. 

Onto another point: The general cosunsus right now is that the Vita needs games. 

Everyone thought that Uncharted would carry the system, but he just doesn't have the brand power of Mario. People thought that other brand names like Resistance, LBP, and Assassin's Creed would carry it.  Call of Duty has the brand power Sony needs, but the game itself is extremely terrible according to reviews. Other than COD, Uncharted, LBPVita, and various PS3/360 ports, the Vita doesn' t have much to offer; granted it hasn't been out that long, but the games should be here by now. 

With that said, come E3 2013, Sony should come out of the gate with a Gran Turismo game (that will actually launch sometime soon ^_^), and be prepared to show people why they bought the Vita (like Nintendo did with the 3DS in '11). At this past E3, all we really got was PASBR and Call of Duty. We already knew about the games on the show floor, but nothing new? At this point Sony HAS to have something in development for it. If E3 2013 doesn't save the Vita, then at that point nothing will. 

And this is me thinking that the vita problem was that no one was buying it. What do I know?

Even if the console had addressed all the issues you mentioned it would still be a poor seller.

And it's poor sales that is any console's biggest problem...regardless of it's capabilities.

Sony needed to released a handheld that was truly revolutionary. Too many consumers the vita is not, that's it's problem.

You know, by saying that you are saying the 3DS has a lack of high quality games? Going off of metacritic both system's games are neck and neck in quality and quantity.

You can ignore metacritic and just state that the Vita just doesn't have a true system seller.

They have custom cards in order to prevent (slow) piracy.  What I think they need, is to make the custom cards cheaper.

Really? I could've sworn that 3DS has more 90+ games. Let's also not forget that Virtue's Last Reward is on both systems. Which Metacritic seems to forget...

EDIT: Oh wait. I guess they added reviews since I last checked. But not all of the 3DS version's reviews are there, so the metascore is skewed. The 9.5 from IGN should be for both versions, according to the review. It even said that both versions were basically equal in that review. :P

The expensive proprietary memory is one of the problems the vita has (and my personal pet peeve) but sadly by far not the only one.