The Vita looks like a fine system, but there is something that the 3DS can have that it can't. Huge franchises.
The PSP's big franchises were console spinoffs, Square Enix games, and Monster Hunter. The spinoffs should continue to do fine, but Square Enix invests in every system on the market except the Wii, suggesting that their games will go on whichever console is more successful. Come to think of it, Square Enix sold more DS games than PSP games. And Monster Hunter is multiplatform nowadays, with 3G already out for the 3DS and 4 only being announced for the 3DS so far.
The DS's big franchises were Mario Kart, Nintendogs, Mario platformers, Brain Age, Pokemon, and Animal Crossing. Nintendogs looks to be smaller this gen, and the Brain Age crowd is on the iPhone nowadays, but Pokemon, Mario, and Animal Crossing are all on the 3DS to stay.
The Vita's best chance is being a console/handheld hybrid, which could essentially replace consoles in Japan if all goes well. Uncharted is a great start, but it also needs to have the same Call of Duty games as its brethren, and a lot of 3rd party multiplatform support.
In other words, the Vita's success depends on 3rd parties, who have a poor record from the PSP era, in comparison to the 3DS, which only needs Nintendo's support.