A fair few different things.
It needs a pricecut. When you look at it next to the 3DS, it looks really expensive for what games are available. In December, in Japan, you could get a 3DS with Mario 3D and Monster Hunter for less than a Vita. Was a no brainer for anyone in the general public.
It needs Japanese support. The PSP and DS thrived because of great JP support, in all kinds of genre. It needs those smaller niche games to go alongside the giants that SE and Crapcom pump out. Western development has never really been big on handhelds, aside from the GTA games... Which brings me to my next point.
It needs follow ups to the games that made the PSP a success. It needs GTA, Final Fantasy, Gran Turismo. It needs God of War and Ratchet & Clank. Most of all, it needs Monster Hunter.
With all these games and a pricecut, Vita would fly off the shelves. Add in a few new colours, especially the white Vita and it would be competing with 3DS. A few smaller things would be cheaper memory cards (?) and actually having good prices for games on PS Store.