Hand held systems over past 25 years outside of Nintendo offerings have struggled. I remember the Game Gear, Lynx, N-Gage, etc. all being seen as failures.
The biggest thing is having 1-3 massive games that really move systems. As much as I love my Vita and its library, the majority of games appeal to the same niche group. Hand held gaming over all was pretty much dead in the mid 90s, and then Pokemon happened and suddenly people were fine dusting off old systems to play a game in late the 90s that looked like an NES title.