Not a big fan of the handheld library overall. But there are a few really good games to mention. Both God of War games, the Patapon games, FFVII Crisis core, and Metal gear solid peace walker.

PSP

• WipEout Pure
• WipEout Pulse
• Daxter
• PixelJunk Monsters Deluxe
• MotorStorm: Arctic Edge
• Hot Shots Tennis: Get a Grip
• LocoRoco 1 & 2

PSV

• MotorStorm RC
• Wipeout 2048

I wish I could say the Patapon series because I loved the art style, concept, characters, and music, but I found it too difficult to get very far. 

I was contemplating not doing a poll because this thread didn't get as much traction as it's console versions but I think we have enough games to create one.  

I feel like turn based rpgs do really well on handhelds.
Same with more "casual" friendly games, like something akin to Loco Roco / Patapon.

Oh come on, how can Killzone Mercenerys not be on that list. Damn, I'd nearly buy a VITA just to play that masterful campaign again.

If you say Playstation handhelds, say no more than Nihon Falcom.

Multiple The Legend of Heroes series.
The Ys series.
Nayuta no Kiseki
Gurumin
Brandish
Tokyo Xanadu

And every single game in this list is great or at least very good.

It is an utter disgrace that Disgaea made it into the poll but no Falcom game. 😛

FF4 The Complete Collection or Tekken Dark Resurrection.

Agreed.

What Falcom game would you guys recommend me to play first? I own a PSP and a PC, and have access to Switch and PS5.

Depends on your tastes. I'm not big into them but the Trails series have long drawn out storylines that span multiple titles. I'm an Ys fan tho and while Ys 1 is my fave game ever. Also short. Oath might be the best Ys game ever made.  Ys 1-II are great but still use the old bump system combat (literally bump into enemies) jarring at first but get used to it fast. Oath is sword swinging. They are fast-paced ARPGs.