Neo-geo pocket and Gamegear. Also there was that south Korean console which was pretty good, but I don't remember the name

Atari Lynx! The only handheld I ever owned!

Wonderswan has, IMO, the best game library. I'd just recommend you to get the Crystal version, I have a monochrome one and the display is absolutely awful, you can play games on it, but it's almost impossible to see anything even during the day, not comparable to a GB/GBC display.

Don't forget these:

PS Vita / PSP

buy a classic pinball machine instead :)

GP32? GP2X? Caanoo? Pretty cool open-source handhelds.

am I really the first person to mention the Turbo Express? Coolest little hand held back in the day.

Well he did say non-Nintendo, so yeah.

I thought the Game Gear screen was a bit blurry. I had the Atari Lynx. It never really caught on but technically it was really good for it's time with some excellent games.