I know you said don't recommend a Switch, buuuuut...I recommend a Switch 
Switch is going to be a WiiU + 3DS + Vita (minus Sony first-party games) so it will satisfy both worlds.
But I understand you want to work through the catalogs of either 3DS or PSV.
I've had great experiences on both platforms. I guess a lot depends on your taste in games. If you're big into indies, visual novels, turn-based RPGs, and Japanese games in general, I think Vita could be a good fit. If you're more into platformers, action-adventure games, strategy RPGs, puzzle games, and dungeon crawlers, then 3DS might be a good fit. For me, Nintendo's first-party content is what gives 3DS a big leg up over Vita, but if you're not crazy about Super Mario, Animal Crossing, Fire Emblem, Zelda, Kirby, Pokemon, and Mario Kart then it's a moot point.
Since I can't be sure which platform you'll get, here's a list of cool games available on both :)
Retro City Rampage
Virtue's Last Reward
Severed
Shovel Knight
