Alot of people blame the price of the PS VITA, which I see as a big contributing factor, cause while the PS Vita was 250$, the proprietary memory cards were insanely expensive so the PS VITA was really well over 300$+ in total for a portable only device. Most people usually don't want to spend over 200$ on a portable only device, so 300$ was far too expensive.
While the Switch is 300$ as well, it's not a portable only device as it's a console as well giving it sort of a value to have 2 consoles in 1 for 300$ with 2 joycons that allows you to play multiplayer anywhere. Plus the portable only Switch is only 200$ and it's able to handle current gen games unlike the Vita which struggled to handle PS3 games.
However, I believe the biggest reasons is the games. The PS Vita tried to bring the "console gaming on the go" idea to life. But imo it felt far short of that, most multi plats on the Vita were nothing close to their console counterparts and really felt like extremely watered down versions of console games not only from a graphical standpoint, but from a game play as well. Borderlands 2 ran like garbage, COD Black ops on the Vita was complete garbage, and even Uncharted Golden Abyss wasn't really the full console uncharted game that might've got people excited as it was only a spin-off. And most developers didn't want to develop games for the vita since it seemed to expensive for a portable device.
Compare it to the Switch where they got full console experiences that didn't feel watered down like BOTW, Mario Odyssey, Smash Bros, Mario Kart, ect.... And that's all Nintendo really needs since most people buy Nintendo systems solely for Nintendo games.
I thought from a hardware perspective the Vita was an awesome system and very well designed. However I don't think Sony tried hard enough.