The Vita launched in a perfect dead area, one year after the 3DS, and one year before the start of Gen 8. People had either already gotten invested in the 3DS, or were waiting for a true next gen experience. Not to mention this thing suffered from an identity crisis. All of its new features were already prominent on smart phones, which everyone already had, and for a while only really had a focus of offering slimmed down versions of PS4 games.
also I feel the need to point out, a large part of why the Atari Jaguar failed was due to its marketing. It was being touted as the 'ultimate' gaming platform, being so amazing that if you didnt buy, you were basically an idiot. People werent exactly flattered being called stupid by an entertainment company