From a sheer sales point of view, the Wii U. Going from a console that sold over 100 million to one that barely sold over 10 million is definitely a bite in the butt. It had some good games, but, for the most part, it just wasn't good enough.
From a marketshare/corporate point of view, the PS3. Sony reigned supreme with around 80% of the marketshare with the PS2. The Xbox and Gamecube sold 40M combined, but the PS2 sold almost 160M! That is incredibly impressive. However, the PS3 ended up with less than 33% of the marketshare and cost Sony so much money they were still in the red until around 2016 (even though the PS4 basically got a running head start).
From a games point of view I would say the Vita. The PSP had outstanding AAA-esque third party games like GTA: Stories, Crisis Core, God of War: CoO and GoS, MGS: Portable Ops and Peace Walker, Kingdom Hearts, etc. Then had popular franchises like Resistance, Killzone, Ratchet and Clank, Jak and Daxter, etc on top of it. It truly felt like a portable PS2. Especially with downloadable PS1 games. The PS Vita however? It started off strong with Uncharted, Resistance, Killzone...but stopped suddenly. There were only 12 total first party games on the system. While it has been chugging along with indies and Japanese games, it never felt like a portable PS3 in my pocket.