I think the main reason for this stigma in the past, is that the Nintendo 64, Nintendo GameCube, and Wii were designed in a very specific, idiosyncratic way. These three consoles were all designed around one thing, doing that one thing, and being that one thing for as long as possible.
Nintendo 64 - Designed to be a cartridge based system with fast load times based around 3D gaming and 4-player multiplayer
Nintendo GameCube - Designed to be a powerful, compact system with a unique design that was meant to just play games the best it could
Wii - Designed to be a newbie friendly standard definition system, built around motion controls that could entertain everybody
While these consoles could be beasts in these purposes, they struggled with being general, all purpose gaming platforms for every game type, which is something PlayStation, Xbox, and even Nintendo's handhelds all excelled at, and its a big reason why Nintendo home consoles lacked substantial third party support for so long. For all the Wii U's faults, making an effort to be an all-in-one gaming system that supported multiple gameplay styles the best it could was one of the few things it got right. If you ignore the screen, the Wii U GamePad was for all intents and purposes, a normal controller. On top of that, you could also use the Pro Controller, Wii Remote Plus and Nunchuck, Balance Board. The Wii U was many things, but lacking in controller options isn't one of them.