Those three consoles, N64, Game Cube and WiiU, also share another thing in common.
They had the weakest third party support.
There's a case to be made for Wii as well in one sense, that it generally missed out on the established franchises. But it got plenty of substitutes.
That said, I don't think it's as simple as pinpointing one single reason as the primary one. And in the case of game library it's a bit of a bad circle. More console sales tends to equate to more incentive to produce games for it. But a weak library can inhibit sales.