Undoubtedly Nintendo does both, and they are at their best when they do both. Wii Sports and Brain Age were nuclear missiles in the video gaming universe. One game set off the fastest-selling console in history; the other began an entire genre and market for casual gaming. But you're right, if Nintendo had taken Brain Age to the next step with microtransactions for various elements, like rushing and retries, etc... they may have been more successful financially. The point is, Nintendo is an innovator.
It's also NOT "Oh, Mario Galaxy is just another Mario game" because it wasn't. The game was completely different from Mario Sunshine; Sunshine wasn't a vast adventure through a variety of locations like Super Mario Bros 3, World (to a lesser extent), and 64; everything was themed similarly and had similar design strategies. Nintendo spent years working on Super Mario 64 II, and that metamorphosized into Super Mario Galaxy 1 and 2. Gravity effects, shooting around between planets... Super Mario Galaxy 1 was the best overall concept, Super Mario Galaxy 2 didn't have the Rosalina framing (they went with their #2 framework), but they developed their more advanced level designs to appropriate completion, and IMO Galaxy 1 may have been the better package, but Galaxy 2 has the best level design in platforming history. These levels are the collection of creative ideas; much of this stuff had never been done before.
That's Nintendo at its best, the creative market leader.
When Nintendo is at its worst... It's when they are making bastardizations of what others are leading on: Gamecube is Nintendo at its worst. It's a PS2, basically, but they made some changes to make it more kid-friendly: the big green button on the Fisher-Price controller, the tiny/barely usable d-pad... There's a reason why 2D never really took off on the Gamecube, plug that controller into a Wii and try to play 2D Virtual Console games with it that have any kind of transitional button stuff... very awkward. The cube-shaped box itself had a handle to appeal to children, make it look like a lunchbox. Nintendo wasn't thinking about "What would Nintendo do?" they were thinking about "What would Sony do to appeal to kids?" And while some people liked Celda, Squirt Gun Mario, and 15-second lap piggy-back Mario Kart, to others it looked like Nintendo was taking a generation off... and that's why not many bought the Gamecube.
I think when Nintendo is adventurous, people like it. I also believe that when Nintendo is showing people a more accessible way to play complex games, they like it. This is where the Wii U failed; it looked like they were trying to make a complex way to play simple games. Gamecube, Wii U, these were not appealing consoles to most. And despite the Wii U having a controller not seen before, it also wasn't used in any way that made the console appealing: asymmetric gameplay? That was the primary marketing thrust of the console, and the only game they showed that did anything with asymmetric gameplay was Nintendo Land, and then nothing really after that. While Nintendo's biggest fuck-up was the N64 (high price, 3rd party alienation, 2nd party evaporation from frugalness and bad relations), at least it was still a Nintendo-feeling console with Nintendoish software. N64 hit 30+ million despite its massive flaws, it was worse than Gamecube and Wii U in terms of the cartridge format and pricing, but better because they didn't sell their soul as they did with Gamecube to try to be the Playstation for children, and they didn't become complacent and arrogant like they did with Wii U.
Anyway, I'm ranting. Bottom line: Nintendo are innovators and creative inventors: consoles are the canvas for that. Some are not very inspirational, not interesting, or not usable (Gamecube and Wii U in particular, Virtual Boy as well) and the audience sees that: they know these are the consoles to skip. Some of their consoles (NES, GB, SNES, DS, N64, Wii, Switch) are canvasses that inspire their devs to make great works of art, things that interest gamers as much as the artists, great Nintendo things.