I think it's a combination of things.

Firstly, they haven't had mass appeal as a console maker since the 5th generation. Nintendo did best when they had no competition (3rd generation) or only 1 real competitor (4th generation). Once Sony came on, Nintendo couldn't hold onto their audience. They didn't make the types of games everyone wanted or didn't know they wanted. Nintendo can do platforms games very well, and they can do some adventure games well like LOZ but other companies were able to come in and offer greater variety of games. I didn't even know I would like a Tony Hawk game until I played it, but if I only bought Nintendo consoles I'd be limited to what I could play.   The Wii's appeal and sales were due to the casual crowd, I think a one time shot.  comparing the Nintendo consoles though, it's been declining and they showed with the Wii U that the market that helped them out last generation has abandonded them.  

Secondly it's 3rd party support. Nintendo pissed a lot of 3rd parties all the way back to the NES, if you don't know there are a lot of good youtube videos and articles about how controlling Nintendo was and how they controlled the cartridge cost and manufacturing so that all risk was on the developers and they were limited to how many games they could make per year. This has carried over from generation to generation as they make the consoles they want and then expect developers to work around them, whereas Sony and MS have worked hard to foster a better enviroment where they can put out their games easier. I think it's telling that looking at 2014 for the Wii U that there are a very small amount of 3rd party support announced so far even 1 month into the year, much less exclusives from developers other than Sega.

Lastly it's their failure to really innovate their games and their milking of their franchises. There have been 6 Mario adventure titles in the last 4 years, I'm not counting other games with Mario in them, just adventure titles. They haven't made a new franchise in over a decade outside of Wii Fit/Sports which is fine for casuals but has little substance to most gamers, they are limited and not deep games, just motion controlled empty calories and not something you'd compare to great games despite their high sales due to casuals. Clearly the new Wii Fit U is not on track to do even a quarter of the sales of Wii Fit because it's come and gone and won't be something with mass appeal any longer.

Nintendo has to take risks with their games. People knock shooters but that's the thing right now, just as arcade games were, or platform games were in the past. They sell well and that's part of the business. While the games Nintendo makes still turn a profit, they have to expand on new franchises in order to get a larger market back. If you didn't buy either of the two mario games on Wii U yet than will a third one get you to? Is that what was holding you back? For the people not interested in Mario or LOZ or Metroid of Dk, they have to bring new unique experience to their platforms each generation, at least two to prevent the sequel stagnation that has been going on lately, another collect a thon Mario game or 8 dungeons of LOZ isn't going to bring in new buyers, it's only going to appease the diehard fans of those franchises who are gradually decreasing and not being replaced or increasing the sales of Nintendo biggest series.

Yeah, an FPS made by Nintendo is really going to lure in all those gamers who buy yearly rehashes of CoD and what not.

You also conveniently ignored the succes Nintendo had last gen. If you look at the top 20 most sold games, most of them are from Nintendo. And there are a lot of recent games in that list.

FPS is the most overrated franchise in the gaming world(in my opinion).

Nintendo's achilles heel is marketing. Most people don't even know the Wii U is Nintendo's latest console. They think the gamepad is an add-on for the Wii. Their games are not the problem. 

Your wish is already true: Disaster Day of Crisis

that thinking is one of the reasons why they are in their current position. Oh yes, I can't wait to play their next known franchises. There is no proff that it won't sell because they haven't tried. If they continue this way, their home consoles will become niche. don't see why the looks of their games can't be a mix of artisitic and realistic. since when is choice a bad thing. every game must not sell 5 million plus for Nintendo.

Sums it up.

where are realistic looking games like last of us, gears, uncharted, asscreed etc with so called 'mature' story lines developed by Nintendo. Zelda is how old now? its not creating any surprises and it doesn't really fit that description.

the company treats it like an adopted IP that's why.

they need to take notes from Rockstar. although gta is their biggest francise, it doesn't stop them from focusing on all their titles almost equally.

I'm not quite sure what you mean by "adopted" IP. Nintendo doesn't act like Metroid doesn't exist. And budget-wise, the Prime games and Other M are some of the biggest projects Nintendo has ever undertaken. Although, when compared to the golden child Zelda, I guess I see what you mean.

You talking about Wii sports/fit/party? ok

what's their sequels doing for the Wii U. 

I'll check that out now. I've heard nothing about it yet.