I really think they need to try and replicate the Wii. Cheap console that offers something unique that would sell to a wider audience. It's very difficult for them to fight for the PS/XBOX market at this point. They'd have to change their image completely. They'd have to make a high spec'd console that will sell for little profit. They'll have to ensure that their console is accepted by 3rd parties in terms of online capabilities. Basically they'll have to do so much for a chance of gaining something little.

A cheap console that sells for a profit from the get go and maybe a unique gimmick that attracts a wider audience is their best bet. Worst case scenario is a console that sells badly but is still profitable.

What I meant is, who is the audience not satisfied with Sony/MS? And how should Nintendo target them? Are they a specific demographic?

The answer is simple. If Nintendo can't compete with Sony and Microsoft and they can't capture the casual audience the only answer left is going third party.

It's not the best option for them but it's the correct choice. For the people of the world to experience nintendo games, once and for all!

I said what you are all thinking, I should get a medal for this shit.

Nintendo needs to do a few things, whether they do this is a whole other thing.
1. They need to adapt more to the West. Meaning allowing NOA and NOE more control on marketing, hardware ideas and games released into the market.
2. They need a console that is on the same playing field as the next PS/XBOX in regards to specs/power/functions. It doesn't need to be exactly as powerful but enough so that it really is not that much of a difference while also being slightly cheaper in cost.
3. Talk to third parties during the building of the console, get them onboard, pay a little money if you need to ensure the first releases on the system are fully optimized and have full DLC so that potential customers know Nintendo is serious this time around.
4. Continue partnering with Third parties with making cross over games with Nintendo IP's. This time work with western and japanese third parties that are exclusive to your console.
5. THIS IS HUGE TO ME. They need to spend some money and maybe buy a few western studios in europe and america and have them focus on making games that solely focus on the western market. Just think Nintendo launches this new system with this new western focused games that are exclusive to Nintendo that have Nintendos quality. You have gamers buying the system to play these games especially if the sysyem is where it needs to be specs wise. This also shows third parties there is an audience for these games which means they will bring their games onto the system.
6. Expand their own studios. For example - Retro should be able to make a game for a Nintendo IP and also at the same time make a new IP for the Western Market. This limits droughts and appeals more the western audience.
7. Have your big guns ready for the launch of the system. A new nintendo system should have at least 2 of these games within the first few months of the systems launch: An epic 3d mario game, an epic zelda game, smash or mario kart.
8. Combine all of this you have a power console that is close enough to the others when it comes to specs, you have two top nintendo ips released at launch, you have third party support with DLC etc, exclusive games for the western audience.

They can do it but will they.

About 32 million of those 3DS sales have come outside of Japan. And yes I know some people own multiple skus so that 32 might go down to 25 million or so, which is still a hell of alot more than the 6.5 million Wii U has sold outside of Japan.

Well I don't believe there is this huge difference between the handheld and console markets that many people seem to think.

Look at the original Gameboy pre-Pokemon, the majority of its popular games were sequels/spinoffs/ports to popular NES games (Mario/Zelda/Metroid/Kid Icarus/Donkey Kong/Mega Man/Final Fantasy/Castlevania/Ninja Turtles/etc), Tetris was the only game that really stood out and sold exceptionally more on handhelds than consoles.

Look at Gameboy/Gameboy Color post-Pokemon, the majority of popular games were still sequels/spinoffs/ports of popular NES/SNES games.

Gameboy Advance, outside of Pokemon, once again the same formula plus a bunch of licensed games based on kids movies/shows.

The majority of popular Playstation Portable games consisted of sequels/spinoffs/ports of popular PS2/PS3 games. Monster Hunter was really the only that sold significantly better on handhelds compared to consoles.

Nintendo DS did have a few huge games just for handhelds, Brain Age/Nintendogs/Pokemon, also Animal Crossing proved to be much bigger on handhelds. But once again a large number of its big selling titles were sequels/spinoffs/ports of games that originated on consoles and some of those big handheld centric titles have pretty much died or sell significantly worse than they used to.

The majority of big games on 3DS are games that originated and have had success on consoles.

Tetris/Pokemon/Monster Hunter/Nintendogs/Brain Age/Animal Crossing, these are the only games that have had mega success on handhelds but not consoles. Tetris/Brain Age/Nintendogs are pretty irrelevant now, Monster Hunter/Animal Crossing have had success on consoles, just not nearly as big. Pokemon has had a decent amount of successful spinoffs on consoles, no reason to believe a mainline title wouldn't be a big success as well.

I believe a big reason why 3DS sells so much more than Wii U comes down to price and software output, 3DS hardware/software is much chesper than Wii U and has significantly more games. A low-cost, high-software output Nintendo console could very well see alot more sales than Wii U. If Nintendo unifies their handheld/consoles to the point that each device costs the same and share a library then I think sales could be something like, console-30 million, handheld-50 million.

Yea and mox xbox's and playstations are aimed at their respective fans.

Their loyal fans.

Everyone.

I would hate this console lol but also the he logic is flawed in 2 major ways IMO:
1.You're not addressing the fact that Nintendo home consoles have to compete with sonys/Microsft.  
2. You're not addressing the amount of cannablisation that will occur with a home console sharing most of Nintendo's handheld library.

-The reason why the Nintendo handhelds perform well is because they offer the best and most recognisable franchises and experiences within that space  the handheld Market-Pokemon, Monster Hunter, Mario, Capcom games, Square Enix games etc). The handheld aspect of the device is in itself a gimmick which sperates it from home consoles, there is demand for such a device and Nintendo is the best at it, there really is little competition. This is why relative to its success in North America and Europe very few 3DS games actually chart over here. PS4 and X1 software sales are both performing much better then the 3DS, even in Europe X1 software sales last year are much higher then what the 3DS achieved at any point in its life (the majority of 3DS software sales are from Japan). The last time we saw something comparable in terms the pattern between software/hardware was the with the wii (high software numbers/very few games making up those numbers/ low compared to the userbase) and again it had a strong gimmick/was not competing in the traditional console space. 

When talking about the home console space you have to acknowledge that sony and MS are there with consoles offering superior output in terms of software and superior value where hardware is concerned. They are doing for consoles what Gameboy/DS done for handhelds. When looking at the west the fall alone saw more major releases happen for the PS4 then the WIi U and 3DS combined for the majority of  2014. If it comes down to picking between the system with Mainline entires like Final Fantasy XV/Resident Evil 7/GTA/COD/Witcher/Batman/Battlefield/Battlefield/Devision/AC Victory/Dragon Age/Elderscrolls/Kingdom Hearts 3/Street Fighter V (i could go on...) and the system with upresed handheld games most as you said are spin offs or games designed for hanheld in mind (Pokemon ORAS/Moster Hunter/Professor Laten etc)people will pick up the home console delivering the cutting edge material.... and if the nintendo's console shared most of its library with a more successful handheld, most people would just get the handheld and skip the console- instead of getting the console offering. Most people don't buy secondary consoles (uneless theres a gimmick), and Nintendo would be positioning themselves as a secondary console with what you're proposing. I think said console will be more profitable then the Wii U but sell in similar numbers and upset fans.