I think you're confunsing the 'casual' markey with the 'familiy' market. Every Nintendo console since the Famicom was mainly targeted at the family market, not just Wii and DS.

The question is how much of that market is left or even needs another $250-$300 console when they already play several hours a week on their phones/tablets for free. 

That will be the problem for Nintendo on this end, Apple/Google/Facebook are not any easier competitors than Sony/MS, maybe even harder truth be told. You can't beat free games. 

The Wii/DS time was different because no one was making games for lapsed/casual gamers, but today smartphone and casual browser games are like 60% of the game market, it's the most over-served segment of the market and there are more people playing games today probably than ever. 

Watching TV I now see like 5x+ more marketing for casual smartphone games than I do for even Playstation/XBox. That would be unthinkable circa 2004. 

You are 100 pecent correct. Look at the DS and it's installed base, 150 million! In one generation the 3DS only has 58 million. Thats almost 2/3 of Nintendo's handheld market that went to mobile. 

Nintendo NEEDS to incorperate mobile phones and tablets into the NX ecosystem or they are at a major disadvantage. 

There's a ton of overlap between those two crowds, a lot of families play their tablet games like this:

It's really gutted Nintendo's market overall. 

In effect maybe that's what NX could be, it's kinda their attempt at creating something that factors in the monstrous impact smartphone/tablets/apps have had. 

I think that NX running Android report from Nikkei was in fact largely correct. It'll probably a custom/propietary version of Android, but it'll be able to have Android apps which Nintendo can then distribute through their store (perhaps for a fee). 

And if you look at a controller on a screen on it ... well by now smartphone chips are so dirt cheap that you could put a $10 chipset in there too that could probably run Android apps even without needing the system. 

Now whether Nintendo would actually find success with this approach is anyone's guess. 

Well Ubisoft did mostly push poorly designed shovelware on the Wii back in the day...
Also with over 60M Consoles already out there, the "Core-Market" is a pretty tough target right now, not to mention that it would include Nintendos image completely shifting.
I dont think family is bad per se: Wii and DS were aimed at familys and I loved both.
Maybe I am part of a dying kind, but I like playing together on the same sofa over getting flamed by strangers online.

I actually have a small theory concering the NX:

It could be absolutly nothing, but I think there are some hints to what kind of controller NX will have hidden in Zelda BotW:
The Shiekah Slate is obviously supposed to look like the gamepad, like so many other key-items in so many other Wii U exclusives.
The thing is: The Gamepad is hardly used in the Game!
Aonuma confirmed, the whole Game can be played with the Pro-Controller!
According to people who played the game at E3, the gamepad simply showed the Controls, which was likely done to help improve the demo.
This means that the Gamepads screen will very likely of little importance in the final game!
Coming back to the Shiekah Slate, its obvious that this is not how it was intended to be.
My guess is that many of the actions that are done on the TV during the demo, like scoping, marking targets, the map in general, etc... all used to be executed via the gamepad.
Why change that? Well, the game was announced for an yet to be shown console not that long ago!
A console that maybe, just maybe does not feature a screen on its controller, because the latter likely has more in common with the other controller that can be used to play Zelda BotW: The Wii U Pro Controller!

Agreed, this is a stretch, but I think we can at least take the absence of obvious gamepad-uses as a confirmation that NX will NOT feature a screen on the controller, and while I personally liked the gamepad most of the time, the image it has gotten over the past few years alone is reason enough to ditch the idea.