My guess is that Nintendo and Sony form a joint venture company and release a PS5tendo and 4GDSvita together with strong hardware, many exclusive AAA games and full 3rd party support.

You make some very good points. Stronger Wii U would not have solved anything. The blue shell would have been in 1080p. That's it. 

I honestly would prefer them going with weaker hardware and a mass market friendly price, that's one of the problems with Wii U, it's priced like a high end device despite not being one.

A sub-$200 console that comes with an improved Wii remote and has a small power gap over Wii U similar to the gap between GC-Wii and has a unified architecture as the handheld that's priced the same is the way to go in my opinion.

To be honest though the XBox One will probably be $299.99 soon and Sony may then have to go down to $299.99 as well all at once. 

So in that context, lets say Nintendo has a "cheap-o" $149.99 console or something, I'm not sure it wouldn't have some of the similar problems the GameCube had. "Yeah it's cheap, but for a little more you can get an XBox or Playstation" type thing. 

In all honesty both would fail:

The NintendOUYA while great on paper will not garner much support. Nintendo can't prosper on first party games and that's a fact. A console which is substantionaly underpowered compared to the PS5 and NXB will barely have any indie support and that's what's practically keeping the WII U alive, minus first party games. No one will buy a console just for a couple of first party games ( See WII U as example ) apart from a couple of million people.

You're already listed why a PS4tendo would be a failure.

The problem is as long as Microsoft is in the market (basically getting nowhere themselves) they hog up too much space for Nintendo to make a traditional style console. There just isn't room for three different consoles doing roughly the same thing.

Nintendo will pivot more towards being a IP brand I think in the next few years, where kids can get introduced to Mario/Zelda/etc. through movies, TV shows, theme park attractions, and smart phone games, and that hopefully broadens the pool for a new wave of kids willing to get into the Nintendo hardware ecosystem (NX). 

I said it to another user, a NintendoOUYA is an extermization, I'm not suggesting they should make a console like it but what I'm suggesting is Nintendo shouldn't go after AAA support with fancy hardware.

You say handleds have a completely different appeal than home consoles, that's correct and that's exactly what I am suggesting. If the appeal of PS and XB consoles is based on graphic fedelity and online capabilities, Nintendo should focus on alternative ways to appeal gamers (and this is not to be confused with hardware gimmicks), an alternative way to appeal is an alternative way to market your console: different strenghts, different weaknesses.

Than it would be so different to spend 399 to play 3rd party games and Sony games or 499 to play everything?

How? I said Nintendo should focus on making an accessible and affordable console, the WiiU is neither accessible or affordable.

In their intention they wanted to get 3rd party support while not alienating the cazuals and families market and, in they end, they failed at doing both things. The WiiU is a very bad and half-hearted attempt at doing a Ps4tendo. The WiiU did so bad, not beacuse they failed to follow a strategy, they didn't have a clear strategy to begin with.

This is fair enough, neither option would make everyone happy. I myself would choose a Ps4tendo over a NintendOUYA.

Just for clarity, when I hypothesize an NintendOUYA as an affordble and accessible console I mean you have everything you need to play for 99$.