I hate to say it, but they're just too much of a kids company I think to ever compete with Sony/MS for the mainstream console core audience which is largely made up of teenagers/college age males and/or young boys who want to play whatever their older brother is playing.
NOT to say they're a kids only company, of course adults can still enjoy their games (I'm certainly one of them), but they are definitely more kids-centric than the other two and that will always be a problem for them in mindshare/marketing. By the time they get to 10/11 even a lot of kids don't want a "kids console" they want to be playing games they shouldn't be playing like GTA. The Wii temporarily solved this issue by bringing in casuals to augment the regular Nintendo audience, but that audience is now gone to smartphones/tablets.
I haven't seen a Nintendo TV commercial in like 6 months since the New 3DS launched, but my nephew was over today and we put on the kids TV network here, and lo and behold within like five minutes there's a Nintendo TV spot for Yoshi's Wooly World hosted by two kids.
They can't compete with Sony/MS who message/market almost exclusively to older core gamers and tailor their entire product line to an older audience, Nintendo will always be stuck trying to compromise between the kids market and trying to hold a slice of the core market.
It's not so much about Nintendo's execution at this point, it's just that the market has changed so much from the 80s/90s where the console market was more kids-oriented. Even if NX is relatively decent spec-ed, what happens in a year or so when Microsoft shows off a much more powerful next-gen XBox or PS5 hype starts to ramp up? There won't be 7-8 year cycles this gen because neither Sony/MS took massive losses early on this gen that required a longer gen to recoup costs.
