They need to shed themselves of the stigma that the Nintendo brand has limitations of technology, 3rd party support, that it depends on Mario/Link too much and that it's for kids. At the same time, they need to rekindle the love of those that were passionate about Nintendo in the past. While calling the system the NN64 would trigger nostalgia for those that loved that system, the N64 also represents Nintendo's fall from market leader, loss of 3rd party support and its first hardware mistake with the lack of CD drive. It also sounds lame.
I would step back a gen or two earlier and return to the Nintendo Entertainment System branding. It sounds adult and mature, while at the same time awesomely retro, back to a time where even the most cynical of modern gamer would admit that Nintendo systems were for the "core" gamer. I would call the system NES3, stating that it is the 3rd system in the NES/SNES line. I would enlist some studios to make sure it launches with non-traditional Nintendo games, such as an RPG from Mistwalker, a shooter, etc, and appeal to the nostalgia of core gamers who bashed Wii or Wii U.
Of course, the system would also have to be true to the NES/SNES roots and have decent power under the hood to play the coming generation of games.
