Yes, they should, imo. They just seem to be living in their own little world right now.

Yet Bayonetta 2 and Devil's Third and Ninja Gaiden 3: Razor's Edge are hyper violent games. Fatal Frame is also M-rated I think. 

So clearly they are willing to make M-rated games, they're just making poor choices in what they greenlight. Instead of niche Japanese titles, they'd be better off making something violent (sure) in a Western fantasy RPG setting. 

How about an open world fantasy game with a Game of Thrones like story? A game that can go to darker places than Zelda can? 

When was the last time Nintendo prioritized storytelling in one their games? Earthbound?

I agree with those above that said adding a creative twist to differentiate themselves is the way to go.

Fire Emblem, Kid Icarus Uprising, Xenoblade, even Bayonetta 2 have cinematic stories with tons of cut scenes (even though Bayonetta gets weird as hell) so the "Nintendo doesn't make cinematic games" isn't even a thing anymore. Maybe if it was 1999 you'd have a point there. Modern Zelda games have a bigger story than Earthbound did. 

None of those games are anywhere near AAA, and, if I'm not mistaken, none of them are 1st party. For M-rated AAA open-world medieval game Nintendo would have to use their own studios, and that's what I don't see them willing to do.

I never said anything about cinematic games. Having cinematics doesn't mean you have a decent story. I'm talking about roleplaying games with a major focus on a well told storyline with excellent writing and morale choice systems a la The Witcher 3 or Fallout New Vegas. None of those games fit the bill, and nothing Nintendo has ever published comes close. It's a large jump from their comfort zone. Their best bet for this sort of thing would be to contract a proven developer like Obsidian and pay them to create a new roleplaying IP for Nintendo. 

Why do they have to use their own internal studio? Retro or NST or Next Level could do it. Or heck, Platinum. 

Which is basically standard industry practice. Nintendo pays developers to develop games for them all the time, it's just a matter of being smarter in the projects they do greenlight. 

Though I do think internally if Nintendo wanted to make a any genre of game, they really could and could probably do it better than most in the industry. 

They are simply extremely talented at building base game play. 

Even when you look at Splatoon ... it's amazing that no one thought of that conceptually before, especially the idea of spraying areas as territory, it is the freshest new idea in that genre in ages and addresses a key issue of "not accomplishing anything if you're not sniping someone's head off" that multiplayer shooters have had. 

And the "swim through the ink" thing, I don't even know how they thought of that, but again brilliant. 

Nintendo is just brilliant at game design, unfortunately a lot of their good ideas are buried in games that today are very hard to market to a mass audience because of the presentation/genre type. 

Umh, isn't Retro 1st party dev? Others - yeah, no, this is medieval open-world WRPG we're talking here about, not that I think Retro would be able to pull it off.

Anyway, why do they have to use their own internal studio? Beacuse if they want to make something AAA like Skyrim or Witcher they would need to cough up a lot od cash for 3rd party exclusive, which I don't see them doing.

That is if they can even find studio that can do it - making open-world WRPG that is suitable for mainstream is really not so easy task. There are smaller devs, like Pirahna Bytes, Obisidian or even Reality Pump who could be able to pull it off with proper funding, but I seriously doubt people currently in head of Nintendo would be willing to invest heavily in such 3rd party game - they just don't seem to me willing to take such a risks.