You mean kind of what Squareenix did in close partnership with Nintendo.
Examples of Nintendo games with intricate storylines:
The Legend of Zelda LttP
The Legend of Zelda Link's Awakening
The Legend of Zelda Oracle of Ages & Seasons
Super Metroid (at the time)
Fire Emblem (I am guessing since I've never played it)
Super Mario RPG (Good example)
There are more but I'm bad at lists.
The IP argument you bring is not valid. The improvement of story in the Mario series only serves to inject life into the franchise and improve the image of Nintendo as a company, since Mario is its icon. Then, the resources required to do this are not greater, this only requires a different direction, not more work. So it's not a tradeoff scenario.
Well I was mostly talking about Mario series in general , it doesnt need to be changed, they have other IPs which they can focus on the story front like you said the Fire Emblem Series (which I will be getting into soon ;) )
The IP argument I was talking about is Mario mostly grew with popularity not because of its storyline, but because of its gameplay aspects, Nintendo has outdone themselves countless times, No one mario game is like another (generation wise and excluding Galaxy/New Super Mario Bros.), they expanded the gameplay elements, changed them up and made them innonative. Storyline did play a part in some of the spinoffs, but the main series line, the gameplay outshined what any story could do for Mario universe.