Nintendo was never strong when it came to the storyline development of their games (for an example how many times do we have to rescue Princess Peach before she learns to defend herself, also those damn toad folks dont know how to defend their own kingdom!)
Nintendo mostly focuses on the gameplay aspects of their games and brings an appealing factor to the gameplay experience which overall attracts people of all ages but mostly young audiences. Not only that but people from older generations will stay close to Mario since they more than likely grew up playing Mario games and once again this is due to the simple yet complex as it grows gameplay style.
I dont see a reason why they should change that at least for Mario. Instead of changing up the formula completely for Mario (in terms of what is he targeting), why not use the available resources to make new IPs that attract people outside of that certain circle or expand on other games that would do better in that category such as Zelda or Metroid.
Storyline wise they could expand it a bit more kinda like what Squareenix did with Super Mario RPG.
You mean kind of what Squareenix did in close partnership with Nintendo.
Examples of Nintendo games with intricate storylines:
The Legend of Zelda LttP
The Legend of Zelda Link's Awakening
The Legend of Zelda Oracle of Ages & Seasons
Super Metroid (at the time)
Fire Emblem (I am guessing since I've never played it)
Super Mario RPG (Good example)
There are more but I'm bad at lists.
The IP argument you bring is not valid. The improvement of story in the Mario series only serves to inject life into the franchise and improve the image of Nintendo as a company, since Mario is its icon. Then, the resources required to do this are not greater, this only requires a different direction, not more work. So it's not a tradeoff scenario.