I actually think that in the next generation, Nintendo's gonna have a small dip in portable sales, but it's not gonna be nearly as bad as with the DS to the 3DS. It might top out at 50 million units, but that's still a lot of units, so oh well. Smartphone/tablet gaming was already blowing up when the 3DS was released, so this means that 50-60 million people consciously chose to get a 3DS over a smartphone/tablet. There are a lot of people like me who would rather quit mobile gaming completely rather than suffer through smartphone/tablet gaming. Some people can live with smartphones. The casuals jumped ship in 2010-2011. The ones still buying handhelds after 2010 have already decided that smartphone gaming is just not for them in it's current form. Since 2010, the main things keeping people from switching to handhelds remain unchanged.
I think there are only 2 things that can cause mobile gaming to completely crush portable gaming to the point where no one would bother. The first is a way to separate the quality games from the shovelware. Right now, the mobile market is an absolute mess. The crap like Flappy Bird is what usually takes over the app charts, and it's extremely difficult for anything else to break through. The game either has to have a pre-existing fanbase, or aggressive promotion elsewhere. Not so much with dedicated handhelds, which have the better games pushed to the top MUCH more frequently. A dedicated gaming platform like Steam is needed.
The only other thing that could completely crush handhelds is a phone dedicated for gaming, with its own first-party games, as well as third-party support. That phone would have to completely blow up in order to have any chance of getting the developer support it needed (*cough*XperiaPlay*cough*). Either that or a button addon that devs can standardize on (which would probably be easier, but it would still have to be worth the money to make games with the controller in mind). The currently fractured market means that you've got some apps that would require one device, others that would require another, and suddenly you've got a huge mess. A device like this would cause me (and many others) to consider switching, but nothing less.
Also, there's the issue of children. Now, Nintendo could fuck things up and start ruining their games with aggressive microtransactions (which is why I would make damn sure that all traces of Pokemon Shuffle are gone before handing it over. Seriously, shame on you, Gamefreak), but as of right now, I can feel confident with handing one to a kid without either a massive credit card bill (if I give them the card), or them coming back and asking me to buy stuff ingame for them (if I don't give it). The kid would probably have a lot more fun with the games too, since they're not built around microtransactions. I can tolerate mild microtransactions in the games I get for myself, but including them in games designed for kids
Until these changes happen, I will NEVER buy a smartphone or tablet for gaming, nor would I give a kid one. And I'm sure most of the current 3DS/Vita owners feel the same. It has nothing to do with graphics (if it was, smartphones would've crushed the 3DS's 2006-level hardware by now). It's the fact that the mobile market is the Atari 2600 all over again.