There's a market for it. But it's a declining market for sure.

The sales are not reflecting that.

This.  Nintendo needs to make an android smart phone with 3DS hardware inside.  Sony's playstation phone failed for the same reason PSP and Vita failed: no support.  A 3DS phone would rule the handheld market.

It's a bit of a misnomer because the DS started so slow due to the GBA being only three years old when the DS launched. 

3DS shouldn't be going through the same sluggish start the DS went through, it's the successor to the DS, launched six years after the DS, it should rocket out of the gate, much like PS2 vs. PS1 (PS1 had slow sales at first because it was obviously a new brand). 

The drop in DS/3DS sales from 2-3 years ago, compared to what the DS was putting is obvious. 

Handhelds won't go away until smartphones find a way of incorporating a decent control system built into the phone. You can have a deep and rich experience on a mobile game, but without decent controls it will never be better.

And there's will always be room for a Nintendo handheld!

I am of the opinion that they are in fact in separate markets.  For one thing, not everyone who is into handheld gaming owns a smart phone or tablet.  A great portion of mobile consumers who own smart phones and download apps for them were never in the handheld gaming market in the first place.  Strictly as a small example, both my boss at work, and my best friend own smart phones.  They both download cheap gaming apps for them like Angry Birds or whathaveyou.  Neither one of them owns a DS, PSP, 3DS, or Vita.  Neither one of them ever planned to either.  So, in those two specific cases, their purchases of mobile games had zero effect on the handheld market, because they were never in it in the first place.  They are just taking advantage of a new feature in the phone market that allows them to play quick, cheap games on said devices.  Secondly, are smart phones going to start releasing gaming geared editions like a Dual Screen or XL smart phone to provide a gaming experience to their consumers that Nintendo doesn't?  I don't think so, because that were reduce the practicality of their primary purpose, which is making telephone calls.  Look at how poorly the N-Gage, hold a taco shaped object to your ear while talking on the phone did.

The handheld market will look like this by the end of the generation IMO:

3DS - 80-85 million
Vita - 25 million

Down a lot from

DS - 150 million
PSP - 70 million

There's still a market for it. It's just a limited market, scaling back. I think the market was going to shrink anyway even without the smartphone factor, because the PSP really did not achieve its destiny of being the "cool" handheld that every 18-32 year old carried around like a gaming iPod. Handheld market is going to still have a place mostly because of younger kids. 

A lot of the people that bought into the hype of the PSP in its early years weren't coming back anyway, they tried it and the novelty of watered down PS2 games on the go didn't hold their attention once the 360/PS3 launched. So Sony at least was likely looking at a large scale regression anyway. For Nintendo the smartphones have probably forced them to retreat back to selling more to the Game Boy market (young boys) as their core demo. Adults if they want something like Brain Training, there are a milion $1 apps that basically do the same thing. 

It's not just about games. Smartphones has fundamentally changed people's lifestyle period. Before you were really limited in what you could do when you got outside the house. You couldn't watch a movie on your cell phone. You couldn't really browse the internet in a satisfying way. You couldn't check social media sites. You generally had to carry a seperate iPod or CD discman (lol) if you wanted to listen to music.

There are a million ways now that a cell phone is basically a mobile entertainment center, you can do all sorts of things with it on top of playing games that eat up any free time you'd have on the go. That's the problem. To the average grown adult, why should they spend $170-$250+ on a dedicated game player, when their cell phone/tablet eats up all their free time on the go as is? If they want to play video games, that's what their 50 inch screen and "real" game console is for. 

It's the ESRB's fault. Nintendo agreed to be a part of the ESRB system, meaning that game developers have to pay a certain amount to get their games rated. Thanks to the relatively small market of the eShop, this means they have to raise prices in order to break even.

iOS games, on the other hand, don't follow the ESRB, since Apple says "meh, screw the rules. We'll rate games ourselves!" They rarely actually do so, but this is enough for governments, so developers can sell their games for as low as they want.

In other words, Nintendo's $2 to $5 eShop games are about as low as you can get while following the rules that have been in place since the 90's. Since phones are not gaming devices, they can get away with murder ignoring the rules.

Some people enjoy playing handhelds at home, around the house. I'm concerned that when the Wii U comes out, some people may find having a 3DS redundant, as the Wii U "play on the tablet" mode would suffice as a "handheld." At least, that's how I see it.

I'm in this boat too, I mostly use 3DS/Vita around the house to be honest. 

Wii U is probably going to dramatically decrease my 3DS/Vita usage. Why play an Assassin's Creed spin-off, when I can just play the real thing?