It's going to be the weakest handheld year since the 2005.
2005: 20.6 million (DS 10.6 million, PsP 9 million). Likely to be a bit higher due to the GBA still selling a bit.
2006: 30 million (DS 20.5 million, PsP 9.5 million)
2007: 42 million (DS 29.5 million, PsP 12.8 million)
2008: 44 million (DS 29.5 million, PsP 13 million)
2009: 37 million (DS 27 million, PsP 9 million)
2010: 30 million (DS 21 million, PsP 9 million)
2011: 30 million (3DS 13.25 million, DS 9 million, PsP 7 million, Vita 0.5 million)
2012: 20 million (3DS 11 million, DS 3 million, PsP 3 million, Vita 3 million). Plus whatever they sell the next few weeks.
In other words, the handheld market is the weakest it has been since the DS launched. Considering 2005 lacks GBA sales, it's likely 2005 was actually higher than 2012.
However, the reason for the drop isn't necessarily smartphones. The total revenue for smartphone games is at just 2 billion dollars a year, while the handheld market is still well above 10 billion.
It's less that the smartphones have stolen the market, and more that the 3DS simply doesn't have the appeal of the DS, and the Vita is nowhere near the appeal of the PsP.
The decrease is due to internal problems, rather than external. Which means there's still plenty of time to turn it around.