Well Nintendo has a long line of successful handhelds. One part of the equation is probably that Nintendo never faced real competition until and only from the PSP as dedicated handheld.
However the pure handheld market these days get heavy competition from smartphones which should explain the drop from other Nintendo handhelds and the 3DS. Another part of the equation is Nintendo's dedication to make great handheld games and some smart deals with third party devs (monster hunter) and that it's pretty cheap to develop for 3DS compared to PS4.
Also handhelds are portable so great for long trips and cheaper than consoles. Since they are weaker than homeconsoles most games are cartoony instead of realistic. So they are great for kids. All those factors attribute to Nintendo is succes when it comes to handhelds.
With home consoles they actually have serious competition. Playstation especially has a great first party line-up and also get all the major third party games. Because of that it appeals to a larger audience than previous core Nintendo homeconsoles, which you buy for the exclusives and little else.
With the Switch Nintendo tries to combine the power of a homeconsole with all the factors I listed above which make them so succesful in the handheld market, so they are three steps ahead of mobiles again, so thst the Switch can outsell 3DS.
Please excuse my (probally) poor grammar
