The real problem is the lack of a fixed term to refer to either home console or handheld.

Back when the first home console was released, they were reffered to as "Video game console", this is not a new term made for the devce, this refers to a device that outputs a game program to a television, and console refers to the *box of switches* used to control it, in much the same way the controls between the front seats of a car for AC, gear changing and general electronic switches and functions are refered to as the "Center console"

Because a specific term was never applied to the device, over time it just got shortened to games console, the term home console only came about when handhelds were released and home was tacked on to differentiate between portable and home, the problem remains that because the original device was never given an appropriate term, neither was the handheld, and instead just tacked on home or handheld, the problem with this is looking at the very reason the term console was used at all.

It was used because the box of switches was used to control a seperate device (monitor), in the same way a center console controls seperate devices (AC, gearbox, lights, etc), just as a sonar console in a submarine is used to control the sonar hardware and have the feedback from this external device display on a terminal video display.

A handheld is a completely self contained device, the switches and buttons it has directly control itself, output is monitored on the device itself, ergo it does not have a "console" that is used to control other devices, it is in itself a single consolidated device.

So, after looking at the background of why "Console" was used at all, it becomes abundantly clear that a handheld cannot be correctly defined as a console, but we do anyway often refer to it as such because nobody ever gave home consoles or handheld consoles their own unique terminology.

Look at other areas.

Phone
Smartphone
Tabletphone
Mini-tablet
Tablet
Tablet PC
Laptop
Netbook
Ultrabook

We give other debices their own singular terminology based on nothing but form factor and basic feature set, yet we are still stuck with the term "console" from the days of the magnavox and colecovision because nobody ever took the time to properly name the then new devices.

Home consoles are borderline "consoles", especially since the era of wireless controllers took wired links away from the base unit and the control unit.
Handheld devices are not "consoles" at all because all input and feedback is done on the same device.