Sony doesn't have the IP to pull it off in the west. Dedicated handhelds need lots of kid-friendly IP, which Sony never developed. Their aim with the Vita was toward older gamer--a segment which has now mostly moved to mobile.
Had they started working on such IP during the life-span of the successful PSP then they might have been in a position to continue with handhelds now but more mature games "on the go" aren't going to do anything at this point.
If Sony really tried, if they really put some resources into dedicated handheld IP, they could probably create a moderate success by today's handheld standards but they're well aware that it's not really worth it. That market is not lucrative compared to other markets and has no hope of growth.
