Nintendo usually goes for cheap with screen resolution but I think this time there will be some concessions here.
I suspect they'll go with a 1280x720 (HD) screen of reasonable quality (ie: better quality than the washed out Wii U displays). Not all content has to be at that resolution mind you, some games could run at 540p or even 480p just fine.
But in this day and age of iPads and iPhones with super high resolutions and 1080p screens becoming the bottom end, I think even for Nintendo I think they recognize their devices look seriously dated even to kids. Apple and Samsung's displays are so much more better looking and vibrant that a 480p screen or something will laughed off from day 1.
A decent 1280x720 LCD that's say 5 to 7 inches (depending on form factor, are they going more mini-tablet or not) will look nice but still be quite cheap come 2016 and a good mobile CPU/GPU should have no problems driving the display.
I have a cheap $160 Samsung tablet and the 8-inch 1280x720 display (far from cutting edge) is light years better looking than a 3DS or Wii U screen.
