I think Sony are going to stay in the handheld market at least another generation, they've learned some lessons from the PSP and earned enough profit to have another stab at it.
I think the next PSP will have both physical games and digital games. All games will require a PSN release, but a physical release will be optional. Hopefully the UMD will die and be replaced with memorysticks in some shape or form.
I think Sony will want to tap into the iPhone-market as well, and make the PSP2 a phone, as they already have the Sony Ericsson division available, and I'm guessing it will be based on Android.
I just hope Sony realize that they can't get away with making a traditional handheld coupled with a phone in it, they need something that will make it unique or it will be butchered by the iPhone and 3DS. And no, additional power doesn't make it unique. Nor does a touchscreen.







