I would love to have portable PS4. Most of my gaming has been done on portable systems (right now Switch, before that it was 3DS + Vita), because I love the convenience and realistically I only have enough time to game on portables. With that being said, I don't think the PS4's architecture would really scale down well into something portable, not even in the future as process nodes shrink (look at Switch, it is using a completely different architecture as opposed to taking what was in the Xbox 360 and PS3 and trying to shrink it). However, we could see new SOC architectures in the future that are designed for mobility but are as powerful as a base PS4 (in fact, it is mostly a matter of "when" as opposed to an "if"; the Switch in handheld mode is about as powerful as PS3 and Xbox 360 and its SOC came out about 10 years after those systems were released).
Thus, the problem here is not the hardware getting there but rather it has to do with the software. Having a portable system that is on par with a PS4 but not based on the PS4's architecture means that it would likely not be software compatible with it. Thus, the system would need to build a large enough base and/or Sony would need to make the porting process so convenient (not to mention bring first-party PS4 games themselves) that devs would be convinced to port their games over.