If they do a PSP3 they need to do something new like the PSP.
If not I'll just keep at the PS4 instead.

Do a 1080p device that plays UMD's, Vita carts and new games, that has 4 shoulder button and have access to PS Store from PS1 to PS3 and you have a perfect companion for the PS5, specially if they go back to expand the PS1/PS2 digital library.
PS5 (having PS4 BC, would love full physical BC from PS1 tho) and this PSP3 combo would mean you have access to all Playstation have has to offer and access to new exciting games in each platform to keep them moving foward.

Honestly that will never happen, tho.
Because why would anyone invest so much in the PSP3 games that are PS4-like instead of making them for PS4/PS5 instead?
Don't think Sony will release another handheld in the next 10 years, if they ever do it again.
PS5 is all they need, and they'll likely profit a lot more keeping focused on just that.
PS5 will full BC is the best we can hope for.