Supporting a handheld with dedicated games is a lot of software investment and sony literally did not have the teams, nor were they ever responsible for their own handheld success. Games like GTA, Final Fantasy etc were. 

Just as we've seen with Switch and Series S, times have changed and tech has too. If Sony can create a 1:1 match for PS5 (CPU/SSD wise) and scale down GPU/RAM down, then they could have a machine that will inheret all of PS5's software with very little effort from developers beyond a patch. Potentially it could even be automated. If PS Vita played everyones PS3/PS4 games, I'm sure it would've been a bigger hit. 

Xbox is in a better position to do this though, especially with the more reasonable speed SSD and the existence of Series S already. They could target a machine 1/2 XSS specs and use some sort of reconstruction to bring these up to 900p/1080p visuals + have a gamepass package as the price of admission. 

Vita struggled when run some ps3 ports, the only ps3/ps4 ports it can run really well are JP AA games and some 1st party games.

A portable PS4 would have made more sense. But the ideal time for that has passed.

I'd say this is still a bold prediction. Sony doesn't have much to gain with a portable device that will only do well (relatively speaking) in Japan.

They're better off with continuing growth in developing markets including China, expanding and improving PS Now, turning their IP's into multi media properties & making a push for the handheld market through games in the mobile space, whether that be through acquisitions, internal growth, or a mix of both. A handheld PS5 feels like a waste of resources.

PS4 success, partially, is because sony focus on the platform. The Ps3 demise, partially, is because sony don't make be a two platform holder, Nintendo have problems with this too. 

Steam Deck basically already accomplishes this and there will almost certainly be a Steam Deck Version 2 with even better hardware by 2023.

Then there's the other elephant in the room ... 5G game streaming is going to become more and more commonplace. There's no point to offering 5G with tiny data caps so providers will start to offer unlimited/gaming centric plans for services like xCloud and we just learned Netflix is starting game streaming too.

So you'll be able to play high fidelity next-gen games on your phone too.

But hey if Sony wants to make a PS5 Portable I guess they could try. Thing is the games couldn't be played directly from the PS5, they'd have to be down ported. 

No way.
The Steam Deck is comparable to PS4 specs, not PS5 or Xbox Series X. I don't see how Sony would release a handheld in 2023 or 2024 and have it perform similar to a PS5.
If Sony releases a handheld (or hybrid similar to the Switch) in the next 2-3 years, it would probably be around PS4 Pro level. Maybe slightly above, but still clearly below PS5 capabilities. Dedicated handheld gaming is still alive almost entirely due to the Switch. I don't think the Steam Deck will sell any more than 10 million units.
Having such a powerful handheld will not drive people to pick it up. That has really only ever worked with home consoles, and even that is debatable.

How feasible would be hardware-wise to turn the PS4 and XBOne into handhelds in a year or two? I mean, they are still getting games, and every game that would release on them would be BC with the PS5/XBoxXS.

Considering how MS/gabe are wanting Gamepass on Steam, and with both Sony/MS releasing their games on Steam already, this would be the most logical and cost effective route.

Sony & MS don't need to pour money into R&D for a new portable device, but they can in turn make money from an existing one on the market with the largest store userbase. 

Some PS folks on twitter are already regarding the device as the new Vita (A bit odd considering it's not Sony based, but whatever floats their boat). 

Yes, any thought of a Sony handheld is ludicrous, but the Steam deck offers up opportunities for PS games to come to the platform and that has to be a good thing.