I will give two examples of how Sony could have sold more systems:

For PSP: Using flash based memory would have been cheaper and the system would have a longer battery life.

For PS3: No Blu-ray. It wasn't needed, and just adds 200 to the cost.

Actually, portable Playstation does exist. It's basically a Playstation, with a removable LCD screen attached to it. Did the removable LCD screen sell well? Nope. And if they made an entirely different portable Playstation, which was not just a Playstation, with a removable LCD attached to it, and that portable Playstation could play Playstation one games, using the original Playstation CDs, and not re-released, smaller CDs, like the Nomad plays using the original Genesis cartridges, that portable Playstation would be in the shape of a Walkman, in order to be big enough to accept those original Playstation CDs. Would it have been successful? No, because the removable LCD screen didn't do anything. Now, if they made a portable Playstation, that did not play original Playstation games, but rereleased Playstation games, on smaller CDs, so that the shape of the Playstation didn't have to be wider than the original Playstation CDs, it would be even worse, because you would have to rebuy all of the games. And besides, we want new Playstation games, not old ones. The Playstation Portable would have done better than any portable Playstation, whether it would be the original, with a removable LCD screen on the top (which does exist), or a portable Playstation that plays original Playstation CDs, or a portable Playstation that plays re-released, smaller CDs. If the Playstation Portable focuses more on the games, than the other things that it could do, it would be more successful.

Simply Sony entered a market they hadn't studied enough; they were unprepared; They should be happy they didn't go the Game Gear way. (they would if they weren't Sony)

I'd Say the most important things to consider with the idea of designing a better PSP are.

Controls
Size
Battery Life
Price

That's why a remake of an orignal PS wouldn't make it, If it read cd's the moving parts would eat the batteries up, Also that would increase the Size.

PSP already still has some control issues. Being able to make an analog stick that is small, works well, and people like is a million dollar idea.. DS went the route of the whole point and click approach which made them shine.

So for improvements I'd agree memory card/Flash based is the way to go it's so cheap, and helps with batterly life. The PSP does have a leg up on the DS by having a memory card that can store multiple things, and games... DS should really combine the Virtual Console and the DS... that would be great.
Price Point was a minor problem for the PSP... I think working on the controls would have helped them a ton... And something to protect the screen. The DS is brillant with it folding in half to provide protection for your screen and more size without making it difficult to use as a portable. If Anything the PSP would have been awesome if It would have placed a wide screen that would be the size of the psp on the top... then the bottom would just be controls... speakers... ext... Maybe some sort of track ball for the analog sticks.. and duel analog sticks... it's supposed to mirror the PS2/3 anyway....

For the PS3 I would say the price is what's killing them. It would have been interesting to include the Bluray DVD drive but offer a download to unlock the ability to watch movies... Sell the upgrade for like 60$... Smaller hard drive option.... I really like the now non-existant 20GB model they had, except I would want wireless internet as an option. Could they have produced at 10 GB model... or a model with no HD, (Remember you can trade the PS3's harddrive out with just about any hard drive ... they don't stick it to you like MS).

Option to not have Bluray forced down your throat 60$
No harddrive, you supply your own 100$
I would have tried to launch the Core PS3 for 250-300.

Price does matter, Lets say a new game comes out on PS3 tomorrow that your like, "man that looks pretty good" and lets also say you don't own a PS3 yet... your not going to pay 500-600$ to play that one game, and you know the price has to go down, so you keep waiting for more games and see if they come.

With the Wii people see one game and they run out and get it ... because it's only 250$...