The PSP is a MASSIVE failure in that it cost Sony it's aura of invinsibilty with game publishers. Most third parties no longer equate the "PlayStation" brand with guarenteed success. This is going to help increase the rate of defection in Japan especially if PS3 sales don't pick up soon.
A much larger % of PSP sales came out of their own console fanbase rather Nintendo's handheld market. So when you factor in that + opportunity cost (the money Sony could of made if it focused it's resources on home consoles instead) it's obvious the PSP was a mistake from any business point of view.
And it's attach rate suck. At least the GC is managed to get into the top 5 once in a while.