The reason PhysX cards don't usually improve performance is because games are not written to perform general physics on the PhysX cards. They perform all general physics on the CPU, and perform "effects" physics on the card. What do I mean by effects physics, I mean things like particles for fire or explosions, cloth blowing in the wind, basically just visuals that don't have any effect on the gameplay. Since the cards aren't offloading any work from the CPU, performance doesn't go up, and it often goes down just from the burden of processing the extra effects.
PhysX cards are never going to be installed in enough computers for game developers to actually use them to accelerate general game physics and improve framerates. If your game engine requires a PhysX card to achieve acceptable framerates for in-game physics, you're basically saying "screw you" to everyone who doesn't have a PhysX card, which is most gamers. Physics isn't something you can just "turn down" in the options menu, it's essential to the core gameplay (except "effects" physics like I described above).
Finally, dual-core CPUs have become standard already, and we're going to see an ever growing number of CPU cores in the future. This means more CPU power available to run physics, and these fully capable CPU cores are far more flexible than a PhysX accelerator. Basically, PhysX cards are already obsolete, not long after they were first announced.
EDIT: To piggyback on what Sqrl said, CPUs may soon evolve to contain specialized cores, something similar to the Cell microarchitecture where you have several cores that aren't exactly the same, some more well-suited to running highly parallelized vector code like the SPUs. These specialized cores will become the backbone of newer game engines, not PhysX expansion cards.
