You should be able to get pretty far by upgrading your graphics card. However, by the sounds of it, you're also going to need a new PSU. At worst, trying to use an underpowered PSU could break your whole computer, I've heard, so you should definitely consider getting a new one before getting a new graphics card because those things consume a lot of power.
After that, you should probably look into getting more RAM. 8 GB should be fine. 4 GB isn't enough for the newest games, but for slightly older games, it should be fine.
I wouldn't probably touch anything else. I don't know how much you can change before Windows considers it to be a new computer and requires you to buy a new license. Also, it might not even be worth the effort and money to upgrade anything else (could end up being a smarter choice to get a new PC altogether if you want to do even more upgrades).







