Problem though is on models like they they cut any corner they can. You'll almost always in a pc that cheap find a powersupply that doesn't provide enough power / has bad 12v rails. The supply will definitely be suited for those machines on their gpuless factory shipping adding a video card alone can send some of them to being unstable powerwise. I also find the cases generally don't have very good airflow design the extra it'll cost you to build it yourself will still always be worth it for just pure reliability if you do it right.

 Maybe, but the last two I linked to has Nvidia video cards. either way, if you're good enought to build a pc, you're smart enough to check the powersupply before you upgrade something as power hungry as a video card.

When I upgraded to a 4200TI I blew my motherboard as I didn't have a good enough Power supply, sometimes you just have to learn learn the hard way :P
I ended up getting a new motherboard, an AMD 1700+ CPU and a very powerful PSU, happily the 4200TI still worked perfectly :)

I upgrade my desktop last month, cost 500
parts:
amd x2 MB, $60
2G RAM, $80
PSU, $40
CPU, AMD 4200, $120
ATI 2600 510MB, $200

The rest of other parts are salvaged.
That was in CAD.

I'm going to be trying to take a e2160 (1.8ghz) and taking it up to 3.0ghz and beyond (hopefully as it'll be my first overclock experience).

 Yes, but no good video cards in prebuilt PCs.

And even if you did buy a video card for it...they're just not good for future upgrading.  It's much easier to add more of the RAM you want, extra HDs, video card, PCI stuff...I dunno.  For a hundred bucks or so more, I find building your own computer much more user-friendly...once you get past building it of course.

Lets not forget all the crap with company built PC's. Unless they've gotten better, I remember getting tons of pre-installed trail crap with our two previous dells.

That was my parent's computer though, I'm custom now.

 exactly and the price of Intel's new processors have been released. Quite a bit cheaper then I expected. 

@SSJ,

Few questions:

1) What parts are you willing to keep and what parts do you want to keep from your current system? 
2) What is the purpose for the computer and what is the most intensive application you plan to run in the forseeable future?
3) Is your budget completely rigid or is there a bit of swing to it?  Aka +/-30 or so.  It makes a difference some times.
4) How important is a fancy case?  Is generic fine?
5) How much Hard Drive space do you need?
6) What kind of speakers do you have and how many PCI slots will you need for existing cards?  Can some of those cards be gotten rid of through current onboard features like NIC, Sound, etc... Some folks really hate onboard sound, some could care less.
7) And finally do you need the cost of a new Windows licence to be included? 

Without the answers to those questions I would suggest the following....with the answers fine tuning can be done accordingly.  Just for the record I pretty much have 1 brand loyalty in computers and its to Seagate for their amazing drives, awesome customer service, fantastic warranty, and in general a very reliable product.  Everything else I try to use a bang for buck mentality.

$120 - AMD X2 5200+ - I like this model because it is the first that upgrades to the 1MB L2 cache per core in this line.
$50 (-$10 MIR) - Generic Motherboard - My title here says it all, you want a budget PC so a solid motherboard without lots frills is what you have to expect.
$85 (-$30 MIR) - 2GB DDR2 800 Corsair - You can go with just about any brand so long as you make sure to get lifetime warranty.
$80 - 320GB Hard Drive - Seagate SATA drive, pretty basic, but still very good 
$40 - Case & PSU - I am picky about PSUs, but in this rig I don't forsee any power issues and this PSU should be able to provide stable clean power, even on the limited rails it has. The case looks pretty good too.  
$25 - CD/DVD Burner - I am not picky about optical drives, if you are then I highly suggest getting what makes you happy since this is a pretty low cost area in general and there is no reason to not be happy with it. 
$90 (-$15 MIR) - 7600 GT 256MB - Note my favorite card but its solid for a $500 system all things considered.  

Pre-Ship Total =  $490 (-$55 MIR)

If you do the Mail-in Rebates  Pre-Ship total will be lowered to $435.

*NOTE* -  This computer is not necessarily my recommendation to you.  The answers to the questions I posed above are pretty important to what kind of computer I would suggest. This computer is just what I would suggest to any generic gamer with no other information to go on other than a $500 budget. 

Personally I am not very satisfied with the video card for a *dedicated* gaming machine, and depending on the gamer I would actually advise purchasing a motherboard with onboard video and saving up to get a nicer card if the reason for the rig was just to game. Some times its worth a month or two of saving to get a nice video card...even if it is tough to do.