This is one place I've been reading about half-length and half-height ("low-profile") cards: http://en.wikipedia.org/wiki/Peripheral_Component_Interconnect#Physical_card_dimensions

Even if I am able to find a decent "low-profile" card, it seems I'm going to also need to upgrade my power supply. And there may be some other additions required that I'm not yet aware of.
The whole point of me considering this upgrade is to weigh my options versus simply buying an Xbox360, for example. 
As I said in my OP, I originally bought this computer for work only, without gaming in mind, so I may have limited myself hadware-wise from the very beginning.

I found a 4550 for $60, this is a list off all neweggs low profile/ low profile ready cards. As to your power supply if your is a standard size it wont cost but $30-40 to upgrade to something more then sufficent. The problem you may have is dell loves using propritary stuff so alot of times they will have normal size power supplies but the screw mounts wont align in any shape or way.

 http://www.newegg.com/Product/ProductList.aspx?Submit=ENE&N=2010380048%201419530408&bop=And&Order=PRICE

Thanks for the link. I had been looking at information on a 4550 just before I posted my previous message.
Also, thanks for info about the power supply. I'll be sure to check that carefully before buying anything.

Oh. What I defined as length, they define as depth. And a max depth of 6.6 inches (167.64 mm) ain't much... My 4850 is 10 inches.

Usually in GPU review articles they encourage people to simply measure the space available in their case for PCI-E cards. I myself usually cut out a hard piece of paper and put it inside the case and see if it fits. You should do that.

The 4550 is a weak ass card.. not recommended. The 4830 is pretty long/deep at 10 inch, but perhaps the 4670 at 7.5 inch will fit?

I think you will have problems to put pretty much anything inside that case...

To put it simply with this setup you have big problem as getting any decent graphic card will be impossible.

If you were to build decent gaming machine then it would require most likely to replace main board, case and power supply (ot to put it simply you would be building new computer and placing inside a few parts from old one).

Cost of it would be:
around 50$ for semi decent computer case:
http://www.newegg.com/Product/Product.aspx?Item=N82E16811119068

85$ for good class mobo with some oc abilities:
http://www.newegg.com/Product/Product.aspx?Item=N82E16813130095

50$ for good class power supply (430W will be enough for single graphic card setups)
http://www.newegg.com/Product/Product.aspx?Item=N82E16817371006

115$ for radeon 4830 card :
http://www.newegg.com/Product/Product.aspx?Item=N82E16814102803

cpu, ram, optical drives, fdd, keybord, mouse from old dell
then for 300$ it's decent pc gaming machine fully capable of playing anything in 1280x1024 (19" lcd native resolution) and as for sharp aquos it will be capable of producing playable framerates at games like mass effect , DMC 4, witcher in FULL HD for sharp aquos.

And if you have more money for 4850 or 4870 radeon graphic cards it will be even better.

The big dowside is you need help of computer savy friend to help you put it together.