I was going to recommend a Lian Li HTPC case until you said won't break the bank.
I don't know if you meant cheap case, or just not excessively expensive.
If it's the latter, go on NewEgg and take a look at some of their horizontally oriented cases.
Also look into getting a Shuttle bare configured PC. They're aimed at the LAN market, but the form factor is small enough that they're good for HTPC as well. They come with a PSU and Mobo pre-installed. Just add memory, CPU and VGA card for the models that don't have integrated graphics.
A good mobo to use for HTPC is the Asus P5Q-EM. Look up the specs. Only drawback is that it's a little pricey for a micro-ATX board (because of the integrated graphics chipset), but it's pretty feature rich for its class. It's also pretty good for overclocking if you want to use it for gaming when paired with a discrete VGA card.







