The best argument in favour of a cheap gaming PC, IMVHO, is that if you have anyway to buy a new PC, mainly for non gaming purposes, you can add a few quids to improve its gaming performances.
This way a PC becomes almost unbeatable with little cost if you are mainly a retrogamer, an entry level with mid-range GPU added is often more powerful than high-end PC's one year older and will run most games already out in bargain edition at full speed. But also most new games using older but reputed engines and games relying on large masses of players, like MMORPG's.
OTOH, if you always need the latest game, and the more power hungry, the better and you need to run it exclusively at highest res with all effects on, your PC gamer life will be always very expensive.