For me the more frustrating part was the uneven performance. First it takes a while to set up if the default settings aren't working right. So I tweak all the settings until it runs decently at the start then have to go back into settings later when it hits a difficult section where other effects or details become the bottleneck.
Everquest, prepare for a raid by turning all the settings down, still stare at the floor in busy places to be able to move. I remember almost not being able to complete Doom 3 because it slowed down so much at the end. Same with TW2, certain scenes with lots of fog or water spray crippled the game on my PC. And FSX so pretty and smooth from high above drifting through the 3D clouds, oh but tweak the settings as part of landing procedures or you'll be touching down at 10fps at a large airport.
And yeah, after you have dialed it all down to get through a section your setup struggles with, you kinda miss it and now see it as a downgraded version, so you go back into settings to turn things on again, and so the cycle continues. I prefer devs doing the balancing for me on console so I never have to check the settings beyond a simple brightness check and spend my time playing instead of adjusting.
Nowadays it's mostly my kids that play on my laptop and I still get to solve problems with mods and settings. Half an hour to set up some mod combination in Minecraft which they play for 5 minutes cause it still runs like crap, ugh.