This is an interesting topic, and I agree with the sentiment in the OP. However, instead of saying what a developer should/should not do I would prefer to say what is good/bad design. How do developers approach difficulty in a way that makes a game well designed?
There are a couple cases I can think of where putting in a difficulty setting makes a game better:
1) Games with high replayability like Mario Kart or Civilization. Mario Kart is designed where a noob comes in and needs a low setting like 50cc, but they'll keep playing and eventually need a 150cc-200cc setting. Difficulty settings objectively make a game like Mario Kart better because it adds a lot of replayability. Even if the Rainbow Road track is objectively harder than the first track across all settings, we still get a lot more replayability from the difficulty settings.
2) Narrative focused (non-RPG) games like The Last of Us. I don't remember if TLoU has setting options at the beginning, but it is the type of game that benefits a lot from them. Some people just want an interactive story and they need an easy setting. Some want something of a challenge but don't play a ton of shooter games and need an average setting. Others play a ton of shooters and need a hard mode. Everyone wants to experience the story, so the game needs to account for different skill levels.
I have a feeling the OP is not really referring to these types of games though. I don't think the OP is talking about competitve multiplayer games either, even though these games don't need a difficulty setting, because the other players provide the difficulty. Instead I think the OP is talking about single player games (or the single player portion) where there is a lot of content, and not a heavy narrative, so the average person will play it only once and they are doing it mostly for the gameplay and overall experience and not for a story. Does a difficulty setting make this game better or worse?
The simple answer is that a difficulty setting at the beginning of the game makes this game worse. First of all, how does a player even know if "normal mode" is right for them? I've encountered games where the normal mode is actually pretty hard, and in other games hard mode still feels kind of easy. Players aren't mind readers. We don't know what the devs think is hard. On top of this, the devs resources are better spent making a higher quality, tailor-made, experience for the players who really like their game. There are so many long, single-player games out there and most of them are forgettable. The way to make a game stand out is to craft the best possible, tailor-made, experience for the type of players that really want that type of game to begin with. The game will be memorable for these players. We know that in clothing, tailor-made means high quality. It is the exact opposite of one-size-fits-all. It is the same with games.
However, making a game hard just for people who like hard games is not good design either. The better way is to design the environment to be hard, but then give the players the tools to both overcome the challenge and adjust the difficulty inside of the game. See, everyone wants to feel powerful and kickass and awesome. Games make a person feel powerful when they overcome a challenge that is tough for them. But everyone is at a different skill level. What is just right for one person is too hard for a second and too easy for a third. Games that are too easy are boring. Games that are too hard are frustrating. A well designed game finds ways to adapt the difficulty to the player, or more precisely, let the player choose the difficulty through gameplay mechanics.
How can it do this? There are actually a ton of tools that developers can use to adapt the difficulty. For example, in Mario games, they often give an easy path and a harder path, but they put better power ups along the harder path. Letting the player choose the path lets them choose the difficulty. Also there are things every game should do like make the controls and UI intuitive.
However, the devs can give players a ton of tools to adjust difficulty simply by making the game an RPG or by giving it RPG elements. Think about Ninja Gaiden vs. Dark Souls. Both series are known for being hard. Ninja Gaiden has been around since the NES days and it has always been hard either as a 2D game or a 3D game. It never had the breakout success that Dark Souls had, and it isn't an RPG. Dark Souls lets you level up. Whenever you level up, the game becomes a little easier. Dark Souls lets you choose your equipment and your build: Knight, Caster, Archer, etc.... These are common things in RPGs. Dark Souls even goes the extra mile and lets you get PC or NPC help for boss fights. All of this lets the player choose the level of difficulty, and they do it in game. This approach is a lot more nuanced than a simple Easy, Normal, Hard mode at the beginning. There are a lot more than 3 settings this way, and because it happens in-game, the player has a good idea of the kind of difficulty they can handle. Most importantly, when the player chooses these things in game they don't even have to know they are adjusting the difficulty. They can just play the game and not worry about it.
So, in the end, the best single player games are tailor-made to the types of players that like them. However, when making a game tailor-made, the devs can give the players tools so they choose their own difficulty level while they play. When players do this they make informed decisions about the level of difficulty they want, while a difficulty setting at the beginning is not an informed decision. RPG games or games with RPG elements especially give players a lot of tools, because these types of games have leveling up and equipment options. When a player is allowed to choose their difficulty in-game then they actually have the best tailor-made challenge possible.
Essay Pro