These are the 3 big things that really matter for every game: 1) Freedom, 2) Challenge, and 3) Content (i.e. what the game is about, and the stuff that it has). Minecraft is a very basic game made by essentially one person and yet it is the best selling game of the 21st century, because it gives people people a whole lot of options about how they want to play (freedom), there is plenty of stuff that can kill you (challenge) and it is in a huge world with lots of different things in it (lots of content). These are the things that actually make a game fun (and also the type of content which is determined by a person's tastes).
So, the reasons why I like the first Metroid more is 1) it's less linear and there is less gatekeeping (more freedom). There are a few powerups in there that you don't even need to beat the game. 2) It's more challenging. I have died a whole lot in Metroid 1. I'm not sure if I ever died in Super Metroid. Instead I played for several hours and then gave up out of boredom. 3) Metroid 1 feels more like a horror game in space to me. That is what I think the game is about (content). A large part of that has to do with the challenge. I think effective horror games kill you a lot. I also like horror games that unsettle you a lot (in a kind of Hitchcockian way). The lack of a map is a plus. The cloning rooms that confuse you are a plus. Slowly running out of ammo and health is a plus. (Metroid 1 probably could have 1 refilling station somewhere, but Super Metroid has far too many.) All of this stuff makes me feel the pressure and puts me on edge. It's disorienting and alienating in a good way.
Metroid 1 is a hardcore game by NES standards. That was the NES version of Dark Souls. It's hard. You will die a lot, and it doesn't hold your hand. I personally like all of that. If you take Dark Souls and make it easier, then it really isn't Dark Souls anymore. That is what they did to the Metroid series. It doesn't feel like Metroid anymore. Super Metroid feels like a watered down and casualized version of the original game.
Great way to start your point. "Here are the three things that matter in games, as proven by these sales numbers". As if implying that those criteria is used by everyone. Speak for yourself.
1. It's nice that Metroid 1 gives players a lot of freedom, but when that freedom consists only of going through a dozen rooms that look the exact same and unable to find anything because the only way forward is one hidden bombable passage in one of the copy-pasted rooms, it feels more limiting than anything. "There are a few power-ups in there that you don't even need to beat the game" is cool, and exists in most Metroid games, but the way it's done in Metroid 1 is also really poor - the most notable example being the Wave Beam, an "upgrade" which removes your Ice Beam, which itself is crucial for all the boss battles and for killing Metroids at the end of the game. In order to get the Ice Beam back, you have to go all the way back to the place you first acquired it, which already was quite hidden. Getting an upgrade should be a good thing, instead finding the Wave Beam in Metroid 1 only gives you problems.
2. Challenge is cool, but it's a lot less cool when your game is filled with cheap stuff such as, but not limited to: pits of lava that you can't get out of once you fall inside (also one of these pits is actually fake and going through it is mandatory haha); enemies hit you during a door animation you can't do anything in, including one such enemy in the final area which always hits you there; bosses throw loads of hitboxes out at once and you have no i-frames upon getting hit, meaning you might take a shitton of damage for what looked like only one hit; dying respawns you super far away from where you were, and always with 30 energy, meaning if you want to have any chance at surviving, you're gonna have to spend the next 15 minutes grinding for energy. Fun.
3. Personally I think the horror in Metroid 1 is a lot worse than it is at most later games. The atmosphere is much weaker, not in the least part because the game starts with *generic catchy videogame song*, but also because there are no backgrounds, there is no sense of place (especially with everything being copy-paste rooms), there is nothing foreshadowed and thus nothing to be afraid of. The only place in the game that feels truly dangerous is Tourian, everything else before that is completely unthreatening, even the two main bosses. Yeah you die a lot, but the whole cheapness and clunkiness of everything makes you always feel like your deaths are because the game sucks rather than because there were frightening enemies or whatever. The enemies in the game are annoying at worst, but really most of them just look, sound, and move pathetically. They only pose a challenge because Samus is also pathetic in this game, and incapable of performing basic human motions such as crouching and turning your arms.
I do agree with you that challenge is important for good horror in a videogame, but if challenge is the only thing you have going for it, and it's cheap kind of challenge, then it's not good enough. I do suppose Metroid 1 has a really strong atmosphere when compared to other games of its era, but when compared to Super Metroid and beyond, not even close. Super Metroid starts off by throwing me into a space station that's been attacked by an unknown enemy, when you get there you see everyone dead and the Metroid from the prologue gone. Then we see Ridley is the cause, and have to fight him but we're no match for him. Then we descend into Zebes knowing that we'll have to face him, and start going into the caverns under the storm, everything is dark and feels dead up until the point you get the Morph Ball, and then everything lights up, you have cameras looking at you, you feel like you're being watched and it's because you are. Suddenly, the space pirates are everywhere. That is some great build-up, the tension is palpable, and there are lots of moments like that throughout the game. Again, I agree it would be more impactful if the game had more challenge, and this is why Fusion has the best horror in the series. But it's already leagues ahead of whatever Metroid 1 was trying to do.
Also, lol at "the entirety of the Metroid franchise does not feel like Metroid anymore". Come on dude.