I know my opinion is far removed from that of the average fan, so I don't think I should speculate as regards what other people appreciate about the series.
I don't think the Metroid 'essence' inheres in the core concept as such - that of a solitary bounty hunter fighting space monsters in the catacombs of an alien world - nor do I think it stems from the general structure of exploration, combat and reward. Rather, when I think of Metroid, I think of a certain sense of dread, solitude and wonder; in other words, an atmosphere.
I think I can exemplify this with Metroid Prime.
What I remember most fondly about Metroid Prime is not the controls, the combat or the plot, etc. What I remember, is the exploration of new locales, not knowing what I would unearth next. Dread, solitude, and wonder. I can't fault any aspect of this game's design, but the three 'world agents' - the Chozo, the Phazon mutagen, and the Space Pirates - I consider quintessential to the game's atmosphere. What they contribute to gameplay, locales and lore cannot be understated, nor can the way they represent different perspectives of the game's events. The sense that 'something is wrong with the world' grows into a struggle between a dying world, a broken empire, a foul poison, and those who would take it all and use it to their own ends... Live through that struggle, and live through Metroid Prime.
Every Metroid game embodies a similar experience. The games build atmosphere through the world and your interactions with it. This is the purpose of the Chozo. This is the purpose of the dead space marine. This is the purpose of that not-so-tiny little larvae that got itself killed. etc.
That, at least, is how I see it.
