You remember wrong, that has never been a thing. It has always been either physical copies shipped + digital sales, or in case of especially big software titles, sell-through to customers.

The only caveat with Nintendo software sales is that titles need to have shipped 1m+ copies in any given fiscal year to get reported, or happen to rank among the top 10 on the given console which is a ranking that is popularly displayed on Nintendo's corporate website; these top 10 will have a visible update every quarter.

This Metroid Prime faces the challenge of launching at the end of the fiscal year, so it's possible that it falls short of 1m and then proceeds to also fall short of shipping 1m+ during the upcoming April to March timeframe. Meaning that this could become a million seller without ever being confirmed as such in a financial report by Nintendo. However, that still leaves the possibility of a mention in the CESA White Paper where Nintendo reports shipment figures of games that haven't made the cut for the financial report in a long time; that's how we know that some smaller IPs on Switch have cleared the thresholds of 2m or 3m in LTD shipments.

 And up until mid last gen CoD games were using a heavily modified engine from 1999.

 I'm glad they didn't because I never wanted inferior versions of those games be the only re-releases like SEGA has with Sonic Adventure. They keep releasing the worse Gamecube version in HD releases. Wii Prime games aside from controls were worse versions. Water physics removed. Particle effects missing. Heat disapation effect from Samus canon missing. MP3 had dialogue censored. ugh.  At least with this version not only was that stuff brought back but better looking than ever.

There are tons of Switch games that don't look great. - Pokemon for example.

The limitations of the Switch games visual make-up tends to become very readily apparent on my 85" 4k TV, more often than not.

That's not to say there aren't games that haven't been handled well... Links Awakening I think has absolutely charming visuals due to the Gameboy-centric art style and material shaders that were put to good use.
But... The low resolution of Switch games tends to hide those smaller details, which is extremely unfortunate.

The game would handle very differently due to different middleware being employed.

An engine does place some technical "limits" on how a game will present, for example.... The Half Life 2 engine for example has hard limits of 10,000 polygons/model, 17,433 vertices, 4096 texture size and 128 bones. - So those models tend to be presented in a certain way to fit inside those constraints... Which are also often dictated by the hardware that it's being run on.

Same goes for how lighting/shadowing and effects are presented... idtech 4 for instance leveraged stencil shadows, it was an engine feature, so shadows felt dynamic, but also extremely contrasty verses baked or shadow maps we get in games today, which gives games on that engine a very unique look.

The game is clearly still presented on the same technical base as it's original release, but you can tell they have made some massive improvements to the engine... They would have likely borrowed technology from their later engines and backported.

It's a smart way of doing it, it reduces development time significantly... Whilst not changing gameplay behaviour to a massive degree... And you still get a big improvement in visuals.

CoD Games were based on id Tech 3... Their latest engine IW 9.0 still has a tiny bit of code from decades prior. - Engine has been rebuilt and overhaul multiple times over the years... And this is actually a common practice.
Epic does it with Unreal, Valve does it with their engines, Bethesda with Creation still has code from Net Immerse.

No point throwing away the entire wheel if you don't need to.

Apparently it's still the same engine today as it was for Metroid Prime or Donkey Kong Tropical Freeze, and probably the same for Metroid Prime 4

 OG Trilogy used Unreal 2. MP4 isn't going to do that. If it's not in house then UE4 is likely.

I remember a recent interview with some ex-Retro devs who talked about how impressive their in-house engine RUDE (Retro Univeral Design Engine) was, and the results certainly speak for themselves; just as they produced some of the best graphics on Gamecube and Wii, Remastered now does the same on Switch.

Engines are "modular" and are actually a group of different "engines" working together. I.E. Physics Engine, Audio Engine etc'. - So they can use the old engine and backport features from newer engines, it happens in the engine industry all the time for remasters.

I think this is probably just because it uses many modern rendering techniques in a way we're not used to seeing on Switch, as most games actually built for the hardware are more cartoonish and stylised.

UE4 is the most common engine used in realistic/semi-realistic modern games, so we tend to associate a lot of these techniques with it.

I'm the opposite, I really don't like remakes that change the original dramatically, always disliked it in movies and am not a fan of games adopting that trend too. It just creates a mess, is my problem with it. Like as someone who's never played a Resident Evil game, RE2 seems like the most appealing one, but both the original and the remake are highly regarded by fans, so which one should I play? Normally I'd go with the newer version, but in this case it's weird because the new one is basically a different game altogether, so maybe I should just go with the original, but then I'm missing out on the nicer presentation and quality of life. So I guess I have to play both of them because they're different games, but they're not different games, they literally have the same name. It's a confused name at that point. I'd rather each game be what it is, if it's gonna be remade, it's still the same but better. Something we already have in a nicer packaging, like you put it. Besides, if you're gonna go through the whole trouble of making essentially a new game, you might as well make it actually an original game instead of a remake.

The ideal remake/remaster for me is one that completely preserves the essence of the original, but with improved presentation, and additional options or quality of life features. Occasionally some gameplay changes can be okay, but only if really necessary, to improve things that needed improving. Metroid Prime Remastered hits that golden standard for me (outside of a few nitpicks), I'm really happy with it, and while I definitely want Prime 4 news and hopefully a release not too far away, I'd love to see Retro remaster the rest of the original trilogy with this level of care.