Despite Sega bragging about doing what Nintendon't (which started by Mega Drive beating SNES to the market), they are indeed perhaps the closest to Nintendo stylistically. And a lot of Sega's cartoony franchises (Sonic, Super Monkey Ball, etc) have found a home on Nintendo either as timed exclusives or true exclusives. 

The closest to Nintendo in financial success? I feel like only PlayStation is on the table. Yes, PS3 lost them ungodly amounts of money. But PS4 and PS5 have made up for it. Xbox makes a lot of money, but a lot of gaming money at Microsoft is Microsoft Gaming across the board, not purely Xbox branding. 

In terms or rivalries as a whole to sum it up more

Nintendo vs. Sega - Similar

PlayStation vs. Xbox - Similar

In fact, far and away the two most prominent "console wars" in history are from those duos. Super Famicom/SNES vs. Sega Mega Drive/Genesis.

And Xbox 360 vs. PS3. 

Yeah pretty much, Sega was to Nintendo what Xbox was to PS, the direct competitor that tries to be the more "cool/hardcore" alternative. (At least in the 6th and 7th gen, modern Xbox feels rather toothless by comparison)

The 16-bit console war was iconic; PS3 vs 360 was a close fight, though in the end both got beaten by the Wii.

A cool side note: 360 beat the Wii that gen in the US. Probably Microsoft's only real sales accomplishment worth bragging about.

Over the past 3m, I've only had time to play: Astro Bot, Zelda EoW, and the remainder of my Zelda TotK 100% completionist run (~25hr). I'm desperately searching for any excuse to use my XBSX lol. Alan Wake, Indiana Jones, Halo CE, or Rare Remix: One of these will be the first games I ever play on an Xbox system.

Also, a higher software sales total, tie-in ratio, and a killing in online subscriptions and transactions. I honestly wonder how awesome Sega's online would've been if Dreamcast survived through that gen and they had a seventh-generation console. 

PS3 won by hardware units sold and it's perfectly rational to argue PS3 had the better library (exclusives and games not on Xbox) but financially Xbox 360 seemed to beat PS3. But Nintendo ate popcorn and laughed, only to fail with Wii U in the next generation. 

Sega. Better than Xbox (and Sony).
And the best Sega era was 1995-2003.

During 2000 until 2003 its internal japanese studios converted to full companies owned by SEGA, and all of them were just amazing. After 2003, being SEGA full third party, were reabsorbed as studios, fused and even dissolved cause economical reasons.

The Magnificent Ten (2000-2003)

1 Wow Entertainment (old AM1)

2 AM2 (always conserved its original name AM2. In the 80s was Studio 128, a special team inside AM1)

3 Hitmaker (old AM3)

4 Amusement Vision (old AM12, an scission from AM3)

5 Sega Rosso (old AM11 aka AM Annex, an scission from AM2)

6 Smilebit (old CS1)

7 Overworks (old CS2)

8 Sonic Team (old CS3)

9 United Game Artists (an scission from Amusement Vision, old AM12)

and finally

10 Wave Master (SEGA's sound studio, also developed some games)

AM# meaning Amusement (Arcade games developer)

CS# meaning Consumer (Domestic games developer)

Originally AM studios only developed for Arcade boards (some did some things in Mega Drive) and CS studios were much less popular (less famous) and only worked for Sega consoles (as I can recall now). Saturn started to erase those roles, and AM1, AM2, AM3 and AM Annex also developed for Saturn, doing ports of its own arcade games to Saturn. The situation became totally mixed when they convert to full companies, doing basically whatever they wanted. 

That's tough. Atari I haven't experienced much besides compillations but with Lynx/Jaguar the few from Atari 50 was a good experience.

For Sega/Xbox though hmm not sure. Gimmicks wise Sega has it there for sure with Dreamcast VMU or Sega CD and 32X, regardless of good or bad they are options.

For games it's tough, I see a lot of good Xbox OG games and go yeah they are interesting. While Sega a lot of unresearched potential I'd say it has over it. Really hard.

I consider niche games more than obvious IPs as to me the deeper it goes the better you know ok this had a lot going for it.

Like to me seeing the not talked about as much games or ones I missed Xbox OG/360 is like wow, I really like what's offered here besides the obvious where they are good but eh, pretty obviously good but I don't care as much anymore for some of them of IPs.

That and if it's just off the obvious games and very few, that's not a strong argument I think, the lesser stuff I think has enough weight to determine that and I don't go off nostalgia, even then I didn't experience much of the OG Xbox, did more 360 and even then I still have yet to get around to like any gen tons of IPs I missed, some obvious ones I've only gotten around to later.

While I haven't experienced a single Sega console only compilations there too but am always impressed with what Sega did with games and hardware despite that.

While the exclusive 1st/3rd parties for them mostly 3rd parties I'm like wow these are excellent.

If by gimmicks yeah Sega, the memory cards and white/black buttons, SmartGlass (remember that) or impulse triggers are cool of Xbox over the different gens but not much. Even if not counting VR/other aspects Sega/Atari had and never made happen still would put them over with those if they did release and flopped because they would be cool still in some way regardless in my eyes, Virtual Boy is, Wii U is, the potential and what did happen was still compelling to me. Even some of the worst consoles by any companies still had appeal I think of hardware/games execution.

Saturn/Dreamcast clearly have interesting libraries. Genesis is obvious of what's there besides a lot probably I have yet to uncover. Master System/Game Gear sure.

Sega has a wider range of IPs though hmm.

I'd probably say Sega even besides the potential I don't know of besides what I've researched.

I would say go with Indiana Jones. Nothing like playing a new system with a new game. You should at least set it up and get some games downloaded so you can jump in when you're ready lol. Master Chief Collection should be soon after so you get some nastagia of Halo greatness.