The reason I say Xbox One is because the OG Xbox had to compete with the PS2 which was practically impossible. With Xbox 360 they pretty much did almost everything right. Even the Kinect was a great play even though it's looked down upon nowadays. The only way the Xbox 360 was going to do better is if PlayStation didn't have their 2010-2013 comeback and if Microsoft had a better late Xbox 360 life in 2013-2015 than PS3. I chose Xbox One because it is easily the one where I can point to the many mistakes made without including a "what-if" Playstation did this bad scenario, as I wanted to point out what Xbox could have done better or not do at all. Since Microsoft was neck n neck with PlayStation by the start of the 8th generation the Xbox One was their golden ticket to really nail it in since Playstation at that point was at a make or break point with PS4 with how much they lost on PS3 and the failure of the PS Vita.

Now that doesn't mean the next Xbox will sell as bad as the Series consoles as we've seen comebacks happen multiple times, but with how gaming has shifted from 7th - 8th generation it really was a bad time to lose to Playstation because of digital libraries and microtransactions being tied to your account, and with a lot of casuals going with what they already are familiar with exclusives or without exclusives.

If the PS3 didn't recover as well and the 360 had a better last few years in terms of first party games then it could've gotten to 100m. It'd be up there with the DS, PS2 and Switch in NA and in the 30's in Europe and would've actually outsold it there which would've been an insane feat. The PS3 would've been down in the 60's worldwide and only in the 20's in Europe in this scenario so Xbox would've been in a fantastic position to stay ahead of Playstation in gen 8 if they released a normal follow-up and if they invested hard into first party studios and invested enough in general would've had a real shot at knocking Playstation out in gen 9 and putting Sony in a terrible position as a company.

Reminder that this is "if the competition was the same", so not if Sony/Nintendo fucked up, rather could they have beaten the historical PS3/Wii/PS4/etc

They absolutely could have won the 7th generation; they were already over 80% of the way there. Exactly how they could have pulled it off is arguable, but here are a few thoughts:

• The 360's technical problems were probably the biggest obstacle to its success. The biggest offender was the RRoD, which was largely caused by overheating. This possibly could have been remedied by simply making the launch model larger, letting it cool more easily. The 360 was designed to be slimmer to appeal to the Japanese market, but we know in retrospect that this was a lost cause.
• Alternatively, Microsoft could have simply delayed the 360 a few months until they fixed the console's design issues. Xbox 360 sales were pretty limited before Q4 2006 (thanks to starting production late), so releasing it in Q1 or Q2 2006 would likely not have been disastrous, provided they were able to fix the overheating issue by then. 
• How the heck did Microsoft let go of Bungie immediately after Halo 3 became their biggest game ever? Even if Destiny were their main focus, at least keep Bungie around to help ensure the transition to the 343 era went smoothly. And then, you get to keep Destiny off of PlayStation!
• Later in its life, the Xbox 360 shifted focus to the Kinect. This was not necessarily a bad thing, but the emphasis on selling Kinect came at the cost of cheaper skus for the base hardware. Barring one time they were getting rid of remaining stock, the 360's base price never dropped below $199.99. Contrast this to the PS2, Wii, DS, etc. The Xbox 360, priced at $149 and later $99, could have been an entry level console you got your kid for Christmas from 2012 to 2014, possibly bundled with Minecraft. 

I don't know if this has been confirmed, but X360 apparently won in software.

In hardware, the Wii was too big for the X360 to beat. I don't think it had a real chance, not with a popular PS3 around. The so called "HD twins" limited each other's sales.

Had Microsoft been as aggressive as Sony during the PS2 days in acquiring more third-party agreements (e.g. CoD), and had Microsoft been more successful in marketing the system in JP and even EU, then XB360 could have certainly overtaken the Wii.

Though XBOne was their best shot IMO… if we ignore Switch ofc. They entered the gen riding high off the XB360 momentum, Sony was struggling to output first-party titles for the first couple of years, and the brand had already built (unlike during the XBOG/XB360 generations). But, Microsoft couldn’t keep the energy due to many marketing and software missteps, and as a result PS4 took off and left Xbox in the dust.

XBSXS could have also beaten PS5 had Microsoft remained committed to exclusivity— though that’d admittedly be a little cheap given how much of Microsoft's first-party output is third-party acquisitions.

I think they actually kind of won the 360 generation, it even felt like it at the time too, being a PS3 owner. But Sony had to drop the ball for that to happen. And when the next gen came along, MS fumbled again with their out of touch visions and the status quo was returned.

They definitely beat PS3 globally in many ways. Less loss on hardware and other things, more software sold, Xbox Live profits, etc. PS3 did beat them by a few million in units, but that's a pyrrhic victory when ignoring Xbox One and PS4. And funny enough Xbox 360's victories over PS3 also become pyrrhic when PS4 kicked Xbox One's butt in almost every metric. 

They did lose billions on RROD. 

They could have won the Xbox 360 Gen but after the first three years they stopped having a lot of exclusives and the pace slowed down.