It's not like 50 million XBox One's is nothing ... that's a larger userbase than what the SNES had. That's a healthy enough userbase size especially for MS which doesn't depend on the game industry to begin with.
If XBox Series S/X can't get to XBox One levels, then the problem is Phil Spencer and the team managing the current XBox hardware division, they are very obviously not getting the job done and need to be replaced.