Unfortunately that's what happens when you don't keep up with the competition on first-party output. Not to mention major issues with your, at the time, biggest IP.
With so many acquisitions in recent years, Xbox still hasn't been able to find their identity or footing really so far this generation. There's been some smaller gems, but they're betting heavily on Starfield to be the title that defines Xbox though and lays the groundwork for their future AAA titles.
I can't imagine a drop that radical, if it is the case, is from that alone though. But I wonder what could have happened over the past 4 years or so if MS had focused on restructuring their studios instead of acquiring new ones. There's no denying that the quality and speed for their developers is subpar and most of them need some kind of overhaul. Other than the consistent quality and output of Forza content it's been bad. Their studios are bloated and have been poorly managed.
343 should never have been allowed to balloon to over 750 employees, then they had the help of 5/6 support studios and Halo Infinite still didn't come out in a timely manner or in a complete state when a game like that should have been achievable with a much smaller studio. Halo Infinite really isn't that ambitious of a title, it's not a sprawling 100 hour open world game.
Microsoft's studios are really not living up to their potential and I'm sure the blame lies fairly high up on the chain.
As nice as it sounds that Xbox is very hands-off when it comes to intervening their studios, it clearly doesn't work as they hope. Half the studios operate just fine whereas the other half struggle.
You never hear issues with Double Fine, inXile, Obsidian, World's Edge, Turn 10, or Compulsion, but there's obvious issues with 343, The Initiative, and Undead Labs that only in more recent times has Xbox tried to resolve. 343 has been completely gutted as a studio finally and good leadership has been put in place, but that's what I meant that Xbox hasn't really found their identity or footing. Some studios are fine, some aren't, it's not unified in the way you would hope, and rather than focusing on those problems to unify the Xbox platform, they're working on a $70 billion acquisition lol.