It could be the fan. I have a PS4 1200 series that makes a high pitch whine when the fan is running at really low RPM's and it's intermittent. When the fan picks up just a little bit, it goes dead silent, and when it picks up a little more, I start to hear the lower deeper 'whoosh' sound instead. This didn't start until a month, maybe two after I bought it. My buddy has the same PS4 and his does the same thing, but his whine is more quiet and less high pitched. I've put my ear right up to the fan before as it's speeding up and down and I only ever hear the whine when the fan is at really low RPM's. I've seen other online forums with 1200 owners that some have this issue as well, and many have taken theirs back, only to end up with the same thing if it's the same model.
I recently set up my PS4 so it's standing up now, and it's basically fixed the problem. The whine still shows up ever so slightly from time to time, but it's minuscule now. Makes me wonder if it has to do with the fan orientation/bearings/lubrication.

Turn up the volume.

I did put up the volume at first. Getting use to it. PS4 pro fan is louder to be honest, just a different kinda noise.

EricHiggin. Thanks

Every so often once or twice a day my fan, or I at least assume it's the fan will make a noticeable whooshing sound but it only lasts a couple seconds at most. I figured the machine was running hot but when I check it feels fine. Haven't thought anything of it in a bad way but still found it odd

Can't say I have ever heard it the few times I have used my Xbox One X console.

Such occurrences do happen more often with electronics that have higher power draw though.

I know your talking XB1, but the fan brands that go into the consoles have been similar over the years. XB hasn't used blower fans before, so with XB1X having one, it's quite possible they are using one or more of the same brand/type of fan that PS4 is using. I've never had or heard of this problem before the PS4 1200 series, so it's possible it's a potential design flaw with that specific type of fan, or just bad batches. It also could very well be coil whine, especially if you say it happens more so under load and not while the console is idling on the home screen. Since XB1X is XB1X, I would suggest taking it back and trying another one and hope for the best. If it happens again, well at least you tried, and apparently you can live with it. I didn't bother taking mine back knowing the odds were good I would end up with the same issue, so I didn't bother. I'm not sure how wide spread the issue is for XB1X, but if it's small, I'd consider trying another. Just my two cents.

It does seem to be quite common for the X just by searching google and youtube.  I also spoke to a guy at work who has one and he has the same noise.  He returned his first one shortly after getting it and his other unit also does it.  It really seems to be more coil noise than the fan though. Again, it's worst under load but my PS4 Pro is much noisier when playing a game such as Horizon.

Cheers,