It is a failure compared to the competition deffinetly, and MS has done little to stop the steady decline in sales. However, if it is turning MS a profit, then no it is not a failure. It has sold well all things considered. My main concern is the lack of exclusives leading many to just go with a PS4/Switch combo, as this will get you access the most games along with a wider range of features. There is almost no reason to own an X1 if you have a PS4 or PC at this point.
So no, it is not a total failure. However the future looks bleak unless they make some changes when it comes to exclusive games.