Kinect could have been fine on Xbox One but it was handled incredibly poorly.
Kinect launched at $149 standalone or $299 for 360+Kinect in Nov 2010 and by holiday 2011 these had deals for $99/$199.
Kinect sold 24 million in a little over 2 years, these people who purchased a Kinect/Kinect bundle for $99-299 from Nov 2010-Feb 2013 were not the same demographic to buy a $499 console.
It was too soon and too expensive for the Kinect audience to move on.
If the Xbox One launched for $399 without a Kinect and kept the $499 bundle with Kinect the Xbox One would have had a stronger start in its early years and overall could have increased its lifetime sales. Would it be at PS4 numbers? Not a chance, but it could have meant more exclusives and LT now at over 50 million. Microsoft messed up big time at the launch for the Xbox One with its mixed messages.