It's not just the Kinect. The xbone APU includes CPU, GPU & ESRAM, all in a single massive die. As die sizes increase, fabrication yields drop, making the cost of manufacturing exponentially higher.
Microsoft made a terribly poor choice from the start, by choosing DDR3 memory not just for system, but for graphics use as well. In order to deal with the bandwidth bottlenecks caused by slow memory, they had to implement some sort of on-die solution. Namely, the ESRAM. But if you increase your die too much, you will then bleed money from low yields. And that is the price you pay, when you make poor design decisions.
To summarize, i am 100% certain that the PS4 has a lower bill of materials than the Xbone, even without including the Kinect. The Xbone is so huge for a reason. That ginormous APU requires massive cooling.