Much much much better than most of the xbone tech threads out there, but you're still making some mistakes.
Basically if we factor in everything, the XBONE does have better CPU performance than the PS4. It's clocked faster, has some work offloaded to the audio chip, exct. However you are wrong in your assertion that using the GPU to "bump up" the PS4 CPU calculations "bumps down" the graphics. It's practically impossible to have the GPU working at 100% capacity when doing just graphics. During the rendering there are lots of holes. With clever and not too complicated coding, these holes can be packed with CPU type calculations.