Have you checked the HDMI Deep Colour option on your TV while connecting your PS4 via the breakout box?
Double check and make sure that option is set to on.
This worked, thank you, you're a legend. As long as the HMD is powered off, I have 4k and HDR working simultaneously. You saved me much time and hassle as I was about to give up and just swap cables to get around it. Thanks again :).
Everything in the above reply is my opinion, from my own perspective and not representative of reality outside of my own head!
-Android user, please be gentle with critique on my spelling.