Have you checked the HDMI Deep Colour option on your TV while connecting your PS4 via the breakout box?
Double check and make sure that option is set to on.
This worked, thank you, you're a legend. As long as the HMD is powered off, I have 4k and HDR working simultaneously. You saved me much time and hassle as I was about to give up and just swap cables to get around it. Thanks again :).