I think the problem is your downloading in the background, but still doing things online. If you download in the background, but then continue to browse the PSN, I dont think it continues to download...it basically waits until you quit using the connection to do something online. So as soon as you go to the XMB it starts downloading again.
Owner of PS4 Pro, Xbox One, Switch, PS Vita, and 3DS