Not only power, cooling as well. My laptop struggles with heat management when using the gpu. A phone is what, 1/10th the size? And my laptop also only lasts for an hour while playing a game on battery.
I'm not sure if you want to use a glowing hot phone in front of your face for VR :)