Pricing and available software. There's only a handful of decent titles for VR use and the prices are ridiculous. Not only the sets themselves, but also the hardware requirements for the kit. The investment is quite simply not worth it, not even close. And by the looks of it, it won't be for a very long time.
HL: Alyx looks amazing, but no one title could spur me to purchase a VR set. I don't think even ten titles of that quality would.