What's the point of that if you're still wearing glasses to see 3D. Sure you can also plaster the walls with tv screens that don't require glasses yet that would be far too expensive for home use.
Lightweight self contained VR/AR glasses are the near future. Eye tracking and foveated rendering will bring the computing requirements down a lot, while mobile chips are getting more and more powerful. The only real bottleneck is battery tech. Still better to wear a portable battery in your pocket than always be tethered to a console or PC.
In a couple decades perhaps holographic projectors will be affordable and something like in the movie Her will become possible.
That would help with motion sickness and the aversion to wearing headsets. The best possible immersion will still be reserved to headsets though.
In the far far future I think a direct brain interface will still be easier than creating a star trek holodeck. At least until we have control of gravity, inertia and matter creation.
What exactly is the appeal of a direct brain interface?
Much easier way to engage all the other senses, as if you are dreaming. I think it will be easier to map and override the inputs to the brain, than it will be to control gravity and innertia and make hard light holograms to build a star trek holodeck.
Perhaps in the far future you can upload your current mindstate, put your own body and mind to sleep and download the new experiences back into your refreshed body in the morning after a night of hanging out in virtual reality. If you still want to go back to reality...