The difference is huge IMO, and it's definitely better than 3D in cinemas if I'm being honest. I was watching a Jurassic Park 3D clip where the T-Rex comes out of the paddock and it really does feel like it's right in front of you and that was a low res clip too (only 1080p). 

There were other clips I tried of 3D video on Youtube that are astounding, one where the camera is moving down a busy street of partiers and stuff and it feels like people are genuinely walking right up to you. The first time I saw it I reflexively moved my head back because it felt so real. 

People also rave on the Vision Pro about things like that Alicia Keys demo and the sports demos (NBA basketball, soccer, etc.), it feels like you are really there, so much so that it feels a tad bit unnerving that you're in a person's "space" like that. 

I'm glad it works for you. However those "getting things close in front of you" is what I call gimmicky and is what gives me a headache :/ That's when I have to take the glasses off in the cinema when things get thrown right in your face.

It's weird, since I do get up close in VR to check things out and there it's fine. The difference is I can look around the object, shift perspective and position. With video you can't, so I'm guessing that perspective being 'stuck' in front of you is what gives me a headache. And for me kills the feeling of being there. Like you said, you reflexively moved you head back, but it has no effect in 3D video. That disconnect turns it into "screen stuck to your face" feeling. I also can't stand it when VR menus move along with you, fixed to your face. Luckily that's mostly a thing of the past, now mostly happens when tracking fails.

I guess everyone is different. My mind must react more to parallax cues rather than stereoscopic. VR still works with one eye, 3D videos don't. Wiggle 3D looks very real to me. More so than pure stereoscopic images. Maybe I messed up my stereoscopic processing by staring at too many stereograms in the 90s lol.

More stuff like the Joshua Bell experience would be awesome, but it's a lot of work to make, plus a lot of local processing power. Volumetric video might be a better way to go for VR videos.

VR is the future. Though I think it will be about twenty years before it becomes mainstream.