It's not quite holodeck technology, but it's not far off. A patent filed back in March by Microsoft describes advanced display technology that extends the gaming environment from the television to all four walls of a room.

Imagine playing Call of Duty or Gears of War and being able to see an enemy approaching from your peripheral vision. How about being able to look over your shoulder to see how close your competitors are in a racing game? Microsoft's patent, simply titled Immersive Display Experience, is geared towards making that happen.

The patent, discovered at Patently Apple, lays out technology that's far more advanced than simply projecting an image around the room.

The tech revolves around a device called the Environmental Display (the webcam-looking device atop the television in the picture), an RGB projector that handles the task of extending the display beyond the television, wrapping the gamer's room in graphics. A depth camera, integrated into the peripheral, could be used to help map the user's environment, with the projected image corrected to compensate for furniture and other objects — instead of seeing a chair painted with light, the projected image would adjust in order to help your furniture blend in.

That sounds excitingly dangerous, doesn't it?

Meanwhile the Kinect, also seen atop the TV in the image, could be used to track a user's movements. Eyes could be tracked so that projected areas would come into focus depending on where the user is looking. The Kinect would also be used to determine where the user is standing so the projector can avoid hitting them in the face with a bright beam of light, which is probably an important thing.

The patent even suggests that the player would be able to enjoy a truly immersive 3D experience by utilizing active shutter glasses that synchronize with both the television and Environment Display.

It's an incredibly ambitious technology that could change the way people play video games. It also sounds like it would be relatively worthless in a well-lit room, but who games in a well-lit room anyway?

Kotaku via Patently Apple via US Patent Office

Very interesting. This seems like it would benefit Kinect games a lot. Espcially in immersion. Imagine Resident Evil 6 with that.

I can't decide if this replaces Project Fortazela, if this is a component of it, or if the two will be used to provide a completely immersive experience. Much of what Project Fortazela was supposed to be capable of (based on the images) the Immersive Display Experience does. But one aspect of Project Fortazela was the capability of projecting within the field of view a 3D image, thereby augmenting the environment.

To provide an understanding of what I mean. If we're considering a Sesame Street interactive game, where Elmo is standing next to a child. Currently with Kinect, there is no means of knowing that, without some way of looking to the left or right. With the Immersive Display Environment, the room may appear similar to the environment on the screeen, but if Elmo is anywhere it's off in the distance on the wall. To truly make the individual feel like Elmo is right there beside him or her, you need a means of projecting an object into your field of view. That requires glasses.

My guess is that the Immersive Display Environment isn't 100% complete yet and there is still one more aspect to it to come.

Not just Kinect games, though they would definitely benefit.  But in a shooter you'd have a 360 field of view around you, rather than having to "look" you just move your head.  Regardless of whether you're sitting or standing, it would provide a gamer with a real-world environment.  It would, I think completely change online gameplay.

This would potentially be amazing. I prefer to sit and relax the majority of the time when I game bit having a semi vurtual reality option would be amazing if done right

Consider it for a game like Forza or Ace Combat. Instead of having a limited field of view, you would get a 360° view of the entire landscape. Not to mention, doing so without having to wear anything on the head. So, not only can you experience it, but everyone else in the room can. Or how about a stadium game like FIFA or Madden NFL.

The one challenge for this is a room with any dark colored walls. Ideally the walls would be white or off white.

I'd rather have a 3D personal viewer with sixaxis using move with motion cameras. Who needs a TV or image projection on the room when you have 3D HD 5.1 surround sound on a virtually 750" screen with 360 degree range of gameplay. Using move you can have the analog stick control for character movement and motion based gestures together.

I came up with that idea months ago and if that is the way gaming is going than this is how I want it to be done. Downsides being rather single player at home while Microsoft's supposed set up could be multiplayer, but projection and gesture controls only don't sound all that great.

this would be a huge 'gimmick' on the level of Wii's motion controls. Not knocking it. I love Wii's controls. But like them, this would probably add a lot to some games and nothing to others but would be something everyone would want to experience.

As cool as it looks, technology like this is still years off. It would be far too expensive for your average person anyway.

Yes, because Microsoft was incapable of taking technology that was still years off, and far too expensive for the average person and selling it for $150 with Kinect.

The basic technology involved here is a projector capable of displaying a 360° image.  Hardly a lot of sophisticated technology.