By using this site, you agree to our Privacy Policy and our Terms of Use. Close

Here is one idea of how a Natal UI could work.

Instead of an interface where there are no graphical displays and control would be done through gestures alone, there could be onscreen buttons that are "clicked".  With both hands you would make a sweeping upwards gesture to bring up the interface, which would contain the buttons corresponding to whatever it is that needs interacting with.  For example a movie interface, would have buttons for forward, pause, rewind, etc. 

Once the hands are swept up, the interface comes on screen.  Natal cannot tell where on the screen you are pointing, so there will be an onscreen effect(a swirling effect, or perhaps a hand) to represent where your hands are so you can move to the buttons.  The way the effect is placed on screen would be determining the hand position relative to the shoulders and extrapolate that over the resolution of the display device.  This way, the person is interacting with the screen relative to themselves and not the screen directly.  IE, the button is on the bottom left of the screen, the person would move their hand down to the bottom left of their body no matter where they are sitting.  This could also be calculated for people sitting in rotations less than 90 degrees from Natal.  People could relate to the effect on screen and quickly interact with whatever it is that needs interacting with.  This will of course make more sense to your brain if your body is at least mostly facing the screen.

The "click" could be performed by moving the hand towards the screen and back twice in succession.  Much like punching, or tapping the button.  Because Natal can detect depth quite easily, the movement does not have to be very much, but in order avoid false positives, a decent amount of movement would probably provide a better experience.

I am not saying a graphic less UI would not work with Natal, but that there is more than one way to do this.  This method could even work for several games.  Imagine a mech warrior type game where you are in a cockpit and all the controls are represented on screen.  You would maneuver by grabbing some joysticks relative to your body.  Push and pull levers to speed up/slow down.  That would be feasible using this method.  So you would have a 1:1 relationship with this virtual cockpit.