New Scientist has an interesting article about NATAL here http://www.newscientist.com/article/mg20527426.800-microsofts-bodysensing-buttonbusting-controller.html
A LONG-lived videogaming skill could be on the way out this year as Microsoft hones an add-on to its Xbox 360 console aimed at making button-studded games controllers obsolete. The device, called Natal after the city in northern Brazil, allows players to control a game using only their body movements and voice. Robbie Bach, Microsoft's president of entertainment and devices, announced at the CES show in Las Vegas, Arizona, this week that Natal would go on sale in November.
Microsoft unveiled Natal in June 2009 at the E3 games industry expo in Los Angeles, but revealed little about how it works. Now the company has allowed New Scientist access to the device and its creators to discover more details.
A player standing anywhere between 0.8 and 4 metres from Natal is illuminated with infrared light. A monochrome video camera records how much of that light they reflect, using the brightness of the signal to approximate their distance from the device and capture their movements in 3D.
This means Natal doesn't require users to wear markers on their body - unlike the technology used by movie studios to animate CGI figures.
Motion capture normally requires massive processing power, and paring down the software to run on an everyday games console was a serious challenge, says Natal's lead developer, Alex Kipman. "Natal has to work on the existing hardware without taking too much hardware processing away from the games developers."
Microsoft collected "terabytes" of data of people in poses likely to crop up during game play, both in motion capture studios and their own homes. Frames from the home videos were manually labelled to identify key body parts, and the data was then fed into "expert system" software running on a powerful cluster of computers. The result was a 50-megabyte software package that can recognise 31 different body parts in any video frame.
"When we train this 'brain' we are telling it: this is the head, this is the shoulder. And we're doing that over millions of frames," says Kipman. "When it sees a new image it can tell you the probability it's seeing a certain body part based on that historical information."
Natal also includes software that has a basic understanding of human anatomy. Using its knowledge that, for example, hands are connected to arms, which are attached to shoulders, it can refine its guesses about body pose to recognise where body parts are even when they are hidden from Natal's camera.
"It correctly positions your hand even if it's held behind your back," Kipman says. "It knows the hand can only be in one place." That's important because during multiplayer games there won't always be a clear view of both players at all times.
He says Natal consumes just 10 to 15 per cent of the Xbox's computing resources and it can recognise any pose in just 10 milliseconds. It needs only 160 milliseconds to latch on to the body shape of a new user stepping in front of it.
The system locates body parts to within a 4-centimetre cube, says Kipman. That's far less precise than lab-based systems or the millimetre precision of Hollywood motion capture. But Douglas Lanman, who works on markerless 3D interaction at Brown University in Providence, Rhode Island, and is not involved with Natal, says that this will likely be accurate enough for gamers.
Lanman is watching closely to see what kind of games Natal makes possible, and how they are received. "Will users find them as compelling as they found the Wii games? Is it important to have physical buttons? We'll know soon."
Those kind of questions, and a desire to move away from the controller-focused interaction that has dominated for decades, are central to Natal, Kipman says. "We think input using existing controllers is the barrier, and by erasing that we can realistically say: all you need to play is life experience."
Some interesting stuff here.
Essay Pro