http://www.eurogamer.net/articles/e3-post-natal-discussion-interview
The highlight of Microsoft's E3 press conference - apart from Gepetto Molyneux and his talking boy - was undoubtedly Project Natal. As you'll know if you've read our hands-on preview, it allows you to play games using your whole body and invisible controllers, via a camera that sits under your TV.
So far, so EyeToy - but there's more to Project Natal than that. So the project director, Alex Kipman, told us while demoing how his shiny new technology can be used to play good old Burnout Paradise. Read on to find out more about how it all works and what the future holds.
Eurogamer: Were you already working at Microsoft when you came up with the idea for Project Natal?
Alex Kipman: I've been working at Microsoft for about eight years, so I've been there for some time. I took on the job of creating an incubation team to think about what's next for Xbox. Don [Mattrick] challenged us to think ahead - to think beyond, to be innovative.
We thought about a wide range of things and made a team effort within Xbox to figure out where we want interactive entertainment to go. After quite a collaborative effort we landed on this, and a lot of research and a lot of sweat blended into something I think we're all very proud of.
Eurogamer: How does Project Natal work with Burnout?
Alex Kipman: Essentially we do a 3D body scan of you. We graph 48 joints in your body and then those 48 joints are tracked in real-time, at 30 frames per second. So several for your head, shoulders, elbows, hands, feet...
Say I'm tracking a wrist, which is what I do for Burnout. I can look at that on a single frame and I can see what direction, acceleration and confidence I have for that joint. Why is that interesting? Because it allows me to not only know where you are, but to know where you're going to be. This is how we do the directing and the predictive behaviour.
If you think about swinging a baseball bat, by the time you're halfway done with the swing, I know not only where you're going to end but when you're going to end. There are very precise and predictable ways so you can have that immediate payoff of my baseball bat hitting the baseball.
Eurogamer: So this technology isn't just for use with Arcade titles or specially designed games like Milo & Kate? It could be used to play regular games?
Alex Kipman: Absolutely. We see there being three types of game. We love the [existing] controller, it's not going anywhere and there will continue to be games that are specifically made to only work with a controller. We'll have games that are specifically designed to work only with Natal - not just arcadey games, but real, hardcore, triple-A titles.
Then you'll have some games that are essentially a hybrid - games that work both with the controller and with Natal. Why is that interesting? Think about a first-person shooter where I'm using the controller but I'm doing facial tracking by just moving around and looking round corners.
Or you could have a hardcore gamer like me playing a game with a controller, while a non-hardcore person sitting next to me enjoys the experience by playing with Natal. I could be having my Halo experience with the controller and the friend next to me, who's not a hardcore gamer, could be throwing grenades or driving the Warthog or doing any number of things with Natal.
We can track up to four players in the same way we track controllers. Each individual player will be able to choose - do I want to bind with a controller, or do I want to bind with my body, or do I want to bind with both?
Eurogamer: I can see you've got the camera under the telly here and it looks like the system is connected up to a laptop. But when people are playing with Project Natal at home, where does the magic happen? Is it in the Xbox?
Alex Kipman: The sensor itself has a lot of magic built in. It wouldn't be interesting for us to go to our developers and say, 'Hey, you can create all these brand new, awesome experiences but you need to do a lot of processing outside of the game.'
So we have a custom chip that we put in the sensor itself. The chip we designed with Microsoft will be doing the majority of the processing for you, so as a game designer you can think about the sensor as a normal input device - something that's relatively free for you as a game designer.
Designers have 100 per cent of the resources of the console and this device is just another input device they can use. It's a fancy, cool, awesome device, but essentially you can just treat it from a free-to-platform perspective, because all of the magic - all of the processing - happens sensor-side.
Eurogamer: A lot of the time, when you're playing a racing game you're holding down the A button to make the car go. So with Project Natal, you're sticking your leg out to make the car accelerate instead. But what if you're playing, say, Tomb Raider? Could you use the camera to play that?
Alex Kipman: Burnout is almost a bad example because it's an old game that wasn't designed for Natal. I would say Tomb Raider would need to be designed for Natal from the get-go. I've presented this to most of our third-party developers and from the creator's perspective you start thinking of brand new game mechanics, brand new ways of interacting with the game.
So the Tomb Raider team would come up with all of their different game mechanics and represent them with different Natal experiences. Lara does a lot of headstands, and I wouldn't expect people in their living room to be doing headstands. As a game designer you'd have to come up with a natural gesture to go mounting into headstands.
How can I make a user in their living rooms feel like Lara Croft without being as fit as Lara Croft? Because none of us are! This is the thing that really excites the game designers we've been talking to, both first-party as well as third-parties. They look at this as a brand new set of paint colours and paintbrushes they can use to paint brand new experiences.
Eurogamer: What if you're lazy? Could you play this Burnout demo sitting down?
Alex Kipman: Game designers will have to come up with what is natural. I can tell you several different options I can think of. You could say, hey, do this to accelerate [mimes pushing a steering wheel forwards] or this [pushes his shoulders forwards] or this to brake [pulls backwards].
And remember I'm tracking 48 joints individually, so there are so many combinations. I just gave you a few I thought of off the top of my head, but game designers could come up with anything. For all I care you could use your head to go forward - it's not very natural, but you could use any number of things as a game designer.
We're not making any predictions about the gestures; we think that's very constraining for game designers. We're saying, we're tracking 48 joints per frame in real time - use the combination of those things to create a rich vocabulary of gestures that allows you to create brand new experiences.
By the way, our system is able to understand these compound gestures in real-time, so you can really live up to this whole "all you need is life experience" idea. You teach the machine to understand the users as opposed to teaching the users to understand the machine.
You do that because there is no single gesture for any action - there will be several gestures for a single action, and as game designers you can manage all of these things and essentially graft them all onto your new experience as game mechanics. So you can have really simple, fun, "jump in" experiences.
