This post describes my first attempt at making Kinect work with a PS3. Microsoft’s new XBOX 360 accessory, kinect has made a powerful entry into the market, becoming the fastest selling gadget of all time. Looks like their “You are the controller” tagline is working. Of course, Sony’s “similar” accessory Move is selling well too but is far behind kinect adoption probably because nothing beats the lure of making something work with just “The Force” .

Now, of course, kinect doesn’t work with PS3 obviously but then if we can’t break the rules, we can at least bend them a little towards our way. I’ve  created a mashup that allows you to use kinect as an input controller for the PS3.
 
Please note that this is pre-alpha quality software currently. I haven’t updated to a lot of recent code for the below libraries and also haven’t done most of the performance/feature improvements yet. The axis performance specially needs lot of tweaking and it works well only while sitting. Putting this out purely as a proof of concept. For the things that I plan to add soon, please see the Todo section below. Follow me at @shantanugoel for latest updates.

VIDEO IN ACTION

The program makes use of several other programs in order to provide this functionality. To compile/use it, you also need the following programs:

1. OpenNI Libraries

2. NITE Libraries

3. PrimeSense Libraries for kinect

4. DIYPS3Controller

Installation/Compilation/Usage

1. Make sure that the above 4 things are installed on your machine and working fine.

2. Download the source of this project to the NITE Samples directory. You can even place it anywhere else but you would need to tweak the makefile to account for the changed paths.

3. Make any changes to the source that you need.

4. Change the included ps3 controller config xml file and place it in one of the locations where diyps3controller can find it.

5. Run make -f kinect-ps3.mak

6. The executable will be available in Samples/Bin directory of NITE.

7. Run kinect-ps3 (without any arguments)

8. Run emuclient (from ps3 controller software) and choose the new xml config file

9. “Become the controller”
Default Config:

Menu Profile – This is the default profile that kinect-ps3 starts with and allows to operate the PS3 menus. Moving hand in any direction will move the ps3 menus as if you were using the DPAD with those directions keys pressed. If you run out of the kinect’s watched space, then just do a backward push with your hand, bring your hand to center and then again do the backward push and continue scrolling. This is like you lift your finger up and then start dragging from edges of a laptop’s touchpad.For selecting any item, make two successive push movements with your hand without changing any x-y direction.
Game Profile – This profile can be switched to by doing 3 successive backward movements of the hand while in Menu profile. Right now I’ve implemented only directional controls in game profile because I’m still struggling with multiple hands detections and skeletal tracking. A full profile will be activated soon enough. Currently, the your hand’s x/y movements and directly translated to right stick x/y movements (e.g. this is looking in different directions in various FPS games). Moving the hand towards or away from kinect (z-axis) translate to left stick y movements (e.g. this is moving fowards or backwards in most FPS games).
Issues/Queries/Suggestions

Please direct them to me by commenting here or emailing me at shantanu AT shantanugoel DOT com

Coming Soon/ToDo

-Full Menu and Game profiles

- Skeletal tracking for better game profiles

- Switching back to menu profiles from game profiles

- Easier way to specify user custom profiles instead of changing code for the same

- Performance improvement for game profiles

Full source code of the project has been released at https://github.com/shantanugoel/Kinect-PS3 under GNU GPL v2.

Not sure why he chose Killzone 3.  He's thrusting his hand forward just to hit X.  No idea how he'll use any of the other buttons.