Kinect Hacking (Article)




About: howitgoes is an account that was created by Instructables staff for a series of articles about technologies relating to DIY.

When Microsoft released it's Kinect system for the Xbox 360 it was hoping to give a boost to the console and sell plenty of games. It certainly hit that target with some brisk sales, but the real surprise was that the Kinect's powerful technology also became a useful tool for programmers. Within a week of its release, the Kinect had become a fantastic high-tech DIY toy with a life of its own and it was all thanks to a worldwide network of hackers and a bounty of cash.

This article is one in a series of Instructables articles about DIY technology. The full list can be seen here.

Step 1: All About Control

For many years, video game controllers were straightforward: push a joystick to move and hit a button to jump or fire. Successive generations of gaming consoles added more buttons, another joystick, and a d-pad or two, but for the most part gamers were still moving sticks around and pushing on buttons. The evolution of the peripheral had a slow and steady pace.

In 2006, Nintendo shook this up with its release of the Wii console. In addition to the buttons and joysticks, the controllers could also detect motion. So instead of button pushes for actions, gamers were mimicking what was happening on the screen. Swings of the controller became bowling, baseball, and even dancing. These intuitive controls enabled lots of non-gamers to play, even becoming popular with senior citizens, and pushed sales of the console past 70 million worldwide.

Two groups were paying close attention to the Wii: Microsoft and hackers. Microsoft saw the potential in freeing gamers from controllers and started the Kinect project, originally known as Project Natal. With the Kinect, a pair of cameras record video and depth and are able to combine the information to precisely see the player's body position. Now no controllers were needed at all and gamers could play just by moving around.

Meanwhile, hackers saw the Wii's controllers as wonderful tools that were dying to be put to use for things besides games, such as head-tracking, virtual whiteboards, and controlling a 15-ton robotic arm. One such hacker, Johnny Lee, made such cool use of the Wii that he was hired on by Microsoft for the Kinect.

Step 2: The Magic of the Kinect

The Kinect looks like a freaky wide webcam. In fact, there are two cameras in it: an RGB camera and an infrared camera. The RGB camera works like any webcam would, picking up the visible light in the room that it's in. The real work starts to happen with the infrared camera.

With an infrared camera the Kinect can see the room it's in no matter what the lighting is like. This is important since many folks like to play games well into the night. To make sure that it has something to read there is also an infrared projector that shines an array of dots into the room. The infrared camera ses these dots and through a process called "structured light" the Kinect can create a depth map.

This video shows the infrared projection into a room.

With this depth information, the Kinect can create a skeleton of the player that has 20 joints. It's all done thanks to lots of R&D on Microsoft's part, analyzing terabytes of data to get a system that works without players even thinking about it. The jointed skeleton then allows for the Xbox 360 to have games that use players motions and gestures fo a more immersive experience.

Step 3: All About the Bounty

So when Microsoft's Kinect was about to be released with its amazing potential to read users' movements, it was inevitable that people would want to use it for their own purposes. It was such an exciting idea that Adafruit Technologies put up a $1,000 bounty for the first person to create open-source drivers for the Kinect. With those, others would have the keys to the Kinect to create their own uses for it.

Microsoft didn't take kindly to the bounty offer and issued a statement saying, Microsoft does not condone the modification of its products. Upon hearing this, Adafruit Industries doubled the bounty to $2,000 and then $3,000 soon after. The race was on around the world, open to anyone with a Kinect and a computer and Microsoft wasn't happy about it.

Where the Wii hacks had helped to nurture a global network of hackers, the hunt for the Kinect drivers took it even further. Adafruit Industries helped out by buying a USB analyser to see all the information that was coming out of the Kinect and posting it online. Chat rooms lit up and updates were constantly being shared.

With so much attention it shouldn't have been surprising that open-source drivers were released on Nov. 10, just six days after the Kinect went on sale. Even more amazing, the winner was Hector Martin, a programmer who wasn't even in the United States, but northern Spain. Since the Kinect went on sale in the EU on Nov. 10, he had made the drivers on the same day that he bought it. Shortly afterwards, Adafruit Industries tested it out and declared him a winner: the search was over.

Step 4:

With the keys to the kingdom out there, Microsoft had a change of heart. In an interview Microsoft's Alex Kipman and Shannon Loftis said that the Kinect interface was left unprotected "by design and they're inspired" by community finding new uses. Whether this was the Kinect team's own desires or Microsoft giving in to the inevitable is up for debate. What's clear is that there's nothing they could do about it, but enjoy the new uses for their own device.

And now there are tons of Kinect hacks that people have created. New videos of cool uses are popping up almost daily. Invisibility cloaks, virtual puppetry, turning people into puffy 3D characters, and so much more. After putting up the $150 for the Kinect, the main limitation is how much creativity and time people can put into it. Do you have an idea that could use this? Then maybe you'll want to make a Kinect hack of your own.

Step 5: Links

DIY Kinect Hacking - Guide by Lady Ada
OpenKinect on GitHub - Share code
OpenKinect Google Group - Discuss
How You Become the Controller - Xbox engineers explain how the Kinect works



  • Sew Tough Challenge

    Sew Tough Challenge
  • Paint Challenge

    Paint Challenge
  • Games Contest

    Games Contest

21 Discussions


6 years ago

Cool article, I didn't know all that background! Thanks for posting


8 years ago on Step 2

The kinect detects distances not by brightness of the dot array, but by the distances between the dots. on the object.

1 reply

8 years ago on Introduction

you've published 3 ibles (maybe more) today, all about different topics and are all just videos you found on youtube. If you want to make a good ible, LEARN ABOUT THE SUBJECT FIRST.

10 replies

Reply 8 years ago on Introduction

Agreed. This showed taught me nothing about how to get started in kinect hacking nor was it anything more than a article. The author did no original work at all in this.

Articles are good, if thet teach you something. If i would have wanted to hack a kinect i would find these videos in 5 minutes on youtube. The author has probably never hacked a kinect, made a drone, printed anything in 3d, or done any of the other 3 " articles". Guess what, i could go on google, type in quantum physics copy and paste the text and make an "article" on it here, what good is it to anyone?


Reply 8 years ago on Introduction

First off, these are a series of articles written by staff at Instructables. Thus the featuring and the guide. Nobody is trying to spam you or lie to you. There is also no other format that we have here for this type of thing so it's in the Instructable format even if it's not a tutorial.

Second, you're really upset about those videos as you've commented on them a couple of times. There's more information here than just the videos. I'm not saying it's an epic article of magnificent proportions, but I did what I could to place the Kinect in context with other things happening and add a bit more to it. It's a story that I've followed for quite a while and find pretty interesting.

As for copying and pasting, I do admit that I copied and pasted the embed code for the youtube videos. Trying to reinterpret that would've led to some errors.

Finally, I hate to break it to you, but many articles that are written are done by people who haven't actually done what they are talking about. Personally, I have a lot of experience with three of the six topics.

If you're still upset, I'll understand. I do appreciate you laying off of the caps lock this time, though. That was nice.


Reply 8 years ago on Introduction

I can perfectly understand his consternation. How was he meant to know that you are a member of staff at Instructables? Your Instructables member page doesn't give any clues that you're a staff member.

Saying "there's no other format that we have here for this type of thing" is a huge cop-out. So they can pay someone to write "articles" but not a developer to implement an "Article" format? Right. Perhaps Instructables could shelve the "Article" concept until they can sort this out.

There are hundreds of other instances where some random (non-staff) Instructables member has posted other peoples' youtube videos, flickr photos and added a bare minimum of original content which leaves everyone wondering why the hell they clicked on the link in their RSS feed.

Many, if not all, of those Instructables were removed. Would they now be allowed to stay, simply by virtue of having the word "article" in the Intro step? This "article" is indistinguishable from any of those other wastes of bandwidth and it is annoying to click on an Instructable in my RSS reader with the title "Kinect Hacking" only to be shown this.

Finally, I hate to break it to you, but many articles that are written are done by people who haven't actually done what they are talking about. Personally, I have a lot of experience with three of the six topics.
This is great, but I, and many others, come to Instructables to see, well....Instructables. i.e. people posting their original project photos, videos etc. with step by step instructions.

If you do only one thing, can you please make sure in future that the titles of these "Articles" to include the word "article"?


Reply 8 years ago on Introduction

Thanks for the feedback and taking the time to write out your thoughts. This series is a bit of an experiment.

No clue as to being a staff member? Good point, will change.

Money for an article, but not for dev time? Yes, yes, and yes. There are lots of things that we'd like to do around here to grow and improve the site. Adding a new form of content would require nav changes, UI fun, and a change to the editor. It looks smaller than it really is.

Others like this? There are such instructables as you mention and they do get removed. However, the reason is often that there is barely anything beyond a youtube video. Also, many of those are people falsely trying to get credit for what was in the photos or videos, passing it off as their own work.

Include "article" in the title to avoid confusion and set expectations? Sure.


Reply 8 years ago on Introduction

I'm not sure how much my input here will matter being that I'm not a very active member and have posted no instructables myself(yet) but I agree with adamazing and at the same time wasn't bothered by any of the problems mentioned as I take everything I read on the internet with a grain of salt anyway.

However, unless I'm missing an entire section of this website(which is possible since I browse at work) there is no instructables blog. Wouldn't something like that even if fed from one of the popular blog sites to instructables cut the cost issue at least a little?

Lance Mt.CazzPhoenix

Reply 8 years ago on Introduction

I approve of this article.

I don't think I can nock up a tutorial for getting into the hacking just yet, but please, if anyone else can do so.

There are six in all. The link in the intro shows a guide for the rest. These articles are all from Instructables. While they aren't Instructables, this is the only way on the site to present an article.

Not sure about the comment about these just being videos. There's information about the history and the bounty which was not in the videos.