Invisible Piano (Keyboard Anywhere, a Kinect Piano)
Intro: Invisible Piano (Keyboard Anywhere, a Kinect Piano)
After writing my previous instructable, I was asked about installing some slightly different software to use with the Kinect. Since I'd already done it, I figured it wouldn't take to long to retrace my steps and write the instructable. After much frustration, I figured out a really easy process to get everything installed and talking.
This instructable with walk you though getting a virtual keyboard working with the current release (11.04) of Ubuntu.
There are other ways of doing this (which I've done in the past), but trying to reaccomplish the task, I found many shortcuts to what I did in the command line previously.
If you have any questions on the command line or getting around in Ubuntu, please see my previous instructable. In this instructable I will assume you have a basic knowledge of the system and how it works. Like my last instructable terminal commands will be given starting with $. Do not copy the $ when transposing.
Also, don't transpose anything to terminal between [ ], it's there for reference.
This instructable with walk you though getting a virtual keyboard working with the current release (11.04) of Ubuntu.
There are other ways of doing this (which I've done in the past), but trying to reaccomplish the task, I found many shortcuts to what I did in the command line previously.
If you have any questions on the command line or getting around in Ubuntu, please see my previous instructable. In this instructable I will assume you have a basic knowledge of the system and how it works. Like my last instructable terminal commands will be given starting with $. Do not copy the $ when transposing.
Also, don't transpose anything to terminal between [ ], it's there for reference.
STEP 1: Materials
STEP 2: Install Ubuntu
For this, I opted to use the newest release of Ubuntu: 11.04.
I downloaded and installed the 32 bit version onto my laptop, then used Update Manager to update the system (there was a couple hundred megs worth of updates).
Bonus info:
For this release, Ubuntu decided to ditch Gnome and move to Unity, their out User Interface.
Like any new system, if you haven't used it before, it's going to take some getting used to. I can say after using it for a while, getting pissed and moving to Debian (and Gnome 3) for a while, then coming back and trying it again, it's not that bad. It's not my first choice in desktops, but that's because it's different that what I'm used to and is going to take me some time to come around. The important bit is that it works, it's useable, and the design is such as to try to make it as user friendly as possible.
With that in mind, if you need help finding your way around, check this out for how to update, install software, etc.
I downloaded and installed the 32 bit version onto my laptop, then used Update Manager to update the system (there was a couple hundred megs worth of updates).
Bonus info:
For this release, Ubuntu decided to ditch Gnome and move to Unity, their out User Interface.
Like any new system, if you haven't used it before, it's going to take some getting used to. I can say after using it for a while, getting pissed and moving to Debian (and Gnome 3) for a while, then coming back and trying it again, it's not that bad. It's not my first choice in desktops, but that's because it's different that what I'm used to and is going to take me some time to come around. The important bit is that it works, it's useable, and the design is such as to try to make it as user friendly as possible.
With that in mind, if you need help finding your way around, check this out for how to update, install software, etc.
STEP 3: Open Kinect
After the system is installed and the updates are complete, The first program you're going to need is OpenKinect.
They have excellent instructions on their webpage to get this installed: click here, and follow the Ubuntu Manual Install instructions.
When you're finished with this step, your Kinect should be visible on your computer.
STEP 4: Python
The Keyboard Anywhere script is written in Python, which OpenKinect can't deal with in its regular installed state. Happily, there is a Python Wrapper that allows the two to work together.
Thankfully, when you installed OpenKinect in your last step, it downloaded the python wrapper on your computer. The problem is that while it downloaded, it didn't install.
If you go to your home (where home is your username) directory: home/libfreenect/wrappers/python you'll see the information about the wrapper. You can open the README to see instructions on installing.
For ease of time and those who like to copy-paste
$ sudo apt-get install cython python-dev python-numpy [these need to be installed before we can install the python wrapper]
Navigate your way, in terminal, to the python folder. If you close and open terminal:
$ cd libfreenect/wrappers/python
and then install:
$ sudo python setup.py install
** Instruction addition as per tdinsmoor in the comments below **
If you are having dependency issues with libexpat1-dev (on Ubuntu 12.04), you'll need to download and install a separate .deb package to fix it.
For AMD64 systems:
http://launchpadlibrarian.net/118383909/libexpat1-dev_2.1.0-1ubuntu1_amd64.deb
For i386 systems:
http://launchpadlibrarian.net/118383396/libexpat1-dev_2.1.0-1ubuntu1_i386.deb
Ubuntu software center may give you an error at first, but ignore this and proceed to install python2.7-dev and python-dev with:
$ sudo apt-get install python-dev python2.7-dev
The packages should now install correctly, and you may proceed to install the wrapper.
************
Thankfully, when you installed OpenKinect in your last step, it downloaded the python wrapper on your computer. The problem is that while it downloaded, it didn't install.
If you go to your home (where home is your username) directory: home/libfreenect/wrappers/python you'll see the information about the wrapper. You can open the README to see instructions on installing.
For ease of time and those who like to copy-paste
$ sudo apt-get install cython python-dev python-numpy [these need to be installed before we can install the python wrapper]
Navigate your way, in terminal, to the python folder. If you close and open terminal:
$ cd libfreenect/wrappers/python
and then install:
$ sudo python setup.py install
** Instruction addition as per tdinsmoor in the comments below **
If you are having dependency issues with libexpat1-dev (on Ubuntu 12.04), you'll need to download and install a separate .deb package to fix it.
For AMD64 systems:
http://launchpadlibrarian.net/118383909/libexpat1-dev_2.1.0-1ubuntu1_amd64.deb
For i386 systems:
http://launchpadlibrarian.net/118383396/libexpat1-dev_2.1.0-1ubuntu1_i386.deb
Ubuntu software center may give you an error at first, but ignore this and proceed to install python2.7-dev and python-dev with:
$ sudo apt-get install python-dev python2.7-dev
The packages should now install correctly, and you may proceed to install the wrapper.
************
STEP 5: Keyboard-Anywhere - Dependencies and Downloads
The python script for Keyboard Anywhere can be found at that link.
All the information below can be found by reading the README provided in the package you can download at the previously mentioned link. There are some discrepancies in the instructions provided which I have fixed for this instructable. (i.e. pyqt4-devtools should read pyqt4-dev-tools and fluid-soundfont-gm doesn't require install because it's part of the fluidsynth package) I've also left out dependencies that have been installed in previous steps.
So before we move on, let's deal with some more dependency issues!
$ sudo apt-get install pyqt4-dev-tools qt4-qmake libqt4-dev python-qt4-gl python-qt4-dev python-setuptools fluidsynth
(if you're asked for special permissions during the install, just click yes)
Finally, if you haven't yet, click downloads at the top and click tar.gz
-unpack it onto your computer where ever it downloaded (I will assume it's the Downloads folder)
If you scroll to the bottom you'll see links to download both QGLViewer and PyQGLViewer
-We're going to skip this for now.
STEP 6: QGLViewer and PyQGLViewer
The instructions given in the Keyboard-Anywhere README for installing these files, is difficult at best, nightmarish at worst. Instead of following those directions, I've opted to install it a completely different way. You can find my instructions at this link.
$ sudo add-apt-repository ppa:christophe-pradal/openalea
$ sudo add-apt-repository ppa:christophe-pradal/vplants
$ sudo apt-get update
$ sudo apt-get install pyqglviewer
--------------------------
If you want to downloaded and install QGLViewer and PyQGLViewer manually, after downloaded and unpacking the files, ensure you complete the following steps to correct errors that will come up.
Navigate to /home/Downloads/libQGLViewer-2.3.9-py/QGLViewer/
Open QGLViewer.pro in gedit (or whatever default application may open in)
Go to the file drop downs:
-Search
-Go To Line
-enter 201
Now, I'm not sure if this is the proper way to fix this error, but it works. Change the line that reads:
system( g++ --version | grep " 3\.2\.0 " > /dev/null )|system( g++ --version | grep " 3\.3\.1\-2" > /dev/null ) {
to:
system( g++ --version | grep " 3\\.2\\.0 " > /dev/null )|system( g++ --version | grep " 3\\.3\\.1\\-2" > /dev/null ) {
One of the only forums I could find that was helpful was this one. If you try to progress in these steps, you'll get that error.
I will not be providing any more directions on manually installing QGLViewer and PyQGLViewer. If you'd like to manually install them, follow the directions in their README/INSTALL files.
--------------------------
$ sudo add-apt-repository ppa:christophe-pradal/openalea
$ sudo add-apt-repository ppa:christophe-pradal/vplants
$ sudo apt-get update
$ sudo apt-get install pyqglviewer
--------------------------
If you want to downloaded and install QGLViewer and PyQGLViewer manually, after downloaded and unpacking the files, ensure you complete the following steps to correct errors that will come up.
Navigate to /home/Downloads/libQGLViewer-2.3.9-py/QGLViewer/
Open QGLViewer.pro in gedit (or whatever default application may open in)
Go to the file drop downs:
-Search
-Go To Line
-enter 201
Now, I'm not sure if this is the proper way to fix this error, but it works. Change the line that reads:
system( g++ --version | grep " 3\.2\.0 " > /dev/null )|system( g++ --version | grep " 3\.3\.1\-2" > /dev/null ) {
to:
system( g++ --version | grep " 3\\.2\\.0 " > /dev/null )|system( g++ --version | grep " 3\\.3\\.1\\-2" > /dev/null ) {
One of the only forums I could find that was helpful was this one. If you try to progress in these steps, you'll get that error.
I will not be providing any more directions on manually installing QGLViewer and PyQGLViewer. If you'd like to manually install them, follow the directions in their README/INSTALL files.
--------------------------
STEP 7: PyFluidSynth
FluidSynth, for some reason, doesn't play nice with the python scripts that are trying to use it. To fix this, I installed pyFluidSynth. It seems to work nicely.
Download pyFluidSynth
Unpackage in your downloads folder
Navigate in terminal to unpacked folder and install
$ cd Downloads/pyFluidSynth-1.2.4
$ sudo python setup.py install
Download pyFluidSynth
Unpackage in your downloads folder
Navigate in terminal to unpacked folder and install
$ cd Downloads/pyFluidSynth-1.2.4
$ sudo python setup.py install
STEP 8: Enjoy!
Everything is now installed, all you have to do is run Keyboard Anywhere.
Navigate to the appropriate directory, and run using the python command.
You need to use sudo, or you don't have enough permissions to do everything.
Unfortunately I've found that sudo, and su both have a timer on them. So after 30-45 minutes, you'll need to close the program and start it up again. I haven't found a way around this yet.
$ cd Downloads/petermoz-keyboard-anywhere-2a8b9ed
$ sudo python keyboard_anywhere.py
The README that is associated with the downloaded Keyboard-Anywhere file has detailed instructions on what to do once running.
If you're comfortable editing python script (it's not too difficult after it's already been written...you're just changing things), you can open the keyboard_anywhere.py file in gedit and change the size of the keys and link to different sound files.
If you have any questions or comments, feel free to leave them below.
Navigate to the appropriate directory, and run using the python command.
You need to use sudo, or you don't have enough permissions to do everything.
Unfortunately I've found that sudo, and su both have a timer on them. So after 30-45 minutes, you'll need to close the program and start it up again. I haven't found a way around this yet.
$ cd Downloads/petermoz-keyboard-anywhere-2a8b9ed
$ sudo python keyboard_anywhere.py
The README that is associated with the downloaded Keyboard-Anywhere file has detailed instructions on what to do once running.
If you're comfortable editing python script (it's not too difficult after it's already been written...you're just changing things), you can open the keyboard_anywhere.py file in gedit and change the size of the keys and link to different sound files.
If you have any questions or comments, feel free to leave them below.
28 Comments
JasonH288 7 years ago
Thanks for the writeup. What is the difference between libfreenect, openkinect and the software you used in your other tutorial? I have a Kinect, want to hook it up in Linux and interact with it using Python. I thought libfreenect is the best way to do this?
LeninH2 7 years ago
Hi, im not sure if someone can help me with a problem... at the moment when i try to use "sudo apt-get install pyqglviewer" i can't download the packages so i try to install it manual but have problems too. I appreciate if someone can help me with that
JacksonY2 7 years ago
Lenin, Did you get this sorted out?
StumpChunkman 7 years ago
One question: what version of Ubuntu are you installing this on?
Also, one place that might help (scroll down to the sudo code):
http://pyqglviewer.gforge.inria.fr/wiki/doku.php?id=documentation
LeninH2 7 years ago
promach 11 years ago
terminal output: transform loaded from file
AND
a blank untitled window output
May I know why ?
StumpChunkman 11 years ago
Please ensure you're following the OpenKinect's read me/install files and that you're installing all of the dependencies. Other then that advice, I'm not going to be any help.
Esra_Juma 11 years ago
My project is Invisible Piano Keyboard anywhere
I downloaded all the software needed as well as the hardware, but I have the following error whenever I tried to run it:
Could you help us please
Thank you in advance
Error: Invalid index [0]
Error: Can't open device. 1.) is it plugged in ? 2.) Read The README
Tracback (most recent call last ):
File "keyboard_anywhere.py" , line 286, in animate depth,
timestamp = freenect.sync_get_depth()
TypeError: 'NoneType' object is not iterable
could not claim interface on motor: -6
StumpChunkman 11 years ago
tdinsmoor 11 years ago
Just wanted to let you know, the install for python-dev did not work for me.
This was on a brand-new install, so it might be broken for everybody. It gave me the following error message on 12.04 Precise:
The following packages have unmet dependencies:
python-dev : Depends: python2.7-dev (>= 2.7.3) but it is not going to be installed
E: Unable to correct problems, you have held broken packages.
Now, I've tried to get the packages they depend on, (python-dev -> python2.7-dev -> libexpat1-dev [ Depends: libexpat1 (= 2.0.1-7.2ubuntu1.1) but 2.1.0-1ubuntu1 is to be installed {I have no idea why this is happening}] -> libexpat1 [which I have already]) and looked all around to try to fix this (especially the "broken packages" part, and tried a reinstall of all my packages after purging), maybe python 2.7.3 is no longer supported on 12.04 (as newest python is 3), but I have little idea. The strange thing is, I got it working before, but I ignored all the warnings and proceeded with the next step anyway. After trying to install GTK3+ (I wanted to try compiling some source code for a 3d-modeling program), my Ubuntu install broke for whatever reason (wouldn't even get to login screen), and I don't want to go through that again.
I'm an Ubuntu noob (trying, though). Could ya help a brother out? Please email me at tdinsmoor13@gmail.com if you or anybody is having the same problem and found a solution, or the possibility of a solution.
Thanks, again.
StumpChunkman 11 years ago
You should also check out the Ubuntu Install over at OpenKinect, it looks like libfreenect causes some issues with 12.04. I think you just need to follow the Official Packages install instructions for that portion.
Once you get that installed, make sure you follow the directions on the Python Wrappers Read Me. The directions there are going to be far more current then the code in my instructable. It looks like the only super requirements right now are getting cython, python-dev, and python-numpy. So install those before you try to compile.
If it doesn't work with following their instructions, then something in 12.04 may have mucked things up. You could try installing everything on an older build of Ubuntu as that might fix it. I know it worked on 11.04, but if it's not Ubuntu's fault, and something was changed in the OpenKinect code with dependancies, then I'm not sure what the problem might be.
I'm not sure if this helped at all, but best of luck. Is your only goal to get the Kinect display working, or do you need this specific app to run?
tdinsmoor 11 years ago
The only reason why I posted here is that I was worried that this led up to my Ubuntu install breaking, because I used manual install then, but ignored the error (being the super-noob I was at the time). After I got it workIng (the keyboard-anywhere) I tried installing GTK3+ so I could try to compile the source code for a 3D model capturing program that needed it.
I was just worried that the error I ignored before led up to my system failure, but that seems unlikely.
Just trying to rule out possibilities, because I don't know what exactly caused it, so that I don't do it again when I try again. (I don't expect help from here, seeing as how this isn't the place. Thank you for your quick response, and the instructabke!
StumpChunkman 11 years ago
tdinsmoor 11 years ago
I'll dump it, maybe you can make some sense of it.
====Trying to setup python..
dinsmoor@Dinsmoor-MU:~/libfreenect/wrappers/python$ sudo python setup.py installrunning install
running build
running build_ext
skipping 'freenect.c' Cython extension (up-to-date)
building 'freenect' extension
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/python2.7 -c freenect.c -o build/temp.linux-x86_64-2.7/freenect.o -fPIC -I ../../include/ -I /usr/include/libusb-1.0/ -I /usr/local/include/libusb-1.0 -I /usr/local/include -I ../c_sync/ -I /usr/lib/python2.7/dist-packages/numpy/core/include
freenect.c:4:20: fatal error: Python.h: No such file or directory
compilation terminated.
error: command 'gcc' failed with exit status 1
====So, I make sure I have all my python packages...
====Now, this is when python-dev refuses to install:
dinsmoor@Dinsmoor-MU:~/libfreenect/wrappers/python$ sudo apt-get install python-dev
Reading package lists... Done
Building dependency tree
Reading state information... Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:
The following packages have unmet dependencies:
python-dev : Depends: python2.7-dev (>= 2.7.3) but it is not going to be installed
E: Unable to correct problems, you have held broken packages.
===And then I try to get python2.7-dev (but python reports that i's already 2.7.3):
dinsmoor@Dinsmoor-MU:~/libfreenect/wrappers/python$ sudo apt-get install python2.7-dev
[sudo] password for dinsmoor:
Reading package lists... Done
Building dependency tree
Reading state information... Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:
The following packages have unmet dependencies:
python2.7-dev : Depends: libexpat1-dev but it is not going to be installed
E: Unable to correct problems, you have held broken packages.
Then I try to get libexpat1-dev...
dinsmoor@Dinsmoor-MU:~/libfreenect/wrappers/python$ sudo apt-get install libexpat1-dev
Reading package lists... Done
Building dependency tree
Reading state information... Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:
The following packages have unmet dependencies:
libexpat1-dev : Depends: libexpat1 (= 2.0.1-7.2ubuntu1.1) but 2.1.0-1ubuntu1 is to be installed
E: Unable to correct problems, you have held broken packages.
====This one says it needs libexpat1, but it gives me a funky thing about it not ====being the right Ubuntu? I'm not sure how to interpret this. Anyway, I try for ====libexpat1..
dinsmoor@Dinsmoor-MU:~/libfreenect/wrappers/python$ sudo apt-get install libexpat1
Reading package lists... Done
Building dependency tree
Reading state information... Done
libexpat1 is already the newest version.
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
====So, it's something to do with my system?
If you could help me out man, I'd be so grateful. I'll post anything you need me to have it spit out. I'm just at a loss, since I got it working once, doing the exact same thing. (minus the system failure)
StumpChunkman 11 years ago
tdinsmoor 11 years ago
I scoured Google for hours looking for a solution, and I've found and tried that solution (and again now, just to double-check for you)
I've tried everything that comes to mind.
Everything on others' minds.
Doesn't make sense to me.
Got any more magic up your sleeve?
Or should I ask the Ubuntu community?
Also, thanks for the quick reply.
StumpChunkman 11 years ago
tdinsmoor 11 years ago
For amd64 systems: https://launchpad.net/ubuntu/quantal/amd64/libexpat1-dev/2.1.0-1ubuntu1
Just had to know what to look for.
Hey, thanks for trying to help me, but here's a solution for some people. You may need to modify your instructions for python wrapper, some of my students were having the same problem, even with a fresh Ubuntu 12.04 install.
Instruction addition:
**
If you are having dependency issues with libexpat1-dev (on Ubuntu 12.04), you'll need to download and install a separate .deb package to fix it.
For AMD64 systems:
http://launchpadlibrarian.net/118383909/libexpat1-dev_2.1.0-1ubuntu1_amd64.deb
For i386 systems:
http://launchpadlibrarian.net/118383396/libexpat1-dev_2.1.0-1ubuntu1_i386.deb
Ubuntu software center may give you an error at first, but ignore this and proceed to install python2.7-dev and python-dev with:
$ sudo apt-get install python-dev python2.7-dev
The packages should now install correctly, and you may proceed to install the wrapper.
**
StumpChunkman 11 years ago
tdinsmoor 11 years ago
I'll look at the links, and see how they like it.
This exercise was more of a "figure it out, fool" type of experiment, a chance for them to do it themselves.
You're welcome and thank you!