loading

This project uses inexpensive HC-SR04 ultrasonic sensors as inputs and generates MIDI notes that can be played through a synthesizer on the Raspberry Pi for a high quality sound.

The project also uses a basic form of gesture control, where the musical instrument can be changed by holding your hands over the two outermost sensors for a few seconds. Another gesture can be used to shut down the Raspberry Pi once you are finished.

The two videos attached to this step are early prototypes - one demonstrates a single instrument playing and the other demonstrates each sensor playing a different instrument. See the in-depth video in a later step for a full demonstration of the finished product!

I created this project in conjunction with The Gizmo Dojo (my local makerspace in Broomfield, CO) to make some interactive exhibits that we can take to local STEM/STEAM events and Maker Faires.

Step 1: Ingredients

For this instructable, you will need the following ingredients:

  • Raspberry Pi (2 or 3) with SD card
  • 8 HC-SR04 ultrasonic sensors
  • Octasonic Breakout Board
  • Bi-Directional Logic Level Converter
  • 32 x 12" Female-Female Jumper Wires for connecting the ultrasonic sensors
  • 13 x 6" Female-Female Jumper Wires for connecting the Raspberry Pi, Octasonic, and Logic Level Converter
  • Suitable power supply for the Raspberry Pi
  • PC speakers or similar

I would recommend using a Raspberry Pi 3 if possible since it has more computing power, resulting in a more responsive and pleasing sound. It can work fine with a Raspberry Pi 2 with a little tweaking but I would not attempt to use the original Raspberry Pi for this project.

HC-SR04 ultrasonic sensors have 4 connections - 5V, GND, Trigger, and Echo. Typically, Trigger and Echo are connected to separate pins on a microcontroller or Raspberry Pi but that means you would need to utilize 16 pins to connect 8 sensors, and this is not practical. This is where the Octasonic breakout board comes in. This board connects to all of the sensors and has a dedicated microcontroller that monitors the sensors and then communicates with the Raspberry Pi over SPI.

HC-SR04 require 5V and the Raspberry Pi is only 3.3V, so this is why we also need the logic level converter that will connect the Raspberry Pi to the Octasonic breakout board.

Step 2: Connect the Ultrasonic Sensors to the Octasonic Board

Use 4 female-female jumper wires to connect each ultrasonic sensor to the board, being careful to connect them the correct way around. The board is designed so that the pins are in the same order as the pins on the ultrasonic sensor. From left to right on the board, the pins are GND, Trigger, Echo, 5V.

Step 3: Connect the Logic Level Converter to the Octasonic Board

The Raspberry Pi and Octasonic Board communicate over SPI. SPI uses 4 wires:

  • Master In, Slave Out (MISO)
  • Master Out, Slave In (MOSI)
  • Serial Clock (SCK)
  • Slave Select (SS)

Additionally, we need to connect power (5V and GND).

The logic level converter has two sides - a low voltage (LV) and a high voltage (HV). The Raspberry will connect to the LV side since it is 3.3V. The Octasonic will connect to the HV side since it is 5V.

This step is for connecting the Octasonic to the HV side of the logic level converter.

See the photo attached to this step showing which pins should be connected to the logic level converter.

The connections from the Octasonic to the Logic Level converter should be as follows:

  • 5V to HV
  • SCK to HV4
  • MISO to HV3
  • MOSI to HV2
  • SS to HV1
  • GND to GND

Step 4: Connect the Logic Level Converter to the Raspberry Pi

The Raspberry Pi and Octasonic Board communicate over SPI. SPI uses 4 wires:

  • Master In, Slave Out (MISO)
  • Master Out, Slave In (MOSI)
  • Serial Clock (SCK)
  • Slave Select (SS)

Additionally, we need to connect power (3.3V and GND). The logic level converter has two sides - a low voltage (LV) and a high voltage (HV). The Raspberry will connect to the LV side since it is 3.3V. The Octasonic will connect to the HV side since it is 5V.

This step is for connecting the Raspberry Pi to the LV side of the logic level converter.

The connections from the Raspbery Pi to the Logic Level converter should be as follows:

  • 3.3V to LV
  • GPIO11 (SPI_SCLK) to LV4
  • GPIO09 (SPI_MISO) to LV3
  • GPIO10 (SPI_MOSI) to LV2
  • GPIO08 (SPI_CE0_N) SS to LV1
  • GND to GND

Use the diagram attached to this step to locate the correct pins on the Raspberry Pi!

Step 5: Connect Raspberry Pi 5V to Octasonic 5V

There is one final wire to add. We need to actually power the Octasonic board with 5V, so we do that by connecting one of the the Raspberry Pi 5V pins to the 5V pin on the Octasonic AVR header. This is the bottom left pin in the AVR header block (this is the 2 x 3 block on the top right of the board). See the attached photo showing where the AVR block is.

See the other attached diagram to find the 5V pin on the Raspberry Pi.

Step 6: Install Software

Install Raspian

Start with a clean install of Raspbian Jessie, then update it to the latest version:

sudo apt-get update
sudo apt-get upgrade

Enable SPI

You must enable SPI on the Raspberry Pi for this project to work! Use the Raspberry Pi Configuration utility to do this.

Install FluidSynth

Fluidsynth is an amazing free software MIDI synth. You can install it from the command-line with this command:

sudo apt-get install fluidsynth 

Install the Rust Programming Language

The Ultrasonic Pi Piano is implemented in the Rust Programming Language from Mozilla (it's like C++ but without the bad bits). It's what all the cool kids are using these days.

Follow the instructions at https://rustup.rs/ to install Rust. To save you time, the instructions are to run this one command. You can accept the default answers to any questions during the install.

NOTE: Since posting this instructable, there are some issues with installing Rust on the Raspberry Pi. Bad timing :-/ but I have modified the command below to workaround the issue. Hopefully they will fix this soon. I am working on creating an image that people can download and burn onto an SD card. If you would like that, please contact me.

export RUSTUP_USE_HYPER=1
curl https://sh.rustup.rs -sSf | sh

Download the Ultrasonic Pi Piano source code

The source code for the Ultrasonic Pi Piano source code is hosted on github. There are two options for getting the code. If you are familiar with git and github, you can clone the repo:

git@github.com:TheGizmoDojo/UltrasonicPiPiano.git

Alternatively, you can download a zip file of the latest code.

Compile the source code

cd UltrasonicPiPiano
cargo build --release

Test the code

Before we move onto making music in the next step, let's make sure that the software runs and that we can read valid data from the sensors.

Use the following command to run the application. This will read data from the sensors and translate them into MIDI notes that are then printed out on the console. As you move your hand over the sensors, you should see data being generated. If not, then skip to the troubleshooting section at the end of this instructable.

cargo run --release

If you are curious, the "--release" flag tells Rust to compile the code as efficiently as possible, as opposed to the default "--debug" setting.

Step 7: Make Some Music!

Make sure you are still in the directory where you downloaded the source code and run the following command.

This "run.sh" script makes sure that the code has been compiled and then runs the code, piping the output into fluidsynth.

./run.sh

Make sure that you have amplified speakers connected to the 3.5mm audio jack on the Raspberry Pi and you should hear music as you move your hands over the sensors.

If you don't hear music and you have an HDMI monitor attached, then the audio output is probably going there instead. To fix this, simply run this command and then re-start the Pi Piano:

sudo amixer cset numid=3 1

Changing the volume

The volume (or "gain") is specified with the "-g" parameter to fluidsynth. You can modify the run.sh script and change this value. Please note that small changes in this parameter result in a big change in volume, so try increasing it by small amounts (like 0.1 or 0.2).

Step 8: Gesture Control

See the video attached to this step for a full demonstration of the project, including how the gesture controls work.

The concept is very simple. The software keeps track of which sensors are covered (within 10cm) and which are not. This translates to 8 binary numbers (1 or 0). This is very convenient, since a sequence of 8 binary numbers makes a "byte" which can represent numbers between 0 and 255. If you don't already know about binary numbers then I highly recommend searching for a tutorial. Binary numbers are a fundamental skill to learn if you want to learn more about programming.

The software maps the current state of the sensors to a single byte representing the current gesture. If that number stays the same for a number of cycles, then the software acts on that gesture.

Because ultrasonic sensors aren't super reliable and there can be interference between sensors, you will need to exercise some patience when using the gestures. Try varying the distance that you hold your hands from the sensors as well as the angle that you hold your hands. You cold also try holding something flat and solid over the sensors to better reflect the sound.

Step 9: Making an Enclosure

If you want to make this a permanent exhibit and be able to show it off to people, you'll probably want to make some kind of enclosure. This could be made from wood, cardboard, or many other materials. Here is a video showing the enclosure we are working on for this project. This is made from wood, with holes drilled out to hold the ultrasonic sensors in place.

Step 10: Troubleshooting and Next Steps

Checkout the README at https://github.com/TheGizmoDojo/UltrasonicPiPiano for the latest version of the instructions and troubleshooting ideas.

If you are still stuck, write a comment on this instructable and I will try and help you out.

Once you have the project working, I recommend experimenting with the code and trying out different musical instruments. MIDI instrument codes are between 1 and 127 and are documented here.

Do you want a single musical instrument with each sensor playing a different octave? Perhaps you'd like each sensor to be a separate instrument instead? The possibilities are almost limitless!

I hope you enjoyed this instructable. Please like it if you did, and be sure to subscribe to me here and to my YouTube channel to see future projects.

<p>I made one as an electronics demo for the grand opening of our new makerspace. After setup, I explained it as an electronic xylophone. :)</p>
<p>Awesome! Thanks for sharing :-)</p>
nice job! one suggestion, you could use photo resistors instead of the ultrasonic sensors to reduce the cost greatly!

About This Instructable

1,946views

23favorites

License:

Bio: I'm a software architect with an interest in Arduino, Raspberry Pi, electronics, robotics and generally making cool projects with my kids.
More by andygrove:Ultrasonic Pi Piano With Gesture Controls! Dalek & Cyberman Voice Changer with Arduino Uno How to make an Ood Translation Sphere 
Add instructable to: