Introduction: Raspberry Pi Visual Theremin

By


Overview:

Learn how to use Simulink to make a Raspberry Pi based theremin that is controlled by a green object. A theremin is an electronic musical instrument that can be played without physical contact. The Raspberry Pi utilizes basic image processing algorithms to generate various tones through its onboard audio jack.


Summary of Functionality:

If built correctly, this project should be able to:

  1. Mark the centroid of a green object with red crosshairs
  2. Generate sound through the Raspberry Pi's audio jack
  3. Change the frequency and gain of the audio based upon the x and y coordinates of the green object

Step 1: Hardware and Software Needed

To construct this project, you will need:

  1. Raspberry Pi
  2. Raspberry Pi Cameraboard

To run the code you will need:

  1. MATLAB
  2. Simulink
  3. Simulink Support Package for Raspberry Pi

If you do not have MATLAB and Simulink, you can get a 30 day free trial by clicking on the link below!

https://www.mathworks.com/programs/trials/trial_re...

Step 2: Hardware

All you need to do for the hardware portion of this project is connect the Raspberry Pi Cameraboard to the CSI connector on the Raspberry Pi.

Step 3: Software

Here are some screenshots of the Simulink model for the visual theremin.

The model can be broken down as follows:

  1. When you first open the file, you will see the Simulink model at the highest level. This contains several MATLAB function blocks as well as various other Simulink blocks to handle the video stream from the Cameraboard.
  2. By double clicking on the MATLAB function blocks, you can view the MATLAB code that they contain.
    • FindCentroid: Determines where the centroid of the green object is.
    • GenerateSawWave: Determines the frequency, period and gain of the audio that will be emitted from the Raspberry Pi's audio jack. This is dependent on the x and y coordinates of the green object's centroid.
    • MarkImage: Superimposes red crosshairs onto the raw video from the Camera Board.

  3. ObjectDetection: This subsystem determines whether or not there is a green object within the video feed

The Simulink files for this project can be downloaded from the following link:

https://www.mathworks.com/matlabcentral/fileexchange/58428-creating-a-visual-theremin-using-raspberry-pi-and-simulink

Step 4: Testing

To test whether your system works, you can run the model in External mode. In external mode, Simulink will generate code that runs on the Raspberry Pi, but at at the same time, you will be able to monitor your system as it operates. This mode makes debugging very easy because you can use the scope and video display to determine whether the centroid is being located and marked correctly.

To run the model in external mode, follow these three easy steps:

  1. Select External mode from the drop down menu at the top of the model.
  2. Make sure the box next to it shows "inf". This stands for infinite and ensures that the model will run indefinitely.
  3. Press the green play button next to the box.

Step 5: Completion

Once you know that your model works in external mode, you can deploy it to your hardware! All you have to do is press the 'Deploy to Hardware' icon which is also located at the top of the model. When you deploy the model to your hardware, Simulink will generate code from your Simulink model and download it to the Raspberry Pi. The algorithm will then run completely independent of Simulink.

**Note: You will not able to see the red crosshairs when you deploy the model to your hardware because there will be no screen associated with your Raspberry Pi.

In just five steps, you have created your very own Raspberry Pi based visual theremin!