In 2016, after being inspired by a video of the Scanman Line Follower on YouTube, I started work on a synthesizer device using the Toshiba TCD1304 linear CCD to synthesize audio from spectrogram data (or graphical data interpreted as spectrogram data) using Michel Rouzic's ARSS code (the source of his Photosounder application). This became overly bulky, hardware-wise, and really didn't work as a standalone controller, so I put it on the back burner.
Recently I became aware that the sensors made by Agilent for optical computer mice do a lot of processing internally, both being able to provide a bitmap image (very slowly) and average darkness along with the change in X and Y using simple serial requests (much faster) rather than having to deal with a high-speed analog to digital conversion like the Scanman / Toshiba sensor. So, I decided to make a simplified version of the CCD synth using a mouse instead of the scanner. By modifying an Arduino library developed by Conor Peterson for reading pixel data from the Agilent sensor to read movement and average darkness I was able to grab the data fast enough for a simple yet responsive standalone gestural synthesizer
The components in this device can be purchased for less than ten dollars and the code is simple enough for almost anyone to modify, making this a quick and inexpensive sound maker for performance or as a prank.
Using the software below, the scrollwheel button switches between modes: 1 - pitch based on X-position, 2 - pitch based on camera input, 3 - a mix of the two. The left mouse button is a momentary trigger and the right is latching. The scrollwheel changes the frequency range and scrollwheel plus left button changes the center point of that range. Left button plus center button toggles volume modulation on the Y-axis.
Teachers! Did you use this instructable in your classroom?
Add a Teacher Note to share how you incorporated it into your lesson.
Step 1: Parts Needed
-24 awg solid wire (multiple colors)
-USB mini cable
-Arduino Nano (or clone)
-Mouse with Agilent sensor A1610 or A2610 (possibly others)
-Mini side cutters
-Mini needle-nose piers
-Soldering iron & solder
-Hot glue gun & glue
-1/16", 1/4" and expanding/stepping bit
-5v USB charger
Step 2: Remove Feet and Screws and Open the Mouse.
Remove the pads from the bottom of your mouse if they are covering and screws. Remove the screws and carefully open the mouse. Be sure to keep the screws where you can find them!
Step 3: Disconnect USB Cable and Remove Encoder Wheel.
Disconnect the mouse's USB cable and discard. Usually there will be a connector but if there isn't, just cut the cable using side cutters, careful not to bridge a connection between the wires (the shared ground touching the +5v could interfere with the operation of the sensor). Remove the scroll encoder wheel so that it doesn't get lost.
Step 4: The Mouse Sensor PCB and Connections.
Here is a Fritzing drawing of the connections and a photo showing the connections made using the Kensington mouse I'm using for the tutorial.
Step 5: Tinning the Wires.
Cut and tin 10 4inch lengths of wire for making the connections. This will make it easier to solder them to the PCB. You can omit the IC ground since it it the same connection as the other ground.
Step 6: Soldering Wires to the Microswitches and Encoder Wheel.
Starting with the ground wire, solder the wires to the underside of the board in the locations described earlier. You can also connect the IC pins on the underside as well. I did these on top because I was referencing the spec sheet while soldering. Flip the board and arrange the wires so that the board is able to seat properly without any additional gaps caused by the wires.
Step 7: Securing the Wires and And PCB With Hot Glue.
Use the hot glue gun to secure the wires to the edge of the board. Don't forget to turn the glue gun on! The connections won't break by accident and it makes them easy to identify when the board is flipped because they are held in order.
Step 8: Soldering Wires to the Optical Mouse Sensor.
I soldered the wires directly to the IC, but they can easily be soldered to the underside of the PCB. I start by tinning the legs of the IC that I need to solder to, and then melt the coated solder on the leg and the wire together with the soldering iron. Secure these connections with hot glue and cut any protruding wires off the top of the board to prevent them from accidentally touching the Arduino Nano.
Step 9: Connecting the Optical Sensor to the Nano.
Cut the wires from the sensor to length and attach them to the Arduino. I go in through the bottom and solder on the top to use as little room as possible. D2, D3, 5v and GND.
Step 10: Connecting the Right and Center Buttons to the Nano.
Cut the right and center button wires to length and solder them to D7 & D8.
Step 11: Connecting the Left Button to the Nano.
Cut the left button wire to length and solder it to D6.
Step 12: Connecting the Encoder Wheel to the Nano.
Cut the encoder wires to length and solder them to D9 & D10.
Step 13: Connecting the Speaker to the Nano.
Finally connect your speaker to the Arduino. The + will go to D5 and the - will go to ground. Since the grounds were taken, I used the USB shielding since it has a lot of solder holding it place. Attach the USB mini cable and feed it through to the opening for the mouse cable. In this example I had to fit it between the scroll wheel and the scroll wheel button, so I stripped a bit of insulation to get it to fit in the narrow gap.
Step 14: Preparing a Smooth Surface for Speaker Mounting,
Inspect the underside of the mouse lid. Usually there will be some kind of spacer and mounting components to keep the mouse from easily collapsing along with something to hold the button assembly on. This mouse has a thin layer of plastic that runs across the entire surface that serves as the buttons which press the inside microswitches. This is held in place by the white plastic piece shown above. I found that I can use that area for the speaker if I hot glue the pivot for the button when gluing the speaker. Clip off anything that might get in the way of the speaker.
Step 15: Drill and Widen Hole for Speaker Mounting.
Mark a spot for the speaker opening and drill through it with a small bit. This pilot hole marks the location for further drilling with a larger bit. If the hole is expanded too quickly the plastic can crack. Start by removing the button assembly and then widening each part separately with a quarter inch bit and then with a conical step bit. Clean up the edges with a knife, deburring tool or a round file.
Step 16: Attach Any Loose Mechanical Components and Mount the Speaker.
First glue down any mechanical parts (such as the button hinge in this example) using the hot glue gun. This might not be necessary, it is dependent of the model of mouse. Then position the speaker and glue around the edges to secure it in place. I usually start with one blog of glue, flip it over while it is still hot to center it and let that dry. Then finish it by following the perimeter of the speaker, being careful not to get glue onto the speaker cover, or covering any of the rear grill.
Step 17: Reassemble, Upload / Edit Code.
Reattach the mouse lid to the body. If it dosn't fit, reposition the wires and make sure that the screw holes aren't covered. Screw it together and connect it to a computer to upload the software using the Arduino IDE. If you are using Nano knockoffs on a Mac you might have to download additional drivers to upload the file. The code can be downloaded from here.
Disconnect from the computer and connect to a USB power supply. Have fun!
Recommended mods: Support for more audio waveforms, rechargeable battery support, Bluetooth functionality, CV output...