The 8-bit style melody card introduced in this tutorial combines simple DIY electronics with downsampled Public Domain images and sounds. The card itself is made of sturdy cardboard, foam board and prints of famous paintings. The paintings have been pixelated in order to give them the look of retro computer graphics. Inside the card is a circuit consisting of an ATtiny85 microcontroller, a small speaker, a LED, resistors, a switch and a button cell battery. The microcontroller contains a surprise melody created from an archival music recording. The mp3 recording has been first converted into MIDI, and the to Arduino code, which was uploaded on the ATtiny. The surprise element comes from this conversion process, as the final melody is quite different from the original. When the card is opened, the melody starts playing. This is achieved by attaching a piece of yarn to the switch - when the card is opened, the yarn is pulled, triggering the melody.
The melody card was originally developed for the "Downsampling Masterpieces Workshop", which I gave in Oct 2014 at the Mozilla festival as part of Europeana Creative's "Redesign Your Cultural Heritage" session. The workshop explored how digital cultural heritage content can be used in tangible works, and what kind of expressive opportunities emerge from "downsampling" images and sounds.
Public Domain content used in the example card:
Front cover:“Het melkmeisje”, painting by Johannes Vermeer, ca. 1660 (original image from Rijksmuseum)
Back cover: “Brieflezende vrouw”, painting by Johannes Vermeer, ca. 1663 (original image from Rijksmuseum)
Inside: Plate from Delf, Vermeer’s hometown, c. 1670 - c. 1700 (original image from Rijksmuseum)
Music piece turned into a surprise melody: “Laulu ilman sanoja”, song by Kuula, Toivo in 1910, performed in 1929 by Erik Cronvall (violin) and Erkki Linko (piano) (original recording from National Library of Finland)
Electronics + tools:
Other materials + tools:
Public Domain images
As explained, the melody card was developed for a workshop that explored creative uses for digital cultural heritage content. Thus, we used as source materials a set of Public Domain images of famous paintings provided by the Rijksmuseum. Following the 8-bit aesthetic of the melodies, the images were pixelated (explained below) and printed in A4 size. You can of course use whatever images or drawings you like if you make a melody card.
In case you are interested in exploring more Public Domain images, the Rijksmuseum provides a lot more beautiful high-resolution images - this is also why we wanted to promote this collection even though we did not need the high-resolution, quite the contrary. You can also check out for example the Public Domain Review and The British Library's Flickr Commons collections.
Pixelating the images in Photoshop
The original images of paintings were pixelated in Photoshop in order to give them a retro computer graphics look. There are probably many ways to pixelate an image, but here is one. First, save a new version of the image in GIF format with only web-safe Windows colors (“Save for Web…”, web-safe colors can be selected from the small menu icon next to the color table). Then, resize the image to very small size, such as 100 px width (“Image” > “Image size”). Next, open the newly created small GIF image in Photoshop and resize it again to about ten times larger. This will create the pixelated look. Here you can see few examples made from the Rijksmuseum images, Johannes Vermeer's Milkmaid (ca. 1660) and Vincent van Gogh's self-portrait (1887). You can find below all the images created for the workshop. If you use them, it would be nice if you attribute the Rijksmuseum as the original provider even though the images are in Public Domain.
As explained, archival recordings of famous music pieces were used in the workshop as a basis of the card melodies. The original 16 recordings were mp3s, and each of them was used to create a different melody. You can find below a list of the all the source recordings. The idea was to create using the mp3s simple Arduino codes that would play the melodies, and to upload these on the small and affordable ATtiny85s using Arduino as an ISP programmer. In order to achieve this, the mp3s were first converted to MIDI, and ultimately to code.
You might wonder why to go though this complicated procedure - why not just make the Arduino code right away? You can surely do this. The code used for the cards is even very simple and consists of mainly tone() commands. One reason why the melodies were created from mp3 files was because the workshop explored creative possibilities in using cultural heritage content from Europeana, and mainly mp3s were available. Another reason for the chosen procedure was that, while playing around with the mp3s, I discovered that the imperfections and unpredictable nature of the conversions to MIDI and subsequently to code resulted in quite delightful surprise melodies. So the surprise element seemed like an advantage, rather than drawback. Below are the melody generation steps in more detail in case you are interested to know more.
The audio-to-MIDI conversion was done using Ableton Live (instructions on their website). Instead of using the whole mp3 recording, which was usually very long, a minute or so from the beginning was used. I tried both "Convert Harmony to New MIDI Track" and "Convert Melody to New MIDI Track" (under right-click) in Ableton in order to see which resulted in a more funny melody. After this I exported the MIDI file. You can find the MIDI files below.
You could of course also skip the audio-to-MIDI conversion, and use right away MIDI as your source file in case you wish to play around with this format. There are plenty of openly licensed MIDI files out there, for example on Wikimedia Commons.
From MIDI to Arduino code
The MIDI to Arduino online tool provided by Dominic Canare (CC BY-SA) was used for converting the generated MIDI files to Arduino code. This tool takes always MIDI track 1 as the one that is converted to the Arduino tone commands. The other MIDI tracks are converted to digitalWrite(), and can be used to e.g. blink a LED. Because of this, ensure that the melody that you want to be played is on track 1. I used an open source program called Aria Maestosa to edit the MIDI tracks so that I had two identical tracks, one numbered 1 and the other 2 (“Settings” > “Channel management” > “Manual” to edit track numbers). Track 1 was played as notes and track 2 provided the commands for a blinking LED.
After I had edited the track numbers, I uploaded the MIDI file to the online conversion tool, and copied the produced code to my sketch. Usually the code was too large for the ATtiny, so I reduced it from the end to just below 8 kilobytes. The conversion tool also created quite a few unnecessary command lines, so I cleaned up most of the extra code before uploading it on the ATtiny (but not all, so the code is not a good example of programming). The melody was also moved into the program loop in order to keep it playing repeatedly. You can find below all the 16 different melody sketches - as you can see they are simple and consist of just lots and lots of tone() and digitalWrite() commands. You can surely make different kind of code if you are up to sound programming - for the melody card the most important thing is that it plays a melody, or a sound of some sort.
Uploading the melodies on the ATtiny using an Arduino
Each of the Arduino melody codes was uploaded on an ATtiny85 microcontroller using a regular Arduino Uno board as an ISP programmer. Here are instructions by Jeko on how to program ATtiny85 (or 45) using the Arduino IDE - remember to also upload the “Arduino as ISP” example on the Arduino to get it working. I used at first a breadboard for programming, but found that making a dedicated shield with programming socket is much more handy. You can see a self-made one in the photo.
As timing commands like delay() do not work straightforwardly on the ATtiny when programming it with an Arduino, I spent some time to adjust the timing parameters (mainly the durations of the delays) in the Arduino code to find a suitable speed for the melody. As a board choice I selected “ATtiny85 @ 1MHz (internal oscillator, BOD disabled)” - this seemed to work in my case.
Another issue that I encountered was that I got an error when trying to upload large sketches on the ATtiny, but resolved the problem with the solution provided in this thread.
List of the Public Domain source recordings:
Before soldering, the legs of the ATtiny were bent flat in order to make it more shallow so that it fits inside the card. A small hole was also drilled through the switch lever so that a yarn could be attached to it later in order to make the switching mechanism (see next step for details).
The electronics were soldered according to the above schematic, which follows the ATtiny85 pin set up in the Arduino sketches attached to this tutorial. If you use a different code, check the ATtiny pins, for example from here. Few things to pay extra attention to:
The basis of the card was made from sturdy, grey cardboard. It consisted of the front and back covers, as well as a thin middle piece. The width of the middle piece was measured so that the electronics compartment made of foam board would fit exactly in between (see next step). The pixelated painting prints were cut to the size of the card and attached with double-sided tape. Lastly, the edges of the card were secured and decorated with a black fabric tape.
Hot glueing the electronics
The electronics compartment inside the card was made from 5mm thick foam board. This height guaranteed that the highest electronics parts could still fit in so that the card could be closed fully. First, a piece of foam board was cut which matched the size of the right hand page inside the card. Then a large hole was cut in this piece for placing the electronics. The hole was cut such that there was space to attach the switch close to the bottom left part of the page (close to the fold of the card). This enabled making easily the switching mechanism that would play the melody when opening the card. The electronics compartment was also decorated with pixel prints and fabric tape, and attached with double-sided tape on the card.
After this, the electronics were hot glued in the compartment. First, a piece of yarn was attached to the hole in the switch lever. The electronics were then taped gently to the card for the time of glueing, so that they would not move, but this is not necessary. However, pay extra attention to the following:
Making the switching mechanism
Once the electronics were tightly in place, the switching mechanism shown in the last two photos was made. The yarn hanging from the switch was attached to the adjacent card page so that it pulled the switch lever to the ON position when opening the card, thus playing the melody. First the length and position of the yarn was adjusted so that the melody would not be triggered too early, but only when the card is almost fully open. Then a small piece of double-sided tape was attached on the optimal position, the yarn was placed over it, and lastly a piece of black fabric tape was attached over the yarn in order to secure it.