Real-time Audio to MIDI Converter.

Introduction: Real-time Audio to MIDI Converter.

Namaste people! This is a project that I worked on for one of my courses ( Real-Time Digital Signal Processing) in my bachelor's program. The project aims at making a DSP system that "listens" audio data and outputs MIDI messages of corresponding notes over UART. Arduino Nano was used for this purpose. Long story short the micro-controller does an FFT on incoming audio data and does some analysis of the peaks and sends appropriate MIDI message. Don't be bothered about the MOSFETs though because they are for some other project (which will be put up later on instructables as well) and are not required for this project. So let's get started already!!

Step 1: Components Required

We'll need the following components to build this project although many of these are generic and can be substituted with their equivalents. Also refer the circuit diagram to work out and hunt for better implementations.

Component Quantity

1. Electret Microphone. 1

2. 30 Kilo Ohm resistor. 1

3. 150 Kilo Ohm resistor. 1

4. 100 ohm resistor. 1

5. 2.2 Kilo Ohm resistors. 3

6. 10 Kilo Ohm preset pot. 1

7. 10 Kilo Ohm trimmer pot. 1

8. 47 Kilo Ohm stereo pot. 1

9. 470 Ohms resistors. 2

10. 0.01uF capacitors. 2

11. 2.2uF capacitors. 3

12. 47uF capacitors. 2

13. 1000uF capacitor. 1

14. 470uF capacitor. 1

15. 7805 voltage regulator. 1

16. Female and Male header strip. 1 each

17. Barrel Jack connector. 1

18. 12 V 1 Amp DC Adapter. 1

19. SPST switch. (Optional) 1

20. Perfboard. 1

Step 2: Technical Specifications.

Sampling frequency: 3840 samples/sec

Number of samples per FFT: 256

Frequency Resolution: 15Hz

Refresh rate: Around 15 Hz

The lower and higher scales of the musical notes are not captured correctly. Lower notes suffer from low frequency resolution where as higher frequencies suffer from low sampling rates. The arduino is already out of memory so there is no way to get better resolution. And better resolution will come at a cost of reduced refresh rate so trade-off is inevitable. Layman version of Heisenberg's uncertainty principle.

The primary difficulty is the exponential spacing between notes (As seen in the figure. Every impulse on frequency axis is a musical note). Algorithms like LFT might help but that's a bit advanced and little complicated for a device like arduino Nano.

Step 3: Circuit Diagrams.

Note: Do not be bothered by the three MOSFETs and the screw terminals in the pictures. They are not required for this project. Notice that the microphone input board is removable or as they call it Modular. A small description of the various blocks is given below.

1) The two 470 ohm resistors combine the stereo audio signal to mono audio signal. Ensure that the ground of signal in goes to virtual ground (vg in the circuit diagram) and not to the ground of the circuit.

2) The next block is a 2nd order sallen-key low pass filter which is responsible for band limiting the input signal to avoid aliasing. Since we are working with only +12v supply we bias the op-amp by making a RC voltage divider. that fools the op-amp into thinking that the supply is 6 0 -6 volts supply (dual rail) where vg is the ground reference for the op amp.

3) Then the output is low pass filtered to block DC offset of 6 volts and coupled with DC of around 0.55 volts because the ADC will be configured to use the internal 1.1 v as Vref.

Note: The pre-amplifier for the electret microphone is not the best circuit on the internet. A circuit involving op-amp would've been a better choice. We desire the frequency response to be as flat as possible. The 47 kilo ohms stereo pot is used to define the cut-off frequency which should be typically half the sampling frequency. The 10 kilo ohm preset (The small pot with white head) is used to tune the gain and the Q value of the filter. The 10 kilo ohm trimmer pot (one with a metallic tuning knob that looks like a small flat head screw) is used to set the voltage to be as close as half Vref.

Note: When you're connecting the Nano to P.C. keep the SPST switch open else closed. Keep special care of this failing to do so can harm the circuit/computer/voltage regulator or any combination of the above.

Step 4: Necessary Applications and IDEs.

  1. For coding the Arduino Nano I went with the primitive AVR studio 5.1 because it seems to work for me. You can find the installer here.
  2. For programming the Arduino Nano I used Xloader. Its really easy to use light weight tool to burn .hex files to Arduinos. You can get it here.
  3. For little bonus mini project and tuning the circuit I used processing. You can get it from here although make there are major changes in every revision so you might have to fiddle with deprecated functions to make the sketch work.
  4. FL studio or any other MIDI processing software. You can get FL studio limited access version for free from here.
  5. Loop MIDI creates a virtual MIDI port and is detected by FL studio as if it was a MIDI device. Get a copy of the same from here.
  6. Hairless MIDI is used to read MIDI messages from COM port and send it to loop MIDI port. It also debugs MIDI messages real-time which makes debugging convenient. Get Hairless MIDI from here.

Step 5: Relevant Codes for Everything.

I would like to thank Electronic Lifes MFG (Website Here!!) for the fixed point FFT library that I utilized in this project. The library is optimized for mega AVR family. This is the link to library files and codes that he used. I am attaching my code below. It includes the processing sketch and the AVR C code as well. Please note that this is the configuration that worked for me and I do not take any responsibility what-so-ever if you damage any thing because of these codes. Also, I had lot of issues trying to make the code work. For example, DDRD (Data Direction Register) has DDDx (x = 0-7) as bit masks instead of the conventional DDRDx (x = 0-7). Watch out for these errors while compiling. Also changing the micro-controller affects these definitions so keep an eye on this as well while dealing with compilation errors. And if you're wondering why the project folder is called DDT_Arduino_328p.rar, well let's just say that it was very dark in the evening when I started and I was lazy enough to not turn on the lights. :P

Coming to the processing sketch, I used processing 3.3.6 to write this sketch. You'll need to set the COM port number in the sketch manually. You can check the comments in the code.

If anyone can help me port the codes to Arduino IDE and latest processing version, I'd be glad and will give credits to the developers\contributors as well.

Step 6: Setting It Up.

  1. Open the code and compile the code with #define pcvisual uncommented and #define midi_out commented.
  2. Open xloader and browse to the directory with code, browse to the .hex file and burn it to nano by selecting appropriate board and COM port.
  3. Open the processing sketch and run it with appropriate COM port index. If everything goes well you should be able to see a spectrum of the signal on pin A0.
  4. Get a screw driver and turn the trimmer pot until the spectrum is flat (DC component should be close to zero). Do not input any signal to the board then. (Do not attach the microphone module).
  5. Now use any sweep generator tool like this to give input to the board from the micro-phone and observe the spectrum.
  6. If you do not see a sweep of frequencies, decrease the cut-off frequency by changing the 47 kilo ohm resistance. Also increase the gain by using the 10 kilo ohm preset pot. Try to obtain a flat and prominent sweep output by changing these parameters. This is the fun part (the little bonus!), play your favorite songs and enjoy their real-time spectrum. ( Watch the video)
  7. Now compile the embedded C code again this time with #define pcvisual commented and #define midi_out uncommented.
  8. Reload the new compiled code onto arduino Nano.
  9. Open LoopMidi and create a new port.
  10. Open FL studio or other MIDI interface software and make sure the loop midi port is visible in the MIDI port settings.
  11. Open hairless MIDI with arduino connected. Select output port to be the LoopMidi port. Go to settings and set the Baud rate to be 115200. Now select the COM port corresponding to Arduino Nano and open the port.
  12. Play some "pure" tones near the microphone and you should hear the corresponding note hit in the MIDI software as well. If there is no response try lowering up_threshold defined in the C code. If the notes are getting triggered randomly then increase the up_threshold.
  13. Get your piano and test how fast your system is!! Best thing is that in the goldy-lock zone of notes it can comfortably detect multiple simultaneous key presses easily.

Note: When the COM port is accessed by one application it cannot be read by another. For example if Hairless MIDI ws reading COM port, Xloader would not be able to flash the board.

Step 7: Results/Videos.

That's it for now guys! Hope you like it. If you have any suggestions or improvements in the project let me know in the comment section. Peace!

Microcontroller Contest

Participated in the
Microcontroller Contest

Be the First to Share

    Recommendations

    • The 1000th Contest

      The 1000th Contest
    • Battery Powered Contest

      Battery Powered Contest
    • Hand Tools Only Challenge

      Hand Tools Only Challenge

    14 Discussions

    1
    ptecsaba
    ptecsaba

    Question 19 days ago on Step 1

    Hi Vibhore,

    What would you change for converting a clarinet to midi? Harmonically rich instrument as you might know. What is the best option for material?

    I have seen this arduino project as well and was wondering what the difference is compared to yours: https://www.arduino.cc/en/Tutorial/AnalogToMidi

    Thanks,
    Csaba

    0
    vibhore
    vibhore

    Answer 17 days ago

    Hello Csaba,
    Your request is something I struggled with while making this project. :P
    So the problem with Harmonically rich instruments is two-folds. First the
    Sampling rate has to be higher even for low notes if the overtones are to
    be detected and second the detection algorithm has to decouple two notes
    separated by octaves knowing the frequency composition of individual notes.
    Its sort of like a vector decomposition. (In this case decomposing the energy
    in frequency domain into a space made of musical notes). This is achievable
    with little effort but I am not sure if a simple arduino is up for it. The other project
    uses very similar concept as this one so I shouldn't see any performance
    improvement except for the fact a better micro-controller is used so accuracy
    would be better.

    Cheers,
    Vibhore

    0
    ptecsaba
    ptecsaba

    Reply 15 days ago

    Thank you very much for the response. In my project, I thought to use a high quality piezo microphone designed for clarinets which takes care of some issues surrounding the analog signal. I have this piezo michrophone: https://www.ebay.com.au/itm/322463860555?ViewItem=...
    Do you suggest I do your project with the better micro-controller?

    Thanks,
    Csaba

    0
    vibhore
    vibhore

    Reply 14 days ago

    The piezo transducer will not solve the problem caused by harmonic richness of the clarinet but it'll give way better signal to noise ratio as it's a contact pickup so there shouldn't be any false note detection. Since your instrument is clarinet (on the higher frequency side) I'd definitely advice you to go with something like an ESP32 along with an external 12 or 14 bit ADC. You'll get plenty of RAM to work with and the high clock speed will mean you will be able to perform those FFT operations very fast without CPU bottle-necking. The DMA will also help move audio samples into buffer without disturbing the CPU. Plus if you feel adventurous you can emulate a bluetooth MIDI device and record your playback live!

    0
    ptecsaba
    ptecsaba

    Reply 14 days ago

    Thanks. Really good insight. I'd like to talk to you in PM. My email ptecsaba@gmail.com. I have something in mind. Let me know if you are interested. Regards.

    0
    vibhore
    vibhore

    Reply 13 days ago

    Sure! I'll ping you soon!

    0
    scuffia74
    scuffia74

    Question 6 months ago

    Hallo, do you think it can work also with magnetic pickup on the audio input?
    thak you Fabio

    0
    vibhore
    vibhore

    Reply 4 months ago

    Hi there, yes it can but the pre-amplifier circuit should be changed suitably as magnetic pickup and electret mics work differently

    2
    aotelin
    aotelin

    Question 1 year ago on Step 7

    Hello,
    My name's Alex A, I'm searching a convert audio to midi.
    Could we imagine, to have a test, or to buy your concept...?
    Thanks for your answer.
    Best regards
    Alex A

    0
    vibhore
    vibhore

    Reply 1 year ago

    Hello Alex, I'm sorry for the late reply. My project was just a concept and works for really specific instruments and frequencies. If you still interested in buying it let me know :P

    2
    fred1896
    fred1896

    Reply 1 year ago

    sup vibhor, would your project work with a basic mp3 recording of a piano song?

    0
    vibhore
    vibhore

    Reply 1 year ago

    Hello fred, this project works for "pure" instruments, by pure I mean those which donot have harmonics or have very low harmonics and piano is harmonically rich instrument so it might not work so well. Flute or violin would work better.

    0
    fred1896
    fred1896

    Reply 1 year ago

    ok thanks

    3
    seamster
    seamster

    2 years ago

    Cool, thanks for sharing your project!