Real-time Audio to MIDI Converter.

Introduction: Real-time Audio to MIDI Converter.

Namaste people! This is a project that I worked on for one of my courses ( Real-Time Digital Signal Processing) in my bachelor's program. The project aims at making a DSP system that "listens" audio data and outputs MIDI messages of corresponding notes over UART. Arduino Nano was used for this purpose. Long story short the micro-controller does an FFT on incoming audio data and does some analysis of the peaks and sends appropriate MIDI message. Don't be bothered about the MOSFETs though because they are for some other project (which will be put up later on instructables as well) and are not required for this project. So let's get started already!!

Step 1: Components Required

We'll need the following components to build this project although many of these are generic and can be substituted with their equivalents. Also refer the circuit diagram to work out and hunt for better implementations.

Component Quantity

1. Electret Microphone. 1

2. 30 Kilo Ohm resistor. 1

3. 150 Kilo Ohm resistor. 1

4. 100 ohm resistor. 1

5. 2.2 Kilo Ohm resistors. 3

6. 10 Kilo Ohm preset pot. 1

7. 10 Kilo Ohm trimmer pot. 1

8. 47 Kilo Ohm stereo pot. 1

9. 470 Ohms resistors. 2

10. 0.01uF capacitors. 2

11. 2.2uF capacitors. 3

12. 47uF capacitors. 2

13. 1000uF capacitor. 1

14. 470uF capacitor. 1

15. 7805 voltage regulator. 1

16. Female and Male header strip. 1 each

17. Barrel Jack connector. 1

18. 12 V 1 Amp DC Adapter. 1

19. SPST switch. (Optional) 1

20. Perfboard. 1

Step 2: Technical Specifications.

Sampling frequency: 3840 samples/sec

Number of samples per FFT: 256

Frequency Resolution: 15Hz

Refresh rate: Around 15 Hz

The lower and higher scales of the musical notes are not captured correctly. Lower notes suffer from low frequency resolution where as higher frequencies suffer from low sampling rates. The arduino is already out of memory so there is no way to get better resolution. And better resolution will come at a cost of reduced refresh rate so trade-off is inevitable. Layman version of Heisenberg's uncertainty principle.

The primary difficulty is the exponential spacing between notes (As seen in the figure. Every impulse on frequency axis is a musical note). Algorithms like LFT might help but that's a bit advanced and little complicated for a device like arduino Nano.

Step 3: Circuit Diagrams.

Note: Do not be bothered by the three MOSFETs and the screw terminals in the pictures. They are not required for this project. Notice that the microphone input board is removable or as they call it Modular. A small description of the various blocks is given below.

1) The two 470 ohm resistors combine the stereo audio signal to mono audio signal. Ensure that the ground of signal in goes to virtual ground (vg in the circuit diagram) and not to the ground of the circuit.

2) The next block is a 2nd order sallen-key low pass filter which is responsible for band limiting the input signal to avoid aliasing. Since we are working with only +12v supply we bias the op-amp by making a RC voltage divider. that fools the op-amp into thinking that the supply is 6 0 -6 volts supply (dual rail) where vg is the ground reference for the op amp.

3) Then the output is low pass filtered to block DC offset of 6 volts and coupled with DC of around 0.55 volts because the ADC will be configured to use the internal 1.1 v as Vref.

Note: The pre-amplifier for the electret microphone is not the best circuit on the internet. A circuit involving op-amp would've been a better choice. We desire the frequency response to be as flat as possible. The 47 kilo ohms stereo pot is used to define the cut-off frequency which should be typically half the sampling frequency. The 10 kilo ohm preset (The small pot with white head) is used to tune the gain and the Q value of the filter. The 10 kilo ohm trimmer pot (one with a metallic tuning knob that looks like a small flat head screw) is used to set the voltage to be as close as half Vref.

Note: When you're connecting the Nano to P.C. keep the SPST switch open else closed. Keep special care of this failing to do so can harm the circuit/computer/voltage regulator or any combination of the above.

Step 4: Necessary Applications and IDEs.

  1. For coding the Arduino Nano I went with the primitive AVR studio 5.1 because it seems to work for me. You can find the installer here.
  2. For programming the Arduino Nano I used Xloader. Its really easy to use light weight tool to burn .hex files to Arduinos. You can get it here.
  3. For little bonus mini project and tuning the circuit I used processing. You can get it from here although make there are major changes in every revision so you might have to fiddle with deprecated functions to make the sketch work.
  4. FL studio or any other MIDI processing software. You can get FL studio limited access version for free from here.
  5. Loop MIDI creates a virtual MIDI port and is detected by FL studio as if it was a MIDI device. Get a copy of the same from here.
  6. Hairless MIDI is used to read MIDI messages from COM port and send it to loop MIDI port. It also debugs MIDI messages real-time which makes debugging convenient. Get Hairless MIDI from here.

Step 5: Relevant Codes for Everything.

I would like to thank Electronic Lifes MFG (Website Here!!) for the fixed point FFT library that I utilized in this project. The library is optimized for mega AVR family. This is the link to library files and codes that he used. I am attaching my code below. It includes the processing sketch and the AVR C code as well. Please note that this is the configuration that worked for me and I do not take any responsibility what-so-ever if you damage any thing because of these codes. Also, I had lot of issues trying to make the code work. For example, DDRD (Data Direction Register) has DDDx (x = 0-7) as bit masks instead of the conventional DDRDx (x = 0-7). Watch out for these errors while compiling. Also changing the micro-controller affects these definitions so keep an eye on this as well while dealing with compilation errors. And if you're wondering why the project folder is called DDT_Arduino_328p.rar, well let's just say that it was very dark in the evening when I started and I was lazy enough to not turn on the lights. :P

Coming to the processing sketch, I used processing 3.3.6 to write this sketch. You'll need to set the COM port number in the sketch manually. You can check the comments in the code.

If anyone can help me port the codes to Arduino IDE and latest processing version, I'd be glad and will give credits to the developers\contributors as well.

Step 6: Setting It Up.

  1. Open the code and compile the code with #define pcvisual uncommented and #define midi_out commented.
  2. Open xloader and browse to the directory with code, browse to the .hex file and burn it to nano by selecting appropriate board and COM port.
  3. Open the processing sketch and run it with appropriate COM port index. If everything goes well you should be able to see a spectrum of the signal on pin A0.
  4. Get a screw driver and turn the trimmer pot until the spectrum is flat (DC component should be close to zero). Do not input any signal to the board then. (Do not attach the microphone module).
  5. Now use any sweep generator tool like this to give input to the board from the micro-phone and observe the spectrum.
  6. If you do not see a sweep of frequencies, decrease the cut-off frequency by changing the 47 kilo ohm resistance. Also increase the gain by using the 10 kilo ohm preset pot. Try to obtain a flat and prominent sweep output by changing these parameters. This is the fun part (the little bonus!), play your favorite songs and enjoy their real-time spectrum. ( Watch the video)
  7. Now compile the embedded C code again this time with #define pcvisual commented and #define midi_out uncommented.
  8. Reload the new compiled code onto arduino Nano.
  9. Open LoopMidi and create a new port.
  10. Open FL studio or other MIDI interface software and make sure the loop midi port is visible in the MIDI port settings.
  11. Open hairless MIDI with arduino connected. Select output port to be the LoopMidi port. Go to settings and set the Baud rate to be 115200. Now select the COM port corresponding to Arduino Nano and open the port.
  12. Play some "pure" tones near the microphone and you should hear the corresponding note hit in the MIDI software as well. If there is no response try lowering up_threshold defined in the C code. If the notes are getting triggered randomly then increase the up_threshold.
  13. Get your piano and test how fast your system is!! Best thing is that in the goldy-lock zone of notes it can comfortably detect multiple simultaneous key presses easily.

Note: When the COM port is accessed by one application it cannot be read by another. For example if Hairless MIDI ws reading COM port, Xloader would not be able to flash the board.

Step 7: Results/Videos.

That's it for now guys! Hope you like it. If you have any suggestions or improvements in the project let me know in the comment section. Peace!

Microcontroller Contest

Participated in the
Microcontroller Contest

Be the First to Share

    Recommendations

    • Game Design: Student Design Challenge

      Game Design: Student Design Challenge
    • Baking Contest

      Baking Contest
    • Cold Challenge

      Cold Challenge

    28 Comments

    0
    AbahArif79
    AbahArif79

    8 months ago

    cool, I'll try to make it tomorrow..thanks for sharing the knowledge :)

    0
    AbahArif79
    AbahArif79

    8 months ago on Step 7

    cool, I'll try to make it tomorrow..thanks for sharing the knowledge :)

    0
    bguess71
    bguess71

    Question 10 months ago

    One additional question. On the schematic, you have off to the side a circuit with two 2.2uF capacitors and two 2.2K resistors. It is not connected to anything. What is that for? See image attached.

    Schematic Question.jpg
    0
    vibhore
    vibhore

    Answer 10 months ago

    Hi there, the circuit creates a virtual ground VG that is used by the op amp circuit. It basically adds a DC shift to input audio. Also it is connected. See "VG" in the circuit.

    New Bitmap Image.bmp
    0
    bguess71
    bguess71

    Reply 10 months ago

    Thanks for the quick response. So forgive my ignorance as I am a hobbyist. So all the VGs would tie into that VG circuit? Excited to try this out for a project I’m working on.

    0
    vibhore
    vibhore

    Reply 10 months ago

    Yes that is correct. We call it netlisting. Basically if a circuit becomes too complicated to connect using wires. We add names to wires/connections and all of those are meant to be shorted together

    0
    bguess71
    bguess71

    10 months ago on Step 3

    What is the Op AMp chip that you are using? It looks like a TI chip L? 35?? something. Thanks.

    1
    bguess71
    bguess71

    Best Answer 10 months ago

    Actually I think I figured it out to be a LF353P

    0
    cameronkkk
    cameronkkk

    Question 1 year ago

    Hi,
    it's very cool project, does it work well with the acoustic piano?
    and how many notes can be correctly transform into midi simultaneously?
    A C13 chord contains 7 notes.


    Thanks

    0
    vibhore
    vibhore

    Answer 1 year ago

    Hi there,
    Piano is a difficult instrument to process real time because it is harmonically rich compared to something like a flute which has almost pure tones in every note. As far as polyphony is concerned I could decode 3 note chords relatively easily but any more is kind of hard with this micro-controller. Having faster processors running better algorithms would be the way to go for polyphony and piano note decoding

    0
    Axo123
    Axo123

    1 year ago on Step 1

    Great work! Happen To sell these? I would need One..

    0
    Axo123
    Axo123

    Reply 1 year ago

    Would use to trigger synths with vocals or something

    1
    sas152ana
    sas152ana

    Question 1 year ago

    Hello Vibhore,
    I want to use 3.5 jack for input signal instead of microphone. Can I just solder it, or I should somehow change scheme?
    Thank you,
    Alex

    1
    vibhore
    vibhore

    Answer 1 year ago

    Hi Alex,
    For giving input from 3.5mm jack you can directly hook up the left and right channels in the schematic given below

    865382bf-7649-4c25-bff9-6879c88409c6.png
    1
    ptecsaba
    ptecsaba

    Question 2 years ago on Step 1

    Hi Vibhore,

    What would you change for converting a clarinet to midi? Harmonically rich instrument as you might know. What is the best option for material?

    I have seen this arduino project as well and was wondering what the difference is compared to yours: https://www.arduino.cc/en/Tutorial/AnalogToMidi

    Thanks,
    Csaba

    0
    vibhore
    vibhore

    Answer 2 years ago

    Hello Csaba,
    Your request is something I struggled with while making this project. :P
    So the problem with Harmonically rich instruments is two-folds. First the
    Sampling rate has to be higher even for low notes if the overtones are to
    be detected and second the detection algorithm has to decouple two notes
    separated by octaves knowing the frequency composition of individual notes.
    Its sort of like a vector decomposition. (In this case decomposing the energy
    in frequency domain into a space made of musical notes). This is achievable
    with little effort but I am not sure if a simple arduino is up for it. The other project
    uses very similar concept as this one so I shouldn't see any performance
    improvement except for the fact a better micro-controller is used so accuracy
    would be better.

    Cheers,
    Vibhore

    0
    ptecsaba
    ptecsaba

    Reply 2 years ago

    Thank you very much for the response. In my project, I thought to use a high quality piezo microphone designed for clarinets which takes care of some issues surrounding the analog signal. I have this piezo michrophone: https://www.ebay.com.au/itm/322463860555?ViewItem=...
    Do you suggest I do your project with the better micro-controller?

    Thanks,
    Csaba

    0
    vibhore
    vibhore

    Reply 2 years ago

    The piezo transducer will not solve the problem caused by harmonic richness of the clarinet but it'll give way better signal to noise ratio as it's a contact pickup so there shouldn't be any false note detection. Since your instrument is clarinet (on the higher frequency side) I'd definitely advice you to go with something like an ESP32 along with an external 12 or 14 bit ADC. You'll get plenty of RAM to work with and the high clock speed will mean you will be able to perform those FFT operations very fast without CPU bottle-necking. The DMA will also help move audio samples into buffer without disturbing the CPU. Plus if you feel adventurous you can emulate a bluetooth MIDI device and record your playback live!

    0
    ptecsaba
    ptecsaba

    Reply 2 years ago

    Thanks. Really good insight. I'd like to talk to you in PM. My email ptecsaba@gmail.com. I have something in mind. Let me know if you are interested. Regards.

    0
    vibhore
    vibhore

    Reply 2 years ago

    Sure! I'll ping you soon!