SonicSurface: Phased-array for Levitation, Mid-air Tactile Feedback and Target Directional Speakers

26,752

240

43

Introduction: SonicSurface: Phased-array for Levitation, Mid-air Tactile Feedback and Target Directional Speakers

About: Build your own cutting-edge devices coming directly from UpnaLab. UpnaLab is the future interactive devices lab working with Ultrasonics, electromagnetism, lasers and more.

We will build an array of ultrasonic emitters. It has 256 emitters arranged in a 16x16 grid operating at 40 kHz with individual phase control. It can focus the acoustic power at controllable positions to create dynamic levitation of small particles and tactile feedback. You can use this for your unique artistic installations or projects. A small showcase is shown at the beggining of the video.

This is a relatively hard and expensive project: requires experience with SMD components and the materials are around 200$.

This research was funded by EU Horizon 2020 - 101017746, TOUCHLESS project.

We note that UltraLeap Ltd. comercialices solutions which are already tested and certified.

The cover image is by @slarripa

Supplies

  • 1x PCB: 183x169mm 2 layers (regular parameter)
    • Stencil for the PCB (can be ordered with the PCB)
  • Solder paste
  • 256x ultrasonic emitters (aka transducers) of 1cm diameter operating at 40kHz. Manorshi provides MSO-P1040H07T at a very good price, minimum order is 500 but they will ship less at a higher price. Also Ningbo has good ones FBULS1007P-T
  • 1x FPGA module: https://www.waveshare.com/coreep4ce6.htm
  • 128x Drivers MIC4127 SOIC8. Other compatible ICs are: TC4427a or MIC4478 (supports 32V but are pricey)
  • 32x shift-registers 74hc595 SOIC16
  • 160x 0.1uf capacitors 50v 0805
  • 1x Arduino Nano or an USB-to-UART adaptor
  • 1x DC barrel connector
  • Header Connectors for the FPGA (2x22) (2.54 pitch)
  • Side connectors either pin or headers (2x4 and 2x2) (2.54 pitch)
  • Spacer (either 3d printed or lasercut)
  • Jumper wires
  • isopropyl alcohol (to clean the PCB and the stencil)

Tools:

  • A computer (Linux or Windows) to run:
    • Ultraino simulator (will need JDK11)
    • Quartus 13.0sp1 (for uploading and compiling the FPGA code)
      • Quartus II Software (includes Nios II EDS)
      • Cyclone, Cyclone II, Cyclone III, Cyclone IV device support (includes all variations)
  • FPGA programmer (Blaster USB ALTERA)
  • Multimeter
  • Reflow oven
  • card, razor or squidgee for spreading the paste
  • Tape
  • cloth rag
  • tweezers
  • soldering iron
  • oscilloscope, ideally with 2 channels
  • regulated power supply, ideally with 2 channels

Step 1: Prepare the Board

Be sure to have all the componentes around and that they fit on the PCB board.

Clean the stencil with alcohol and a rag.

Put other boards at the sides of the target board and tape them.

Put the stencil on top and align it so that only metal pads are seen (if you cannot fit it, perhaps you need to flip it), tape it.

Apply the solder paste. Spread it and remove the stencil (clean it in case that you want to use it again).

Step 2: Place SMD Components

Place the drivers, make the marks coincide so that they have the correct orientation. Be aware that the orientatino changes every row.

Place the shift registers, they also have orientation.

Place the ceramic capacitors, they do not have orientation.

Carefully, put the PCB in the oven and use the profile indicated by your paste.

Would be good to check for shortcuts on the 5v rail and 18v rail after baking. The 18V rail is also called the power rail and it is the voltage that powers the emitters, with something between 6v and 16v. The 18v rail is accesible through pins and the DC connector at both sides.

Step 3: Soldering Through-hole Components

Cut the FPGA header connectors to the correct length (22x2 each). You can use the FPGA as a guide to keep the connectors spaced properly, you do not need to fully connect the FPGA. Solder the FPGA connectors.

Solder the power rail connectors, the ports, and the DC barrel at the sides.

Step 4: Programming the FPGA and Checking the Channels

Connect the power cable (usb to very small dc barrel) and the programmer to the FPGA (without being connected to the board or anything else).

Open the programmer: you can open directly the Programmer or the Quartus environment.

Select the primaryNoCalib.jic provided file, select program configure, select your programmer, and click start. This file is for a primary board and does not contain phase calibration for the emitters.

When it finishes, switch off and on the FPGA. L4 should be blinking.

Connect the FPGA into the board paying attention to its orientation (there are white marks on the PCB). Check for shortcuts again.

Put 5v into the 5V rail and 9V in the 18V rail.
Get ground on the board for your scope and check that each channel is generating a 40khz 9vpp signal. Be aware that the signal pin on each circle changes position every two rows, one pin is signal and the other ground.

If the signal is 3.3v there is a problem with the driver. if there is no signal, the problem can be with the driver or the shifter. It is normal to have a couple of channels broken even after trying to repair them.

Step 5: Place and Check the Transducers

We put a lasercut or 3d printed spacer so that the emitters sit flat but this one is not completely necessary if you leave a small gap between transducers and board.
Put all the transducers. You can mark the polarity of the transducers (never trust the manufacturer, check Step 4 of the Acoustic Levitator) and place the marked leg in the signal hole. If you do this, you do not need to do the calibration later on.

Once the transducers have been placed, carefully flip the board using a rigid plank. Solder the transducers.

Put 5v intot he 5v rails and 9v in the power rail. Regular consumption is 0.1A for 5v and 0.9A for the power rail.

Check with a spare transducer connected to the scope that all the transducers are emitting.

Step 6: Calibration

Testing control from the computer

We will use the arduino as a cheap usb-to-serial so that the computer can send commands to the FPGA array. Upload an empty sketch on the Arduino. Provide power to a single board; and connect the Arduino ground, RX and 5v to the board as shown in the schematic.

Run the Ultraino Software. Load the emptyboard simulation.

In the devices tab, select chainnedFPGA protocol, click connect and select the arduino port (usually the last one).
Now press A to select all the transducers (or select all in the Transducers Tab), press S to switch them off, or W to switch them on. You should hear a clicking noise and see a change in the power consumption.

Calibrating the phase deviation and polarity of the emitters

This step is not necessary if you marked the polarity of the transducers and placed them with the marked leg in the signal hole before soldering. You can still check to see if the polarity is correct for every emitter.

You will need a 2 channel oscilloscope, connect one channel to port D and GND to get the reference signal from the board, make this channel the trigger. Connect the other channel to a spare transducer.
With the ultraino software running and connected to the board select the first transducer and switch it on, put the spare transducer on top of it and you should be able to see the reference signal (square) and the received ultrasonic wave (sine) in the oscilloscope.

You will need to adjust the received signal phase to match the reference signal by using the keys H (forward), J (backward) or K (180º change). When the reference signal raises, the received wave should be at its peak. Calibrate all the transducers (M to go to the next one).

You can set this calibration into the simulation file and export it to other programs (e.g. the FPGA firmware). Apply your calibration only on the FPGA or in the simulation but not in both (otherwise they will cancel each other).

If you do not want to keep a simulation file with the calibration for each board, you can hardcode the calibration into the FPGA code. Export it, and copy the numbers into the Distributor block of the FPGA (double click on it for opening the code)

You need to recompile the FPGA code, it will take a long time.

You need to convert the compiled code (sof) to JTAG format (jic). Open the convert program option in the menu, select jic, EPSC16, cyclone IV E, EP4CE6. Select the previously generated sof file, click generate, and use the programmer to upload the generated file into the FPGA. Now you do not need to worry about calibrations for this board.

Step 7: Single Board Levitation

With one board you can levitate over a reflective surface.

Put the board above a surface with a separation of 16 cm, add a slice at 16cm (you can change this distance as your project requires).
Click on the slice and now you can focus at any place that you click (you will need to select the correct checboxes). You can levitate particles at that point (you will need to connect to the board before this).

For multiple points you need to select a complex algorithm, you can use the tab move to control all the particles or individual ones.

Step 8: 3D Levitator

You can use a primary and a secondary board together to levitate in 3D, they are placed one in front of each other.

Use the supports provided and connect the boards as shown in the schematic.

Load the simulation file twoOpposedArrays.

Connect and test that you can switch on and off the top board and the bottom board individually (shortcuts T - select top, B - select bottom, W - switch on, S - switch off).

Add 4 points and create traps there.

You can use a ruler to place the particles at the correct height. The points can be moved in 3D.

Step 9: Miscellaneous

ESP32 controller

For calculating focal points and sending the phases to the boards, it is necessary to run the Ultraino simulator in a PC. For more portable systems, we have programmed some of the basic algorithms into an ESP32 controller. It is possible to program sequences that are trigger with a button. Or to send through uart or wifi/http the position of the desired focal points.

The code for the ESP32 controller can be found here.

It is recommended to use a step down that takes the 18V pins and steps it down to 5V to power the ESP32 and the logic of the boards.

Tactile feedback

For perceiving the focal points with your hand, the sound should be modulated at 200Hz, we provide an FPGA code that does this modualtion but by sequentially switching on and off the different transducers, in order to minimize the audible sound.

Connecting multiple boards in parallel

In the previous step, the boards are connected in series. But it is also possible to connect them in parallel by having a USB-to-UART for each board. Then in the Ultraino code the protocol should be SimpleFPGA for the first board (usually the one on top), and then click on "Con Extra" to connect the rest of the boards.

Make it Fly Challenge

First Prize in the
Make it Fly Challenge

5 People Made This Project!

Recommendations

  • Eggs Challenge

    Eggs Challenge
  • Fiber Arts Challenge

    Fiber Arts Challenge
  • Build a Tool Contest

    Build a Tool Contest

43 Comments

0
aedenGB
aedenGB

Question 5 weeks ago

Thank you for the detailed instructions! I have got my board all together, all channels working. I soldered in the transducers, programmed the FPGA, connected to the 3D Acoustic Sim, but I am having very weird issues. If I switch all transducers on, they all turn on, but if I try to turn individual transducers on one at a time, only ~10% turn on. Weirdest behavior is some of those transducers that don't turn on can be turned on by turning on a combination of that and other transducers. Ex. transducer 256 doesn't turn on if I command it on in the 3D acoustic sim, but it turns on if I command transducer 256 and 253 at the same time (but 253 doesn't turn on).

I'm pretty lost, any chance anyone knows what might be going on?

0
UpnaLab
UpnaLab

Answer 4 weeks ago

What software version are you using? what simulation file? What you burnt into the FPGA?

Thanks.

0
UpnaLab
UpnaLab

Reply 26 days ago

And in the Devs tab which protocol you select "SimpleFPGA" "ChainnedFPGA"?

0
aedenGB
aedenGB

Reply 25 days ago

I've selected "ChainedFPGA". Should I also be selecting "SimpleFPGA" somewhere?

Screenshot 2022-07-20 125730.pngScreenshot 2022-07-20 125730.png
0
UpnaLab
UpnaLab

Reply 19 days ago

For a single array any of those would work "ChainnedFPGA" or "SimpleFPGA" you can try on that combobox where Chainned FPGA appears. Do you know that all the emitters are on or off because of the power consumption? at 9v it should be around 0Amps and 1.1Amps respectively.

0
aedenGB
aedenGB

Reply 14 days ago

I tried the SimpleFPGA and ChainedFPGA... unfortunately no difference. They have the exact same issue. I can tell all the emitters are on because of the current draw, and I have checked them all with an oscilloscope, both with an additional transducer in front of them, and also with the oscilloscope probes directly on the pins on the pcb.

Are there other things I can try?

0
UpnaLab
UpnaLab

Reply 13 days ago

We will solve it together, no worries. Can you send me an e-mail to asier.marzo @ unavarra.es attach the simulation file that you are using and would be very useful a video (with the phone for example) of the steps that you are taking.

0
aedenGB
aedenGB

Reply 4 days ago

Just sent you an email with all the details.

0
deanfishman
deanfishman

Question 3 months ago

Hi,

I'd like to firstly thank you for this detailed instrutable.
I am currently working on an honors thesis on the haptic application of this hardware. I was hoping you could provide a little more guidance on how to utilize this hardware to achieve mid air tactile feedback. My topic is on projecting haptic Braille for the visually impaired via two ultrasonic arrays.

Kind regards,
Dean

0
wangyuanyi86
wangyuanyi86

7 months ago

Hello I have some problems in Step 4. I open Quartus II 15.0 Programmer and select the PrimaryNoCalib.jic. After the download is complete, L2 is on and L1 is blinking.

FPGA.jpg
0
UpnaLab
UpnaLab

Reply 6 months ago

I am an idioat, I meant L1 should be blinking.

0
wangyuanyi86
wangyuanyi86

Reply 6 months ago

but L2 is on while L1 is blinking. Is it correct?

0
Pier Holtrop
Pier Holtrop

8 months ago

Hello, where or how do i buy the pcb and stencil ? are the base and pillars for a double setup ready to print from stl file ? Thanks

0
Beamformer
Beamformer

Tip 10 months ago

So there was a bug in the software for the opposed array configuration when 'Con Extra' was used (for parallel control) but I found a fix. Add the following lines of code to the SerialComms.java file under the List<String> code:



By default, when the 'Con Extra' button is pressed, a port ID of -1 is assigned, which corresponds with a port index of -2, which throws an error and does not connect to the secondary board. The above code just assigns the second serial port (assuming there are 2, and you are using the first port for the primary board) to the secondary board. I tested this fix and it works correctly for selecting individual transducers on either board. Will be testing the levitator tomorrow.

0
dbeaven1
dbeaven1

Question 10 months ago

If possible, could you post the Altium project files on the github? I would love to export a 3d model of the board so that I can use it in my project mock up.

0
DanB24
DanB24

Question 11 months ago

Ive made one of the boards and am at the stage of testing the outputs. I have about 7 dead channels, and 3 are adjacent. Should i focus on troubleshooting or is this acceptable and still functional?

0
Beamformer
Beamformer

Answer 11 months ago

Hello, I am making these boards too but am having trouble downloading the Quartus software. Did you download the Quartus II subscriber edition v13.0sp1 or the most latest (v21.1)?

0
DanB24
DanB24

Reply 11 months ago

I just so happened to have a copy at work we used a year or two ago. I had problems with the intel link too. When i get back from vacation ill see if i can post it somewhere for you.