Introduction: How to Build a LipSync

Picture of How to Build a LipSync

Touchscreen devices have become a staple piece of technology in this day and age. In 2016, upwards of 1.6 billion people in the world are using smartphones. Most people cannot get through the day without using their smartphone. It keeps us connected. It keeps us organized. It is an integral part of our social lives. For people in wheelchairs whom experience difficulties with fine upper body motor control, the usage of mobile devices can be very challenging. Statistics show there are over 1 million people in the United States and Canada that have very limited or no use of their hands, making touchscreen devices very difficult or impossible to use.

The "LipSync" is an assistive technology device which is being developed to allow quadriplegics and other people with limited hand use the ability to use touchscreen mobile devices by manipulation of a mouth-operated joystick with integrated sip and puff controls. The user is able to manipulate a cursor on their device screen using a mouth-operated joystick with integrated sip and puff controls to simulate the actions of "tap" and hitting the back button, respectively. With longer sips and longer puffs, additional secondary features are enabled including a "tap and drag", "long tap and drag" and the possibility of more specialized functions as per the user's needs. The LipSync is design specifically for portable devices, it does not require AC power, but it will work with any device including desktop and laptop computers that support mice through a universal serial bus (USB).

There are 2 main aspects we will be addressing in our project:

Developing an easy to build, but robust electronics assembly that novice to experienced makers can build.

Developing a device housing which can be 3D printed by makers either at home or at dedicated facilities.

FUTURE LIPSYNC PROJECT GOALS

Apr-Dec 2017: Support a network of makers to connect directly with people with disabilities and creating 150 LipSyncs through the Pacific Northwest

Jan-Dec 2018: Scale maker-disability matchmaking service across North America, to support 1,500 LipSyncs being created at the local level, as well as developing and applying other assistive technologies.

In addition, the Neil Squire Society has been working on a service delivery model for the technology. It will soon be launching the Makers Making Change initiative. Makers Make Change has the goal of creating one-on-one relationships between makers and people with disabilities. The Makers Making Change initiative will provide the infrastructure to connect makers, open assistive technologies, and people with disabilities in their community to make a local impact. Makers will be able to meet people with disabilities in their community and work with them to deliver a solution.

GET INVOLVED

If you want more information on the LipSync, track our development closer or work with us in creating (or getting) a LipSync, please connect with us. We have a newsletter and request form on our webpage: http://www.neilsquire.ca/lipsync

Step 1: Obtaining Parts and Components

Picture of Obtaining Parts and Components

Download all the necessary files and directories to assemble a LipSync:

https://github.com/makersmakingchange/LipSync/arch...

LipSync-master\Hardware\Housing_design: Contains all 3D printing files in STL format and its respective print settings

LipSync-master\Hardware\PCB_design: Contains schematic and layout for the LipSync printed circuit boards

LipSync-master\Software\LipSync_Firmware: Contains microcontroller code LipSync-master\LipSync_BOM.csv: Contains LipSync Components in CSV format

Components should be ordered at least 2 weeks in advance to allow for sufficient shipping time. See below for the full list of components and its purchase links.

Component List:

  1. [1] LipSync Circuit Board Pack

  2. [1] Arduino Micro / Genuino Micro Microcontroller

  3. [2] Right Angle Push Buttons

  4. [2] 300 Ohm 1/4W Through Hole Resistors

  5. [4] 10k Ohm Mini Axis Through Hole Resistors

  6. [1] NXP Pressure Sensor [component label: MPXV7002DP]

  7. [1] 0.01uF Ceramic Capacitor [capacitor label: 103]

  8. [1] 1.0uF Ceramic Capacitor [capacitor label: 105]

  9. [1] 470pF Ceramic Capacitor [capacitor label: 471]

  10. [2] 17 Position Straight Female T/H Headers

  11. [2] 3 Position Right Angle Male Headers

  12. [2] 3 Position Straight Female Headers

  13. [2] 4 Position Straight Female T/H Headers

  14. [1] 10 Position Break Away Male Headers

  15. [1] 5mm Bi-Colour T/H LED

  16. [1] Shunt Connector

  17. [1] 2N3906 PNP-Type T/H Transistor

  18. [1] 6 Position Straight Female T/H Headers

  19. [1] 1 Position Straight Female T/H Header

  20. [1] Front Housing

  21. [1] Joystick Front Plate

  22. [1] Joystick Rocker

  23. [1] Joystick Base Plate

  24. [1] Rear Housing

  25. [2] Buttons

  26. [1] Mouth Piece

  27. [1] Filter

  28. [1] Luer Lock Fitting

  29. [4] Force Resistive Sensors

  30. [4] Screws Philips Head M3

  31. [6] Compression Springs

  32. [2] Standoffs M3 10mm

  33. [2] Standoffs M3 15mm

  34. [1] Luer Lock Nut

  35. [4] Bumpers

  36. [4] Nuts M3

  37. [1] Tygon Tubing 3/32" ID x 45mm

  38. [1] Light Pipe

  39. [1] Threaded Insert 5/8"-27

  40. [1] Threaded Adapter 5/8"-27

  41. [1] USB Cable

  42. [1] USB Adapter

  43. [1] Cable Tie

Step 2: Obtaining the Right Equipment

To complete this LipSync Build, you will need the follow tools:

  1. Small Spool of 60Pb/40Tn Solder Wire

  2. Temperature Adjustable Soldering Iron (350°C)

  3. Needle Nose Plier

  4. Fine File or Sandpaper

  5. Baby Powder

  6. Super Glue

  7. Medium Strength Threadlocker

  8. Desolder Pump

  9. Safety Glasses

  10. Hobby Knife

  11. Flush Cutters

  12. ABS or High Strength PLA Filament 100g

Step 3: Print Out 3D Components

Picture of Print Out 3D Components

The 3D Components include:

1 x Rear Housing

1 x Front Housing

1 x Joystick Rocker

1 x Joystick Base Plate

1 x Joystick Top Plate

These .stl files can be found on the hackaday site: hackaday.io/project/13424-lipsync

Step 4: Begin LipSync Build Part 1

Picture of Begin LipSync Build Part 1

For this part you will need the following components:

  1. 2x 300 Ohm Resistors
  2. 2x Push Buttons
  3. 10 Position Breakaway Headers
  4. Shunt Connector
  5. Transistor
  6. Bi-Colour LED
  7. 6 Position Female T/H Header
  8. 2x Right Angle Male Headers
  9. PCB Board 1

Step 5: Insert 2x 300 Ohm Resistors and Solder

Picture of Insert 2x 300 Ohm Resistors and Solder
  1. Snip the 300 Ohm Resistors using wire cutter
  2. Using Needle Nose Pliers, bend the leads of the resistors at a 90 degree angle.
  3. Insert the leads of the resistors into the holes of the PCB
  4. Bend the leads on the other side to ensure they don't fall out
  5. Flip the PCB and solder the resistors to the PCB
  6. Snip the ends of the resistors, ensure flush cut (cut it as close to the PCB as you can).

Step 6: Insert the 2 Push Buttons and Solder

Picture of Insert the 2 Push Buttons and Solder
  1. Insert the leads of the buttons into the holes of the PCB
  2. Flip the PCB and solder the push buttons to the PCB

Step 7: Snap Away 2 of the 10 Position Breakaway Headers and Solder

Picture of Snap Away 2 of the 10 Position Breakaway Headers and Solder
  1. Using a needle nose plier, snap away 2 of the 10 position header as shown.
  2. Insert the 2 position breakaway headers.
  3. Flip the PCB and solder the 2 position breakaway headers to the PCB

Step 8: Insert the Shunt Connector

Picture of Insert the Shunt Connector

Carefully place the shunt connector on the 2 position break away headers

Step 9: Insert the Transistor and Solder

Picture of Insert the Transistor and Solder
  1. Snip the transistor using wire cutters
  2. Place on board as shown
  3. Insert the leads of the 2 position breakaway header into the holes of the PCB
  4. Flip the PCB and solder the transistors to the PCB
  5. Snip the ends of the transistor, ensure flush cut (cut it as close to the PCB as you can).

Step 10: Insert Bi-Colour LED and Solder

Picture of Insert Bi-Colour LED and Solder
  1. Snip the LED using wire cutters
  2. Orienting it with the flatside facing you, bend leads of the LED at 90 degrees (as shown)
  3. Insert the leads of the LED into the holes of the PCB, push all the way in
  4. Flip the PCB and solder the LED to the PCB
  5. Snip the ends of the LED, ensure flush cut (cut it as close to the PCB as you can).

Step 11: Insert 6 Position Female T/H Header

Picture of Insert 6 Position Female T/H Header
  1. Insert the leads of the 6 Position Female T/H Header into the holes of the PCB
  2. Flip the PCB and solder one lead of header to the PCB
  3. Flip the PCB and ensure that the header is sitting flat on the board (if it isn't, unsolder the one lead and fix)
  4. Flip the PCB and solder the rest of the leads

Step 12: Insert 2 Right Angle Male Headers

Picture of Insert 2 Right Angle Male Headers
  1. Insert the leads of the 2 Right Angle Male Headers into the holes of the PCB
  2. Flip the PCB and solder one lead of each header to the PCB
  3. Flip the PCB and ensure that the headers are sitting flat on the board (if they aren't, unsolder the one lead and fix)
  4. Flip the PCB and solder the rest of the leads

Step 13: Begin LipSync Build Part 2

Picture of Begin LipSync Build Part 2

For this part you will need the following components:

  • NXP Pressure Sensor
  • 0.01uF Ceramic Capacitor [capacitor label: 103]
  • 1.0uF Ceramic Capacitor [capacitor label: 105]
  • 470pF Ceramic Capacitor [capacitor label: 471]
  • PCB Board 2

Step 14: Insert Capacitors and Solder

Picture of Insert Capacitors and Solder
  1. Snip the leads of the capacitors using wire cutters
  2. Place the the 103 to the far left, the 105 in the middle and the 471 to the far right.
  3. Push them in all the way and ensure they sit flat on the PCB
  4. Flip the PCB and solder the capacitors to the PCB
  5. Snip the ends of the capacitors, ensure flush cut (cut it as close to the PCB as you can).

Step 15: Prepare the Pressure Sensor and Solder

Picture of Prepare the Pressure Sensor and Solder
  1. Apply a small amount of solder to one lead of the pressure sensor as shown
  2. Apply a small amount of solder to the corresponding solder pad on the PCB
  3. Place the pressure sensor on the PCB with all leads in line and on top of their corresponding solder pad
  4. Place the solder tip on the lead to reflow the solder on the two components. You will be able to feel the pressure sensor should “drop” onto Board 2.
  5. Apply solder to the rest of the leads and be careful to ensure that they have connection to the solder pad

Step 16: Begin LipSync Build Part 3

Picture of Begin LipSync Build Part 3

For this part, you will need the following components:

  1. PCB Board 1
  2. PCB Board 2
  3. 2x 4 Position Female T/H Headers
  4. 2x 4 Position Breakaway Headers

Step 17: Insert 4 Position Female T/H Headers/ 4 Position Breakaway Headers and Solder

Picture of Insert 4 Position Female T/H Headers/ 4 Position Breakaway Headers and Solder
  1. Insert the 4 Position Breakaway Headers into the 4 Position Female T/H Headers as shown
  2. Insert the leads of the 4 Position Female T/H Header into the holes of the PCB Board 2
  3. Insert the leads of the 4 Position Breakaway Headers into the holes of PCB Board 1
  4. Solder one lead of each of the 4 Position Breakaway Headers into Board 1
  5. Solder one lead of each of the 4 Position Breakaway Headers into Board 2
  6. Check to ensure that the headers are sitting flat on the boards (if they aren't, unsolder the one lead and fix)
  7. Solder the rest of the leads on all the headers

Step 18: Begin LipSync Build Part 4

Picture of Begin LipSync Build Part 4

For this part, you will need the following components:

  1. PCB Board 1
  2. 2x 17 Position Female T/H Headers
  3. Arduino Genuino Microcontroller

Step 19: Insert 17 Position Female T/H Headers and Solder

Picture of Insert 17 Position Female T/H Headers and Solder
  1. Insert the microcontroller leads into the two 17 Position Female T/H Headers.
  2. Carefully remove the Board 2 (with pressure sensor) as shown
  3. Insert the leads of the 17 Position Female T/H Headers into the corresponding holes on the PCB as shown
  4. Flip the PCB and solder one lead of each of the two headers to the PCB
  5. Flip the PCB and ensure that the headers are sitting flat on the board (if they aren't, unsolder the one lead and fix)
  6. Flip the PCB and solder the rest of the leads
  7. Re-Insert Board 2

Step 20: Begin LipSync Build Part 5

Picture of Begin LipSync Build Part 5

For this part, you will need the following components:

  1. Joystick Front Plate
  2. Joystick Base Plate
  3. Joystick Rocker
  4. 4x 10k Ohm Mini Resistors
  5. 3 Position Straight Female Headers
  6. Luer Lock Fitting
  7. 4x Force Resistive Sensors
  8. 4x Screws Philips Head M3
  9. 4x Compression Springs
  10. 2x Standoffs M3 10mm
  11. 2x Standoffs M3 15mm
  12. Luer Lock Nut
  13. 4x Bumpers
  14. 4x Nuts M3
  15. Tygon Tubing 3/32" ID x 45mm
  16. Sandpaper
  17. Baby Powder
  18. Super Glue

Step 21: Insert 4x Mini 10K Ohm Resistors and Solder

Picture of Insert 4x Mini 10K Ohm Resistors and Solder
  1. Snip the Mini 10K Ohm Resistors using wire cutter
  2. Using Needle Nose Pliers, bend the leads of the resistors at a 90 degree angle.
  3. Insert the leads of the resistors into the holes of the PCB
  4. Bend the leads on the other side to ensure they don't fall out
  5. Flip the PCB and solder the resistors to the PCB
  6. Snip the ends of the resistors, ensure flush cut (cut it as close to the PCB as you can).

Step 22: Remove Support Material From 3D Printed Parts

Picture of Remove Support Material From 3D Printed Parts
  1. Using an exacto knife, carefully and thoroughly remove the support material from the 3D printed joystick parts

Step 23: Insert the 10mm Standoffs

Picture of Insert the 10mm Standoffs
  1. Place the 10mm standoffs into the raised slot as shown
  2. Insert a nut on both of the standoffs and (using needle nose pliers) tighten the nuts
  3. Apply super glue to the connecting areas of the nut and standoff
  4. Wait 5 to 15 mins for the super glue to dry

Step 24: Insert Luer Lock Fittings and Nut

Picture of Insert Luer Lock Fittings and Nut
  1. Using fine sandpaper, sand the rounded surface of the rubber bumpers
  2. Apply a small amount of baby powder to the sanded surface of each rubber bumper
  3. Insert (twist in clockwise) the Luer Lock fitting into the rocker with the smaller end of the fitting coming out on the flat side of the rocker (as shown)
  4. Insert the nut from the flat side of the joystick rocker
  5. Ensure the nut sits flush to the rocker, once flush, twist the luer lock fitting in the rest of the way.
  6. Apply a small amount of super glue to the connections between the luer fitting and nut, as well as the nut and rocker. - Wait 5 -15 minutes for the glue to dry.

Step 25: Place Rubber Bumpers

Picture of Place Rubber Bumpers
  1. Using fine sandpaper, sand the rounded surface of the rubber bumpers
  2. Carefully remove the rubber bumpers from their packaging and place in the indented circles on the joystick rocker - ensure you place the higher part of the bumper towards the center.
  3. Apply a small amount of baby powder to the sanded surface of each rubber bumper

Step 26: Insert 3 Position Female T/H Headers and Solder

Picture of Insert 3 Position Female T/H Headers and Solder
  1. Insert the leads of the two 3 Position Female T/H Headers into the holes of PCB Board 3
  2. Flip the PCB and solder one lead of header to the PCB
  3. Flip the PCB and ensure that the header is sitting flat on the board (if it isn't, unsolder the one lead and fix)
  4. Flip the PCB and solder the rest of the leads

Step 27: Insert Force Resistive Sensors and Solder

Picture of Insert Force Resistive Sensors and Solder

**WARNING** - Force Resistive Sensors (FSRs) are very fragile - Be careful not to touch with soldering iron or they will melt. Be careful not to bend at a radius of less than 2.45mm as this will cause the FSRs to malfunction. Be careful to solder the FSRs quickly as prolonged heat at the leads will conduct heat to the plastic of the FSR and cause it to melt.


  1. Insert 1 FSR at a 30 Degree angle away from the centre of the board (as shown)
  2. Hold the board with a clamp and point the out to the side (do not lay board down as this can cause the FSR to bend and break)
  3. Quickly and carefully solder the the leads of the FSR
  4. Repeat steps 1-3 for the next 3 FSRs

Step 28: Assemble Joystick Baseplate

Picture of Assemble Joystick Baseplate
  1. Insert 15mm Standoffs as shown
  2. Place Board 3 onto the Joystick Base Plate
  3. Insert 2 nuts onto the standoffs above Board 3 and tighten using needle nose pliers
  4. Insert the head of the FSRs through the corresponding slots in the Joystick Base Plate as shown, ensure the black adhesive is facing the centre of the base plate.
  5. Using an exacto knife, carefully remove the plastic covering (as shown) of the adhesive on the head of an FSR, be sure not to remove the black adhesive while trying to perform this step.
  6. Carefully push down the head of the FSR so that the adhesive sticks to the Joystick Base Plate within the designated outline.
  7. Repeat steps 6 and 7 for the rest of the FSRs
  8. Apply super glue to the connection between the nuts and the standoffs, wait 5 to 15 minutes for the glue to dry

Step 29: Assemble Joystick

Picture of Assemble Joystick

*Note* - Use the joystick stand holder to assist with assembly

  1. Place Base Plate onto Joystick Stand Holder
  2. Place Rocker (the side with the tygon tubing facing down) onto the Base Plate as shown
  3. Place 4 springs into the indented circles
  4. Place 4 screws into the Joystick Top Plate
  5. Plate Joystick Top Plate onto of the Rocker + Springs
  6. Tighten the screws until the Top Plate sits on the standoffs
  7. Connect the 3 Position Right Angle Male headers from Board 1 into the 3 Position Straight Female Headers on Board 3, there is a slot in Board 1 which allows the tygon tubing to carefully be sent from one side of Board 1 to the other. (As shown)
  8. Connect the tygon tubing to the outermost connector on the pressure sensor

Step 30: Begin LipSync Build Part 6 (Final Part!)

Picture of Begin LipSync Build Part 6 (Final Part!)

For this step, you will need the following components:

  1. Rear Housing
  2. Front Housing
  3. Fully Assembled Joystick (with Board 1, 2, and 3)
  4. 2x 3D printed Buttons
  5. 2x Springs
  6. Light Pipe
  7. Cable Tie
  8. Filter
  9. Mouth Piece
  10. USB Cable

Step 31: Insert the Light Pipe

Picture of Insert the Light Pipe
  1. Insert the light pipe into the hole on the Rear Housing

Step 32: Insert Cable Tie Onto USB

Picture of Insert Cable Tie Onto USB
  1. Insert Cable Tie onto the Micro USB end of the USB cable
  2. Cut the tail of the cable tie that is sticking out

Step 33: Feed USB Cable Thru

Picture of Feed USB Cable Thru
  1. Feed the USB Cable (Micro USB end first) half way through rear housing.

Step 34: Insert Springs and 3D Printed Buttons

Picture of Insert Springs and 3D Printed Buttons

  1. Using tweezers or needle nose pliers, insert the 2 springs into the corresponding slots in the Rear Housing
  2. Insert the buttons, narrow end first, into the centre of the springs as shown

Step 35: Final LipSync Assembly

Picture of Final LipSync Assembly
  1. Connect the Micro USB side of the USB Cable to the Micro USB Port on the microcontroller of the Joystick
  2. Slide the Joystick Assembly along the internal tracks inside the Rear Housing
  3. Push in all the way until the Push Buttons on the Joystick Assembly sit inside the 3D printed buttons. - Check from the back by pushing the buttons, if they click well, then buttons have been correctly inserted into the 3D printed buttons - otherwise carefully push the assembly up and re-adjust
  4. Twist on the the Front Housing onto the threads of the Rear Housing (ensure flush and tight fit)
  5. Lightly twist on the filter (do not twist on too tight)
  6. Lightly twist on the mouth piece (do not twist on too tight)
  7. Insert the 5/8"-27 Threaded Insert

Step 36: Uploading the Microcontroller Code

  1. Download and install the latest version of Arduino IDE: https://www.arduino.cc/en/main/software
  2. Download and save the latest LipSync microcontroller code: https://raw.githubusercontent.com/makersmakingchan... Right-click on the hyperlink of microcontroller code, select “Save Target As” (Internet explorer) or “Save Link As” (Chrome or Firefox) and save the microcontroller code (LipSync_Firmware) to the computer directory or folder of your choice.
  3. Open the Arduino IDE and click on Tools -> Board -> Arduino/Genuino Micro

  4. Connect to the PC using the LipSync USB cable.

  5. Click on Tools -> Port

  6. Click the right arrow at the top left of the Arduino IDE to upload the microcontroller
    code to the LipSync device.

  7. Once upload is complete, a message saying “Upload Successful” will show at the
    bottom left of the Arduino IDE.

Step 37: Calibrate the Joystick

Picture of Calibrate the Joystick

**Note** - You must calibrate your joystick before using it the first time.

Allow the LipSync to initialize for 3 seconds before moving the mouthpiece each time after it’s plugged into a device.

  1. Press both speed adjustment buttons on the back of the LipSync for a few seconds until LED comes on

  2. LED slowly blinks red & green 5 times

  3. While LED blinks green 6 times, move the mouthpiece to the 12 o'clock position (up direction)

  4. LED blinks red 1 time when position is measured

  5. While LED blinks green 6 times, move the mouthpiece to the 3 o'clock position (right direction)

  6. LED blinks red 1 time when position is measured

  7. While LED blinks green 6 times, move the mouthpiece to the 6 o'clock position (down direction)

  8. LED blinks red 1 time when position is measured

  9. While LED blinks green 6 times, move the mouthpiece to the 9 o'clock position (left direction)

  10. LED blinks red 1 time when position is measured

  11. LED rapidly blinks red and green 6 times when calibration is complete

Comments

renar2d2 (author)2017-08-31

Any ideas for a one handed gaming mouse or controller? Have 2 people in my family that only have one hand.

jfryar30272 (author)2017-08-28

This is amazing. I've been working with a friend who works with kids that have disabilities. They have had problems with tablet interaction, so this would be awesome. Thank you so much for sharing! I'm amazed at the simplicity of it, while still intrigued by it's complexity. You are awesome!

fernw (author)2017-08-26

The "Lip Sync" name and the capsule shaped form of this device were developed by ACCT, a private company (no longer active) that licensed internal electronic technology from Neil Squire Foundation but independently worked to produce and market the complete device. The continued use of that name and the external form of this device without acknowledgement or permission is effectively copyright infringement.

Swansong (author)2017-08-21

Amazing first Instructable and awesome system! That could help a lot of people :)

About This Instructable

6,760views

51favorites

License:

More by RashmiP11:How to Build a LipSync
Add instructable to: