Whisker Sensory Extension Wearable

4,855

38

18

About: Attempting to extend the human perceptual apparatus one wearable at a time!

Intro: Whisker Sensory Extension Wearable

This project, like my previous Ultrasonic Sensory Extension Wearable, was designed/created for multiple purposes, a couple of which being: wearable-assisted (See Figure 1) pet care and empathy research and application (specific to animals with vibrissae, see Figure 2) as well as basic sensory augmentation research (specifically, the tactile-to-tactile variety).

Here's a cursory overview of the components and their functions:

- Two sets of custom built flex sensor whisker devices (total of 8, 4 per side) receive tactile information (bend, flex, etc.) from objects in the user's immediate environment. The initial voltage/resistance information received by each sensor is then converted to bend angle information (e.g., a bend angle of 10 degrees). This bend angle information is subsequently converted to proportional pulse width modulation output and sent to corresponding vibrotactile displays on the user's forehead.

- Each whisker sensor has its own SparkFun ProMini 3.3V/8MHz microprocessor that does the transducing/converting. There are two circuits operating on each ProMini (this approach was necessary to alleviate motor EMI disturbance issues). One functions to input whisker information while the other outputs PWM commands to the coupled vibrating mini motor disc located in the vibrotactile display situated on the user's forehead. Each circuit is powered independently through one of the outputs of a dual output battery.

- Two vibrotactile displays supply tactile stimuli to the user's forehead. Each motor housed within the two displays is coupled to its own whisker (in the same orientation as the whiskers on the face), functioning in such a way as to reflect the whisker's bend angle in the form of vibration information. Vibration intensity is proportional to whisker bend angle (e.g., 10-degree bend angle --> 40 units PWM output, etc.).

Thanks to sensory substitution/augmentation phenomena (neuroplasticity; intra-modal plasticity), it's plausible that, after a given training period, an individual could extend their pre-existing somatosensory apparatus into the realm of these new whisker-like appendages, as blind individuals utilizing tactile-to-vision sensory substitution do (e.g., the use of a walking cane, see Figure 3). It's plausible that the somatosensory (SS) cortex (mediated through stimulation of forehead skin receptors) could, over time, develop a new "whisker representation" housed somewhere in SS cortex. This type of sensory extension has already been shown in multiple studies (e.g., Sensory Augmentation for the Blind [tactile-to-direction sensory augmentation through the use of a vibrotactile display worn around the waist], Embedding a Panoramic Representation of Infrared Light in the Adult Rat Somatosensory Cortex through a Sensory Neuroprosthesis [although brain-invasive techniques are used here, this supports the concept of cortical re-mapping], etc.). Now let's make some whiskers!

Step 1: Whisker Assembly

It took me a while to develop a whisker sensor apparatus that was flexible enough to mimic real whiskers, yet rigid enough to consistently return to a straight, unbent position. I ended up using a 4" unidirectional bend/flex sensor from Flexpoint Sensor Systems (See figure 1). A JST plug is soldered to the legs of the sensor, then a 14" long, 0.08" wide, 0.03" thick polystyrene strip (I procured mine at a local hardware store) is silicone glue-adhered to the sensor, heat shrink is applied, and a protective coating of Sugru is molded around the entire base of the whisker unit. Here are the detailed instructions:

- Take the plug end of the 3-pin JST connector and remove the center wire (See figures 2-4)

- Snip the plug wires so you have ~1.5 cm of wire remaining, then strip and solder these leads to the sensor pins (remembering the plug/sensor orientation). I used heat shrink to provide insulation (See figures 5, 6)

- Mount the polystyrene strip to the sensor with some sort of flexible adhesive (I used Liquid Nails silicone glue). Make sure to secure the strip to the sensor well (See figures 7, 8)

- Take your Sugru (I used a single 5g pack) and mold it around the base of the sensor/strip/plug making sure to encase all of these components. Also, be sure to apply the Sugru high enough to fully secure the strip, but not too high as to restrict the sensor's ease of movement/bend. Take your time. You'll have at least 30-45 minutes until the Sugru begins to harden. Before you let dry, make sure that your plug fits properly into the receptacle side of the JST connector (See figures 9-13)

- Lastly, I adhered labels to the whisker apparatus. Side (L/R) and number position (1-4) were used (See figures 14, 15)

- Make 7 more (or whatever number of whiskers you desire). Be sure to create each whisker in the same way. This will help with sensor calibration later.

Step 2: Whisker Mount Assembly

So, once the whiskers were completed, I needed a way to mount them to the helmet. I decided to design and create a curved arm with a disc mounting point at the far end. I designed the component in Adobe Illustrator and used 1/16" thick clear acrylic as the material. After laser cutting, I embedded my custom whisker wiring harness to the disc portion of the mount in Sugru. Lastly, I made sure that there was some sort of way to attach the arm to the helmet. When designing the vector I added a cutout in the portion of the arm that inserts into the helmet. Two screws/bolts were used to secure the arm to the helmet (one above the helmet insert point and one below). Here are the detailed instructions:

- Take the 2-pin JST wire assembly, twist, and strip the wire in prep for soldering to the ribbon wire (See figures 1-3)

- Take the receptacle end of the 3-pin JST connector and remove the center wire. Then strip the remaining wires in prep for soldering to the ribbon cable (See figures 4-6)

- Make sure your ribbon is long enough and has enough wires to accommodate the number of whiskers you are implementing. I used 4 whiskers on each side, so each whisker wiring harness required 8 wires. Solder both JST wire assemblies to each side of your ribbon and apply heat shrink (See figure 7)

- Open figure 8 for my whisker arm vector file (PDF)...keep in mind that these dimensions were based on the particular helmet (more specifically, the helmet's hearing protection insert slots) that I chose to work with. Of course, you can design your own if you wish. I used 1/16" clear acrylic and cut with my shop's laser cutter. I then used a heat gun to (carefully!) shape the arm to fit flush against the cheek (See figure 7)

- Place your wiring harness on the acrylic arm and temporarily secure with tape. Then take your Sugru and mold the JST receptacles in place on the disc portion of the arm. I used around 5-6 single Sugru packets. This is going to take some time to get right both structurally and aesthetically. Just make sure that the receptacles are positioned in such a way that won't cause whisker overlap when they're plugged in. Again, you'll have about 30-45 minutes of mold time. Once complete, dry/cure overnight (See figures 9-13)

- Finally, procure a couple M3 bolts/nuts so that you can secure the arm to the helmet. If anyone comes up with a better way to secure the arm, please let me know:) (See figure 14)

Step 3: Vibrotactile Display Assembly, Helmet, & Battery Setup

This vibrotactile setup is pretty similar to the display used for my other sensory extension wearable project (Cody the ultrasonic translator dog). The only major differences being the overall display positioning (one to either side of the forehead) and the orientation of individual vibrating motor discs within each array (one-to-one representation of whisker orientation (a diamond pattern composed of four motors). The whole system is housed on/supported from a typical construction helmet and is powered by a single 5V dual output battery. Here are the detailed instructions:

- Twist the extender wire together in prep for soldering. Make sure your extender wire is long enough for your application (See figures 1, 2)

- Twist the 2-pin JST wire assembly in prep for soldering (See figures 3, 4)

- Take your extender wire, JST wire assembly, and vibrating motor disc and solder together. Make sure to apply heat shrink. Also, adhere a velcro disc (hook side) to the back of the motor. Repeat 7 times. Lastly, cut two short pieces of velcro cable ties/straps so that the collection of motors wires can be bound together and velcroed to the side of the helmet (See figure 5, 6)

- Cut out two 5 cm diameter discs of velcro (pile side). These will be the mounting pads for the vibrating motors. Adhere (I used super glue) the discs to the helmet's headband (mine are spaced ~2 cm apart) and velcro the motors onto the mounting pads in the same orientation as you embedded the whisker receptacles on the whisker arms (See figure 7)

- Adhere velcro to the top of the helmet and to the underside of the battery. Velcro the battery in place so that the outputs are positioned toward the back of the helmet (See figures 8-10)

- Adhere miscellaneous velcro (pile side) in order to support remaining helmet fixtures. Add bands of velcro to each side of the helmet IOT support microprocessors. Add velcro discs to the back of the helmet (beneath the battery and between the two side bands) IOT support the power distro boards. Add small pieces of velcro to each side of the helmet (immediately in front of the upper protrusion of the whisker arm) IOT support the bound motor disc wiring.

Step 4: Microprocessor, ProtoShield, & Power Distribution Board Assembly

I chose to build a microprocessor for each whisker. The factors that led to taking this approach ranged from the benefits of modularity, to the want for parallel ADC simultaneity. This unit consists of (1) SparkFun Arduino Pro Mini (3.3V/8MHz), (1) Pro Mini Protoshield, (1) BC337 NPN transistor, (1) 1N4001 power blocking diode, (1) 220 Ohm resistor, (1) 10k Ohm resistor, (4) 2-pin JST plug/receptacle assemblies, (2) 8-pin female headers, (1) 6-pin right angle header, (2) 4-pin headers, (2) 1-pin headers, (2) female-to-male jumper wires - 3", solid core wire - 22 AWG, (2) USB-A plugs, (2) power distribution boards, and velcro (hook & pile). You can do this in whatever order you desire, but I'll detail my build below.

- Solder the 8-pin female headers onto the ProMini. I placed mine from Raw-A0 and TXO-5 (See figures 1, 2)

- Solder the 6-pin right angle header in place perpendicular to the 8-pin header (from BLK-GRN) (See figures 1, 2)

- Adhese the velcro (hook) to the back of the ProMini (See figure 3)

- Solder the 4-pin headers onto the wings of the Protoshield. Make sure to orient the board in the right direction (See figures 4-6)

- Bend the legs of the 2-pin JST receptacle 90 degrees in prep for soldering (See figures 7-9)

- Place and solder the JST receptacle GND pin into place. (this will require bending the pin again once inserted into the board; See figures 10, 11)

- Insert the diode and bend/snip the legs according to figures 12-14 in prep for soldering

- Insert another 2-pin JST receptacle and bend one leg according to figures 15, 16

- Make sure all three pins create one solder point, then solder (See figure 17)

- Bend the emitter pin of the transistor slightly away from the flat face of the component in prep for board placement (See figure 18)

- Place the transistor according to figures 19, 20

- Bend the remaining diode leg and transistor emitter pin according to figure 21. Make sure they create one solder point and solder together (See figure 22)

- Place the 220 Ohm resistor onto the board and solder one resistor leg and the transistor base pin together. Also, solder the remaining transistor collector pin to GND (See figures 23-25)

- Cut a piece of solid core wire and solder into place connecting the diode/emitter pin solder point to the remaining leg of the JST receptacle (See figures 26, 27)

- Place a 1-pin header onto the board and solder to the remaining leg of the 220 Ohm resistor (See figures 28-30)

- Bend the legs of another JST receptacle 90 degrees and place onto the board according to figure 31

- Bend the JST receptacle GND pin again and solder into place (See figure 32)

- Cut a piece of solid core wire and solder into place between the remaining pin of the newly placed JST receptacle and RAW (See figure 33, 34)

- Insert the last JST receptacle into the board and bend one of its pins onto VCC and solder in place (See figures 35, 36)

- Place the 10k Ohm resistor onto the board according to figure 37

- Cut a piece of solid core wire and solder into place between one leg of the newly placed resistor and GND (See figures 38, 39)

- Insert another 1-pin header onto the board and solder to the remaining leg of the 10k resistor (See figures 40, 41)

- Cut one last piece of solid core wire and solder between the remaining pin of the most recently placed JST receptacle and the 1-pin header/10k resistor solder point (See figures 42, 43)

- The board is finished...figure 44!

- Plug in the 3" female-to-male jumper wires (left side: whisker 1-pin to A0; right side: 5 to motor 1-pin) (See figures 45, 46)

- Make 7 more whisker microprocessor assemblies!

- Twist the 2-pin JST plug wires together (See figures 47, 48)

- Solder cannibalized USB-A plug onto the power distro board input (See figure 49)

- Solder 8 twisted JST plug wire assemblies onto the power distro board outputs (See figure 50)

- Adhere velcro (hook side) to the backside of power distro board (See figure 51)

- Make another power distro board assembly (Figure 52 shows both power distro boards on helmet)

- Attach four microprocessors to each velcro band on either side of the helmet and plug in all inputs/outputs (See figures 53, 54)

Step 5: Implement the Code

Ok, now it's time to upload the code. There are a few things that you'll need to tweak (on each individual microprocessor) before you're ready to whisk the world. Also, since we're working with a ProMini, you'll need an FTDI breakout board in order to connect the microprocessor to your computer.

- Remember, these instructions will need to be carried out on each whisker microprocessor individually.

- First, use a multimeter to measure both the VCC output voltage and the resistance across the 10k resistor. Input these values into their respective spots in the code.

- Then, double check that all other variables are set to the correct inputs/outputs (e.g., mtr, flexADC, etc.)

- Then, power up your motor circuits (plug in the power distro board that is running to your motors), plug your FTDI board (connected to USB/computer) into the first ProMini, and upload the code.

- Once you're up and running, you'll see in the serial monitor that Resistance, Bend, and Forward Intensity will print. Now it's time to calibrate the whisker (each whisker is unique and will have a slightly different baseline resistance). Set the STRAIGHT_RESISTANCE variable to whatever the baseline resistance (i.e., unbent whisker position) is printing as. Then, set the BEND_RESISTANCE variable to STRAIGHT_RESISTANCE + 30000.0. In the original code, this variable was meant to reflect the flex sensor resistance output at a bend of 90 degrees. Since our whiskers don't get anywhere close to a full 90-degree bend (at least in typical situations), adding 30000.0 ohms to the baseline resistance works just fine. Feel free to set the bend resistance to whatever works best for you application though. If you've set everything correctly, then you see that when the whisker is unbent, a bend angle of 0 degrees (more or less) will print. Once you've calibrated all whisker units, you can disconnect the FTDI board form the last microprocessor and power up your whisker circuits (plug in the remaining power distro board that is running to the whiskers).

Just to go over a few points in the code:

- Once voltage and resistance are calculated, two different mapping functions are carried out. The first maps voltage/resistance onto a bend angle (i.e., 60000.0 ohms --> 0.0 degrees bend) and the second subsequently maps bend angle onto motor intensity (i.e., 0.0 degrees bend --> 30.0 PWM units motor intensity). You might need to tweak the second mapping function a bit to get it just right. The bend parameters are initially set to 5.0 and 80.0 and the motor output parameters are set to 30.0 and 130.0. These values worked for my purposes. Anything less than 5.0 degrees on the low end causes problems because the sensor fluctuates approximately +/- 3.0 degrees at baseline. Anything more than 80.0 is unnecessary because the sensor rarely flexes beyond this angle. The vibration disc that I used has a threshold of about 40.0 PWM units (i.e., it won't noticeably vibrate until output reaches about 40.0). Starting the range at 30.0 helps buffer the initial vibration level a bit. I set the upper range to 130.0 because this level is more than intense enough for upper-level forehead display intensity.

- I have the intent to further modify this so that continuous flex sensor self-calibration is carried out. Currently, when whiskers are in use the baseline resistance can fluctuate significantly beyond the initial set value. This can lead to reduced whisker sensitivity/functionality. If a continuous self-calibration were to be implemented, then this deleterious phenomenon would hopefully be eliminated. For now, you may have to re-calibrate the whiskers from time to time.

Step 6: Train & Learn...

This step is "under construction"

Share

    Recommendations

    • Fix It! Contest

      Fix It! Contest
    • Furniture Contest 2018

      Furniture Contest 2018
    • Tiny Home Contest

      Tiny Home Contest

    18 Discussions

    0
    None
    qorlis

    5 months ago

    I'm sorry, but I got lost after the first paragraph. I, like many others who like Instructables, see 'ibles like this as technobabble that is difficult to follow, let alone understand.

    I'm trying to be nice here, but can this not be explained a bit more simply and concisely? It would be nice to understand the purpose of this 'ible and it's construction for those of us who have less technical vocabulary. I believe Instructables is supposed to be a forum for those who would like to share their creations with others so they can be duplicated. I could be wrong, though.

    5 replies
    0
    None
    afcurlinqorlis

    Reply 5 months ago

    I as a not very technological guy understand where your coming from but I think for the sake of making it possible for other people to recreate something this complicated he needs to give the techies all the details.

    0
    None
    qorlisafcurlin

    Reply 5 months ago

    Supplying the details is fine, but it's not always the techies that would be interested in these types of 'ibles. The technological jargon serves to obscure the instructions. I appeal to the simpler way of expressing them. Surely the techies would understand as well.

    0
    None
    NeilRGqorlis

    Reply 2 months ago

    The road would be longer and the learning curve steeper for those who only see jargon, but the challenge can be met by individually googling the the terms that are capitalized and asking questions as they arise. Some of the "jargon" are technical terms that belie a deeper level of understanding that the author has that would be beyond the scope of a single instructable.

    If someone were to have a need for this project or want to duplicate it, it is not unreasonable to ask them to invest in their own learning.

    Having been an engineer my entire professional life, I can attest to the value of a needs driven learning approach. ie you acquire knowledge in pursuit of a particular goal. It tends to stick with you better than if you tried to read a textbook from cover to cover.

    In closing, I would like to mention that this project is in many ways an aggregation of skills and technology available on instructables across a wide variety of projects.

    Best,

    Neil

    0
    None
    Antzy Carmasaicqorlis

    Reply 4 months ago

    What I could gather from a quick read, this is like a walking cane for blind, except for the head. The whiskers have sensors that detect whenever they are bent and activates a vibrator in the helmet proportional to the amount of bend. Each whisker has a different vibrator so wearer can differentiate between different whiskers triggered.
    As to the application, I believe that blind people are proficient with walking canes which help them navigate terrain and avoid obstacles placed on ground such as vehicles, sidewalk, other people, walls etc. They are very vulnerable to overhanging obstacles hitting their head such as low tree branches, low door frames or staircases with "mind your head" warning(which they obviously can't read). This can help them prevent nasty accidents by providing a way to detect objects around their head.

    0
    None
    qorlisAntzy Carmasaic

    Reply 4 months ago

    Now, the purpose is clear. Thank you. Seems like a step in the right direction. Isn't there some sort of sonic device that 'sees" obstacles ahead and gives either haptic feedback in the form of vibrations or a verbal warning to the wearer? I think I've seen something like sunglasses that had a sonar/radar function that told the wearer where an obstacle is located so it can be avoided. It works a little like the echolocation used by bats and dolphins.

    Having said that, I know that Instructables is for those who want to try making something themselves and not buying something already manufactured. That's why I like it anyway. I'm a big fan of DIY. I'm also a big proponent of the K.I.S.S. (Keep It Simple, Stupid or is it Keep it Stupid Simple?)method.

    0
    None
    JackDefay

    4 months ago

    The idea of extending our sensor capacity through technology like this is so interesting, and the whisker aesthetic is fantastic! I've used haptic motors for feedback from an ultrasonic sensor before, but I had to use a separate haptic motor driver for each one. Is your haptic system binary? Or was the haptic motor driver part of the custom board you made?

    0
    None
    ChristineC194

    4 months ago

    I think this is absolutely amazing! I have some experience with 'extending my consciousness' into an inanimate object.....that's how I know exactly where my tires are on the ground when I drive. Imitating the sensory abilities of animals could give us so much insight into the world around us!

    0
    None
    wrose7

    4 months ago on Step 6

    Great idea! You're probably aware it's going to need some polishing... depending on how sensitive it is you might need to compensate for spurious signals generated by head movement, the wind, etc. (you undoubtedly already know that).

    How about adding a microswitch to the end of each sensor... it could sense the immediate touch, before any strain develops. Also, the idea of placing whiskers on footwear or other portions of the body might have merit.

    0
    None
    nedchurch

    5 months ago

    Chindogu at its best! Great for when you have to crawl around in claustrophobic small dark spaces.

    0
    None
    Gadisha

    5 months ago

    Lol, I love this idea!

    Makes me think of this TED talk I recently viewed

    Although I don't think that we will need technology to enhance our perception, it's an interesting way of widening our view and exploring other ways to perceive our surroundings.

    0
    None
    lukewoodbury

    5 months ago

    All kinds of awesome!

    0
    None
    Dr H

    5 months ago

    Brilliant, I love it.

    Have you ever tried to use it to find your way in the dark?

    0
    None
    gast

    5 months ago

    I like it when people think outside the box. Who knows what this could lead to plus you were able to exercise your brain and use your skills.

    0
    None
    inconceivable1

    5 months ago

    wow this is really weird... cool!

    0
    None
    remael

    5 months ago

    This. This is amazing. I have no words, should have sent a poet...
    *starts slow clap*