Talking Joystick Mouse

Picture of Talking Joystick Mouse
I work in Assistive Technology, which is technology designed to help people with disabilities stay independent.

We encountered a man who had quadriplegia due to a motor vehicle accident.  He had a little motion in one hand and wanted to use a computer, but he could not use a regular mouse.  If he could use the computer, he could go online, surf the internet, research whatever struck his fancy, chat with other people, email, etc. - it would open a lot of doors for him and greatly increase his standard of living.  We tried a commercially available joystick mouse, and he could use it successfully.  The problem was cost: the commercial model was $550 (since then, it has dropped to $400 ) which neither he nor his family could afford.

I built a joystick mouse for him out of a USB gamepad and arcade machine components.  This mouse had two features lacking in the commercial model: 1) It could launch programs or commands, greatly increasing the efficiency of computer use  2) it could talk, giving auditory feedback regarding which button was pressed, as he could not move his head to look at his hands. 

We have since used descendants of the first joystick mouse with several people who had cerebral palsy or other conditions that made traditional computer use difficult.  The cost to build one of these is about $45.

Remove these adsRemove these ads by Signing Up
Great project! I made something very similar a couple years ago but your design is much nicer looking. I used the "JoyToKey" freeware, which is very good, but "AutoHotkey" looks like it's more flexible.
I was thinking of having the buttons countersunk, and then have hinged flaps that could flip in from the sides (or flaps that are just added on as needed with velcro) to cover up buttons for clients that are not as able to avoid hitting them accidentally while manipulating the joystick. A deluxe feature (for models that may be used by multiple individuals) could included adding alternative 1/8" switch jacks along the sides. This would allow you to plug in whatever switches the client finds most accessible (head array switches or whatever).
shadowwynd (author)  gabriellaigo1 year ago
The best thing about having the buttons defined through software is that they don't have to do anything. Instead of countersinking / covering with velcro, you can just disable the buttons through software. Hitting the wrong button = nothing.

On the other hand, countersinking the buttons might help act as a keyguard, and hiding the buttons completely could help when working with people who don't have the mental ability to use six buttons.

I like the idea of the 1/8" switch jacks on the side - that will certainly be an addition to the next micro-production run. This design is wood about 1/2" thick on the sides (thicker than any panel jacks), but it could be milled down. I have thought about making a switch-interface out of just the game controller board and putting it in a plastic enclosure box. In my design, I use the crimp connectors on the arcade parts (rather than soldering into place) so that the design can be physically rewired easily if needed. The hard part is adapting the controller board - everything else (switch jacks, etc) is relatively easy.
Paint_Man2 years ago
have you tried "JoyToKey," or "Xpadder?"

They can program both keyboard and mouse. JoyToKey is free, but Xpadder costs 5 dollars for each latest version.
Tomcat943 years ago
This is a great instructable! It's a unique method of assistive technology that takes something as simple as a joystick and some buttons and turns it into something that helps the people that need it most. Very nicely done.

This can also be used for MAME cabinets as well, except you use it for a MAME cabinet instead. Most MAME cabinet instructions are a bit vague when describing how to put the joystick together, so this actually kills two birds with one stone...
kelseymh3 years ago
Awesome project set! I wish I understood why the commercial product costs ten times your component cost: economy of scale ought to work in the other direction. I've added this project to I'bles Assistive Technology group; hope that's okay.
shadowwynd (author)  kelseymh3 years ago
Thanks for adding it to the AT group!

We have been puzzled by the economies of scale issue ourselves. However, this is true for most of Assistive Technology - much of the commercial equipment is insanely expensive (it is not hard to find things in the $5000+ price bracket). This is sad, because most people with disabilities don't have a lot of money (or the money they do have goes for therapies, medicines, etc.) It may have to do with being classed as a "medical device" or something - apparently there is 10x-20x jump in price if something is classed as medical or therapeutic.

Great work shadowwynd!

I've been a AAC (Augmentative & Alternative Communication) technologists for almost 10 years.

From what I've seen is that the companies that don't charge enough don't survive. To be effective their product must:

1.) Be strongly made.
2.) Good support must be offered.
3.) Solid research & development must be on going.

Take DynaVox as an example. They are still in business after all these years!

I think if something is classed as "medical or therapeutic" or sold as such, the manufacturer or distributor takes on the liability of that product. And you know how lawsuits and the abuse of such, and greed in the healthcare coverage/billing comes into play. The person with a real need is in a catch-22. If you market this as a game or computer accessory, you would have no problems. Thanks for sharing.
YoshiDan3 years ago
That's pretty cool.

I was thinking you could probably use the board out of a USB numeric keypad for this instead of a game controller. That way you wouldn't need any 3rd party software; you could just turn on mousekeys in Windows.
shadowwynd (author)  YoshiDan3 years ago
Great idea! The board from a USB Keypad + Mousekeys would work very well and could be an easier project than the gamepad. We have used the boards from full-size keyboards for other projects (as well as the board from mice).

Two notes, though:

1) For our original user, it would still need additional software to tell him what was pressed because he couldn't move his head to see his hand. For someone who didn't need this, the keypad would work great, especially if they didn't need variable speed.

2) There exists the possibility of blocking keys due to simultaneous presses. A keyboard is designed for very fast sequential presses, but doesn't work well if multiple keys are pressed at the same time. A keyboard uses a matrix to of rows and columns to detect key presses. If the right multiple keys are pressed, other input is ignored. For instance, open a text editor. Hold down the "A" and "S" keys simultaneously. Then try typing "W" or "X" while holding the "A" and "S" keys down - most keyboards won't do it because the same column is still selected. A numeric keypad would be the same.

A joystick normally has each button go to a separate pin on the microcontroller. This means that all the buttons can be used independent of the others (they can all be held down and one could be toggled successfully). For *this* project it shouldn't make a difference due to the limitations of the user. For some projects (for instance, if you were using a joystick board as an interface to a bank of switches) this property becomes important.
I'm working on an arcade machine with all the same parts. Are you sure the hole is the same size for the buttons and joystick? it seems like the joystick should be bigger. Does the 1 1/8th inch hole for the joystick allow full movement? My lack of a 1 1/4 inch bit is the only thing really holding me back right now. (And the fact that I'm in college and my machine is back home, but that's besides the point)
shadowwynd (author)  Super Cameraman3 years ago
It is not intuitive, but yes - for the Happ Competition/Ultimate line of joysticks the hole needed is 1 1/8th, same as the buttons. This allows for full motion. Other brands and models might differ, refer to manufacturer's instructions. You do have to mount the joystick so the neutral position is centered under the 1 1/8th hole. If you go slightly bigger it should be fine for the joystick - too big of a hole for the buttons and they may not mount.

good luck3 years ago
chto za hren, pochemu net kililici
good luck3 years ago
модный гаджет
zidakano3 years ago
Something I personally would do to improve longevity/reliability of this product would be to solder in a female usb port and mount it in the case and use a plain usb cord. That way there would be no problem if it were "yanked" and if the cord ever went bad (as my current mouse has done) it could easily be replaced with off the shelf parts rather than opening it up and having to re solder and/or replace the cord.

While making something "self-serviceable" is bad practice in mass production I consider it a must in "good quality"

But either way it is a good ible just the same.

shadowwynd (author)  zidakano3 years ago
A USB port / detachable cable is one of the improvements we want to have for the next model/small production run (we build these 2 or 3 at a time). I think we are Model IV (not counting the prototype).

I would prefer to see more mass-produced things be self-serviceable, myself.
p1xel3 years ago
Nice =)
ynze3 years ago
Very cool!
Great job on the 'ible! Very clear and straightfoward!
ChrysN3 years ago
Wonderful project!