PC Mouse Made With Arduino Uno and Joystick

by Syntaxian

40K4980


Intro: PC Mouse Made With Arduino Uno and Joystick

Hi! Welcome to my first Instructable.

I recently began tinkering with my new Arduino Uno and decided to find an application for a PS2 joystick module. I thought it would be nifty to turn my Arduino into a joystick controlled mouse for my PC.

I must confess: I thought the task would be easy using the "mouse" library I found online, little did I realize that this class only works with Arduino Leonard and Micro (and perhaps a few others), but not the ubiquitous Uno. I was discouraged, but I decided to use the Uno's serial communication as well as my rudimentary Java abilities to "hack" the Uno into a functional mouse for Windows. It surprisingly worked! Here's how:

STEP 1: Materials

This project does not require many materials:

1 Arduino Uno

5 male to male wires

5 female to female wires (to connect to joystick module and to add extension length for joystick.

1 Joystick (I used the SainSmart PS2 joystick module and would recommend it)

STEP 2: Setting Up the Arduino Uno

The setup of the Uno can be seen in the materials picture, and here's the instructions:

Connect the five female to female wires to the pins of the joystick module. Now, connect five male to male wires into the ends of the female wires and connect them to the Arduino in this way:

1. The Ground on the joystick to Arduino Gnd

2. The +5V on the joystick to Arduino 5V

3. The UPx on the joystick to A0 on the Arduino

4. The UPy on the joystick to A1

5. The SW pin (the digital click switch) to digital pin 7 on the Arduino

STEP 3: Upload the Joystick Program to Arduino

Connect the Uno to your PC and upload the joystick code seen here (please note I did not create this code originally):

int pushPin = 7;       	// potentiometer wiper (middle terminal) connected to analog pin 3
int xPin = 0;
int yPin = 1;
int xMove = 0;
int yMove = 0;
			// outside leads to ground and +5V
int valPush = HIGH;     // variable to store the value read
int valX = 0;
int valY = 0;
void setup()
{
	pinMode(pushPin,INPUT);
	Serial.begin(9600);         //  setup serial
	digitalWrite(pushPin,HIGH);
}

void loop()
{
	valX = analogRead(xPin);    // read the x input pin
	valY = analogRead(yPin);    // read the y input pin
	valPush = digitalRead(pushPin); // read the push button input pin
	
	Serial.println(String(valX) + " " + String(valY) + " " + valPush);    //output to Java program
}

STEP 4: Setting Up Java Program

Now that the Uno is set up, we need to connect it to my Java program which is capable of taking the Uno's serial output values with the special library RxTx and moving the mouse with the library collection JNA. Both of these libraries are included for download at the end of this step. Please note that the only part of the code I changed from the example RxTx was adding the method that moves the mouse in a way that I calibrated for my joystick. It's a bit crude, but it served my purposes.

I used BlueJ as my IDE, but whichever Java IDE you use, install RxTx and JNA libraries for this project, which I named "Mouse". Once that's done, created a project and include this code:

import java.awt.*;
import java.awt.event.InputEvent;
import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.io.OutputStream;
import gnu.io.CommPortIdentifier; 
import gnu.io.SerialPort;
import gnu.io.SerialPortEvent; 
import gnu.io.SerialPortEventListener; 
import java.util.Enumeration;


public class Mouse implements SerialPortEventListener {
    SerialPort serialPort;
        /** The port we're normally going to use. */
    private static final String PORT_NAMES[] = { 
            "/dev/tty.usbserial-A9007UX1", // Mac OS X
                        "/dev/ttyACM0", // Raspberry Pi
            "/dev/ttyUSB0", // Linux
            "COM4", // Windows**********(I changed)
    };
    /**
    * A BufferedReader which will be fed by a InputStreamReader 
    * converting the bytes into characters 
    * making the displayed results codepage independent
    */
    private BufferedReader input;
    /** The output stream to the port */
    private OutputStream output;
    /** Milliseconds to block while waiting for port open */
    private static final int TIME_OUT = 2000;
    /** Default bits per second for COM port. */
    private static final int DATA_RATE = 9600;
    
    int buttonOld = 1;
    
    public void initialize() {
                // the next line is for Raspberry Pi and 
                // gets us into the while loop and was suggested here was suggested  http://www.raspberrypi.org/phpBB3/viewtopic.php?f...
                //System.setProperty("gnu.io.rxtx.SerialPorts", "/dev/ttyACM0"); I got rid of this
        CommPortIdentifier portId = null;
        Enumeration portEnum = CommPortIdentifier.getPortIdentifiers();
        //First, Find an instance of serial port as set in PORT_NAMES.
        while (portEnum.hasMoreElements()) {
            CommPortIdentifier currPortId = (CommPortIdentifier) portEnum.nextElement();
            for (String portName : PORT_NAMES) {
                if (currPortId.getName().equals(portName)) {
                    portId = currPortId;
                    break;
                }
            }
        }        
        if (portId == null) {
            System.out.println("Could not find COM port.");
            return;
        }
        try {
            // open serial port, and use class name for the appName.
            serialPort = (SerialPort) portId.open(this.getClass().getName(),
                    TIME_OUT);
            // set port parameters
            serialPort.setSerialPortParams(DATA_RATE,
                    SerialPort.DATABITS_8,
                    SerialPort.STOPBITS_1,
                    SerialPort.PARITY_NONE);
            // open the streams
            input = new BufferedReader(new InputStreamReader(serialPort.getInputStream()));
            output = serialPort.getOutputStream();
            // add event listeners
            serialPort.addEventListener(this);
            serialPort.notifyOnDataAvailable(true);
        } catch (Exception e) {
            System.err.println(e.toString());
        }
    }
     /**
     * This should be called when you stop using the port.
     * This will prevent port locking on platforms like Linux.
     */
    public synchronized void close() {
        if (serialPort != null) {
            serialPort.removeEventListener();
            serialPort.close();
        }
    }
    /**
     * Handle an event on the serial port. Read the data and print it. In this case, it calls the mouseMove method.
     */
    public synchronized void serialEvent(SerialPortEvent oEvent) {
        if (oEvent.getEventType() == SerialPortEvent.DATA_AVAILABLE) {
            try {
                String inputLine=input.readLine();
                mouseMove(inputLine);
                System.out.println("********************");
                //System.out.println(inputLine);
            } catch (Exception e) {
                System.err.println(e.toString());
            }
        }
        // Ignore all the other eventTypes, but you should consider the other ones.
    }
public static void main(String[] args) throws Exception     {
        Mouse main = new Mouse();
        main.initialize();
        Thread t=new Thread() {
            public void run() {
                //the following line will keep this app alive for 1000 seconds,
                //waiting for events to occur and responding to them (printing incoming messages to console).
                try {Thread.sleep(1000000);} catch (InterruptedException ie) {}
            }
        };
        t.start();
        System.out.println("Started");
    }
    
    // My method mouseMove, takes in a string containing the three data points and operates the mouse in turn
    public void mouseMove(String data) throws AWTException
    {
        int index1 = data.indexOf(" ", 0);
        int index2 = data.indexOf(" ", index1+1);
        int yCord = Integer.valueOf(data.substring(0, index1));
        int xCord = Integer.valueOf(data.substring(index1 + 1 , index2));
        int button = Integer.valueOf(data.substring(index2 + 1));
        Robot robot = new Robot();
        
        int mouseY = MouseInfo.getPointerInfo().getLocation().y;
        int mouseX = MouseInfo.getPointerInfo().getLocation().x;  
        
        if (button == 0)
        {
            if (buttonOld == 1)
            {
                robot.mousePress(InputEvent.BUTTON1_DOWN_MASK);
                robot.delay(10);
            }
        }
        else
        {
            if (buttonOld == 0)
                robot.mouseRelease(InputEvent.BUTTON1_DOWN_MASK);            
        }
        
        if (Math.abs(xCord - 500) > 5)
            mouseX = mouseX + (int)((500 - xCord) * 0.02);
        if (Math.abs(yCord - 500) > 5)
            mouseY = mouseY - (int)((500 - yCord) * 0.02);
        
        robot.mouseMove(mouseX, mouseY);
            
        buttonOld = button;
        System.out.println(xCord + ":" + yCord + ":" + button + ":" + mouseX + ":" + mouseY);
        return;
    }
}

STEP 5: Troubleshooting

Getting the Java program to work may be difficult. I've got some tips if you're stuck:

-Change the "Com4" string in the PORT_NAMES[] to the port your arduino Uno is connected to. (I changed to Com4 from the default Com3 in my Java program)

-Comment out the line relating to Raspberry Pi (if you copied my program, I already did this)

-Click "Rebuild Package" or your IDEs equivalent

-Reset the Java Virtual Machine in your IDE. Maybe even reset the program before using the mouse the first time.

STEP 6: Conclusion

I hope this project works for you and that you can improve upon it. Ultimately, the easiest solution is to use an Arduino Leonard or Mini that can function as a system device for mouse inputs, but I found it fun to make the Uno function in a way it was not designed--a mouse--by using my limited Java knowledge.

I learned a lot alone the way and hope to add several features in the future:

-Right Click button. The joystick has one button which I reserved for the left click.

-Real device driver for this project. I'm not sure if this is possible, maybe someone can enlighten me on the subject!

Thanks for reading!

56 Comments

DUPLICATE_Eeeeee 2 years ago

I dont understand what the problem is. I followed downloading the different RXTX files from SeanY25 however I could not evade the RXTX mismatch, I don't even know if that is the issue or if it is something else. for prior knowledge I am using a windows computer on IntelliJ IDE.





WARNING: RXTX Version mismatch
Jar version = RXTX-2.2pre1
native lib Version = RXTX-2.2pre2
Started
#
# A fatal error has been detected by the Java Runtime Environment:
#
# EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x0000000180003990, pid=25204, tid=29972
#
# JRE version: Java(TM) SE Runtime Environment (16.0.2+7) (build 16.0.2+7-67)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (16.0.2+7-67, mixed mode, sharing, tiered, compressed oops, compressed class ptrs, g1 gc, windows-amd64)
# Problematic frame:
# C [rxtxSerial.dll+0x3990]
#
# No core dump will be written. Minidumps are not enabled by default on client versions of Windows
#
# An error report file with more information is saved as:
# C:\Users\"Username"\ideaprojects\joystick\hs_err_pid25204.log
#
# If you would like to submit a bug report, please visit:
# https://bugreport.java.com/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#


Please let me know anything
Thank you.

Afghanibaby 2 months ago

you have to download a RXTX version 2.2pre2


Syntaxian 2 years ago

It seems that your version of Java environment already has its own RXTX version 2.2pre2, and you downloaded a Jar version RXTX-2.2pre1, which is why you are getting a mismatch. Maybe delete the version you added, since it might be included. The new version may not be compatible, but it probably is.

DUPLICATE_Eeeeee 2 years ago

Do you mean delete the RXTXcomm file from the IDE? Because when I do that it only seems to create a lot of compilation errors (due to the fact that it doesn't recognize the imports from "gnu...")

Afghanibaby 2 months ago

Got it working only problem is i gotta add a delay to the arduino IDE code and its all choppy if i dont add a delay the joystick doesnt move. Is there anyway around it and also i got some stickdrift if anyone could also help with that

liorsalter1 3 years ago

java.lang.UnsatisfiedLinkError: C:\Users\USER\Joystick\rxtxSerial.dll: Can't load IA 32-bit .dll on a AMD 64-bit platform thrown while loading gnu.io.RXTXCommDriver
java.lang.UnsatisfiedLinkError: C:\Users\USER\Joystick\rxtxSerial.dll: Can't load IA 32-bit .dll on a AMD 64-bit platform
at java.base/java.lang.ClassLoader$NativeLibrary.load0(Native Method)
at java.base/java.lang.ClassLoader$NativeLibrary.load(ClassLoader.java:2430)
at java.base/java.lang.ClassLoader$NativeLibrary.loadLibrary(ClassLoader.java:2487)
at java.base/java.lang.ClassLoader.loadLibrary0(ClassLoader.java:2684)
at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2649)
at java.base/java.lang.Runtime.loadLibrary0(Runtime.java:829)
at java.base/java.lang.System.loadLibrary(System.java:1867)
at gnu.io.CommPortIdentifier.<clinit>(CommPortIdentifier.java:123)
at Mouse.initialize(Mouse.java:42)
at Mouse.main(Mouse.java:104)

what is the problem here?

mrunal bhimte 4 years ago

Hello there, I was trying to make this project and I faced a single problem at last step after compiling blue j program and uploading Arduino code....

This was written on terminal window after running the program.

Started
#
# A fatal error has been detected by the Java Runtime Environment:
#
# EXCEPTION_ACCESS VIOLATION (0xc0000005) at pc=0x0000000180005b00, pid=7856, rid=5448
#
# JRE version: OpenJDK Runtime Environment (11.0.2+9) (build 11.0.2+9)
# Java VM: OpenJDK 64-Bit Server VM (11.0.2+9, mixed mode, tiered, compressed oops, g1 gc, Windows-amd64)
# Problematic frame:
# C [rxtxSerial.dll+0x5b00]
#
# No core dump will be written. Minidumps are not enabled by default on client versions of Windows
#
# An error report file with more information is saved as:
#F:\Document\BlueJ\yo\hs_err_pid7856.log
#
# If you would like to submit a big report, please visit:
# https://bugreport.java.com/crash.jsp
# The crash happens outside the Java Virtual Machine in native code.
#. See Problematic frame for where to report the but.
#

Please provide solution for this problem plzzz.....

CyberCommander 4 years ago

I have finished the Java and Arduino Programs, but somethings not working. I've checked the serial monitor and it is outputting the correct data, but it seems that the java program isn't picking it up. I've created my JAR file and everything. Please help.
Thanks

reptooyep 5 years ago

Hi,
You did a great job.
I want to do quite the same with a useless raspberry pi.
I want to make a minecraft gaming station and use two ps2 joysticks :
one for arrows and another for mouse view.
For the action buttons, i'll make a ATMEGA328p standalone board with V-USB library recognised as a HID keyboard which send keyboard shortcuts while key pressed.
For the mouse view, i'd like to use your methode but my 2nd standalone chip will be connected on raspberry GPIO by I2C or serial (whatever). I would like to have your advices for this. I can also use the same library (V-USB) and it will be recognize as a HID mouse by the raspberry.

BeefStew34 5 years ago

Im still having a bit of trouble running the program How do you run it in Blue J I am on the final stage and are getting these errors

java.lang.ClassNotFoundException: gnu.io.RXTXCommDriver thrown while loading gnu.io.RXTXCommDriver

java.lang.UnsatisfiedLinkError: no rxtxSerial in java.library.path

at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)

at java.lang.Runtime.loadLibrary0(Runtime.java:870)

at java.lang.System.loadLibrary(System.java:1122)

at gnu.io.CommPortIdentifier.<clinit>(CommPortIdentifier.java:126)

at Mouse.initialize(Mouse.java:41)

at Mouse.main(Mouse.java:103)

SeanY25 5 years ago

I had the same problem, here's what I did to make it work. I am on MacOS.

I think the reason is because BlueJ is unable to find the two Java library files required for the code to work, "RXTXcomm.jar" and "librxtxSerial.jnilib". You can get the 2 files from (http://rxtx.qbang.org/wiki/index.php/Download). I downloaded binary "rxtx 2.2pre2 (prerelease)" because I was faced with RXTXcom files mismatched, which requires both your Arduino and Java IDE (BlueJ) having the same version (2.2) of RXTX. Now, you have to move the 2 files into your Java directory (which is hidden by default, can be open with Terminal by the following code).

cd /Library/Java/
open .

Move the 2 files into the Extension folder. (Admin password will be required)

Next, make sure you have directed BlueJ towards 2 libraries, RXTXcom.jar (which is the one you just installed in the Java directory) and the jna-3.2.7-sources.zip (from the tutorial above). You do so by -> BlueJ -> Preferences -> Library -> add files. Now BlueJ should know where to find those correct 2 libraries for the code to work. Restart BlueJ, compile the code, and right click on the "mouse" module -> void main (string[] args) -> "OK". It should work by now. Good luck! (took me 8 hours to figure this all out!)

Now, i'm off to try to make this work on Processing...

BeefStew34 5 years ago

Thx I haven't Tried yet but It looks like it should work Thx again for your help

Nikhil saiS 5 years ago

heyy beefstew did you find any answer to this errors.I am also getting those errors. If you got any answer please mail me.mail:nikhilsai6076@gmail.com

Syntaxian 5 years ago

It looks like you might not have installed the libraries I included in the instructions. RxTx class is contained in one of the files so your IDE should find them if you import the libraries.

Asher Lego Films 5 years ago

Hi, it doesn't work in eclipse ide... will you help me?

here is the console log

Exception in thread "main" java.lang.NoClassDefFoundError: SerialPortEvent

at java.lang.Class.getDeclaredMethods0(Native Method)

at java.lang.Class.privateGetDeclaredMethods(Unknown Source)

at java.lang.Class.privateGetMethodRecursive(Unknown Source)

at java.lang.Class.getMethod0(Unknown Source)

at java.lang.Class.getMethod(Unknown Source)

at sun.launcher.LauncherHelper.validateMainClass(Unknown Source)

at sun.launcher.LauncherHelper.checkAndLoadMain(Unknown Source)

Caused by: java.lang.ClassNotFoundException: SerialPortEvent

at java.net.URLClassLoader.findClass(Unknown Source)

at java.lang.ClassLoader.loadClass(Unknown Source)

at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)

at java.lang.ClassLoader.loadClass(Unknown Source)

... 7 more

I am not very good at java (or using eclipse) so I don;t really know what to do

Syntaxian 5 years ago

I would recommend checking to see if the libraries are correctly added to your IDE and also I would check to see whether the Arduino is connected to the same serial port on your computer as mine. Your computer will likely have different options to see the Arduino, perhaps COM3 or COM4 or something like that. If you can figure out which one it is and change the code if necessary that could help.

arduino_nano 7 years ago

its posible to do this movements automatic without joystick? anyone can respone please i need this for a important project thanks

More Comments