How to Control a Simple Processing Game With Arduino





Introduction: How to Control a Simple Processing Game With Arduino

Hi all,

In this Instructable, we will see how to create a "link" between a Processing sketch and an Arduino card. In this example, an ultrasonic module will be used to control a plane in a simple game. Again, this tutorial is just an example, you may use it to create something else, an other game or other fun project ! Just take a look at the different kinds of sensors supported by the Arduino and imagine what kind of projects you can make !

The goal of the game is simple : control the height of a plane, and try to avoid the clouds. The height of the plane is given by your hand.

As always, please tell me if you find English mistakes ! Feel free to ask questions.

Step 1: Material List

Ok, you will need :

• an Arduino card (UNO in this example)
• an ultrasonic measurement module
• breadboard wires
• Arduino software installed on the computer (link here)
• Processing software (link here)
• the Arduino and Processing sketches (see next steps)

Step 2: Arduino Wiring

Ok, please load the attached sketch into the Arduino IDE.

The ultrasonic module wiring is simple:

  • Vcc -> to Arduino 5v
  • Trig -> Arduino pin 3
  • Echo -> Arduino pin 2
  • GND -> to Arduino GND

How this module works ? It contains two ultrasonics emitters and receivers (the gray cylinders). The emitter sends a sound vibration, the vibration bounces on the closest object (your hand) and the signal is received by the module. By computing the time between the emission and reception, the module will deduce the distance between him and your hand.

Check the wiring, and upload the code on the Arduino. Then open the Serial Monitor, and move your hand above the module. The serial monitor should display a list of numbers... The distance in cm between him and you hand.

Still in this window, get the serial port number. In my case, COM16. (bottom right side of the window)

Step 3: Processing Configuration

Ok, open Processing and open the attached zip file. It contains the source sketch and some pictures. Please leave them in the same folder.

In the code, check the line 52. We will put our serial port number here.

Ok it's strange, Processing doesn't directly works with the COM port number, but with an other numbering. If your serial port is 1, the Processing number is 0. COM 2 -> Processing number 1, ... In my case, COM16 is the number 2. (In case of problems, just upload and run the attached sketch "serial_ports.pde" in Processing to search for existing serial ports.)

The Processing sketch game will get the serial data provided by the Arduino, and the values will be used to move the plane.

How it works ?

The Arduino card send data only if the player moves its hand. As soon as data is received, a special event in the processing code is activated :

So when the player move its hand, a new distance value is sent. The Processing sketch gets the value, check the range and apply the new value to the height variable. Meanwhile, the game continues...

For debugging issues, the incoming distance is printed into the Processing console.

Step 4: Play !

Ok, if everything is OK, select Run in the main toolbar.

Put the emitter horizontally on a table, and put your hand at about 20 cm above to start. The plane should move as you move your hand... The height values are displayed in the Processing console.

Remember that this game is just an example, you can't lose or win... (but looks very hard ?)

That's all, a simple way to use Arduino sensors to drive, in this case, a game written in Processing. But imagine what you can do with other sensors : an accelerometer (with think about the wiimote games), digital buttons, gyroscope sensor, sound detectors, piezzo sensors... What kind of games you will create ?

Thanks for reading !

UPDATE : Check the new next step to learn how to replace the plane drawing by a nice picture.

Step 5: Optional Step : How to Replace the Plane by a Picture

I receive a lot of feedback about this project. Thanks everybody, it's always a pleasure.

Some people asks me how to replace the (ugly) plane model by a picture, like the clouds. (That's right, the initial model was not really beautiful...)

To replace it by a picture, you just have to replace the line coordinates by a picture call. So you have to load it first, and to make sure that the picture translates and rotates fine.

Check the attached ZIP file. It contains the new project file, with the new plane picture. More explanations are in the program. Just ask if you have additional questions.

Thanks for reading !!

Step 6: A Score System

UPDATE 07/12/15 :

Hi readers,

A lot of people was asking how to add a score system when you hit (a bit violent no ?) the bird.

A simple way to achieve this is to compute the distance between the plane and the bird. When this distance goes under a given value (40 pixels in my code), the score is increased by one and the bird position is resetted to the right side of the screen.

The score is also displayed on the screen.

The only con with this method is that it doesn't really take care of the plane position. But it works.

If you want something harder, just reduce the threshold distance, try 20 pixels.

Check the attached code. Unzip the file and make sure to keep the pictures in the same folder.

2 People Made This Project!


  • Paper Contest 2018

    Paper Contest 2018
  • Pocket-Sized Contest

    Pocket-Sized Contest
  • Science of Cooking

    Science of Cooking

We have a be nice policy.
Please be positive and constructive.




Good job, I like the game but ... how to do it multiplayer?

Sorry for my delayed answer.

You have different ways to add an other player :

-you can use a second ultrasound module wired on the arduino. Of course, you will have to send the two information to the game, and add an other plane
-or you can play with the mouse or keyboard, instead of adding an other ultrasound device.

Let me know if you need more details !

Hello! I am having some problems with the serial port code, first it says " Duplicate field ultrasound_plane.myPort the if i change the port says that the comand Void() is duplicated, what am i doing wrong?

Strange... What is your current Processing version?

Hey Yoruk, how do I code this to fit to a mac? My Arduino is name COM3 on a windows but my mac read it as usb/port, so I can't put any values on myPort.


I don't know anything on macs.... Please take a look at this page :

They talk about mac and serial ports. Hope this helps...!

My Arduino Serial COM3, and I Can't find the Processing Serial
what is the problem ?!!


What version of Processing do you have ? 32, 64 bits ?

Hi Yoruk, I seem to have the same problem. Mine is a 64 bit. And processing version 3.0.2 Please help asap.

The serial function doesn't works with the 64 bits version. You have to use the 32 bits instead. Try it out ! (both versions can fit on the same computer)