How to Control a Simple Processing Game With Arduino




About: French engineer in robotics. I love Arduino projects / coding / guitar / various electronics. Oh, I like cooking also ! PS : English is not my main language, please inform me in case of grammar mistakes ! ...

Hi all,

In this Instructable, we will see how to create a "link" between a Processing sketch and an Arduino card. In this example, an ultrasonic module will be used to control a plane in a simple game. Again, this tutorial is just an example, you may use it to create something else, an other game or other fun project ! Just take a look at the different kinds of sensors supported by the Arduino and imagine what kind of projects you can make !

The goal of the game is simple : control the height of a plane, and try to avoid the clouds. The height of the plane is given by your hand.

As always, please tell me if you find English mistakes ! Feel free to ask questions.

Teacher Notes

Teachers! Did you use this instructable in your classroom?
Add a Teacher Note to share how you incorporated it into your lesson.

Step 1: Material List

Ok, you will need :

• an Arduino card (UNO in this example)
• an ultrasonic measurement module
• breadboard wires
• Arduino software installed on the computer (link here)
• Processing software (link here)
• the Arduino and Processing sketches (see next steps)

Step 2: Arduino Wiring

Ok, please load the attached sketch into the Arduino IDE.

The ultrasonic module wiring is simple:

  • Vcc -> to Arduino 5v
  • Trig -> Arduino pin 3
  • Echo -> Arduino pin 2
  • GND -> to Arduino GND

How this module works ? It contains two ultrasonics emitters and receivers (the gray cylinders). The emitter sends a sound vibration, the vibration bounces on the closest object (your hand) and the signal is received by the module. By computing the time between the emission and reception, the module will deduce the distance between him and your hand.

Check the wiring, and upload the code on the Arduino. Then open the Serial Monitor, and move your hand above the module. The serial monitor should display a list of numbers... The distance in cm between him and you hand.

Still in this window, get the serial port number. In my case, COM16. (bottom right side of the window)

Step 3: Processing Configuration

Ok, open Processing and open the attached zip file. It contains the source sketch and some pictures. Please leave them in the same folder.

In the code, check the line 52. We will put our serial port number here.

Ok it's strange, Processing doesn't directly works with the COM port number, but with an other numbering. If your serial port is 1, the Processing number is 0. COM 2 -> Processing number 1, ... In my case, COM16 is the number 2. (In case of problems, just upload and run the attached sketch "serial_ports.pde" in Processing to search for existing serial ports.)

The Processing sketch game will get the serial data provided by the Arduino, and the values will be used to move the plane.

How it works ?

The Arduino card send data only if the player moves its hand. As soon as data is received, a special event in the processing code is activated :

So when the player move its hand, a new distance value is sent. The Processing sketch gets the value, check the range and apply the new value to the height variable. Meanwhile, the game continues...

For debugging issues, the incoming distance is printed into the Processing console.

Step 4: Play !

Ok, if everything is OK, select Run in the main toolbar.

Put the emitter horizontally on a table, and put your hand at about 20 cm above to start. The plane should move as you move your hand... The height values are displayed in the Processing console.

Remember that this game is just an example, you can't lose or win... (but looks very hard ?)

That's all, a simple way to use Arduino sensors to drive, in this case, a game written in Processing. But imagine what you can do with other sensors : an accelerometer (with think about the wiimote games), digital buttons, gyroscope sensor, sound detectors, piezzo sensors... What kind of games you will create ?

Thanks for reading !

UPDATE : Check the new next step to learn how to replace the plane drawing by a nice picture.

Step 5: Optional Step : How to Replace the Plane by a Picture

I received a lot of feedback about this project. Thanks everybody, it's always a pleasure.

Some people asks me how to replace the (ugly) plane model by a picture, like the clouds. (That's right, the initial model was not really beautiful...)

To replace it by a picture, you just have to replace the line coordinates by a picture call. So you have to load it first, and to make sure that the picture translates and rotates fine.

Check the attached ZIP file. It contains the new project file, with the new plane picture. More explanations are in the program. Just ask if you have additional questions.

Thanks for reading !!

Step 6: A Score System

UPDATE 07/12/15 :

Hi readers,

A lot of people was asking how to add a score system when you hit (a bit violent no ?) the bird.

A simple way to achieve this is to compute the distance between the plane and the bird. When this distance goes under a given value (40 pixels in my code), the score is increased by one and the bird position is resetted to the right side of the screen.

The score is also displayed on the screen.

The only con with this method is that it doesn't really take care of the plane position. But it works.

If you want something harder, just reduce the threshold distance, try 20 pixels.

Check the attached code. Unzip the file and make sure to keep the pictures in the same folder.

2 People Made This Project!


  • Made with Math Contest

    Made with Math Contest
  • Multi-Discipline Contest

    Multi-Discipline Contest
  • Robotics Contest

    Robotics Contest

24 Discussions


Reply 4 weeks ago

For an other sensor you will have to do the same : writing an arduino code that can read this sensor then send the data to the Processing sketch.

YorukGuillermo J.G

Reply 2 years ago

Sorry for my delayed answer.

You have different ways to add an other player :

-you can use a second ultrasound module wired on the arduino. Of course, you will have to send the two information to the game, and add an other plane
-or you can play with the mouse or keyboard, instead of adding an other ultrasound device.

Let me know if you need more details !


Reply 4 weeks ago

Well this code is not very written to easily add another plane. The actual solution is to duplicate all the drawing code and apply new coordinates variables. The better way should be to create a "plane" class that you can instantiate two times.


Question 1 year ago

Hi there, I was wondering how to add an obstacle element to the game, so when the plane hits a cloud the score is decreased by one.

I have tried using the same 'check distance' coding as you used for checking the distance between the bird and plane to increase the score, however it does not work for the distance between a cloud and plane.

1 answer

Answer 1 year ago

You are right : an obstacle and the bird collision detection works the same, by computing the distance.

The problem is that is there is only one bird at a time, but we have man clouds. If you want to work on multiple collision items, you will have to create an array of "distances" to store the distance between the plane and all the items.

Let me know if you need more details. You can also post some code.


3 years ago

Hello! I am having some problems with the serial port code, first it says " Duplicate field ultrasound_plane.myPort the if i change the port says that the comand Void() is duplicated, what am i doing wrong?

1 reply

Reply 3 years ago

Strange... What is your current Processing version?


3 years ago

Hey Yoruk, how do I code this to fit to a mac? My Arduino is name COM3 on a windows but my mac read it as usb/port, so I can't put any values on myPort.

1 reply

Reply 3 years ago


I don't know anything on macs.... Please take a look at this page :

They talk about mac and serial ports. Hope this helps...!


Reply 3 years ago

Hi Yoruk, I seem to have the same problem. Mine is a 64 bit. And processing version 3.0.2 Please help asap.


Reply 3 years ago

The serial function doesn't works with the 64 bits version. You have to use the 32 bits instead. Try it out ! (both versions can fit on the same computer)


3 years ago

Hi Yoruk! Thanks for sharing this example project. I met problems when i tired to do it myself. As i am using ping ultrasonic sensor that is differ to your sensor. It only has ping pin so as a result i have massive numbers println.

distance = pulsetime/58 < this still gives me numbers over 100

but when i tried divide pulsetime by large numbers, the range from the sensor shorten to only 2 digits.

What should i do to fix this problem!!? I really hope you to reply me. Please!

1 reply

Reply 3 years ago

Hmmm.... Can you show me a pic of this sensor ? Have you got it's datasheet ?

When you move your hand over it, do you see differents values ? Range ?


3 years ago

Hello there! Thanks for the instructions, I'm a bit confused with the code

"Angle = mouseY-300; //uncomment this line and comment the next one if you want to play with the mouse

//Angle = (18- DistanceUltra)*4; // you can increase the 4 value..."

my ultrasonic sensor cannot control the plane but my mouse does instead, i tried changing the code but it doesn't work

1 reply

Reply 3 years ago


You just have to comment the first line (used to play with the mouse) and to enable the next one (to play with the sensor). So, you will have :

//Angle = mouseY-300; //uncomment this line and comment the next one if you want to play with the mouse

Angle = (18- DistanceUltra)*4; // you can increase the 4 value..

This should solve the problem, let me know !