Introduction: Flight Simulator With Arduino and Python
Fly virtually around the world; tilting the Accelerometer forward, back, left and right to control the Plane's Pitching and Banking.
Objectives:
* Learn how to send Serial data from Arduino to Python
* Learn how to manipulate strings and lists in Python to read and interpret the data sent from Arduino
* Learn how to use the pywin library for Python to control Windows functions such as Mouse and Keyboard
We use Python as the interface between Arduino and Google Earth. In my Arduino / Raspberry Pi Internet Radio Instructable (https://www.instructables.com/id/Arduino-Raspberry-Pi-Internet-Radio/) we used nanpy on both Python and Arduino so that Python could issue commands to Arduino and read responses. Life would be boring if we always did stuff the same way, so in this project Arduino talks over Serial USB to Python by use of the pyserial library for Python.
If you don't have Google Earth on your PC download it from here:
http://www.google.com/earth/index.html
BTW, you if you don't want to agree to Google Chrome becoming your Default Browser UNCHECK the box at the License Agreement stage.
Step 1: You Will Need
You will need:
* A Windows PC
* Google Earth installed on the PC
* Python 3.2 installed on the PC
* An Arduino Uno
* An Accelerometer (e.g. http://goo.gl/ZddGy )
* A prototyping system or breadboard to wire the Accelerometer to the Arduino (e.g. http://goo.gl/1E1iI)
* A small push switch
Step 2: Install the Required Python Libraries
If you don't have Python already installed, get it from here:
http://www.python.org/download/releases/3.2.3/
I recommend Python 3.2 (as opposed to Python 2.7) because it's the supported version going into the future and many of the popular libraries such as pyGame, nanpy, pyserial, have been ported to 3.2. I also recommend the 32bit version for increased compatibility with libraries (even on a 64bit machine).
1.
We'll be using pyserial to communicate with the Arduino. There's an excellent guide to installation here:
http://goo.gl/L5o78
and you can find the library here:
http://pypi.python.org/pypi/pyserial
2.
Python will control the flight simulator by emulating the PC's mouse. To do that we need the pywin32 library for Python.
Take a look at how to install it here:
http://sourceforge.net/projects/pywin32/files/
I used the win32 installer for python3.2 (32bit) from here:
http://goo.gl/lO0DB
Step 3: Wire the Accelerometer and Switch to the Arduino
Wire the accelerometer to the Arduino as shown in the Diagrams below:
* 3V => 3.3V
* GND => GND
* X => A0
* Y => A1
* Z => A2
BEWARE: The Power to the ADXL335 is 3.3V. Connecting it to 5V will KILL it.
Add a Push Switch between Digital Pin0 and GND
Step 4: Arduino Tilt Control
There are several components to this project:
1. Tilting the Arduino Accelerometer sends data to Python on the PC
2. Python moves the PC's Mouse
3. Google Earth runs the Flight Simulator according to the Mouse Position
We'll start with getting the Arduino to sense the Tilt of the Accelerometer and send the data over the serial port (over USB) to the PC.
The full code is shown in the Next Step. For now let's see the main ideas that make the code work.
1.
At the start of the code we need to establish what readings from the Accelerometer correspond to level. We need to do this so we'll know when the device has been tilted away from level.
We write a function called average() using a for loop to measure the relevant data from the pin and average it as follows:
// Get the Average of 8 readings from 'pin'
int average(int pin) {
int Ave = 0;
for (int i=0; i<8; i++) {
Ave = Ave + analogRead(pin);
}
return Ave/8;
}
We call average() for the x, y and z axis measurements during the setup() section that is run once, at the beginning of the program.
2.
We establish the device is tilted by a simple subtraction as follows:
xTilt = analogRead(xpin)-xAve;
3.
We need to communicate the State of the Pushbutton Switch to Python so it can takeover mouse control when pressed and relinquish it when pressed again.
To do this we need to enable a Pullup on swpin in the setup() section:
digitalWrite(swpin, HIGH); // Enable Pullup on Switch Pin
and read the pin in the loop() section:
swState = digitalRead(swpin); // swState = 0 when Pressed
4.
We communicate with Python on the PC via the USB Serial connection by writing a pre-determined String Format:
"Data, xTilt, yTilt, zTilt , swState\n"
which is a String starting with "Data", the tilt values separated by commas, the switch State and a newline character.
So if the xTilt, yTilt, zTilt and swState where 2, 5, -1 and 0 respectively the string sent would be:
"Data,2,5,-1,0\n"
As all the data goes out over the Serial Port you can use the Serial Monitor <Ctrl+Shift+M> to view it.
5.
Go to the Next Step, copy the code into a blank Sketch and Upload to the Arduino.
Step 5: Full Arduino Code
/* Sense the Tilting action of an ADXL335 Accelerometer and
outputs the data on the Serial port (USB).
(c) Anthony Kelly, 2013 */
/*
A0 : X-Axis
A1 : Y-Axis
A2 : Z-Axis
*/
int xpin = A0, ypin = A1, zpin = A2;
int swpin = 0;
int swState = 0;
int xAve, yAve, zAve;
int xTilt, yTilt, zTilt;
void setup() {
Serial.begin(9600); // Fast Baud rate to reduce lag
pinMode(swpin, INPUT);
digitalWrite(swpin, HIGH); // Enable Pullup on Switch Pin
// Calibrate the sensor for LEVEL position by taking the Average of 8 readings
xAve = average(xpin);
yAve = average(ypin);
zAve = average(zpin);
}
void loop() {
xTilt = analogRead(xpin)-xAve;
yTilt = analogRead(ypin)-yAve;
zTilt = analogRead(zpin)-zAve;
swState = digitalRead(swpin); // swState = 0 when Pressed
// Send the Data as a Serial String as follows:
// "xTilt, yTilt, zTilt, swState \n"
// This String will be read by Python with lines separated by '\n'
Serial.print(xTilt, DEC);
Serial.print(",");
Serial.print(yTilt, DEC);
Serial.print(",");
Serial.print(zTilt, DEC);
Serial.print(",");
Serial.print(swState, DEC);
Serial.println();
}
// Get the Average of 8 readings from 'pin'
int average(int pin) {
int Ave = 0;
for (int i=0; i<8; i++) {
Ave = Ave + analogRead(pin);
}
return Ave/8;
}
Step 6: Python Mouse Control
We'll be using Python3.2 with the pywin32 library to gain control over some of the Windows functions such as Mouse Control. The pyserial library will allow us to read what Arduino is sending us on the USB Serial Port.
The full code is shown in the Next Step. For now let's see the main ideas that make the code work.
1.
At the start we open the serial port:
ser = serial.Serial(COM-1, 9600) # Open the Serial Port
But first we need to figure out which COM port it's on.
To do that we try each one in turn. If it fails we deal with the exception and keep looking.
# Find out which COM port the Arduino is on
for com in range(15):
try:
ser = serial.Serial(com, 9600) # Open the Serial Port
print("Found on COM",com+1) # If you got here the COM port was opened OK
break # We found it so stop searching
except :
print("Nothing on COM",com+1)
2.
swState is the variable we will read the switch state into (from the Serial port from Arduino).
Actually we don't want to know the switch state. We want to know when the button has been pressed.
We can do this by keeping track of the Previous value and subtracting the present value from it. We put that in a function:
def swPressed():
return swStateD1 - swState # 0,1,-1 : Same, Just Pressed, Just Released
i.e.
swState : 1,1,1,0,0,0,1,1,-
swStateD1 : -,1,1,1,0,0,0,1,1
swPressed(): -,0,0,1,0,0,-1,0,
As the comment says : when the button is pressed swPressed() returns 1
3.
We read the Serial data being sent by the Arduino using ser.readline() as follows:
This line reads a line up to and including the 'newline' characters '\r\n'.
rx = ser.readline().decode().split('\r\n')
and we split it based on commas
data = rx[0].split(',')
Finally we index each piece of data fom data (a List):
xTilt = int(data[0])
yTilt = int(data[1])
zTilt = int(data[2])
swState = int(data[3])
4.
Next we calculate the x,y position for the cursor based on the "centre" position (i.e the cx,cy position of the cursor when we pressed the switch to Activate cursor control - more on that later), and the tilt. We also give some control over the sensitivity i.e. how much the x,y values move for a given Tilt value:
x = cx + xTilt/Sensitivity
And we limit the x,y values. Nicely done in Python as follows:
x = max(min(xMax, x), 0) # Limit to the Screen co-ordinates
5.
Finally we check if the switch went from NOT Pressed to Pressed:
sw = swPressed() # Check the Button
If it was just pressed and we are in control of the cursor (enCursor ==1) then we disable the cursor and Break out of the while loop ending the program. But before we Break we emulate the pressing of the SPace Key to PAUSE Flight Simulator as follows:
win32api.keybd_event(0x20, 0,0,0) # PRESS the SPACE Key to Pause Flight Simulator
win32api.keybd_event(0x20,0 ,win32con.KEYEVENTF_KEYUP ,0)
Otherwise if the switch was just pressed and we are NOT in control of the cursor (enCursor ==0) then we have to enable the cursor (enCursor==1), update the cx, cy variables to the present position
(cx,cy) = win32api.GetCursorPos()
we get the cx, cy values so we can move the cursor + or - of this point in x and y as the accelerometer is tilted.
and CLICK the LEFT Mouse Button using the win32 library:
win32api.mouse_event(win32con.MOUSEEVENTF_LEFTDOWN,int(x),int(y),0,0)
win32api.mouse_event(win32con.MOUSEEVENTF_LEFTUP,int(x),int(y),0,0)
Now if we're in control of the cursor, we move the cursor to the x,y position we calculated earlier from the Tilt:
if enCursor: win32api.SetCursorPos((int(x),int(y)))
=================
Copy the code in the next step into a New Window in IDLE (Python IDE) and save it as mouse.py
Step 7: Python Code
import win32api, win32con
import serial
# Initial Cursor Position
cx = 0
xMax = 1900
cy = 0
yMax = 1000
Sensitivity = 0.1 # Higher numbers make the mouse movement less sensitive to Tilt
# Find out which COM port the Arduino is on
for com in range(15):
try:
ser = serial.Serial(com, 9600) # Open the Serial Port
print("Found on COM",com+1) # If you got here the COM port was opened OK
break # We found it so stop searching
except :
print("Nothing on COM",com+1)
ser.flushInput() # Discard Buffer Contents
enCursor = 0 # Start with the Cursor Disbled
swState = 0 # The State of the Switch
swStateD1 = 0 # The State of the Switch the Last Time
def swPressed():
return swStateD1 - swState # 0,1,-1 : Same, Just Pressed, Just Released
while 1:
# Read a line and separate the newline characters from the rest
rx = ser.readline().decode().split('\r\n')
# print(rx) # Debug
# Now separate the data from the commas
data = rx[0].split(',')
xTilt = int(data[0])
yTilt = int(data[1])
zTilt = int(data[2])
swStateD1 = swState # Remember the old swState before we update it
swState = int(data[3])
x = cx + xTilt/Sensitivity # x-Cursor = centre value (when button was pressed) + Tilt value
x = max(min(xMax, x), 0) # Limit to the Screen co-ordinates
y = cy - yTilt/Sensitivity # y-Cursor = centre value (when button was pressed) + Tilt value
y = max(min(yMax, y), 0) # Limit to the Screen co-ordinates
sw = swPressed() # Check the Button
# Disable the cursor if it's Enabled now and the switch is pressed
if (enCursor and sw==1):
enCursor = 0
win32api.keybd_event(0x20, 0,0,0) # PRESS the SPACE Key to Pause Flight Simulator
win32api.keybd_event(0x20,0 ,win32con.KEYEVENTF_KEYUP ,0)
break
# Enable the cursor if it's Disabled now and the switch is pressed
elif (~enCursor and sw==1):
enCursor = 1
# Click the LEFT-Mouse Button
(cx,cy) = win32api.GetCursorPos()
win32api.mouse_event(win32con.MOUSEEVENTF_LEFTDOWN,int(x),int(y),0,0)
win32api.mouse_event(win32con.MOUSEEVENTF_LEFTUP,int(x),int(y),0,0)
if enCursor: win32api.SetCursorPos((int(x),int(y)))
# We're Done. Close the Serial Port
ser.close()
Step 8: Ready for Takeoff !!
1. Plug in your Arduino and upload your Arduino code (if you haven't done that already).
2. Start Python IDLE and open mouse.py in theIDLE editor
3. Check that the COM variable in the Python code is set to the COM port used by the Arduino IDE
4. Press F5 to Start the Python code running
5. Open Google Earth
6. Start the Flight Simulator mode (Tools-> Enter Flight Simulator) or (Ctrl+Alt+A)
7. Best to Select the SR22 Aircraft to start with. It's easier to control.
8. Centre your mouse in the Google Earth Screen
9. Press the Button on the Arduino to Take Control of the Mouse
10. Tilt your Arduino to Control the Plane. Backwards to Climb, Forwards to Descend etc...
KeyBoard Controls:
PgUp : Speed ++
PgDn : Speed--
Space: Pause
More details in Google Earth Help
11. Press the Button on the Arduino again to Relinquish control of the Mouse, Pause Flight Simulator Mode and Terminate the Python Program.
Have fun!!
Here's a Video of me flying...
Step 9: Next Steps
Why not use the ideas from this Instructable to do something else?
* Use a "pattern" of motion on the Accelerometer to start a Program on your PC
- Different "swipe patterns" could work like Hot Keys to perform a function e.g. CUT, PASTE etc..
* Use Python and matplotlib to graph the movement of the Accelerometer and display it on the screen
* Use "SensorMonkey" to plot the Accelerometer movement on the Internet
https://sensormonkey.eeng.nuim.ie/
