Intro: Wireless Gimbal Control With Google Glass
This instructable serves as an shortened version of a longer document on the installation and operation of a wireless gimbal control system using Google Glass.
The methods and results are typically in their beta-testing phase and will not necessarily be user friendly. Follow the instructions documented here to implement the project on any 2/3 axis gimbal with a newer BaseCam Electronics Alexmos gimbal controller and Google Glass. I understand Google Glass is discontinued, but it is also possible to use the Android code provided and implement it on another Android device (basic Android programming skills will be required, and is not covered in this instructable).
Take note that a certain amount of self-study will be necessary pertaining to activities that are not directly covered by this manual, but are required to achieve wireless control (such as gimbal stabilisation & PID calibration of the gimbal).
A note for experienced tinkerers and developers:
If you are proficient in Android development (I am an mechatronic engineer, not a software developer so please excuse the messy programming), then you should find it relatively easy to apply the Android code on most other Android devices, other than Glass. However, the UI and other subtleties affect the Android code so it is not simply a matter of copy-paste to apply this project to a tablet or phone.
The Arduino code is also not perfect, but it should be relatively simple to figure out whats going on if you have played with Arduino before. This project is still a little rough around the edges, so bare with me.
Below is a link to my website which documents the progress of this project (my undergraduate engineering thesis) a little bit more thoroughly:
Gimbal Control by Google Glass
Step 1: Hardware Required
The hardware required to implement gimbal control are as follows:
- 2 or 3 axis camera stabilising gimbal frame. A more rigid frame is better
- Brushless gimbal motors of the right size corresponding to the gimbal frame and payload size
- A BaseCam Electronic SimpleBG Alexmos 2 or 3 axis stabilising gimbal controller, preferably with a second IMU for better yaw control
- An Arduino Uno or Mega
- An Arduino Wi-Fi Shield with integrated antenna
- Google Glass
- 3 Cell (11.1V) Li-Po battery
Step 2: Software Packages Required
The following software packages needs to be
downloaded and installed in order to upload the necessary programs:
- Arduino IDE – Arduino IDE Download (for Step 4)
- Android Studio – Android Studio Download (for Step 5)
The following library package needs to be downloaded and placed in the Arduino library directory, which is automatically created when the Arduino IDE is installed:
- BaseCam Electronics SBGC API Library – Library Link to Github
The following code and scripts needs to be downloaded and placed in the respective Android Studio or Arduino project directories (the place where a new project is saved). The two Arduino libraries also need to be paced in the Arduino library folder.
Google Drive link to code
- MyGimbalControl folder needs to go to Arduino project folder directory
- ControlGimbal folder needs to go to Android project directory, and must be imported in Android Studio IDE.
Step 3: Hardware Setup
· The gimbal frame should be as rigid as possible to avoid any delayed fluctuations or movements reaching either of the IMU sensors (if two are used).
· The gimbal should be balanced properly on all axis with the payload attached. Make sure that the gimbal controller (frame IMU) and camera/payload IMU is mounted correctly.
· Tune the gimbal controller PIDs with SimpleBGC GUI. The tuning process can be extremely tedious, time consuming and frustrating. There is sufficient literature and help in online forums focussed on gimbal PID tuning using Alexmos controllers, so refer to those resources to achieved good gimbal stabilisation (which will be required for responsive and vibration free gimbal control with the Google Glass).
· The Arduino microcontroller with the Wi-Fi shield attached needs to be mounted on the gimbal below-yaw (as shown in the picture above) in such a way that it can be connected to the Alexmos gimbal controller via UART.
· The Arduino and its Wi-Fi shield can be powered from the UART’s 5V/Ground lines coming from the gimbal controller. The gimbal controller receives power from the 3 cell Li-Po battery.
· The UART port should be labelled on the gimbal controller (it is also referred to as the “Serial Bluetooth Port”). A single signal line (normal conductive wire) connects the Arduino’s transmitting (Tx) UART pin to the Alexmos board’s receiving (Rx) UART pin. Communication only takes place one-way, since the Arduino sends the gimbal controller commands on how to orientate itself, but does not receive anything in return.
· The Arduino’s UART Tx pin number can be specified, since it is a software serial port (the Wi-Fi shield already uses the hardware serial ports). The pin number is specified in the SBGC Serial API library header file (further discussed in step 4). The default pin number when the Arduino Mega is being used is Pin 49 (this will need to be changed when using the Uno, since it does not have that many pins).
Step 4: Software Setup - Arduino
Once all the software packages are installed, the code can be opened and uploaded using each respective IDE.
Copy the MyGimbalControl.ino file and it's containing folder into your PC’s Arduino project directory (you should have downloaded it from G-Drive in Step 2), and open it using the Arduino IDE. Make sure the “SBGC_Arduino” and “SBGC_lib” library files are both in the library folder in the Arduino directory. Next, make sure the Arduino is connect via USB. If an Arduino Mega is being used, the code can be uploaded as is. (If you get an upload error, check that the correct Arduino Device is selected in “Tools”, as well as the correct port).
If an Arduino Uno is being used, the UART Tx pin needs to be changed to one that is available on the Uno. To change it, open the SBGC_Arduino.h file in the “SBGC_Arduino” Arduino library folder (use notepad or wordpad to open). The very first line of code is “#define SBGCTx 49,” to change the UART Tx pin on the Arduino Uno, change the number to the available pin that connects to the UART Rx pin on the gimbal controller. Upload the code to the Arduino.
Once a program is uploaded on to the Arduino, it will stay there unless erased or changed, even when power is removed.
Step 5: Software Setup - Android Code
Open Android Studio and import the ControlGimbal project folder (you should have downloaded it in Step 2).
- Android Studio looks quite overwhelming for an IDE, but it is quite simple once you get to know it. Fortunately for this application it is only necessary to upload the code on to the Google Glass. Once project folder has been imported, connect Google Glass to your computer via its micro USB port.
- Each Wi-Fi shield has a default IP address, it is necessary to change the IP address of the Arduino Wi-Fi shield in the code in Android Studio. To do this, navigate in the toolbar on the left to the following file: app/java/josuablom.controlgimbal/SensorActivity. Double click on SensorActivity, the first line of code declares the string variable “SERVERIP,” change its value (the IP address in green) to the IP address of the current Arduino Wi-Fi shield being used. Save the code (Ctrl + S). Finding the Arduino Wi-Fi shield’s IP address will be discussed in Step 6, so keep Android Studio open for now and continue.
- Once the Google Glass is connect, click the green play button on the top toolbar, or use the keyboard shortcut (shift + F10). This should upload the application on to Google Glass, wait a moment until it is finished uploading. At this point, disconnect Google Glass from the computer, the Control Gimbal application home screen should be visible on the Google Glass, if not, navigate to the application.
Step 6: Wireless Network Connection Setup
Unfortunately the Arduino Wi-Fi shield has some firmware/hardware limitations which prevent it from being a Wi-Fi server/hotspot itself. In order to connect the Google Glass and the Arduino Wi-Fi shield to the same network, one needs to make use of a cell phone or tablet’s Wi-Fi Hotspot capability. Most modern cell phones have this capability and it is easy to set up.
- Turn on your cell phone/tablet’s Wi-Fi hotspot and set the network credentials to the following:
Network SSID: “ControlGimbal”
Once this is set, your Arduino Mega with the Wi-Fi shield attached should connect to the Wi-Fi hotspot created by the tablet/cell phone when it is powered up. The Wi-Fi shield takes a few second to connect and an orange LED will be on while it is connecting. A solid connection is indicated by a single green light next to “LINK” on the Wi-Fi shield.
- To get the IP address of the Arduino Wi-Fi shield, connect it to your computer via USB and open the code in the Arduino IDE. Make sure a Wi-Fi hotspot is created by a cell phone/tablet with the credentials above. If the code isn’t already uploaded, do so, then click on the serial com port (in the top right hand corner of the Arduino IDE). Wait for the Arduino to connect to the hotspot network, when it is connected it will display all the Wi-Fi shield’s credentials as well as the IP address, this IP address should match the one in the Android Studio code as discussed in step 5.
- To connect the Google Glass to the same Wi-Fi network, go to the following website:
Click on “My Wi-Fi networks,” and enter the credentials as mentioned above. Click “Generate Code.” A barcode will appear on the screen. Next, navigate to Wi-Fi settings on the Google Glass itself, and select “Connect to Network.” Swipe all the way to the right and select “Add Wi-Fi network.” With the Google Glass facing the computer screen, it should read the barcode and connect to the network. If you have previously connected to the network with these credentials, it should be saved as one of the network options on the Glass, and can simply be selected. Glass will say “Connected” if it successfully connected to the Wi-Fi Hotspot created by the cell phone/tablet.
Step 7: Wireless Control Setup
At this point, both devices are connect on the same network and can communicate with each other. To test the communication, simply open the Arduino code in the Arduino IDE and (with the Arduino connected to the PC), click on the serial button in the top right hand corner. Navigate to the Control Gimbal application on Google Glass, and select the “Control Gimbal” option by tapping twice. In the Arduino serial window you should now see the orientation data of the Google Glass (also visible in the Glass’ HUD) being displayed at a rate of 10Hz. This means that the system functions correctly. The only step that’s left is to connect the Alexmos controller’s UART Rx port to the Arduino’s UART Tx port (the port number is as defined in step 1a).
When the Alexmos gimbal controller is powered on, it should power on the Arduino and its Wi-Fi shield as well (assuming that a 5V and Ground wires are connected to the Arduino from the Alexmos). The Arduino should power up with the script (or code) loaded on to it, and initialise the gimbal controller. Once the Arduino, gimbal controller and Wi-Fi shield is ready and connected to a Wi-Fi hotspot, the Google Glass can be used to control the yaw, pitch and roll axes of the gimbal, assuming that all the axes’ PIDs are properly tuned and set up.
Step 8: Wireless Control Characteristics
Owing to the nature of this project and the fact that it is a final year engineering project, much of the components are still in their ‘beta’ phase. This means that the control characteristics of the gimbal with the Google Glass isn’t perfectly smooth. For some reason, wireless UDP packages tend to go missing (this is a fundamental characteristic of UDP communication) more than is generally expected. This results in the gimbal “freezing” for a second or two every now and again, until it starts receiving data again.
The sensor that controls Google Glass’ yaw control is a magnetometer, or compass. It is very sensitive to external magnetic interference. This means that it is not always possible to control the gimbal on its yaw axis by looking left or right (denoted as “yaw control” in the Control Gimbal application). As a contingency, the Control Gimbal application on Glass has another way of controlling yaw by tilting your head to either shoulder, while still looking forward (denoted as “Roll Control”). These two modes can be selected while in the Control Gimbal application and tapping on the Glass’ touch interface. To stop controlling the Gimbal, swipe down. To send the gimbal to its home position, select the “Home Gimbal” option in the Control Gimbal application. The application is very straight forward and intuitive.
Step 9: Feedback and Resources
Any feedback will be appreciated. I will interactively support anyone who is willing to delve into a project like this, if the instructable does not suffice. I also have several resources which helped me achieve success in this project, which I am happy to share upon request. Goodluck!
Thank you for perusing and happy tinkering!