Introduction: Autonomous Drone Using RPi
Over the last decade, the commercial UAV industry has grown exponentially. Nowadays UAVs are being used for mapping, infrastructure analysis, navigation, food and package delivery, photography and film making, pest spraying and much more.
Also during this decade we have seen Alexa and Siri taking over our daily lives, providing automation to our day to day routine. Like dimming out the lights, playing songs, calling mom and even reading out recipes while we cook.
Similarly if we can fuse this concept of automation with UAV, wouldn’t that be awesome?
So here it is, an instructable that focus on how you can make a drone autonomous (level 3). This instructable is only meant to get you started with the concept. Consider this the “hello world” example of autonomous drones.
In this tutorial, all we’ll be doing is integrating a LiDAR with our drone so the drone gets a capability to detect an obstruction in front of it and plan an action accordingly. In this tutorial we'll land once we see an obstacle in front of our vehicle. You might wanna go beyond that like increasing altitude to go over the obstruction, scan area using Computer Vision algorithm to find another way out.
Once you get the hang of how a drone can be controlled using external sensors, you can just go on an implement your own codes like following red color car, reading ARUCO markers for inventory management in a warehouse, using thermal camera to assist firefighters regarding the best pathway inside the burning building….
So go on and conquer the sky, cause sky isn’t the limit.
The method described in this instructables can be implemented on planes, VTOL, Hexacopter, Octacopter, Quadcopter, helicopter, and basically anything being controlled with Pixhawk flight controller. The reason for using quad in this case is because its one of the easiest one to get started with.
In order to get started with you’ll need an ArduPilot supported flight controller (APM/Pixhawk/Pixhawk 2.1). The reason for this is because the code/platform is open-source and doesn’t need developer license. This allows average person (like me) to grow, understand, explore and educate others regarding the new horizon.
Step 1: Part Selection of Quadcopter - Concept
So first thing you need to do is get yourself a ArduPilot UAV. If you already have one of them then you can skip forward. Just make sure it has enough room to carry Raspberry Pi (Zero/ Zero-Wireless/Model 3 B/ Model 3 B+).
So you don’t have a vehicle to work on. That’s okay. Literally just Google DIY Pixhawk quadcopter and you’ll find a lot of build guides and configuration and you’ll even find whole kit available on Amazon, Ebay and AliExpress.
For those of you who are still curious on how to select parts, just draft up a chart and start writing down the weight of the objects that you'll be using (frame, ESC, motors, propellers, battery, Flight controller, receiver). Using the weight list verify (using motor thrust table) that your combination of motor and propeller would be able to lift of the total weight at 50% throttle. If yes, then you're heading in the right direction.
Anyway, I will not be covering on how to select parts for your quadcopter. Finding a perfect combination can be tiring. It is an iterative process. To be honest with you guys, I have even spent almost 2 months on a hexacopter configuration, even after having some experience with selection of items.
Step 2: Part Selection - My Approach
If you need some place to start then what I went through while selecting the parts might help you.
ecalc.ch, an online calculator which helps you select motor-prop combination, lets you estimate battery time and range, and much more.
Ecalc helps you to have an idea on how to select parts by playing around with different configurations. Though in real world, ecalc data generally deviates by 15%. As long as you are a beginner, this would work fine, by if you're thinking of making a drone for your company to do a geological survey, then you might wanna think again.
Anyway, this is what I did for my built:
Frame - GF 400 (alternate: F450 or any other frame that weights around 300g and can support 10" prop)
Motors - DJI 2212 920KV (alternate: Any BLDC 2212 motor having KV rating from 920-1100)
Electronic Speed Controller (ESC) - EMax BLHeli 30A (alternate: SimonK ESC having >25A limit - with BEC so you can power up your quad)
Battery - Gens Ace 3s (11.1V) 5000mAh 45C LiPo (alternate: capacity can vary from 3000-6000 at ~40C, remember bigger battery increases the weight - effecting the flight time (battery capacity is not directly proportional to flight time!!!) You have to find the sweet spot.
Flight Controller - Pixhawk 2.1 Cube with Here GNSS (alternate: Pixhawk 1 with M8N GPS)
Transmitter and receiver (for manual flight, initial calibration and taking manual control if anything goes wrong) - Taranis FrSky X9D+ with Taranis X8R (alternate: Flysky FS-i6 with FS-iA6B)
Putting together the part is like adult version of LEGO blocks. Its easy and fun (as long as you don't mess up the polarity of the ESC)
Step 3: Configuring Quad - Making Ready-to-Fly
So now you have assembled a quadcopter with the right wiring.
Cool, now lets begin configuring initial parameters so it can fly. For that download and install Mission Planner on you PC.
Mission Planner is open source software that can be used (and is generally used) to configure Pixhawk related items. Mission Planner is available on Windows and Linux. Mac users can use QGround Control instead.
One of the major misconception I hear is that you need to ‘program’ your Pixhawk flight controller. That is not true. All you need to do is follow a bunch of steps to properly calibrate stuff together.
Mission Planner and QGroundControl are very much user friendly and as a matter of fact they literally guide you through the whole setup, with Mission Planner giving a more user configurable platform and QGroundControl providing a cleaner user interface. Still if you need help, there are a lot of tutorials on setup.
My favourite and easiest to follow is Pixhawk series by Painless360.
If you need help then do see this video series, it'll help you a lot.
Step 4: Configuring Raspberry Pi
Okay so now you have a quad that flies using RC, and you want want to make it fly on its own… First thing you need is a Raspberry Pi (with a PiCam for video (optional)). Preferably one with wireless WiFi connectivity.
In my setup I used Raspberry Pi Model 3 B which has built in Wi-Fi. This makes it very easy to SSH into the device. Don’t worry if you have a model which doesn’t support Wi-Fi. Using a Wi-Fi dongle will work too at the expense of a USB port.
The big idea is to transmit and receive real time flight data on your Ground Station (PC or Mobile phone) and also receive and view live video and run python scripts to control your drone without manual RC input.
So assuming you know how to use RPi (visit here for how to setup RPi), download and install APSync image on your RPi (download)
APSync will let your RPi work like a WiFi hotspot. The default WiFi name is 'ardupilot' and password: 'ardupilot'. More information onhow to install here.
Connect your PC to this network. Once connected, open up you web browser and type 10.0.1.128:8000 in address bar. Use this page to video streaming on/off using this webpage (If you have attached RPi camera)
Step 5: SSH Into Raspberry Pi
Once connected with that WiFi acces point, use PuTTy to SSH into your Raspberry Pi. Use IP address: 10.0.1.128 and Port: 22
The default SSH login details are username: apsync and password: apsync
Now you’re in... so you're now in a command terminal. Write 'pip list' to ensure you have dronekit, MAVProxy and pymavlink installed (It is a part of APSync package
Step 6: Connecting RPi and Pixhawk
The connections are quite simple. The Pixhawk 2.1 has multiple UART which we can use to interface with our Pi.
There are a number of ways to connect RPi to Pixhawk:
- Using a custom cable to connect RPi UART (GPIO 14 and GPIO15) to TELEM2 (/dev/ttyAMA0 or /dev/ttyS0)
- Using custom cable to connect TELEM2 to RPi's USB port through a converter (/dev/ttyUSB0)
- Using a micro-USB cable to connect Pixhawk directly to RPi's USB port (/dev/ttyACM0) - Not recommended
To make sure that you're connecting to the right port, use the command ls /dev/tty*
Use the Telemetry cable that comes with the Pixhawk and splice it to add male headers to the other end. Remember Tx on one device goes to Rx at the other end and vice versa.
More info here
Step 7: Running Python Script on RPi
Now that you have everything setup, all you have to do is run the python code you made.
First copy the python script you made on the Pi. Open the terminal and navigate to the directory where you saved the file. Then simply type the following command
python your_file.py --connect /dev/ttyAMA0 (/dev/ttyAMA0 will be replaced with whatever comes at your end)
This command will control your vehicle, but in order to view real time in flight data and fly using python script, open two SSH terminal windows. First type in
python mavproxy.py --master /dev/ttyACM0 --out=udp: 10.0.1.128:14550 --out=udp: 10.0.1.XX:14550 --out=udp: 10.0.1.XX:14550
Here replace XX with the assigned IP of the devices where you want to view it (PC/Phone or even both). Also 10.0.1.128 is the IP of RPi in ardupilot network. If everything went good then you'll be receiving flight data. In order to fly your quad using python, use the other terminal window to run the following command
python your_file.py --connect udp: 10.0.1.128:14550
Now you are ready to fly your drone using a custom script. But first lets make one.
This is a very simple code to initialize your drone and make it hover at a place. So in this hover and land code, all you do is:
- Arm and takeoff
- Wait for some time
- Switch flight mode to land
Customize your code to go into any defined waypoint.
You can also connect a LiDAR to measure the distance and avoid obstacles. Instructions on how to connect it can be found here. Also here is a raw code for measuring distance, if distance is below a threshold then braking and landing.
Add functions and make your own code to suit your needs.
For detailed information visit:
Step 8: (Optional) - Bench Testing of Code
Before you conquer the skies with your python scripts, I highly recommend that you should setup a software simulator.
Setting-up simulator is easy. Below is a video I made to explain the process
Step 9: Credits, Acknowledgement, About Me
I, Jawwad Zahed, have been working as full time researcher at National University of Sciences and Technology (NUST) for a year. Building, flying and testing UAVs is not just my hobby, its my job as a researcher. I have been around autonomous drones for 3-years.
Accompanying me is Sohaib Ahmed Jalali, my friend and also my colleague here at NUST. Back in 2015 we had this dream of making flying robots, and by the end of our degree program we were able to achieve it. During our Final Year Project, we faced a lot of criticism and negative comments from the judging panel, be we didnt quit. Our project, Autonomous drone, got the highest marks and was even nominated for gold medal.
During this 3-year journey, Rashad Reyaz, a junior of mine, helped me a lot. He is the team leader for NUST Airworks. NUST Airworks recently won IMechE UAS Challenge 2019 and were declared Grand Champions this year.
Its surprising that how far we have come....
Participated in the
Make It Fly Challenge
Question 10 months ago on Step 2
can you send me a link to the assembly instructions?
Question 1 year ago
Why is it not recommended to connect the raspi and pixhawk over micro-usb? That's what I have been doing and it seems to be working fine, the only issue is that there is often a delay between when I send a command from the raspi and when the pixhawk actually reacts to it. Will switching to the telemetry port on the pixhawk solve this or is there some other issue I am missing? (I am using the pixhawk cube orange with a raspi 3b+)
1 year ago
I cannot find in your script the Lidar that u're using
Question 1 year ago
I want to use Here GPS to work with raspberry to get location. I don't need flight controller for my work. Do I need to buy Pixhawk controller for connection, or I can work with Here+ GPS and telemetry module? I am new to this field, please suggest.
Answer 1 year ago
No. Just get Here+ documentation and really just use UART to decode the information being sent by the GPS module
1 year ago
What software did u use the flowchart or circuit diagram that is used in the pdf???
Reply 1 year ago
I used draw.io, Fritzing and MS PowerPoint
Question 2 years ago
1 last thing. I've spent a long time researching and buying parts on Amazon, and I'm struggling to find some pieces. Is it possible to send the links to the project above? I think it would be helpful for other users and me. Thanks!
Answer 2 years ago
Try HobbyKing. I buy most of the stuff from there.
2 years ago
Does the battery need to be 11.1V or can it be lower/higher?
Best Answer 2 years ago
Yes you can! Depends on the motor configuration. Motor manufacturers provide thrust data and along with that they provide you with information for that battery that is to be used. In case of 2212/1100KV, You can use 4S LiPo (4-Series (14.8V)) or 3S (3-Series (11.1V)).
In Some FPV drones, it is recommended to use 4S/6S LiPo - But again its the motor manufacturer that tell you the compatible ones
Reply 2 years ago
Can I use this battery?
Reply 2 years ago
for a combination that could house and lift RPi? No. This battery wouldnt be able to provide the required power hence your copter wouldnt take-off.
Reply 2 years ago
Also, LiPo is recommended as it provides the required Power-Density and Energy-Density requirements.
(Images source: https://www.gelest.com/applications/batteries/)
Answer 2 years ago
IT depends on the SBC or the microcontroller used.
Reply 2 years ago
No! SBC and Microcontroller work on 3.3V/5V/6V/12V which is a standard with electronics domain. SBC and microcontroller are low powered items (5V-3A compared to 11.1V-30A motor) and cannot dictate the overall voltage of your system. Usually SBC and microcontroller are already fitted with in-built voltage regulators.
Also if you haven't noticed, RPi and Pixhawk requires 5V/6V while I recommended the use of 3S LiPo battery (11.1V/14.8V).
Usually your ESC (unless its OPTO) has voltage regulator built in. What it does is, it takes input from the battery, and outputs 5V/6V from the servo connector. This output is used to power the SBC and flight controller.
If your SBC doesn't support 5V/6V then you can always buy a suitable BEC (Battery Eliminator Circuit) to match the output requirement.
Motors are the key element that consumes the most power and dictates the required voltage of the system. You need to make sure you match the manufacturer's requirement otherwise your SBC and flight controller would be working fine, but your vehicle would either burn its motor or wouldnt take off
2 years ago
My partner is interested in making an autonomous drone with lidar using raspberry pi. This is not information I have knowledge enough to understand where to begin buying parts to build, however, I would like to purchase parts for this purpose. Could anyone direct me on where to buy these pieces/parts? We already a raspberry pi 3.
Reply 2 years ago
I prefer buying stuff from UAV related products from HobbyKing (https://hobbyking.com).
2 years ago
Very nice and very helpful. Can you explain how the raspberry Pi is powered? Is it getting power via the pixhawk TELEM port? Can that supply enough power for both the RPi and Lidar? Thanks!
Reply 2 years ago
TELEM port has a power limit. I used to power RPi but takeoff and sudden maneuvers would restart RPi. I recommend using a BEC to be plugged in to servo rail and powering you RPi through the rail (BEC). By the way, LiDAR doesnt take much power compared to RPi (almost negligible)