I choose this project because I heard news about people who passed away because we missed the golden time and other places that are just too big for a human to perform search and rescue. I wanted to do something about it. So I started brainstorming and came up with "If I can combine machine learning with flying wing it will be able to travel a large area in short amount of time while being able to search for any people who might be injured or missed by first-aid workers."
So I made a flying wing using styrofoam since it is light and cheap. I installed raspberry pi on board with a camera that runs on OPEN CV and Tensor flow. After finishing the build, I got my friends to hide in the local forest and other areas. I sent the airplane from my backyard and able to locate every single one of them. My result shows that the flying wing was able to detect accuracy up to 99% by scoring 148 out of 150 when locating people in Forest, Large open Space and Plain field(Each 50, three at a time). I want to help the first-aid workers to locate people who are injured faster than our previous method which is using a massive number of people to search a vast land. By using the Flying Wing that I built, it will reduce the amount of total time it takes to locate people who are injured.
Teachers! Did you use this instructable in your classroom?
Add a Teacher Note to share how you incorporated it into your lesson.
Step 1: Question / Proposal
How can we get to people who are injured as fast as possible?
It is the golden time that matters the most when saving a person's life. If a person is located in a place where first-aid workers can not get to in time or hard time trying to find them all of the first 10 minutes is going to pass away. I made a 'Machine Learning Search and Rescue Flying Wing' to help the first aid workers to locate people as fast as possible. For example, someone could be hurt in the national forest or park and the area of it is very huge. First-aid workers are not going to be able to locate them in 10 minutes. So, instead of sending drones which drains the battery fast and highly expensive, why don't we send Flying Wing which is fast, efficient and cost about a quarter of how much a search and rescue drone would cost.I think that my UAV will able to locate faster than drone while can be on the air longer than any other drone with the same specifications.I am going to run many different types of test including Nearby rivers, Forest and Trails and see how many of my friends, I can locate in less than 10 minutes. Technology advanced so much that now we use drones to locate people who are endangered in a place where first-aid workers cannot get to. Why don't we improve on that by using my idea?
Step 2: Research
Before we start, we need some basic research on our topic, so here is what I came up with!
https://www.ted.com/talks/raffaello_d_andrea_the_a... I first got inspired by the idea of UAV search and Rescue drone five years ago when Raffaello D'Andrea'Autonomous systems pioneer.' have developed a drone that cooperates and reacts to its surrounding environment. I thought that if we use the same idea to save people we can make a world such a better place! Now I realize that my project could improve on all those things. While mine does not react to its environment(since it is going to be up high in the air) but looks for people by itself while flying about 30 to 45 minutes(Depending on the wind against it).I could improve their great foundings that could be used by real first-aid workers and people to look for injured people if a natural disaster occurred. These researchers have already been conducted, but no one seems to be improving them what so ever. So, I decided that maybe, I can improve on it. Instead of making a drone which uses four motors I will make a flying wing which only uses two engines while creating enough force to lift itself and package that could go it(Such as the first-aid kit). It is going to fly for longer since it's a glider with a motor so it can glide through the sky while being aerodynamic as possible since it is flat so there is less wind pushing towards it. My primary goal is to use this flying wing to locate people who are lost in a big forest or to send the first-aid kit if first-aid workers cannot get there in time. It is going to be hard for EMS to send 100 helicopters into one single area, so instead, we can send 100 of these flying wing and still get the similar effects while only spending about 1% of how much an EMS helicopter cost. This flying wing will be able to save money and time for the government and can be used anywhere on the Earth, and any first-aid workers can fly this flying wing without any training. I think that the world has a lot to benefit from this project and that more lives will be saved by this flying wing.
Step 3: Basic Designing of the Flyingwing
Before making the actual Flyingwing, we need to see which type of air wing will fly the fastest. In order to choose the right one, I tested 97 different NACA airfoil design on a computer software called ‘xflr5’.The software helped me to understand the aerodynamics, how weight affects the speed and the airflow of the wing. In order to get the maximum speed with the least power, I had to dig through every possible factor that could affect the wing and the flying wing. After many trials, I decided to go with NACA 2408 (naca2408-il) Since it is one of the most used types for flying wing and airflow really smooth when the flying wing is in the air which means it uses less power.
After choosing the wing I decided to make a 3D diagram of the Flying wing. This is a rough 3D model I built using GrabCAD. Instead of having two motors, it would have one electric motor. I wanted the flying wing light as possible while being able to build it for the cheapest price. I decided to build it using styrofoam since Styrofoam weighs 0.05 gram per cubic centimeter. Carbon fiber might be strong but Carbon, solid weighs 2.266 gram per cubic centimeter. I wanted the flying wing light as possible so it flies as fast as possible.
Before we start making the Database and doing all sorts of crazy coding, we first have to how Tensorflow and OPEN CV will work on the Flying Wing. In order to do so, we must install all the necessary software on to the drone.
First, the Raspberry Pi needs to be fully updated. Open a terminal and issue:
sudo apt-get update sudo apt-get dist-upgrade
Next, we’ll install TensorFlow. In the /home/pi directory, create a folder called ‘tf’, which will be used to hold all the installation files for TensorFlow and Protobuf, and cd into it:
mkdir tf cd tf
A pre-built, Rapsberry Pi-compatible wheel file for installing the latest version of TensorFlow is available in the “TensorFlow for ARM” GitHub repository. GitHub user lhelontra updates the repository with pre-compiled installation packages each time a new TensorFlow is released.
Now that we’ve got the file, install TensorFlow by issuing:
sudo pip3 install /home/pi/tf/tensorflow-1.8.0-cp35-none-linux_armv7l.whl
TensorFlow also needs the LibAtlas package. Install it by issuing (if this command doesn't work, issue "sudo apt-get update" and then try again):
sudo apt-get install libatlas-base-dev
While we’re at it, let’s install other dependencies that will be used by the TensorFlow Object Detection API. These are listed on the installation instructions in TensorFlow’s Object Detection GitHub repository. Issue:
sudo pip3 install pillow lxml jupyter matplotlib cython sudo apt-get install python-tk
TensorFlow’s object detection examples typically use matplotlib to display images, but I prefer to use OpenCV because it’s easier to work with and less error prone. The object detection scripts in this guide’s GitHub repository use OpenCV. So, we need to install OpenCV. To get OpenCV working on the Raspberry Pi, there’s quite a few dependencies that need to be installed through apt-get. If any of the following commands don’t work, issue “sudo apt-get update” and then try again. Issue:
sudo apt-get install libjpeg-dev libtiff5-dev libjasper-dev libpng12-dev
sudo apt-get install libavcodec-dev libavformat-dev libswscale-dev libv4l-dev
install libxvidcore-dev libx264-dev
sudo apt-get install qt4-dev-tools
Now that we’ve got all those installed, we can install OpenCV. Issue:
pip3 install opencv-python
And now, OpenCV is installed on to our raspberry pi!!!
Step 5: Coding-part2
So now, we must install protobuf so that camera can understand what it's seeing.
First, get the packages needed to compile Protobuf from source. Issue:
sudo apt-get install autoconf automake libtool curl
Then download the protobuf release from its GitHub repository by issuing:
Configure the build by issuing the following command (it takes about 2 minutes):
Build the package by issuing:
The build process took 61 minutes on my Raspberry Pi. When it’s finished, issue:
This process takes even longer, clocking in at 107 minutes on my Pi. According to other guides I’ve seen, this command may exit out with errors, but Protobuf will still work. If you see errors, you can ignore them for now. Now that it’s built, install it by issuing:
sudo make install
Then move into the python directory and export the library path:
cd python export LD_LIBRARY_PATH=../src/.libs
python3 setup.py build --cpp_implementation python3 setup.py test --cpp_implementation sudo python3 setup.py install --cpp_implementation
Then issue the following path commands:
export PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=cpp export PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION_VERSION=3
That’s it! Now Protobuf is installed on the Pi. Verify it’s installed correctly by issuing the command below and making sure it puts out the default help text.
Now that we’ve installed all the packages, we need to set up the TensorFlow directory. Move back to your home directory, then make a directory called “tensorflow1”, and cd into it.
mkdir tensorflow1 cd tensorflow1
Download the tensorflow repository from GitHub by issuing:
git clone --recurse-submodules https://github.com/tensorflow/models.git
Next, we need to modify the PYTHONPATH environment variable to point at some directories inside the TensorFlow repository we just downloaded. We want PYTHONPATH to be set every time we open a terminal, so we have to modify the .bashrc file. Open it by issuing:
sudo nano ~/.bashrc
Move to the end of the file, and on the last line, add:
Now, we need to use Protoc to compile the Protocol Buffer (.proto) files used by the Object Detection API. The .proto files are located in /research/object_detection/protos, but we need to execute the command from the /research directory. Issue:
cd /home/pi/tensorflow1/models/research protoc object_detection/protos/*.proto --python_out=.
This command converts all the "name".proto files to "name_pb2".py files. Next, move into the object_detection directory:
I really suggest to watch videos from youtube. Espically "https://www.youtube.com/watch?v=npZ-8Nj1YwY&t=1s" for better understanding :)
Step 6: Coding Improvements
Alright, this is the part where people skips cause it's too difficult to even explain what each of the codes does. Everyone else's codes are different and that is why I am not going to explain too much here. You still can use the Machine Learning raspberry pi from the last 2 steps but this one allows you to modify what your camera does. Steps are not really difficult, you just have to upload these codes and change the file path of the machine learning or raspberry pi into the directories you saved the codes into.
Here is the Python file: https://docs.google.com/document/d/1hmIHigzf2_1ZZ...
Here is the Java file: https://docs.google.com/document/d/1hmIHigzf2_1ZZ...
and the C++ File: https://docs.google.com/document/d/1hmIHigzf2_1ZZ...
Skip this step if you are not familiar with coding :(
Step 7: Making the Flying Wing!(Home Made)
Let's get started!!!
My design uses the flitetest spear(More in the next step) as basic fundamental but my main wing design is different so it can go faster and uses less battery. So here we go.
- I cut the Styrofoam box following the my prototype design using cutting professional cutting knife.
- I first made a base which is 1.041 meter long(Wingspan).
- I then made the top base which is 52.1cm in length to cover the top.
- The skeleton body of flying wing was attached together using Polyurethane tape.
- I made a rectangular empty space in the front to put the raspberry pi and its camera for main detection set up.
- When all the built was completed this was the skeleton form of the flying wing.Two servo motors are placed but had not been connected yet.This one had a case but I choose to remove it in order to get better view of inside and give more space for computer storage.
- Total weight is grams without any battery or anything placed in but the 8 grams servo motor was 687 grams.
- Then I worked on wiring connecting flight control deck to 30A ESC and connecting ESC to a battery, Main motor. I then bind the flight deck to a controller
Sorry, I do not have a full video or time-lapse video of me making it but it is very simple!
Step 8: Making the Flying Wing!(Store Made)
I guess this is the time where we get to do something fun!!!
Here is what I suggest:
and here is the parts link to it;
Step 9: Putting Things Together!
When you are all done making the Flying-Wing and the machine learning camera, you can put it together in any way you would like!
The picture shows how I put it. I added more stuff such as GPS and other cool things but that is only an OPTION and brings up the cost by 150 dollars which is a bit out of our goal of keeping things cheap!
Step 10: Testing!
Objective: To locate humans in 100 different locations with an accuracy of minimum 75% and a maximum time of 1 hour.
Environment variables were not controlled to simulate a real-life situation. Human variables were not controlled to simulate a real-life situation. Method:
- The data was hidden in a given area without any location information.
- If one is located, it is to move on to a different spot within 50 seconds for time sake.
- If 100 times(Counting every time when a person switches to a different position) were reached the experiment was over.
- The data were collected to evaluate the accuracy and the time it took to locate them.The scores(Results) were calculated and turned into script text from code.
- All scores(Results) were calculated to find the average of accuracy and the average time it took to locate 1 person.
The results of this experiment were unbelievable. As you can see, out of 100 different possible location of victims the Flying Wing successfully located 83% of them. Rest 16% were reachable, the machine was not too sure if it was human or not. It only missed 1 out of 100. This also means 99% Detection rate with 93% of high-degree accuracy. This test also reaches the main objective, Flying Wing had a higher detection rate than 75% within 1 hour of given time.
This uncontrolled test backs up my main thesis. This proves that Flying wing is indeed faster at locating possible victims than what normal human would have done.
Step 11: Applications!
In the future, the 'Machine Learning Search And Rescue Flying Wing' would include new hardwares and software. These includes:
- Better camera to get higher Frames Per Second (FPS)
- Bigger database for faster and more accurate comparison
- Better software algorithm(Ruby), instead of box, fitting curved lines to perfectly match the outline
- Additional Thermal Camera to detect human by heat trackingI would also like to work with the International Search and Rescue Advisory Group so more lives can be saved around the world using my invention.
I would also like to work with MIT to improve my invention so it could go above and beyond of whats expected of a First-Aid UAV.
Also, Search and Rescue is not the only option. As I mentioned before, the machine learning technology has no limits. It is already part of our daily lives and has been benefiting us everywhere.The application for my UAV is limitless as well.It could be used as:
- Search And Rescue Operations(The Whole Project)
- Detecting Lost Hikers(Part Of Project)
- Finding Injured Army(Future Application)
- Use it to track animals(Endangered Species)
- Protection And Security Purposes(Border Security And Unknown Vehicle Tracking)
- Land-Protection Services(National Park)
Combining Machine Learning Algorithm wit Flying Wing is indeed most effective way to Locate and Save peoples life than any other UAV out there. The Flying Wing archived the goal of locating possible victims within the given time. By combining Machine Learning technology the Flying Wing was able to successfully complete the task of 'Finding Human' while simulating a real-life situation. Multiple computer simulations, Real-life photo comparison, and Online data training show the Machine Learning Camera was able to tell the difference between human or not from many different given angles and speed of Flying Wing moving.
Step 12: Conclusion
I used Google’s Tensorflow software and created my own DataBase and coded raspberry pi to look for human beings. I made raspberry pi to take a picture every second and made to to go through a 3 database ,Human outline tracking(Where the computer look at the image and make an outline of the human),Box Outline tracking(Where the computer look at the image and surround the human with a fitting box) and Image Comparison Tracking(Where the computer make a comparison between the photo that was taken and the give images of human beings).
All of this technology will go on a flying wing where it would fly over an area that needs help(Natural disaster) to locate people who are isolated or who are not yet found and injured. It will send the current camera view to the controller while looking for humans so if the camera misses it we have a second look by a human being. My main goal is to help the first-aid workers and rescuers to locate more humans so they can get to them within the ‘Golden Time’ and since being able to locate humans is the most important, I made the plane light and fast as possible by using computer software to analysis my wing design for the plane.
Step 13: About Me!
I love making things even it cost everything that I own.
I enjoy flying drones and doing science experiments I got into STEM when I got my first LEGO set when I was a kid. It was just amazing to me that you can make something that is based on your thinking and your creativity and from there on I got interested in Science and Math even more which changed the perspective on how I look at my future. I admire Nikola Tesla most. He is the one who changed the world and who shifted the perspective on how we look at the world today. He came up with ideas that most people afraid to try and he never gave up in his hard time. My dream career future is getting 45/45 in IB and then going into computer engineering in MIT. I want to graduate and then work for Google in software engineer. I want to show my ideas that I worked on as a kid to other engineers when I get in. The prize does not matter to me; what matters the most is that I am in the contest and that I have participated and tried my best. I want the experience, not money. You can never trade money with an experience that will be much valuable as this!. By entering this contest, it will better shape my future, and it will be the most valuable memory to me.
Participated in the
Make It Fly Challenge