Introduction: Build Your Own Self-Driving Car - (This Instructable Is Work in Process)

About: Have enjoyed a mostly fun ride in an electronics and software career. Also enjoy latin dancing.


If you take a look at my other Instructable on Drive Robot With Remote USB Gamepad, this project is similar, but on a smaller scale. You can also follow or get some help or inspiration from the Robotics, Home-Grown Voice-Recognition, or Self-Driving Car playlists on Youtube.

I began with the large robot (Wallace 4), but since I started a local Meetup group, I needed something on a smaller scale, and the group was very interested in computer vision.

So I came across this Udemy course: Build Your Own Self-Driving Car that gave me the idea for this project.

If you are interested in the Udemy course, you can keep checking back there; it goes on sale for a huge discount from time to time. Note: there is Part 1 and Part 2 - you need to do some investigation on how to get the two courses as package (discounted).

The purpose of this instructable is two-fold. First, to give some pointers and alternatives to certain portions of the course (such as the parts and hardware). And second, to expand on the course.

The main purpose of the Udemy course:

is to be able to get a small wheeled robot car to self-drive on a scaled-down two-lane road.

It has to recognize the lane stripes, and when it has reached the end of the road.

It has to recognize a stop sign (and stop).

Also, a RED and GREEN traffic signal.

It must also recognize and maneuver around an obstacle (another car).

What this Instructable adds to the course:

Drive the little car with a remote USB Gamepad, much the same way as in this other Instructable.

Give some alternatives to what the course provides.

You may not even have to buy the course:

This Instructable may be all you need to get started.


The essential (suggested) parts:

A robot chassis

Four motors


Raspberry Pi (3, 3B+, 4)

Camera (USB Webcam, or Picamera module)

Battery power

On/Off switches

jumper wires

stand-offs (plastic and maybe metal, too)

Please review the entire Instructable and also the videos before attempting to purchase parts.

After having done this project, I realize that the exact parts are not that critical.

Step 1: More Details About the Parts...

The associated video goes into some detail about the parts, and some issues that I found.

  • Look around for different chassis / motors
  • The motors should already have wires soldered to them
  • You may want to have a drill and drill bits, OR a chassis with more holes
  • Keep in mind that weight is an issue. Everything should be as light as possible.
  • The L298 H-Bridge motor driver works great. NOTE: get one with the screw terminal blocks(see photo)
  • You will probably want both plastic and metal standoffs, size M3 is probably the best choice.

Plastic standoffs are good for mounting the boards onto the chassis (motor driver, Arduino, Raspberry, power pcb, on/off switch, etc).

Metal standoffs are good for assembling the chassis (strength), and also especially when you are developing (programming, testing). For development, the metal standoffs can serve as stilts. Just like if you were working on a real car, you want to raise car so that the wheels are in the air and can move freely. This is very important! You will make mistakes and you don't want the car to just take off and crash.

Drill + drill bits

I really want to emphasize the use of a drill, if you are able to, and using standoffs instead of double-sided sticky tape. You're very likely going to end up removing and re-positioning your boards, etc, several times during this project, and using the tape becomes very messy.

Using a drill makes it very easy to re-position (especially if the chassis is plastic) and it looks more professional.

Step 2: Powering the Car During Development

In my opinion, the fastest, easiest way to get started with this project, is:

  • for software Arduino sketch development, just connect the Arduino to your computer via USB
  • for software Raspberry Pi, you should have a 5V USB power that can supply at least 3 Amps. And it should have an on/off switch. Unless you have a good, powered USB hub connected to your computer, you probably will not be able to power the Raspberry directly from your computer.
  • For when you are ready to test the motors/wheels, the easiest is (see photo) a good power supply. However, those are not inexpensive.

My point with this section is to say that you do not want to be using battery power during development, because that will greatly slow down your progress.

Also, by doing something similar to the above suggestions, you do not have to worry (yet) about exactly how you will power the car. You can delay that decision for later in the project.

Step 3: Powering the Car During Actual Use

If you decide to follow the course (or what I have done) for 5V power to the logic, then be aware that not all 5V USB powerbanks are good for this project.

The main point here is that you need 5V but you need at least 3 Amps! Think of it this way - you want a powerbank that will power a laptop computer (perhaps).

If you live in the U.S.A. I think one of the best ways to do this is to buy from Best Buy. Why? Because of their 14-day money back policy for returns.

I actually had to try three different powerbanks before I found one that would work. The other ones cause the Raspberry Pi to complain about under-voltage.

I had started with the least expensive powerbank, and just kept trying the next model (that cost more), until I found one that worked.

How to power the Arduino

In the Udemy course, the author chose to power the Arduino directly from the powerbank (through a custom pcb that he made) and he used power pins on the GPIO connector of the Arduino.

I, however, chose to just power the Arduino directly from the Raspberry Pi, through the USB cable.

You will have to decide which is better.

How to power the motors/motor driver

In the Udemy course, the author chose to power the motors/driver directly from the 5V powerbank. There are two considerations if you use that approach.

  1. When the motors first begin turning, they draw the most current. This can(will) cause the power voltage to sag(dip) below 5V, and cause the Raspberry to reset.
  2. Using only 5V to power the motors means that you are not providing as much power as you could to the motors, and the car will move slower (more sluggish). I have tested the motors (with that power supply)(see photo) to at least 9V. They work fine at 9V.

Observations about 9V (or more)

If you took a look at all the photos and videos for this Instructable, you noticed that I assembled a custom PCB to create my own 9V power source. I have learned a few things along the way.

Right now I am using several (3) 9V battery cells in parallel, to power the motors. I have used both alkaline and NiMH rechargeable batteries.

Learning Experience #1: It takes a long time (many hours) to properly charge NiMH 9V batteries.

Possible Solution: Invest in a multi-battery NiMH charger. It should be a "smart" charger.

Disadvantage: They're not inexpensive.

Learning Experience #2: 9V batteries are actually made up of several small interior cells. If one of those cells dies, the entire battery is useless. I have NOT had this problem, but I read about it.

Learning Experience #3: Not all 9V batteries are the same voltage. This one is important. Because the higher the voltage, the more speed is possible. Some battery cells (and chargers) are only 8.4V. Some even less. Some are 9.6V.

Learning Experience #4: 9V batteries, espeically the NiMH ones, are light in weight. A good thing. However, most of them only provide mA of output current. That is why I had to place them in parallel. You need a total current capacity of almost 2 Amps, even for brief periods of time.

Learning Experience #5: There exist 9.6V battery packs, used for things like radio-controlled cars. I have not used one yet, but I believe they provide more current than doing the parallel 9V batteries like I did. Also, you can charge the single unit. The packs come in different sizes. And there's a weight consideration. And then, do you use the pack to power the entire car, or just the motors? If for the entire car, you will then need a 5V step-down regulator for the Raspberry Pi.

The L298 H-Bridge does have the ability to output 5V for this purpose, but I'm concerned about how much current it can produce for the Raspberry Pi, and if it will be too much strain on the L298 board.

If you decide to have two separate power sources, then you may have a weight issue (too heavy).

Step 4: Software Programming for Gamepad Driving

I think I covered a lot of this section already in the Robot Driven Via Remote USB Gamepad Instructable, so I will not repeat that here.

The programming/software sections in that other Instructable are just a suggestions. I think one learns more by trial-and-error.

Step 5: Adding a Camera

In the Udemy course, I believe the author uses round wooden dowels and a glue gun to construct a way to elevate the camera.

You will want to elevate the camera so that it looks down onto the two-lane road, so it can more easily recognize the lanes.

Where I live in the U.S.A., the wooden dowels were very inexpensive. You can buy them at either Lowe's or Home Depot. I chose square dowels instead of round dowels.

I also chose to make a more sturdier base for the camera tower, and I made the entire tower removable from the car, so that I can play around with and experiment on what is the best position for it on the car.

Also, I made the tower with the idea in mind that I will start with a USB webcam, but possibly later move on to using the Picamera module.

You may want to invest in a fish-eye type camera.

I bought a very inexpensive hot glue gun, but I wanted to reinforce the base of the tower better, so I pre-drilled some screw-holes and added screws to hold everything together better.

Then I bolted the base onto the chassis of the car.

If later, I want to move things around, I just unbolt the base from the chassis, drill new holes in the new location of the chassis, and re-bolt the tower onto the chassis.

I brought over the "follow-me" Python and Node.js code from the large robot (Wallace Robot 4) as a way to test everything. Please see the photos in this section for the list of youtubes that give a lot more detail about "follow-me".

As I mentioned, it was easier to first mount a USB webcam. Later I can mount the Picamera module.

Step 6: Face Recognition - Determine Position

This part is not the focus of the Udemy course, but it was a fun exercise.

If you do some web searching for "python opencv face recognition", you will find many good examples on how to do that, and they all pretty much follow the same steps.

  1. load the "haar" face file
  2. initialize the camera
  3. start a loop where you grab a frame
  4. convert the color image to gray-scale
  5. feed it to the opencv to have it find face(s)
  6. start an inner loop (for each face found) (in my case, I add code to abort if more than 1 face)

For this purpose here, once we have detected a face, we know the X, Y, W, and H of the imaginary square that outlines the face.

If you want the robot to move forward or backward, you just have to consider W. If W is too big (too close), have robot move back. If W is too small (too far), have robot move forward.

Left/right movement is just a bit more complicated but not crazy. Take a look at the image for this section that details how to determine the left vs right face position.


If you run any of the web OpenCV examples, they all show the actual view of what opencv is "seeing", with the face outlined in a square. If you will observe, that square is not steady(constant), even if you are not moving.

Those changing values would cause the robot to be constantly in motion, forward or back, left or right.

Thus, you will need to have some sort of delta for both forward/back and for left/right.

Let's take left vs right:

Once you have calculated left, and right, then get the difference (delta):

delta = abs(left - right)

You need to take the absolute because you don't know which one will be the larger number.

Then you add some conditional code to only attempt to move if the delta is greater than some minimum.

You would do the same thing for forward vs back.

Step 7: Face Position - Moving Robot

Once you know that you need the robot to move left or right, forward or back, how do you do that?

Since this Instructable is a work in process, at the moment, I just copied over the code from my large robot to use for this project. Please check out my Robotics playlist on youtube where it details all of this.

In brief, I have the code in layers.


Python face-recognition script makes http requests to Node.js server


Node.js server listens for http requests for move directions, converts those into custom serial protocol


Custom serial protocol between Node.js server and Arduino


Arduino sketch that does the actual commands to move the robot

The Udemy course does not do it like above. But since I wanted to make good progress and focus on the actual image-recognition, I re-used my previous code for now.