ESP32 Camera Robot - FPV

The ESP32 Camera module is an inexpensive and powerful PLC. It even includes facial recognition!

Let's build a First Person Viewpoint robot that you drive through an on-board web interface!

This project uses the Geekcreit ESP32 module with OV2640 Camera. It's based on the AIThinker module.

There are many different ESP32 Camera clones out there. Some work, some don't. I'd suggest you use the same module I did so you have a good opportunity to be successful.

The robot works as follows.

The ESP32 broadcasts a web URL to your network that presents the live video stream with some check-boxes for operating some functions of the camera. It also receives key presses sent to the web page from the keyboard that are directional commands for the robot. You may wish to build the USB joystick shield so you can drive the robot with the joystick rather than typing keyboard commands.

When the ESP32 receives key presses, it forwards those bytes to the Arduino Nano which then drives the motors to make the robot move.

This project is of moderate-high difficulty. Please take your time.

Let's get started!

Supplies

• ESP-32 Camera module with OV2640 Camera - I'd recommend the Geekcreit product
• External snap-on antenna for the ESP-32 to maximize signal strength
• Arduino Nano
• Arduino Leonardo for the Joystick module (we need USB keyboard emulation provided by the Leonardo)
• Generic Joystick module
• L293D Quad H-bridge chip
• DC-DC Buck Coverter with 5V output to power the ESP32
• FTDI Serial Adapter for programming the ESP32
• A generic robot chassis with two geared motors - any chassis will work. 3 to 6V motors are recommended
• 2 x 7.4V 1300mAh LiPo batteries (or similar) to power the ESP32 and the motors
• 1 x 9V battery to power the Arduino Nano

Step 1: Program the ESP32 Camera

Using a breadboard, connect your ESP32 Camera to the FTDI adapter as follows:

FTDI ESP32

3.3V-------------> 3.3V

GND------------->GND

TX -------------> U0R

Rx -------------> U0T

Additionally, connect pin IO0 ("eye-oh-zero") to GND. You need to do this to put the ESP32 into programming mode.

Unzip the esp32CameraWebRobotforInstructable.zip file.

There are 4 files in this project:

esp32CameraWebRobotforInstructable.ino is the Arduino sketch.

ap_httpd.cpp is the code that manages the web server and deals with setting the camera features from the web page and receiving key presses from the web page.

camera_index.h contains the HTML/JavaScript code for the web application as byte arrays. Modifying the web application is well beyond the scope of this project. I'll include a link for how to modify the HTML/JavaScript later.

camera_pins.h is the header file pertaining to the pin configuration of the ESP32 camera.

To put the ESP32 into Programming Mode, you must connect IO0 ("eye-oh-zero") to Ground.

Launch your Arduino IDE and go to Tools/Boards/Boards Manager. Search for esp32 and install the esp32 library.

Open the project in your Arduino IDE.

Put YOUR router's network ID and YOUR password into the lines highlighted in the above picture. Save the project.

Go to the Tools menu and make the selections as shown in the picture above.

Board: ESP32 Wrover

Upload Speed: 115200

Partition Scheme: "Huge APP (3MB no OTA)"

and choose the port that your FTDI adapter is connected to.

Click the "Upload" button.

Now, sometimes, the ESP32 won't start uploading. So be ready to press the RESET button on the back of the ESP32 when you start to see the ...---... characters appear in the console during upload. It will then start uploading.

When you see 'press RST' on the console, the upload is complete.

DISCONNECT IO0 from Ground. Disconnect the 3.3V line between the FTDI adapter and the ESP32.

The ESP32 Camera requires a lot of current to work well. Connect a 5V 2A power adapter to the 5V and GND pins on the ESP32.

Open the Serial Monitor, set the baud rate to 115200 and then watch as the ESP32 reboots. Eventually, you'll see the URL for the server.

Go to your Browser and enter the URL. When the web site loads, click the 'Start Streaming' button and the live video stream should begin. If you click on the 'Floodlight' checkbox, the on-board flash LED should light up. Watch out! IT'S BRIGHT!

Step 2: Build the Robot

You need a two wheeled robot chassis. Any one will do. Assemble the chassis as per the manufacturer's instructions.

Then wire up the robot as per the diagram.Leave the battery connections for now.

The L293D is used to control the motors. Notice that the half-notch on the chip is TOWARDS the ESP32.

Typically, 6 pins are needed on the Arduino to control two motors.

This robot only requires 4 pins and still works fully.

Pins 1 and 9 are connected to the 5V source of the Arduino so they are permanently HIGH. Wiring the robot this way means we need two fewer pins on the Arduino to control the motors.

In forward directions, the INPUT pins are set to LOW and the motor Pulse Wave Modulation pins are set to values between 0 and 255 with 0 meaning OFF and 255 meaning maximum speed.

In reverse directions, the INPUT pins are set to HIGH and the PWM values are reversed. 0 means maximum speed and 255 means off.

Unzip and upload the ArduinoMotorControl sketch to the Arduino Nano.

Step 3: HEY! Wait a Second! Why Do I Need an Arduino Nano?

You are probably thinking, "Hey! There are at least 4 IO pins available on the ESP32 camera. Why can't I use them to control the motors?"

Well, it's true, there are pins on the ESP32 as follows:

IO0 - needed for putting the ESP32 in programming mode

IO2 - available

IO4 - the Flash LED

IO12, IO13,IO14,IO15,IO16 - additional GPIO pins.

If you just load a basic sketch onto the ESP32 to control the pins with PWM commands, they do work.

HOWEVER, once you activate the CAMERA libraries in your sketches, these pins are no longer available.

So the easiest thing to do is to just use a Nano to control the motors via PWM and send the commands from the ESP32 by serial communications over one wire (ESP32 U0T to Arduino Rx0) and GND. Very simple.

Step 4: Wire Up the USB Joystick (Optional)

You can drive the robot by sending key presses to the web page as follows:

8 - Forward

9 - Forward Right

7 - Forward Left

4 - Rotate Left

5 - Stop

1 - Reverse Left

2 - Reverse

3 - Reverse Right.

The USB joystick sketch translates joystick inputs into key presses and sends them to the web interface which forwards them to the Arduino to drive the robot.

Connect the joystick to the Arduino LEONARDO as follows:

Leonardo Joystick

5V ------------> VCC

GND ------------> GND

A0 ------------> VRx

A1 ------------> VRy

Open the usbJoyStick sketch, select Arduino Leonardo as the board and upload it to the Leonardo.

If you want to test it, just open a text editor on your computer, click the mouse in the window and start moving the joystick. You should see the values from 1 to 9 showing in the window

Step 5: LET'S RIDE!

Take some time and go over your wiring to make sure all's correct.

Next, connect your batteries as follows.

1. Power up the ESP32 Camera. It needs a few seconds to start the web server.

2. Power up the Arduino Nano.

3. Power up the motors.

Launch your browser and go to the URL for the ESP32.

Click the Start Streaming button.

Click your mouse somewhere in the browser screen so that the screen is now the focus.

Start driving your robot with the joystick (or keyboard).

I've found that the default frame size works okay for broadcasting the live video fairly responsively over WiFi. However as you increase the framesize, the stream will become more choppy because you're trying to broadcast bigger images.

This is a challenging project that gives you the opportunity to start working with live video streaming and driving a robot over WiFi. I hope you found it fun!

NOW GO AND MAKE SOMETHING WONDERFUL!

January 2020 Update - The last photos show the final version of the robot, hard-soldered and securely mounted to the chassis.

The three front-mounted switches are as follows:

Left - Motor power battery

Centre - Arduino battery

Right - ESP32 Camera Battery

I could use one big battery with some buck-boost transformers (I use one for the ESP32 - it's in the lower right of the front-view photo), but for the sake of simplicity, I am just keeping the 3 batteries.

Robot now on Access Point!

I find it cumbersome to demonstrate this robot outside my home because my school enterprise network does not allow me to connect the robot web server to it. As a solution, I did research on using the Access Point feature of the ESP32 web server. It takes some work, but requires fairly minimal changes to the main robot sketch to make the ESP32 broadcast it's own IP address. It's not as powerful as a dedicated high speed wifi hub (sometimes hangs if you move too fast), but it works quite well and now I can demonstrate the robot anywhere I want without having to connect it to a network! Once you get the robot working, try converting it to Access Point yourself!

Step 6: Details of How to Modify the HTML/Javascript Code for the Webserver

This is not necessary, but I've had some requests.

I've provided this Google Doc with details on how to use CyberChef to convert back and forth between HTML/Javascript and the byte array representations in the camera_index.h file.