Voice and Gesture Controlled Mecanum Wheel Robot

This project is a development of my earlier project named "Mecanum wheel robot - bluetooth controlled". As we all know well a meacanum wheel robot can move across without any difficulty like no other robot. To make it even more impressive in this instruction we will control the robot with voice and gestures.

Step 1: The List of Parts

The robot can be controlled via gesture and voice (bluetooth control). So that we can control the robot with hand gestures we will need gesture detection attitude sensor APDS-9930. Because this sensor requires 3.3V voltage, we need to use a 3.3 / 5V logic level converter to connect it to Arduino. All these elements were connected on mini prototype breadboard. In addition, the robot was equipped with a mini wifi IP camera.

Parts needed in this project:

1. Mini Wifi Module Camera CCTV IP Wireless Camera

2. APDS-9930 Gesture Detection Attitude Sensor

3. TTL Bi-directional Logic Level Converter 3.3V/5V

4. Mini Solderless Prototype Breadboard 170 Points

5. T Plug Connector For RC ESC Battery

Step 2: Connection of Electronic Parts

The connections of electronic modules are the following:

APDS-9930 -> Arduino Mega 2560

VCC - 3.3V (Power)
GND - GND (Ground)
SDA - A4 (I2C Data)
SCL - A5 (I2C Clock)

The SDA and SCL pins are connected via 3.3/5V logic level converter as shown in the photo.

Step 3: Arduino Mega Code

I've prepared a sample Arduino program that will allow you to control the robot by gestures and voice (available at GitHub): Gesture_and_Voice_controlled_mecanum_wheel_robot. The voice commands are processed by the BT Voice Control for Arduino app from Google Play and further sent via bluetooth to Arduino. The Arduino program checks in the main loop - "void loop()" whether the new command (character) has been sent from Android app via bluetooth. If there is any incoming character from bluetooth serial the program read the serial data until it encounters the end of the voice instruction "#". Then it starts execution of "void processInput() " function and depending on the voice command a specific direction control function is called.

Step 4: Testing of Mobile Robot

This video show tests of mobile robot based on program from the previous section (Arduino Mega Code). The commands controlling the direction of movement of the robot are called by voice. In addition, the speed of the robot can be regulated by the movement of the hand.

If you would like to see my other projects related to robotics please visit: