Introduction: Smart Robot With Vision and Voice Control

How to make an intelligent robot with computer vision, voice control, voice synthesis and more, using your Android phone!

Step 1: Parts

Hardware

1. An Android phone that supports OTG with Android KitKat or later installed. Most modern phones support OTG. (I used a UMI Emax here cos it's powerful and cheap!)

2. An Arduino Uno based robot platform. I recommend the MakeBlock Starter Robot Kit (you can use either IR or Bluetooth version).

3. The top part of a dashboard mobile phone holder (I used a BlackFox Digidock CR-3104)

4. An OTG cable (easily found on ebay or Amazon)

5. A standard micro USB cable (preferably a very short one, or adapter module if you can find one)

6. A magnetic periscope lens (easily found on Amazon or ebay)

Software

1. Arduino IDE

2. DroidScript IDE

Step 2: Make the Robot Base

1. First build the standard tracked robot model according to the MakeBlock instructions.

2. Check that you can drive the robot using the included infra-red controller.

Step 3: Reconfigure the Hardware

1. Remove the battery box and strap it below the robot using two of the cable ties.

2. Take off the ultra sonic sensor and it's metal plate, then re-attach them in the lower position shown.

3. Attach the ultra-sonic range sensor to port 4 on the controller board.

Step 4: Attach the Phone Holder

- Attach the phone holder to the top of your robot using cable ties, double sided tape or Velcro pads.

- Place your phone in the holder and attach the periscope lens to your phone's camera.

- Connect the OTG cable and micro USB cable together and then connect them between your phone and the robot's control board. Make sure that the OTG cable goes into your phone and not the other way around or it won't work.

Step 5: Program the MakeBlock Orion (Arduino Uno) Board

- Install the Arduino IDE

- Install the MakeBlock Arduino libraries:

- Connect the robot controller board to your computer using a standard micro USB cable.

- Select "Arduino Uno" as the board type from the 'Tools' menu.

- Create the following new sketch (i.e. program):-

//-------------------------------------------------------
// Smart Rover - Arduino Sketch for controlling a 
// Makeblock robots from DroidScript via OTG cable.
//
// This program can be tested using the Aurduino serial
// monitor.  
//
// Examples: 
//   type 'lft100x' to turn left at speed 100.
//   type 'buzy' to turn buzzer on, 'buzn' to turn it off.
//
// Copyright: droidscript.org
// License: Creative Commons Attribution ShareAlike 3.0
//--------------------------------------------------------
 

#include <MeOrion.h>
#include <Arduino.h>
#include <SoftwareSerial.h>
#include <Wire.h>
//Global variables.
char g_version[] = "0.10\n";
MeDCMotor g_motorL( M1 );  
MeDCMotor g_motorR( M2 );
MeUltrasonicSensor ultraSensor( PORT_4 ); 
unsigned long timer = 0; 
//Setup the hardware.
void setup() 
{
  //Setup USB serial comms.
  Serial.begin( 115200 );
  Serial.setTimeout( 100 );
}
//This function is called forever.
void loop() 
{
  //Read serial commands.
  while( Serial.peek() != -1 )
  {
    //Read 3 character command.
    char cmd[4] = "---";
    Serial.readBytes( cmd, 3 );
    
    //Execute command.
    if( strcmp( cmd, "buz" )==0 ) Buzzer();
    else if( strcmp( cmd, "stp" )==0 ) Stop(); 
    else if( strcmp( cmd, "fwd" )==0 ) Forward();
    else if( strcmp( cmd, "rev" )==0 ) Reverse();
    else if( strcmp( cmd, "lft" )==0 ) Left();
    else if( strcmp( cmd, "rgt" )==0 ) Right();
    else if( strcmp( cmd, "ver" )==0 ) GetVersion();
  } 
  
  //Report status every second.
  if( (millis()-timer) > 1000 ) 
  {
     timer += 1000;
     Report();
  }
}
//Get software version.
void GetVersion()
{
  Serial.print( g_version );
}
//Send status back to DroidScript App.
void Report()
{
  int dist = ultraSensor.distanceCm();
  Serial.println( String("dist:") + dist + String(";") );
}
//Control buzzer.
void Buzzer()
{
  //Read on/off parameter.
  char onOff;
  Serial.readBytes( &onOff, 1 );
  
  //Start of stop buzzer.
  if( onOff=='y' ) buzzerOn();
  else buzzerOff();
}
//Turn vehicle left.
void Left()
{
  //Read speed parameter.
  int speed = Serial.parseInt();
  
  //Turn on motors.
  g_motorL.run( -speed );
  g_motorR.run( speed );
}
//Turn vehicle right.
void Right()
{
  //Read speed parameter.
  int speed = Serial.parseInt();
  
  //Turn on motors.
  g_motorL.run( speed );
  g_motorR.run( -speed );
}
//Drive vehicle forward.
void Forward()
{
  //Read speed parameter.
  int speed = Serial.parseInt();
  
  //Turn on motors.
  g_motorL.run( speed );
  g_motorR.run( speed );
}
//Reverse vehicle.
void Reverse()
{
  //Read speed parameter.
  int speed = Serial.parseInt();
  
  //Turn on motors.
  g_motorL.run( -speed );
  g_motorR.run( -speed );
}
//Stop vehicle.
void Stop()
{
  //Turn off motors.
  g_motorL.run( 0 );
  g_motorR.run( 0 );
}

Step 6: Create the Phone App

- Install the free DroidScript IDE App from Google Play.

- Navigate to the following link using your phone's browser and download the DroidScript project (.SPK) file from here:- http://androidscript.org/demos/rover

- Use a file browser such as Explorer+ or ES File Explorer and open the downloaded .spk file. The .project file will then be automatically installed into DroidScript (Select 'Yes' when it asks you if you trust the source).

Step 7: Take It for a Spin!

- Place batteries in the controller board and turn on the sliding power switch.

- Insert (or re-insert) the OTG cable into your phone (select DroidScript as the default USB program if prompted)

- Start the 'Rover' App by touching it's icon in DroidScript.

- Press the "fwd" key and then the "3" key, then press "Run" button (your rover should drive foward for 3 seconds).

- Press the "Voice" key and try speaking to it (Note: its often best to download offline voice recognition in the Android Settings if you don't have a fast internet connection).

- Try putting your hand in front of the ultrasonic sensors while it is moving foward.

- Try holding colored objects in front of the camera.

Step 8: Further Experiments

There are loads more cool things that can be added to this robot using DroidScript, such as GPS navigation, compass orientation, face tracking,motion detection, remote camera streaming etc. Just look through the DroidScript samples and see what you can find.

The best way to program your robot is to use DroidScript's WiFi IDE. This lets you connect to your phone via WiFi and remotely edit your programs using using a web browser running on your PC or Mac (just press the little arrow button at the top left on the DroidScript App to connect).

Have fun!

Phone Contest

Participated in the
Phone Contest