Raspberry Pi Cam Pan-Tilt Control Over Local Internet

On previous tutorials we explored:

How to stream to the internet a video captured by a camera: VIDEO STREAMING WEB SERVER

and,

How to position a camera (or anything) using: PAN-TILT MULTI SERVO CONTROL

In this tutorial, we will combine what we have learned before, controlling our camera position thru internet, as shown in the example:

The above gif shows the camera controlled by buttons, pre-programmed with fixed Pan/Tilt angles. In this tutorial, we will also explore other alternatives to control the camera position thru internet.

Below the block diagram of our project:

Step 1: BoM - Bill of Material

Main parts:

Raspberry Pi V3 - US$ 32.00
5 Megapixels 1080p Sensor OV5647 Mini Camera Video Module - US$ 13.00
TowerPro SG90 9G 180 degrees Micro Servo (2 X)- US$ 4.00
Mini Pan/Tilt Camera Platform Anti-Vibration Camera Mount w/ 2 Servos (*) - US$ 8.00
Resistor 1K ohm (2X) - Optional
Miscellaneous: metal parts, bands, etc (in case you will construct your Pan/Tilt mechanism)

(*) you can buy a complete Pan/Tilt platform with the servos or build your own.

Step 2: Installing the PiCam

1. With your RPi turned-off, install the Camara on its special port as shown below:

2. Turn on your Pi and go to Raspberry Pi Configuration Tool at main menu and verify if Camera Interface is enabled:

If you needed to Enabled it, press [OK] and reboot your Pi.

Make a simple test to verify if everything is OK:

raspistill -o /Desktop/image.png

You will realize that an image icon appears on your Rpi desktop. Click on it to open. If an image appears, your Pi is ready to stream video! If you want to know more about the camera, visit the link: Getting started with picamera.

Step 3: Instaling Flask

There are several ways to stream video. The best (and "lighther") way to do it that I found was with Flask, as developed by Miguel Grinberg. For a detailed explanation about how Flask does this, please see his great tutorial: flask-video-streaming-revisited.

On my tutorial: Python WebServer With Flask and Raspberry Pi, we learned in more details how Flask works and how to implement a web-server to capture data from sensors and show their status on a web page. Here, on the first part of this tutorial, we will do the same, only that the data to be sent to our front end, will be a video stream.

Creating a web-server environment:

The first thing to do is to install Flask on your Raspberry Pi. If you do not have it yet, go to the Pi Terminal and enter:

sudo apt-get install python3-flask

The best when you start a new project is to create a folder where to have your files organized. For example:

from home, go to your working directory:

cd Documents

Create a new folder, for example:

mkdir camWebServer

The above command will create a folder named "camWebServer", where we will save our python scripts:

/home/pi/Documents/camWebServer

Now, on this folder, let's create 2 sub-folders: static for CSS and eventually JavaScript files and templates for HTML files. Go to your newer created folder:

cd camWebServer

And create the 2 new sub-folders:

mkdir static

and

mkdir templates

The final directory "tree", will look like:

├── Documents
       ├── camWebServer
               ├── templates
               └── static

OK! With our environment in place let's create our Python WebServer Application to stream video.

Step 4: Creating the Video Streaming Server

First, download Miguel Grinberg's picamera package: camera_pi.py and save it on created directory camWebServer. This is the heart of our project, Miguel did a fantastic job!

Now, using Flask, let's change the original Miguel's web Server application (app.py), creating a specific python script to render our video. We will call it appCam.py:

from flask import Flask, render_template, Response

# Raspberry Pi camera module (requires picamera package, developed by Miguel Grinberg)
from camera_pi import Camera

app = Flask(__name__)

@app.route('/')
def index():
    """Video streaming home page."""
    return render_template('index.html')

def gen(camera):
    """Video streaming generator function."""
    while True:
        frame = camera.get_frame()
        yield (b'--frame\r\n'
               b'Content-Type: image/jpeg\r\n\r\n' + frame + b'\r\n')

@app.route('/video_feed')
def video_feed():
    """Video streaming route. Put this in the src attribute of an img tag."""
    return Response(gen(Camera()),
                    mimetype='multipart/x-mixed-replace; boundary=frame')

if __name__ == '__main__':
    app.run(host='0.0.0.0', port =80, debug=True, threaded=True)

The above script streams your camera video on an index.html page as below:

<html>
  <head>
    <title>MJRoBot Lab Live Streaming</title>
    <link rel="stylesheet" href='../static/style.css'/>
  </head>
  <body>
    <h1>MJRoBot Lab Live Streaming</h1>
    <h3><img src="{{ url_for('video_feed') }}" width="90%"></h3>
    <hr>
    <p> @2018 Developed by MJRoBot.org</p>
  </body>
</html>

The most important line of index.html is:

<img src="{{ url_for('video_feed') }}" width="50%">

There is where the video will be "feed" to our web page.

You must also include the style.css file on the static directory to get the above result in terms of style.

All the files can be downloaded from my GitHub: camWebServer

Only to be sure that everything is in the right location, let's check our environment after all updates:

├── Documents
       └── camWebServer
               ├── camera_pi.py
               ├── appCam.py
               ├── templates
               |     └── index.html
               └── static
                     └── style.css

Now, run the python script on the Terminal:

sudo python3 appCam.py

Go to any browser in your network and enter with http://YOUR_RPI_IP (for example, in my case: 10.0.1.27)

NOTE: If you are not sure about your RPi Ip address, run on your terminal:

ifconfig

at wlan0: section you will find it.

The results:

That's it! From now it is only a matter to sophisticate a page, embedded your video on another page etc.

Step 5: The Pan Tilt Mechanism

Now that we have the camera working and our Flask WebServer streaming its video, let's install our Pan/tilt mechanism to position the camera remotely.

For details, please visit my tutorial: Pan-Tilt-Multi-Servo-Control

The servos should be connected to an external 5V supply, having their data pin (in my case, their yellow wiring) connect to Raspberry Pi GPIO as below:

GPIO 17 ==> Tilt Servo
GPIO 27 ==> Pan Servo

Do not forget to connect the GNDs together ==> Raspberry Pi - Servos - External Power Supply)

You can have as an option, a resistor of 1K ohm in serie, between Raspberry Pi GPIO and Server data input pin. This would protect your RPi in case of a servo problem.

Let's also use the opportunity and test our servos inside our Virtual Python Environment.

Let's use Python script to execute some tests with our drivers:

from time import sleep
import RPi.GPIO as GPIO
GPIO.setmode(GPIO.BCM)
GPIO.setwarnings(False)

def setServoAngle(servo, angle):
	pwm = GPIO.PWM(servo, 50)
	pwm.start(8)
	dutyCycle = angle / 18. + 3.
	pwm.ChangeDutyCycle(dutyCycle)
	sleep(0.3)
	pwm.stop()

if __name__ == '__main__':
	import sys
	servo = int(sys.argv[1])
	GPIO.setup(servo, GPIO.OUT)
	setServoAngle(servo, int(sys.argv[2]))
	GPIO.cleanup()

The core of above code is the function setServoAngle(servo, angle). This function receives as arguments, a servo GPIO number, and an angle value to where the servo must be positioned. Once the input of this function is "angle", we must convert it to an equivalent duty cycle.

To execute the script, you must enter as parameters, servo GPIO, and angle.

For example:

sudo python angleServoCtrl.py 17 45

The above command will position the servo connected on GPIO 17 ("tilt") with 45 degrees in "elevation". A similar command could be used for Pan Servo control (position to 45 degrees in "azimuth":

sudo python angleServoCtrl.py 27 45

The file angleServoCtrl.py can be downloaded from my GitHub

Step 6: Controlling Camera Position Via Web - Using Buttons

Let's start creating a new directory calling it:

mkdir PanTiltControl1

On this directory we should have the following environment and files:

├── Documents
       └── PanTiltControl1
               ├── camera_pi.py
	       ├── angleServoCtrl.py
               ├── appCamPanTilt1.py
               ├── templates
               |     └── index.html
               └── static
                     └── style.css

The file camera_pi.py is Miguel's script and angleServoCtrl.py, both used before. You can download both from my GitHub, clicking on correspondent links.

Now we need the appCamPanTilt1.py, the index.html and style.css. You can download those files from my GitHub, clicking on correspondent links. Pay attention to its correct position on your directory.

Let's see the index.html:

<html>
  <head>
    <title>MJRoBot Lab Live Streaming</title>
    <link rel="stylesheet" href='../static/style.css'/>
  </head>
  <body>
    <h1>MJRoBot Lab Live Streaming</h1>
    <h3><img src="{{ url_for('video_feed') }}" width="80%"></h3>
	<hr>
	<h4> PAN:  
		<a href="/pan/30"class="button">30</a>
		<a href="/pan/45"class="button">45</a>
		<a href="/pan/60"class="button">60</a>
		<a href="/pan/75"class="button">75</a>
		<a href="/pan/90"class="button">90</a>
		<a href="/pan/105"class="button">105</a>
		<a href="/pan/120"class="button">120</a>
		<a href="/pan/135"class="button">135</a>
		<a href="/pan/150"class="button">150</a>
		==> Angle: [ {{ panServoAngle }} ]
	</h4>
		<h4> TILT: 
		<a href="/tilt/30"class="button">30</a>
		<a href="/tilt/45"class="button">45</a>		
		<a href="/tilt/60"class="button">60</a>
		<a href="/tilt/75"class="button">75</a>		
		<a href="/tilt/90"class="button">90</a>
		<a href="/tilt/105"class="button">105</a>		
		<a href="/tilt/120"class="button">120</a>
		<a href="/tilt/135"class="button">135</a>		
		<a href="/tilt/150"class="button">150</a>
		==> Angle: [ {{ tiltServoAngle }} ]
	</h4>
	<hr>
	<p> @2018 Developed by MJRoBot.org</p>
  </body>
</html>

The index.html was created from the previous file, used when we streamed our video. The new bunch of lines on this new file is related to each one of the buttons that appear on the page.

Let's analyze one of them:

<a href="/pan/30"class="button">30</a>

This is a simple HTML hyperlink TAG, that we have styled as a button (the button style is described in style.css). When we click on this link, we generate a "GET /<servo>/<angle>", where <servo> is "pan" and <angle> is "30 degrees". Those parameters will be passed to the Web Server App (appCamPanTilt1.py).

Let's see this part of code on appCamPanTilt1.py:

@app.route("/<servo>/<angle>")
def move(servo, angle):
	global panServoAngle
	global tiltServoAngle
	if servo == 'pan':
		panServoAngle = int(angle)
		os.system("python3 angleServoCtrl.py " + str(panPin) + " " + str(panServoAngle))
	if servo == 'tilt':
		tiltServoAngle = int(angle)
		os.system("python3 angleServoCtrl.py " + str(tiltPin) + " " + str(tiltServoAngle))
	
	templateData = {
      'panServoAngle'	: panServoAngle,
      'tiltServoAngle'	: tiltServoAngle
	}
	return render_template('index.html', **templateData)

In this example, "servo" is equal to "pan", and the 2 lines below will be executed:

panServoAngle = int(angle)
os.system("python3 angleServoCtrl.py " + str(panPin) + " " + str(panServoAngle))

What we are doing here is the same as we did when we tested the servo position on Pi Terminal. PanPin will be translated by "27" and panServoAngle to "30". The app will generate the command:

python3 angleServoCtrl 27 30

Note that we do not need use "sudo" in this case, because the app was already started using "sudo".

The video shows the project working and how the GET requests appear on Pi Terminal:

Step 7: Using Incremental - Decremental Angle Buttons

Sometimes what we only need is a few buttons to move our servos in steps:

Pan: Left / Right
Tilt: Up / Down

We can also use +/- buttons (Incremental - decremental angle), your choice. Let's create a new directory:

mkdir PanTiltControl2

On this directory we should have the following environment and files:

├── Documents
       └── PanTiltControl2
               ├── camera_pi.py
	       ├── angleServoCtrl.py
               ├── appCamPanTilt2.py
               ├── templates
               |     └── index.html
               └── static
                     └── style.css

The files camera_pi.py and angleServoCtrl.py are the same used before. You can download both from my GitHub, clicking on correspondent links or use the ones that you have downloaded before.

Now we need the appCamPanTilt2.py, the index.html and style.css. You can download those files from my GitHub, clicking on correspondent links. Pay attention to its correct position on your directory.

Let's see the NEW index.html:

<html>
  <head>
    <title>MJRoBot Lab Live Streaming</title>
    <link rel="stylesheet" href='../static/style.css'/>
  </head>
  <body>
    <h1>MJRoBot Lab Live Streaming</h1>
    <h3><img src="{{ url_for('video_feed') }}" width="80%"></h3>
	<hr>
	<h4> PAN Angle: <a href="/pan/-"class="button">-</a> [ {{ panServoAngle }} ] <a href="/pan/+"class="button">+</a> </h4>
	<h4> TILT Angle: <a href="/tilt/-"class="button">-</a> [ {{ tiltServoAngle }} ] <a href="/tilt/+"class="button">+</a> </h4> 
	<hr>
	<p> @2018 Developed by MJRoBot.org</p>
  </body>
</html>

The index.html is very similar to the previous one. The bunch of lines used on the last index.html was replaced by only 2 lines, where we will only have now 4 buttons Pan [+], Pan [-], Tilt [+] and Tilt [-].

Let's analyze one of the 4 buttons:

<a href="/pan/-"class="button">-</a>

This is also a simple HTML hyperlink TAG, that we have styled as a button (the button style is described in style.css). When we click on this link, we generate a "GET /<servo>/<Increment or decrement angle>", where <servo> is "pan" and <-> is "decrease angle". Those parameters will be passed to the Web Server App (appCamPanTilt2.py).

Let's see this part of code on appCamPanTilt2.py:

@app.route("/<servo>/<angle>")
def move(servo, angle):
	global panServoAngle
	global tiltServoAngle
	if servo == 'pan':
		if angle == '+':
			panServoAngle = panServoAngle + 10
		else:
			panServoAngle = panServoAngle - 10
		os.system("python3 angleServoCtrl.py " + str(panPin) + " " + str(panServoAngle))
	if servo == 'tilt':
		if angle == '+':
			tiltServoAngle = tiltServoAngle + 10
		else:
			tiltServoAngle = tiltServoAngle - 10
		os.system("python3 angleServoCtrl.py " + str(tiltPin) + " " + str(tiltServoAngle))
	
	templateData = {
      'panServoAngle'	: panServoAngle,
      'tiltServoAngle'	: tiltServoAngle
	}
	return render_template('index.html', **templateData)

In this example, "servo" is equal to "pan", the lines below will be executed:

if angle == '+':
	panServoAngle = panServoAngle + 10
else:
	panServoAngle = panServoAngle - 10
os.system("python3 angleServoCtrl.py " + str(panPin) + " " + str(panServoAngle))

Once the "angle" is equal to "-", we will decrease 10 from panServoAngle and pass this parameter to our command. Suppose that the actualpanServoAngle is 90. The new parameter will be 80.

So, PanPin will be translated by "27" and panServoAngle to "80". The app will generate the command:

python3 angleServoCtrl 27 80

Note that we do not need use "sudo" in this case, because the app was already started using "sudo".

The gif shows the webpage working :

Step 8: Using "POST" Approach

Sometimes could be interesting to send specific angle commands like:

Pan Angle ==> 35 degrees
Tilt Angle ==> 107 degrees

What means that or you will define your increment to "1" on the last step, what will take a lot of time to reach the required position, or you create a POST from your webpage. Let's explore this last possibility.

Let's again create a new directory:

mkdir PanTiltControl3

On this directory we should have the following environment and files:

├── Documents
       └── PanTiltControl3
               ├── camera_pi.py
	       ├── angleServoCtrl.py
               ├── appCamPanTilt3.py
               ├── templates
               |     └── index.html
               └── static
                     └── style.css

The files camera_pi.py and angleServoCtrl.py are the same used before. You can download both from my GitHub, clicking on correspondent links or use the ones that you have downloaded before.

Now we need the appCamPanTilt3.py, the index.html and style.css. You can download those files from my GitHub, clicking on correspondent links. Pay attention to its correct position on your directory.

Let's see the NEW index.html:

<html>
  <head>
    <title>MJRoBot Lab Live Streaming</title>
    <link rel="stylesheet" href='../static/style.css'/>
  </head>
  <body>
    <h1>MJRoBot Lab Live Streaming</h1>
    <h3><img src="{{ url_for('video_feed') }}" width="80%"></h3>
    <p> Enter Pan Tilt Servo Angle:
	<form method="POST">
		PAN:  <input type="text" name="panServoAngle" value= {{panServoAngle}} size="3">
		TILT: <input type="text" name="tiltServoAngle" value= {{tiltServoAngle}} size="3">
		<input type="submit">
	</form>
   </p>
   <hr>
	<p> @2018 Developed by MJRoBot.org</p>
  </body>
</html>

The index.html now is a little bit different from the previous one. We will create here a "Form", which method will be POST. An HTML form input TAG, will be used to pass as parameters the Pan/Tilt angle values digited by users. Those parameters will be passed to the Web Server App (appCamPanTilt3.py) when the button "submit" is pressed.

Let's see this part of code on appCamPanTilt3.py:

@app.route('/', methods=['POST'])
def my_form_post():
	global panServoAngle
	global tiltServoAngle

	panNewAngle = int(request.form['panServoAngle'])
	if (panNewAngle != panServoAngle):
		panServoAngle = panNewAngle
		os.system("python3 angleServoCtrl.py " + str(panPin) + " " + str(panServoAngle))

	tiltNewAngle = int(request.form['tiltServoAngle'])
	if (tiltNewAngle != tiltServoAngle):
		tiltServoAngle = tiltNewAngle
		os.system("python3 angleServoCtrl.py " + str(tiltPin) + " " + str(tiltServoAngle))

	templateData = {
      'panServoAngle'	: panServoAngle,
      'tiltServoAngle'	: tiltServoAngle
	}
	return render_template('index.html', **templateData)

What we will do is to check which angle was changed generating the correspondent command similar what were done before.