Introduction: Raspberry Pi Linux Motion Google Photos Integration
The idea is to upload photos and videos made with motion activated cam connected to Raspberry Pi to upload files to a cloud. ‘Motion’ software supports uploading to Google Drive through PyDrive. In this article ‘Motion’ is used to upload to Google Photos.
Hardware:
Raspberry Pi 3B+
USB Webcam Logitech C920
Hardware selection was not determined, I just took what was at hand.
Prerequisites:
For convenience Raspberry pi should be in your local network – to control it without monitor/keyboard and upload/download files. For this you should have ssh agent on your PC (eg putty).
Many thanks to ssandbac for a great tutorial. If you need more info on how to setup environment check out this article. I borrowed motion installation and configuring steps from it and added some changes. In particular, instead of emailing files and alerts this example uses uploading to google photos shared album and get notifications kind of “ added photos” in notification bar.
Here are the steps:
Step 1: Install Linux Motion on Raspberry
1.1 Update pi
pi@raspberrypi:~ $ sudo apt-get update
pi@raspberrypi:~ $ sudo apt-get upgrade
1.2 Download motion
pi@raspberrypi:~ $ sudo apt-get install motion
1.3 Now edit this file with the following changes
pi@raspberrypi:~ $ sudo nano /etc/motion/motion.conf
# Start in daemon (background) mode and release terminal (default: off)
daemon on
# Use a file to save logs messages, if not defined stderr and syslog is used. (default: not defined)
logfile /var/log/motion/motion.log
# Image width (pixels). Valid range: Camera dependent, default: 352
width 1920
# Image height (pixels). Valid range: Camera dependent, default: 288
height 1080
# Maximum number of frames to be captured per second.
framerate 30
# Specifies the number of pre-captured (buffered) pictures from before motion
pre_capture 5
# Number of frames to capture after motion is no longer detected
post_capture 5
# Output 'normal' pictures when motion is detected (default: on)
output_pictures off
# The quality (in percent) to be used by the jpeg compression
quality 100
# Use ffmpeg to encode movies in realtime
ffmpeg_output_movies off
# or the range 1 - 100 where 1 means worst quality and 100 is best.
ffmpeg_variable_bitrate 100
# When creating videos, should frames be duplicated in order
ffmpeg_duplicate_frames false
# Bool to enable or disable extpipe (default: off)
use_extpipe on
extpipe ffmpeg -y -f rawvideo -pix_fmt yuv420p -video_size %wx%h -framerate %fps -i pipe:0 -vcodec libx264 -preset ultrafast -f mp4 %f.mp4
target_dir /var/lib/motion
# Command to be executed when a movie file
; on_movie_end sudo python3 /var/lib/motion/photos.py %f.mp4 &
Leave the last one with semicolon for now (commented) to uncomment after ensuring that video recording and uploading works.
1.4 Then change
pi@raspberrypi:~ $ sudo nano /etc/default/motion
pi@raspberrypi:~ $ start_motion_daemon=yes
Step 2: Setup Google Photos API for Python
2.1 Recommended to create a new account for this purpose to share an album with your main one to get notifications when new files added, plus more storage space. Enable Google Photos API for the account you are going to use for uploading.
You should have credentials.json file after this.
2.2 Python environment setup
Basically the environment setup is required on raspberry only. But it requires oauth authorization which is more convenient to accomplish on PC. To do this on raspberry you need to connect monitor/keyboard to it or setup some remote desktop UI. I just installed the same environment on raspberry and PC both. So the steps 2.2.1..2.2.3 were made on PC, 2.2.1, 2.2.2, 2.2.5, 2.2.6 on Rpi
2.2.1 install Python 3
2.2.2 Install google api packages according to manual*(see 5.1)
On PC
pip3 install --upgrade google-api-python-client google-auth-httplib2 google-auth-oauthlib
On raspberry
pi@raspberrypi:~ $ sudo pip3 install --upgrade google-api-python-client google-auth-httplib2 google-auth-oauthlib
2.2.3 Check out script uploading to google photos.. It is placed on my github. Put it into the same directory with credentials.json.
2.2.4 Take some picture and test uploading
python3 photos.py image.jpg
Install missing dependencies if there are and try again. As a result you should get token.pickle in the script’s directory and also a new shared album created in your Google Photos web interface with image.jpg. As you get token.pickle you don't need credentials.json for photos.py in the same directory anymore.
2.2.5 Share the album with the account on what you’d like to get notifications on new media added. Add this account to your phone.
2.2.6 Put photos.py and token.pickle in /var/lib/motion on raspberry. ‘pi’ user can not write to the ‘motion’s dir’s so upload to /home/pi first
scp photos.py token.pickle pi@IP:/home/pi
Then login to raspberry and move the files under sudo
ssh pi@IP
pi@raspberrypi:~ $ sudo mv photos.py token.pickle /var/lib/motion
2.2.7 Check how uploading works on raspberry. Take some picture with fswebcam and try to upload it
pi@raspberrypi:~ $ sudo fswebcam /var/lib/motion/image.jpg
pi@raspberrypi:~ $ sudo python3 /var/lib/motion/photos.py /var/lib/motion/image.jpg
There sould be image.jpg in the album “helloworld” and notification in the phone’s bar.
Step 3: Test
3.1 Start Motion service
pi@raspberrypi:~ $ sudo service motion start
You can change the command to "stop", or "restart"
3.2 Enable motion logs
pi@raspberrypi:~ $ tail -f /var/log/motion/motion.log
3.2 View the camera output on another device connected to the same local network. Enter in browser:
http://IP:8081
3.3 Looking at the logs wait until the motion detected and the file NAME.mp4 is written in /var/lib/motion. Then launch uploading script manually
pi@raspberrypi:~ $ sudo python3 /var/lib/motion/photos.py /var/lib/motion/NAME.mp4
Check the python traces. Wait until event_end shows up in motion.log. Then go to the “helloworld” album in your google photos and check if there is a video uploaded.
3.4 If uploading is successful uncomment in /etc/motion.conf the line:
pi@raspberrypi:~ $ sudo nano /etc/motion.conf
# Command to be executed when a movie file is ready
on_movie_end sudo python3 /var/lib/motion/photos.py %f.mp4 &
pi@raspberrypi:~ $ sync
pi@raspberrypi:~ $ sudo service motion restart
3.5 Looking at the motion logs and in the album check if the video uploaded automatically.
3.6 Optionally share the album with your main account to get notification when the new video or photo added.
Step 4: Optional: Configure Web Access to Real Time Streaming Camera
This step is based on Michel’s Parreno tutorial. I just chose FreeDNS instead of NoIP as recommended here.
4.1 Configure authorized access to video streaming motion server:
pi@raspberrypi:~ $ sudo nano /etc/motion/motion.conf
# Set the authentication method (default: 0)
# 0 = disabled
# 1 = Basic authentication
# 2 = MD5 digest (the safer authentication)
stream_auth_method 2
# Authentication for the stream. Syntax username:password
# Default: not defined (Disabled)
webcontrol_authentication username:password
# Maximum framerate for stream streams (default: 1)
stream_maxrate 30
# Restrict stream connections to localhost only (default: on)
stream_localhost off
If you are not going to use web control interface from external network leave it disabled (as by default)
# Restrict control connections to localhost only (default: on)
webcontrol_localhost on
Also, since raspberry goes online, I recommend to change default raspberry password
pi@raspberrypi:~ $ passwd
Although ssh port 22 is not rerouted to the raspberry, still.
4.2 Go to FreeDNS site
4.3 Sign up
4.4 Add subdomain (For Members -> Subdomains)
4.5 Choose DNS client to install on Raspberry (For Members -> Dynamic DNS -> Synamic DNS Resources -> Dynamic DNS Clients)
I chose wget_script update.sh from Adam Dean (in the bottom of the page)
There are placeholders _YOURAPIKEYHERE_ and _YOURDOMAINHERE_. To get them go to (For Members -> Dynamic DNS)
And on the page below you’ll find examples of scripts with your APIKEY and DOMAIN (the one added in 4.4). I took these values from Wget Script and replaced _YOURAPIKEYHERE_ and _YOURDOMAINHERE_ in update.sh
4.6 Then run update.sh on raspberry. It may require dnsutils for nslookup. Install it then:
pi@raspberrypi:~ $ sudo apt-get dnsutils
4.7 Then configure your router to reroute external world requests to 8081 port to the raspberry’s ip
4.8 Reserve the ip for your raspberry’s MAC in DHCP setting so Rpi will always have the same ip
4.9 Then enter in browser on a device which is not connected to the local network:
yourdomain:8081
Enter your credentials that you defined in motion.conf.
Test how the video works.
4.10 in order to update DDNS automatically setup cron task. See quick_cron_example on (For Members -> Dynamic DNS)
Step 5: Tips
5.1 Be attentive installing python packages on raspberry. I spent a day debugging this – the issue was that from console the script ran well, but being called from motion event callback did not. What made it worse was that traces from the script were unavailable in the latter case.
The reason was that following the guide I installed the packages for ‘pi’ user (which by default is in /home/pi directory and restricted for other users) but to run the script as a child of ‘motion’ service the packages must be available for ‘motion’ user as well. So finally I fixed it installing the packages as
sudo pip3 …
This is not a proper way still works. Installation without sudo as pip3 --system was giving me errors for some reason.
Correspondingly the script is called also under sudo (see motion.conf).
During this throubleshooting I did a lot of unnecessary changes and not sure what are necessary and now too lazy to roll them back incrementally and see when it stop working. Particularly, granted motion administrator rights:
pi@raspberrypi:~ $ groups motion
motion : motion adm sudo audio video users netdev pi
pi@raspberrypi:~ $ sudo cat /etc/sudoers.d/010_pi-nopasswd
pi ALL=(ALL) NOPASSWD: ALL
motion ALL=(ALL) NOPASSWD: ALL
Also was changing files owners and permissions similar to uploading to Google Drive. Probably it may help you in a case you have a similar issue.
5.2 Google Photos API lets to add files to shared albums only so anyone with the link can access it. Do not share it by link and delete old movies or move them to trash or from album. In latter case they remain in the account.
5.3 Google photos assistant detects faces, which is quite useful if camera quality is fine. As a bonus it makes fancy media kind of compilations and gifs etc.
5.4 I tried to use 4G LTE USB modem for internet access and here are my results.
5.4.1 Huawei E3372h-153 works with raspberry without issues and extra software 5.4.2 Also enabled hotspot so Rasperry shared internet connection through wifi. There is https://howtoraspberrypi.com/create-a-wi-fi-hotspot-in-less-than-10-minutes-with-pi-raspberry/ a really easy guide how to do it using RaspAP. 5.4.3 Dynamic DNS didn’t work in 4G network of my carrie. There is explanation why
5.5 After using this system for a couple of weeks turned out, although videos are more convenient to view and upload, Google Photos works better with images. Eg it enables things/faces grouping analyzing images only, and only then looking for faces/things from the images in videos, but not the opposite. So I'm going to test images uploading rather videos.