Introduction: Use the Raspberry Pi Camera to Detect and Classify Objects in Pictures
Give Your Raspberry Pi Smart Vision
Use Google Cloud Vision on the Raspberry Pi to take a picture with the Raspberry Pi Camera and classify it with the Google Cloud Vision API. First, we’ll walk you through setting up the Google Cloud Platform. Next, we will use the Raspberry Pi Camera to take a picture of an object, and then use the Raspberry Pi to upload the picture taken to Google Cloud. Finally, we can analyze the picture in the cloud, and the cloud will try to label the picture (tell us in text what’s going on in the picture). Skynet here we come!
The Google Cloud Account is free for 60 days. After 60 days, you will be charged based on usage. You can create an account here with either your Google or Gmail login. Google offers the first 60 days for free.
These directions assume you’re using Raspbian for Robots on your SD Card. You can buy an SD Card with Raspbian for Robots on it, or you can download it for free and install it using our directions here.
In this picture we’ve asked Google Cloud Vision to analyze the GoPiGo on the left, with the returned text response on the right. It returns “labels” (the things it sees in the picture) such as “vehicle”, “wheel”, “toy”, “product”, and “model car”. Pretty good job Google!
Teachers! Did you use this instructable in your classroom?
Add a Teacher Note to share how you incorporated it into your lesson.
Step 1: Google Cloud Set Up
Setting up a Google Cloud Account
The first step is to set up a Google Cloud Account. You can create an account here. You can create an account here with either your Google or Gmail login. Google offers the first 60 days for free.
After you’ve created your account, you can set up a project and enable billing. First, click here to go to the Google Cloud Projects Page.
Create a new project.
In our example, we named it “vision1”
You will need to enable billing for your account (you won’t be charged). You can do this by clicking here. Go to the Cloud Platform Console here. Select the project “vision1” and click the hamburger in the upper left hand corner of the page.
Select Billing on the left hand side. From here, you can enable billing on your Google Cloud account. Finally, we’ll enable the API. Click here to enable the API. Select the “vision1” account we created. Click “Continue”. You should get a message that API is enabled.
Optional: Test the Account
Now that the account is setup, why not upload an image from your computer and test it here? You can follow this tutorial to try out a few images and see what Google thinks they are!
Use our picture above. Is it an animal? A reptile? Google will tell you!
Step 2: Getting a JSON Key
Now we want to get a JSON key to put on our Raspberry Pi. This JSON key will handle all the authentication to use our Google Cloud Account. Instead of a password, we can use the file to authenticate our account on the Raspberry Pi.
Caution: You should be careful where you store this key. Be sure to change the password on your Raspberry Pi before putting the JSON key on it, leaving your JSON key unprotected could expose you to something malicious! In the command line type:
And follow the prompts to change the password on the Raspberry Pi.
- Head back to the Console in Google Cloud. Find the box titled “Use Google APIs” and click “Enable and manage APIs”
- Click “Credentials” and Create Credentials. Credentials is on the left hand side, with a picture of a key next to it.
Select “Create an Service Account Key”. See more about what we’re doing here.
Under “Service Account” Select “New Service Account”. We’ll give it a name, “vision”
- Finally, create a role. We’ll give it full access, so select “Project” and “Owner” to give the Pi Full access to all resources.
- A popup should come up telling you you have created a new key, and an automatic download of the JSON key should begin. Keep track of this file!
That should be it, you should have a service account key!
You should now use an FTP program (such as FileZilla) or Samba (see our tutorial here!) to move the JSON file over to your Raspberry Pi. Place the JSON file in the home directory.
Step 3: Set Up the Hardware
These Raspberry Pi Camera setup directions assume you’re using Raspbian for Robots on your SD Card. You can buy an SD Card with Raspbian for Robots on it, or you can download it for free and install it using our directions here. If you’re not using Raspbian for Robots, you will need to take additional steps to enable and setup your camera; here are some directions on setting up the camera.
Step 4: Prepare the Raspberry Pi
In this project, we’re assuming you’re using our image, Raspbian for Robots. If you’re not, you can download it here!
Next, we’ll upgrade Pip. Pip is a package manager for python language installations. Note, you should have Pip installed (it comes installed on Raspbian for Robots); if you don’t upgrade, you will get an error on installation!
sudo pip install --upgrade pip <br> sudo apt-get install libjpeg8-dev
Next, install Google API Python Client. Again in the command line, run:
sudo pip install --upgrade google-api-python-client
Next, install Python Imaging Library. Again in the command line, run:
sudo pip install --upgrade Pillow
Install Python Picamera:
sudo apt-get install python-picamera
Turn on Super User. In the command line, type the command “su”:
You’ll be prompted for your password; this is the password you used to login to your Raspberry Pi.
In the home directory, we will make the JSON file available to any application we’re running. Run the command:
Be sure to substitute your JSON filename in this command with the name of the file you have on your Raspberry Pi.
Get the Example Code
Get the example code: we have three python examples on Github here.
sudo wget https://github.com/DexterInd/GoogleVisionTutorials
Step 5: Label Pictures
One of the cool things you can do with Google Vision is get the cloud to figure out what’s in the picture you take. Let’s take a picture of a GoPiGo Wheel and see what the cloud thinks it is.
Set up the camera and run the sample program. Again, we’ll be holding the camera in place with the GoPiGo robot kit.
With the camera set up, we’ll take a test picture. Looks like the wheel is in there.
In this sample program we’ll return the entire JSON response. We get a few interesting results from the picture: “yellow” (with an 80% score), “wheel”, “product”, “tire”, “toy” and “automotive tire”.
Step 6: Conclusion
Google Cloud Vision is a really fun service to just play around with! Google Cloud Vision on the Raspberry Pi is perfect for projects with the Raspberry Pi and Raspberry Pi Camera, and adds in-context vision for the Raspberry Pi.
Learn more about our projects here
Questions? Refer to our tutorial here