Conversations with a toaster have always been very one sided. Until now, meet Ted the talking Toaster!

Below an overview of the content:

Project video
Toaster
Electronics
Chat/Voice bot
Remo.tv
World domination

Supplies

Hardware components:

Software apps and online services:

Raspberry Pi Raspbian
Google Dialogflow
Google Cloud Platform

Hand tools and fabrication machines:

As an Amazon Associate I earn from qualifying purchases you make using my affiliate links.

Step 1: Project Video

Step 2: Toaster

To make a talking toaster the first thing we need is, a toaster.
I got this one from a friend, but thrift stores are a great place to find yourself a broken toaster with good looks.

The first thing we're going to do is remove as much of the internals as possible, while keeping the outside looking nice.

Some fiddling and the right amount of violence is needed to make this happen, but in the end you'll have plenty of space

Step 3: Electronics

With all that space, we can start filling it up. Our aim is to make a talking toaster, so that's what we wil start with.

We're going to use a Raspberry Pi 3 b+ with a Google AIY kit.
This kit consists of a hat, microphone and speaker. They provide a great tutorial on how to set it all up.
For this project we've used an older version of the kit, but the newer ones will work as well.

This covers the talking hardware for our little bread burner, but we also want him to express himself. To make this happen we're going to drill two holes in one of the front panels.

Through these holes we put two little servo's, they're the start of his eyebrows.

Now we can wire the servo to the AIY hat, first we need to solder some pins and then we can easily attach them. There's some more info on the possible extensions (see image).

The last offline step is to glue on some fuzzy pipe cleaners as eyebrows and googly eyes as, you guessed it, his eyes.

And with that, we've got a very good looking little heating element. He looks the part, so he deserves a name, let's call him Ted.

Step 4: Chat/Voice Bot

At this point you can keep following the AIY tutorial to make the toaster your personal google assistant, or write your own custom code. In this case we opt for creating a custom chatbot, using Dialogflow. They also provide a good tutorial on how to setup a chatbot.

After some creative writing our chatbot is done, but at this point Ted is just a toaster and the chatbot doesn't do much. To change this we open up the Dialogflow chatbot API. This means we can access the chatbot from our Raspberry Pi. And now the fun begins The flow of data is as followed:

1
The microphone picks up someone speaking and saves the audio

2-3
Using some Google magic (Speech-To-Text) we extract the text from the audio

4-5
This text is sent to our chatbot (Dialogflow) and is matched with an intent.After which one of the possible answers is send back to the Raspberry Pi (Ted)

6-7-8
Using Text-To-Speech, the text is converted to audio and played via the speaker You can find the full code for this attached to the article.

Ted can now listen and talk, but how is he going to show the right emotion for each response? When fetching the response from the Dialogflow API , it also sends back the intent name. Ending every intent with "_emotion" makes sure we can extract the emotion linked with each response and move the servos accordingly.

Example, our intent is called "climateChange_angry" he will extract everything after the "_".This means "angry" , and then moving the servo to look angry.

Now Ted can listen, talk and look frisky, that's one small step for Ted, one giant leap for toasterkind!

ted_code.py
Download

Step 5: Remo.tv

Ted might be a tad evil, who would think a toaster could be out for world domination?
As his creators we're responsible for his well being, and since he demanded to be connected to the internet to spread his message, let's do exactly that.

It turns out we're not the only ones who want to connect robots to the internet, Remo.tv is way ahead of us. This means that all we have to do is attach a Pi Camera and follow their instructions.

Now people can see Ted, but talking to him is tricky. However, Remo.tv allows you to implement your own chat handeling.So removing some parts of our previous code and adding the custom chat handler does the trick. Now we have a new the data flow (see image).

1
Chat message is received from Remo.tv
2-3
This message is sent to Dialogflow and the response with emotion is returned.
6-7
The response text is converted to audio via the Text-To-Speech.
8
The audio is played via the speaker and immediately picked up by the microphone.
9
This audio is sent to Remo to be broadcast to everyone who's tuned in.