This Instructables show how to build a simple chatbot pet.

It is motion responsive and gives positive message responses while chatting.

You'd be willing to chat with it for half an hour. Or you can leave it standby near you, sometimes it will give you some positive message responses while you chat with others.

Supplies

Arduino Nano RP2040 Connect dev board
JR6001 voice module
Motor Driver module
Electronic Pet bird, dancing robot or all other moving robot

Step 1: What Should Chatbot Pets Do?

Many times, chatbot pets just need companionship rather than answering your questions intelligently. One of a good example is Haro in Gundum Anime. It can give random positive message response is already very good enough.

Step 2: Designs & Requirements

Arduino IDE compatible
Speedy speech recognition, offline engine preferred
Voice output
positive message response
motion response

Note:

Arduino IDE just my personal preference, if you choose other platform like micro-python, then the hardware selection and software design may be very different.

Step 3: Arduino Speech Recognition Engine

If you browse the Arduino Official Store, you can find a Speech Recognition Engine for their dev board. It is a pre-trained offline engine so it can help to development a chatbot quickly.

There are various pre-trained speech recognition engine in the market but seldom support Cantonese language. So this time I will use this engine with Cantonese as a demo.

The engine can support Nano 33 BLE Sense Rev2, Portenta H7, Nicla Vision and Nano RP2040 Connect. I picked Nano RP2040 Connect in this project.

Ref.:

https://store-usa.arduino.cc/products/speech-recognition-engine

Step 4: Trial Mode

The Arduino Speech Recognition Engine can run in trial mode before any payment.

Here are the steps to try it:

Install DSpotterSDK_Maker_RP2040 Library in Arduino IDE Library Manager
Follow the steps to get the dev board serial number: https://tool.cyberon.com.tw/ArduinoDSpotterAuth/DocHowToGetBoardSerialNumber.php
Start the trial at: https://tool.cyberon.com.tw/ArduinoDSpotterAuth/CTMain.php
Fill the email, serial number and select desired language
Decide a Trigger words, this is the hint you start the conversation, just like "Hey Siri" or "Hello Google"
This engine can recognize at most 20 Commands, you can treat it as 20 keywords in your conversation that the chatbot will response
Confirm filled details and wait the reply email
Open example in Arduino IDE File Menu -> Examples -> DSpotterSDK_Maker_RP2040 -> Voice Recognition
Save the example
Copy the email attachment to the example folder
Change follow 2 #include line to the new file names

#include "CybLicense.h"

and

#include "Model_L0.h"             // The packed level zero model file.

Then upload the program to test. As show in the above video, trial mode requires wait 20 seconds in the middle.

Step 5: Voice Output

After speech recognition, we need voice output response.

The RP2040 dev board can develop with 2 board cores.

The earlephilpower's arduino-pico board core support PWMAudio and I2S output. However, Arduino Speech Recognition Engine requires Arduino official Mbed OS board core.

So we need another way for voice output.

Step 6: Voice Module

Direct playing voice with MCU may not the best option. Decoding and playing voice use up the processing power, and the voice clip use up the limited storage easily. Better alternative is offload the job to a voice module. The voice clips, in MP3 format, preloaded in the voice module so you don't need to concern the storage and how to decode it. And it also built-in amplifier, simply connect it to a speaker and send a command will play the voice loudly.

Note:

I am using voice module JR6001, if you are using another voice module please follow the data sheet modify the send message function accordingly.

Step 7: Motion Response

Most robot toy use a DC motor emulate bionic motions. Simply use a motor driver module can reproduce a motion response easily.

Step 8: Wire Connection

Since Arduino Nano RP2040 Connect is highly integrated many feature in one dev board. The wire connection in this project is relatively simple. Just few wires connect to 2 modules.

Here are the connection summary:

RP2040    Voice Module    Motor Module
======    ============    ============
GND    -> GND          -> GND
3V3    -> VCC
GPIO15 -> BUSY
TX     -> RX
RX     -> TX
          SPK+/SPK-                     -> Speaker
GPIO28                 -> IN2
GPIO29                 -> IN1
VIN                    -> VCC
                          OUT1/OUT2     -> Motor

Step 9: Sample Code

I have uploaded my sample code to the Github:

https://github.com/moononournation/PositiveResponseChatbot.git

However, the engine model needs to match the dev board serial number. You need import your own trial or release license and generated model files before using it.

Step 10: Positive Message Response Map

This is the most important step in this project. I would like to have positive message response for each keyword, or called command in this engine. To avoid message repeat too much in the conversation, each keyword can have multiple response and appear randomly. At the same time some message can reuse in multiple keywords. All relationship managed in the responseMap.h.

Here are some response example:

"Good Morning" will reply "Good Morning"
"Hello" will say back "Hello", "How are you?" or "Let's play together!"
"It's great!" will echo "great!" or "I think so."
"I am cold" will considerate "wear more clothes"
"It taste good" will reply "I wanna taste it"

Step 11: Making Voice Clips

After decided the response message map, it's time to make voice clips. You can record clips one by one with recorder app. Or you can generate it by TTS engine.

I invited a little girl help me record it and use Audacity tune to a higher pitch. Then export all voice clips in MP3 format and name it in number according to the response map. At last copy all files to the voice module.

Note:

JR6001 requires all voice clips name in 2 digits number and place into the ZH folder.

If you are using Mac OS, you need to double check with Terminal no any "." leading files in the folder:

ls -la ZH

Step 12: Test Run

Step 13: Off the Shelf Robot

The electronic part and the software are ready, but we still need a robot look. There are many off the shelf robot toy in the market, simply buy one or reuse one in your hand can do the job ;P