BIA (AI) Vocal Assistant (2/3 : BIA on Raspberry PI)

Hello

Hi there :) My nickname is TylerMaker, I live in France and am happy to share with you the lines below. This Instructables is the second in a serie of three.

What is BIA (AI) ?

BIA is an AI vocal assistant. BIA is an all in one software : ChatGPT, voice command, YouTube, Netflix, TV, Radio and web navigation. BIA is multi language (English, French, German, Spanish, etc) and can interact via the voice or command line. Can run on any laptop, or on a Raspberry.

Example of what you can do with it :

Chat with ChatGPT
Ask Bia to open a web page, YouTube, a radio, a TV channel or Netflix
Ask Bia to take a picture
Ask complex instruction with a If This Then That logic
Save complex instructions as a macro
And some more

Extra cool functionality, BIA also provides IA to the educational Otto robot. Otto is a world-wide known open source robot used for educational and fun purpose. What you do is giving vocal instructions : ask the robot to walk, dance and so on, then Bia will transform your voice request on the fly into Arduino Code... Let the magic be ... Otto will execute your request :) :)

Example of what can be done with Otto :

Ask the robot to walk,
Dance,
Be happy or not,
Ask complex actions like "if this then that",
Create macros
Take pictures,
And some more

Available Instructables ?

BIA (AI) on Laptop (#1/3) : the goal of this first instructable is to install and run the program on laptop.
BIA (AI) on Raspberry Pi (#2/3) : the second instructables explains how to install BIA on Raspberry Pi.
BIA (AI) and Otto the robot (#3/3) : in the third instructable is detailed how Otto the robot can be used together with BIA

IMG_2210.MOV
Download

Supplies

You need a Raspberry PI :)

The program was tested on a good old Raspberry PI 1B Plus, the latest version of Raspberry PI OS and Python version 3.8. Should run also on Windows or Linux, to be tested still :)

Shopping list :

A Raspberry, any modern version should work, like the Raspberry PI 4 or 5
I needed for the network a WIFI USB dongle TP-LINK
For the audio, you can use headphones with jack. I ordered a mini loudspeaker
Microphone for voice input
Camera : a small one Raspberry compatible
You will need also a touch screen. We used an LCD TFT 3.5 touch screen

Note : the touch screen is optional, VNC viewer can be used instead. BIA can be fully used from your smartphone or laptop with VNC

Prepare the Raspberry PI :

Follow the instructions to install Raspberry PI OS : link
Install the WIFI network, please browse available ressources on google
Enable SSH. Ressources are available on google
Install the microphone, please browse also ... available ressources on google
Install the camera, please browse...
Install the touch screen, please again .. google .. ;) ;)
On your laptop install VNC viewer

We need a case for the Raspberry PI .. (option)

Steps :

Download 3d models, in general one for the top case and one for the bottom case : link
slice them with the software Cura
and print the 2 cases (top and bottom) with a 3d printer

We are using ChatGPT, we need an OpenAI key..

As a prerequisite you need to create an open account in order to use ChatGPT. Then you need to request an API Key.

Check the instructions directly at the OpenAI help page : here.

Note : the usage of the API of chatGPT is not free but indeed very cheap. In general a few cents by month only. After 3 months developping and testing the program I was charged only 14 cents.

Take note of the API key and go to the next step.

We are using Python, we need to install it on your Raspberry PI..

Download and install Python 3.8 : link

Step 1: The Concept

Concept

A double layer IA mechanism provide fast and accurate answers :

First layer

The first layer is powered by OpenAI (chatGPT), ittakes care of answering the questions

Second layer

The second layer is processed by a native AI, it takes care of :

Cache management : questions and answers are cached in a local database. The goal of it is to get rid of the network latencies whenever possible (requests to chatGPT are network dependent, requests to the cache are not). The goal of it is ensuring faster answers
Categorization : the input is categorized with a NLP (Natural Language Processing) technique into categories like question, feedback, action, etc
Paraphrases : if for example you ask "how are you", and then later "how are you doing" both questions will be considered similar and the same answer will be picked up from the cache
Emotions : the user input is scored on several axis (positive, negative, neutral). Based on the emotion the previous answer will get promoted or deprecated in the cache
Scoring : each prompt/interaction is scored, in this way only the best answer is given
Smart actions (complex requests), like If This Then That requests, instructions to Otto Robot, etc

Step 2: Install the Python Dependencies

All the steps below have been done via SSH. Please check the web on how to connect to Raspberry with SSH :)

Prerequesites :

Download the nltk ressources :

pip install nltk 
python3
>> import nltk
>> nltk.download('punkt')
>> nltk.download('vader_lexicon')

Install one by one the libraries :

sudo apt-get install libjpeg8-dev
sudo apt install espeak 
sudo apt install python3-opencv 
sudo apt-get install flac 
pip install openai 
pip install websockets 
pip install argostranslate 
pip install py3langid 
pip install pyautogui 
pip install pyaudio 
pip install speechrecognition 
pip install pyttsx3 
pip install parrot 
pip install customtkinter 
pip install pyduinocli

Install the languages :

argospm install translate-de_en
argospm install translate-en_de
etc with all the languages needed
Check the microphone configuration : https://www.pofilo.fr/post/2018/12/16-mic-raspberry-pi/

Step 3: Install the Arduino Client

Programs to manage the Arduino of the robot

Install arduino-cli and arduino avr. Important : the Arduino-cli file must be in the bin folder

<<python package>>/bin/.

The command to install arduino-cli and arduino avr are :

curl -fsSL https://raw.githubusercontent.com/arduino/arduino-cli/master/install.sh | sh
./arduino-cli core install arduino:avr

Step 4: Install the Python Package Biaspeech

Biaspeech package

Biaspeech is the BIA engine core program. It is available online on PyPi.

pip install biaspeech

Run file

Create a python program to run the program. Enter the code as below. Save it as run.py

import os
os.environ['OPENAI_API_KEY'] = "xxxx" # openAI API key
os.environ['OS'] = "raspberry"
from biaspeech import biaspeech

Step 5: Play With It

Basic usage of BIA, app mode :

Open a terminal, then type :

python3 run.py

The BIA app will open, just push the button and ask a question. BIA will answer ... Enjoy :)

Advanced usage of BIA, command line mode :

Open a terminal, then type :

python3 run.py "what can I do a friday afternoon in Paris?"
python3 run.py -help
python3 run.py -version

BIA will answer ... Enjoy :)

Developer usage of BIA, keyboard mode :

Open a terminal, then type :

python3 run.py

Write a question. BIA will answer ... Enjoy :)

Note : the UI parameter under the [main] section must be set to "keyboard".

The config file can be found under :

<<python package>>/utils/config.cfg

Example of some prompt :

Quelle est la capitale de la France?
How are you today?

Step 6: Play With It, Other Options

Advanced prompts :

I do not like this answer => a negative prompt will decrease the scoring of the latest answer, it will go down in the cache
I like this answer very much => a positive prompt will increase the scoring of the latest answer, it will stays up in the cache

Skills. The special prompts are the following :

arduino : ask the otto robot to do something => Example : Arduino dance the salsa
camera : take a picture => Example : Camera now
macro : save the last command as a keyword => Example : Macro salsa
netflix : run netflix and search for a movie => Example : Netflix breaking bad
python : run a combination of prompts => Example : Python if the capital of France is Paris then Arduino walk one meter
radio : open the web radio site and search for a station => Example : Radio deutschlandfunk
tv : open molotov tv => Example : TV m6
web : open a website => Example : Web google.com
youtube : open youtube and search for some => Example : Youtube dire straits

Remote control

You can remote control BIA with VNC, this is explained in the next Instructables.

Step 7: Next Steps

Check those instructables

BIA (AI) on Laptop (#1/3) : the goal of this first instructable is to install and run the program on laptop.
BIA (AI) and Otto the robot (#3/3) : in the third instructable is detailed how Otto the robot can be used