Connection-less German Speech Recognition and Synthesis With Arduino or Rasberry PI

MOVI stands for 'My Own Voice Interface' and is a kickstarter-backed Arduino Shield that makes it very easy to build your own speech dialogs to control devices within the Arduino IDE. The shield does not need any connection to the Internet, a WiFi, a PC or any other outside server.

You might have seen the featured Instructables on the voice-controlled Christmas tree or the voice-controlled alarm clock.

Unfortunately, until now, MOVI was only available in English. A new firmware update to 1.10, however, allows to load other speech recognition models onto the board in the open-source format produced by Voxforge.org. Among the languages supported on MOVI's website right now are Spanish and German. This Instructable shows how to update and install the German language pack, change the speech synthesizer to German, and what to pay attention to when programming German dialog.

This Instructable assumes basic familiarity with MOVI under Arduino or Rasberry PI. If you haven't used MOVI before, read either the Instructable on MOVI on Arduino or MOVI on Rasberry PI first.

Otherwise: Let's go!

Step 1: Ingredients

As mentioned above, this Instructable assumes your MOVI is setup and running with either a Rasberry PI or an Arduino. On top of that you need:

• A computer that can read and write an SD Card under either Linux, Mac OS X, or a current version of Windows. Since MOVI uses a Micro SDCard, you will most likely need an adapter as shown in the figure above (image curtesy Tomasz Sienicki).

• If you are using Arduino, you need the latest version of the MOVI library. As of writing this Instructable, Rasberry PI does not have a library yet. Download it from here. Or check their website if this link is broken.
• You (probably) need the firmware update to 1.10 or higher from MOVI's website. Download it from here. Or check their website if this link is broken.
• You need the German language pack from MOVI's website: Download it from here. Or check their website if this link is broken.

If you are using Windows you also need to install the following software:

• Disk Imager is free and open source and well-recommended in the Internet of Things community. We will use it to backup the SD card.
• BootICE is a tool that works around a Windows bug that does not allow an SD card to have multiple partitions.

Step 2: Read a Few Words of Caution

The most important thing is to back up your MOVI SD card first before installing the firmware update or adding a language pack. MOVI's SD cards are board specific and replacing them requires you to send in your board to the manufacturer!

Feel free to ask questions below and definitely provide feedback about your experience but this Instructable comes with absolutely no warranty.

As of writing this Instructable, no version of Microsoft Windows supports accessing multiple partitions on an SD Card. This makes the process of backing up and updating MOVI with a Windows computer a lot more cumbersome. Some people might find it easier to use a Linux or a Mac than to go through the procedure under Windows.

in this Instructable, follow all steps, depending on your operating system, e.g. "Windows only". If a step is not marked it is valid for all operating systems.

Step 3: Backup Your SD Card (Linux Only)

• Unplug all power connections, as well as the USB cable from the Arduino.
• Remove MOVI’s SD Card by pressing on it gently before pulling it out. Leave the shield on the Arduino.
• Boot up your computer and put the SD card into it. You may have to use an adapter. If so, put the adapter into 'lock' mode so the SD card is made read-only. Make sure your computer is booted up before you plug the SD card as the SD card is bootable and therefore might confuse your system.
• The best way to back up your SD card is to copy the entire image in one piece. The images are usually created as files with a .img extension.
• Follow these steps:

1. Find the device file, the SD card is associated with with below command
   

   sudo fdisk -l

2. Let's say your SD card is /dev/sdb (just like in our figure) then you can create image using
   

   dd if=/dev/sdb of=./MOVI_Backup.img 

3. If you ever want to restore the data then you can use the same command on an unmounted SD card but with input and output swapped:
   

   dd if=./MOVI_Backup.img of=/dev/sdb 

   
   I recommend restoring to a new SD card right away and then using this backup SD card from now on.You can use any micro SD Card with at least 4GB. So using a new one for every update is an even safer alternative. Store the original SD card in a safe place!

Step 4: Backup Your SD Card (Mac Only)

• Unplug all power connections, as well as the USB cable from the Arduino.
• Remove MOVI’s SD Card by pressing on it gently before pulling it out. Leave the shield on the Arduino.
• Boot up your computer and put the SD card into it. You may have to use an adapter. If so, put the adapter into 'lock' mode so the SD card is made read-only. Make sure your computer is booted up before you plug the SD card as the SD card is bootable and therefore might confuse your system.
• The best way to back up your SD card is to copy the entire image in one piece. The images are usually created as files with a .img extension.
• Follow these steps:

Find the device file, the SD card is associated with with below command

diskutil list

Let's say your SD card is /dev/sdb (just like in our figure) then you can create image using

sudo dd if=/dev/rdisk1 of=./MOVI_backup.img bs=1m

If you ever want to restore the data then you can use the same command on an unmounted SD card but with input and output swapped:

sudo dd of=/dev/rdisk1 if=./MOVI_backup.img bs=1m

I recommend restoring to a new SD card right away and then using this backup SD card from now on.You can use any micro SD Card with at least 4GB. So using a new one for every update is an even safer alternative. Store the original SD card in a safe place!

Another nice tutorial on this step can be found here btw.

Step 5: Backup Your SD Card (Windows Only)

1. Unplug all power connections, as well as the USB cable from the Arduino.
2. Remove MOVI’s SD Card by pressing on it gently before pulling it out. Leave the shield on the Arduino.
3. Boot up your computer and put the SD card into it. You may have to use an adapter. If so, put the adapter into 'lock' mode so the SD card is made read-only. Make sure your computer is booted up before you plug the SD card as the SD card is bootable and therefore might confuse your system.
4. Open Windows Explorer. It shows a disk after inserting MOVI’s SD card but this disk must not be modified in any way! It contains system files vital to MOVI. Changing them will lead to MOVI not working. Placing update files in there will not work either. However, remember the drive letter, here D:.
5. 4. Start DiskImager or a similar tool (see Figure 2) and save an image of the SD card onto your computer. The image will take 4GB of space. Save it in a save spot.
   
   I recommend restoring to a new SD card right away and then using this backup SD card from now on.You can use any micro SD Card with at least 4GB. So using a new one for every update is an even safer alternative. Store the original SD card in a safe place!

Step 6: Put the Update File And/or the Language Pack on the SD Card (Linux/Mac OS Only)

• Once your backup is done, eject (Mac OS X) or unmount (Linux) the SD card and take the card out of the computer.
• Either put the backup SD card into your computer or the original back. However, this time we will write on the card, so make sure the adapter is in 'unlocked' mode.
• The card should mount automatically and the mounted partitions look like in our picture (Mac OS). Under Linux it might be necessary to mount partition 4 (e.g. /dev/sdb4) manually.
• The partition we need is called MOVI UPDATE and is a DOS/FAT32 partition. When the partition is mounted for the first time, the partition should be empty and have a little more than 100MB of space. If you see any files on the partition, you mounted the wrong partition.
• If you don't run MOVI 1.1 yet, copy MOVI's firmware update file onto partition 4. For example with:
  

  cp ~/Downloads/update-101.movi /Volumes/MOVI\ UPDATE/ 

  Obviously, the source and target paths of this command depend on where you downloaded the update file to and you where mounted the SD card partition.

• Now, switch into the directory on the SD card and unzip the German language pack with something like:
  

  unzip ~/Downloads/german-movi.zip

  Again, the location of german-movi.zip will depend on where you stored it.

• If everything went well, an
  

  ls -l

  in the SD card directory should show the files just like in our second screenshot.
  

• Change out of the directory and unmount/eject the SD card.

Step 7: Put the Update File And/or the Language Pack on the SD Card: Step I (Windows Only)

• Once your backup is done, eject the SD card and take the card out of the computer.
• Either put the backup SD card into your computer or the original back. However, this time we will write on the card, so make sure the adapter is in 'unlocked' mode.
• You are familiar with this line: Open Windows Explorer. It shows a new disk after inserting MOVI’s SD card. As said before, this disk must not be modified in any way! It contains system files vital to MOVI. Changing them will lead to MOVI not working. Placing update files in there will not work either. However, remember the drive letter, here D:.
• In order to see the right portion of the SD card, we can write the update file and the language pack to, we need to use BootICE which was mentioned to be downloaded in step 1.
• Open BootICE – Select the MOVI SD card (3.7GB) (the drive letter from the previous step, here D:) and click on Parts Manager.
• Select the MOVI UPDATE partition and click Set Accessible (see screenshots).
• Go back to the Windows Explorer. It must now show an empty drive of about 105MB. It must not show the uImage and script.bin files seen earlier. If you see them, redo and double check this step.

Step 8: Put the Update File And/or the Language Pack on the SD Card: Step II (Windows Only)

If MOVI is not running Firmware 1.1 or later yet, copy the MOVI update file you downloaded earlier (update-101.movi). See screenshot.

Now unzip the german-movi.zip file whereever it currently resides and move the file modeldir as well as the directory German onto the drive. This should occupy about 18MB of space on the drive now.

Go back to BOOTICE and select the MOVI BOOT partition and click Set Accessible to restore the SD Card into it’s old state. This step is very important, otherwise MOVI will not boot. Compare to the screenshots.

Go back to Windows Explorer. You should now see the original partition again wich looks just like the first screenshot of the last step.

Eject the drive and take the SD card out.

Step 9: Boot MOVI

• Re-insert the SD Card into MOVI by gently pressing the card in until it locks.
• Connect speakers to MOVI as it is wise to listen to the messages during the update process.
• Do not connect any USB cable but power up the Arduino/MOVI combination using a reliable external power supply and listen to the status messages on the speakers. The update will be performed automatically. The speakers will let you know when the process is finished.
• It is very important not to unplug MOVI while the update is in progress. Partial updates might make MOVI unusable. If this happens, restore the SD Card from the backup.
• Once the update finished, MOVI will automatically restart.
• With the new models in place, MOVI will take a little longer to boot. You will know that everything has been successful when you hear MOVI saying: "MOVI is ready using alternative Models".

Step 10: Program MOVI

The new language models are loaded and so MOVI will be very confused about the program that is on the Arduino right now as it's speaking a language it doesn't now. Let's fix that.

The good news is that MOVI can be used the exact same way as before except we now have to train German sentences. Also, we need to switch the synthesizer to German if we want German responses.

So the standard light switch program needs to be changed to look like this:

recognizer.addSentence("Es werde Licht");     // Add German sentence 1
recognizer.addSentence("Gute Nacht");         // Add German sentence 2  
recognizer.train();                           // Train (may take 20seconds) 
recognizer.setSynthesizer(SYNTH_PICO,"-l=de-DE"); // Switch the synthesizer to the new one and to German. 

The full program can be downloaded below. Add an LED to PIN13 or just watch the Arduino LED go on and off to German -- just like in the intro video! Congratulations. For tips on how to make MOVI work with German in more complex scenarios, see the next step.

Step 11: Tips and Tricks for Operating MOVI With German

Here are a couple tips that will improve your German experience with MOVI.

• The call sign will still be English. So chose a call sign that's language independent, e.g "Arduino", "Computer", "Mohvee" or something like that.
• Unfortunately, Arduino does not support umlauts as it only supports 7-bit ASCII. Also umlauts cannot be transcribed in the standard way (e.g. ä-$gt;ae). They need to be transcribed sort-of phonetically.

  Umlaut	Transcription	Example
  ä	e	Äpfel -> Epfel
  ö	o, sometimes oi	schön -> schoin
  ü	u	fünf -> funf
  β	s or ss	Straβe -> Strasse

• While the Voxforge.org models have been trained with different speakers, the models aimed at 'hochdeutsch' and therefore will not support regional dialects like Bavarian, Austrian, or Schweitzerdeutsch. Having said that, you can always try to train words of these by yourself, like "Kassa".
• You can also use the old speech synthesizer (espeak) in German. To do that change the synthesizer command from the last step to:
  

  recognizer.setSynthesizer(SYNTH_ESPEAK,"-vde");
  

• You will notice that the German models are a bit slower and not exactly as good as the English models that come with MOVI. The good news is: You can work on that. Check with the Voxforge.org community to improve the model quality.
• If you get tired of German and want MOVI back in English, all you have to do is to long-press MOVI's button to perform a factory reset. You can also do that factory reset programmatically. The factory reset will rename the file called modeldir to modeldir.disabled. So if you want to go back to German again at some point, all you have to do is rename the file back to modeldir by putting the SD card back into your PC and following the steps of this Instructable starting from Step 6.

MOVI has a quite active community on the manufacturer's website. So I recommend checking out MOVI's forum in case you encounter any trouble.

The cool image for this step is from this blog btw. Hopefully, MOVI can help your German practice...