Wouldn't it be nice of someone read you a book aloud when you were feeling lazy to read it yourself?
Have you ever wanted to get that Kindle Book into another format, or just copy the text? Have you ever wanted to get all of your highlights or notes off your Kindle?
In this project we make an ebook reading robot with the BrickPi.
We use the BrickPi to control the LEGO Mindstorms to turn the pages, a RaspberryPi camera to take pictures of each page, and the Raspberry Pi to convert the text to speech. The result is an e-book reader that can store text, search for selected text, or read the ebook aloud to you.
Teachers! Did you use this instructable in your classroom?
Add a Teacher Note to share how you incorporated it into your lesson.
Step 1: Parts Required:
Step 2: Introduction
If you want to digitize a book, there are mechanisms available. However, most are too big, too error prone or too expensive for personal use.
The BrickPi Bookreader strikes a balance by using the Raspberry Pi to do the heavy processing and the BrickPi as the interface to the real world, controlling the NXT motors to handle page turning.
To make an automated system that reads a book aloud we need a few tools (some that already exist):
1. A software setup which can take a good picture of a page, perform Optical character recognition(OCR) on the image to convert it to text, and a Text to Speech(TTS) engine that can read the text aloud.
2. A mechanism which can turn each page, automating the system.
For step 1, we have some open source software that does the job very well. With the Raspberry Pi as the brains of the operation and the official Raspberry Pi camera as its eyes, the size of whole setup is considerably smaller than Google’s.
The second part is a bit tricky. When we started, we assumed there were be some decent projects out there that did the page turning.
So after digging around we found nothing so we set out to build our own. The biggest challenge of the project was creating a good page turning mechanism. After some research we found the Google Book Scanner, which turns the pages well but is beyond the scope of almost anyone but a corporation. We also found Scanbot, which works well but requires a lot of moving parts with a lot of precision timing. Building the contraption with LEGO’s is naturally easier.
Instead of diving straight into building a mechanism for turning the pages on a physical book, we decided to build a platform which could read from the Kindle app on a Nexus 7.
Step 3: Setting Up the Camera
The first thing to get our Bookreader up and running is to get the Raspberry Pi camera up and working. The Raspberry Pi camera packs a lot of punch, there are a lot of options, it’s easy to set up, and the image quality is acceptable for our project.
After connecting the camera, there is one more thing to do: change the focus of the Raspberry Pi camera. The Raspberry Pi camera comes with its focus fixed at infinity, and since it is a fixed focus camera you have to manually change it. Here are some helpful links to do focus the camera:
Step 4: Testing the Camera
After setting up the camera, take a test image to see that it is properly focused. In rig we built, we have the camera about 10.5 inches above the tablet (choose a height which is comfortable for you and take a few test images to check if the images are clear and the whole screen of the tablet is captured).
Now fix the camera into it’s adapter next to the Ethernet Jack. Here is a great guide to setting up the Raspi Camera. It should be helpful in setting up bot the hardware and software.
After the camera is set up , test it to see if it works:
raspistill -o image.jpg
If the camera is initialized properly is you’ll see a new file image.jpg in your present folder. Open it to see the image.
Now secure the camera at the desired height and place your tablet or book under it. Take an image. You may need to readjust the focus of the camera and angle at this point.
Black text on white background works the best so select that from the text options and keep the text size sufficiently large. The larger the text is, the better results will be from OCR.
Step 5: Setting Up the Text to Speech
First test if the audio is working on the Raspberry Pi. Plug a headphone or speakers in the audio jack and run the following command:
If you are able to hear the sounds, move to the next step! If not, this tutorial may help you setup the audio.
Next, install espeak. Run the following in terminal:
sudo apt-get install espeak
after it successfully installed, run the following command. (disregard error messages on the terminal if you can hear sound):
If you are able to hear “hello” from the headphones or the speakers then move to the next step.
Step 6: Installing the Optical Character Recognition (OCR) Engine
The OCR engine converts the image file we take of the book into text. We are using Tesseract OCR Engine. It runs well on the Raspberry Pi, it does not require an online connection, and it reliably converts images to text.
First, install tesseract:
sudo apt-get install tesseract-ocr
Next, test the OCR engine.
Take a good image of a piece of text, in a Book or from an ebook and run tesseract:
tesseract image.jpg o
where image.jpg is the image which was taken by the raspberry pi camera and o is the file in which the text will be saved(tesseract will make it o.txt so no need to add the extension).
Now, wait a few minutes, the OCR takes a lot of processing power.
When its done processing, open o.txt. In our experience, the recognition was >90% and works better with larger font size. If the OCR did not detect anything at all, try rotating the image and running the tesseract again.
Step 7: Building LEGO Platform and the BrickPi Mount
By taking pictures a few times from various height’s we found that we needed the camera above 10 inches for getting a good and clear image of the text.
Our first build was not elegant: we just looked for LEGO blocks to build a platform on which we can place the Brick Pi casing along with the battery pack and the camera (we cheated with a little bit of tape to fix the camera in the correct position).
Step 8: Building the Robotic Arm
Probably the hardest part of this build is to get a robotic arm to turn the pages on a capacitive touch screen.
After testing a couple of materials, the best candidate for a capacitive stylus was a bit of antistatic foam (the one that comes with IC’s from Mouser!). You can also use normal kitchen sponge, but it has to be moist, so just put a few drops of water on it. We found performance improved with a conductive wire with the foam and wrap it around the arm. We also found that it helped to connect a 47 nF capacitor between the foam and the ground on the BrickPi.
Attach the piece of foam on an arm stretching out from the motor and connect the capacitor to it. We also attached a small LEGO block, tilted, which helps making an easy contact with the touch screen.
Place the tablet in a proper position so that the arm is able to move up and down properly and turn the pages on the screen.
On the Github repo, there is a test code called arm_test.py to help you calibrate the arm for perfect movement. Just connect the motor to Port C of BrickPi and change the values of ‘t’ and ’sp’, until you get flawless movement.
Step 9: Putting It All Together
With the camera and all software’s working and the platform ready with the motor arm, it now time to bring the Bookreader to life.
Make sure that the camera is calibrated to take focused images of the tablet here with the text clearly visible and that the motor mount is moving correctly and turning the pages.
Stay tuned, we'll be releasing a BrickiPi Real Book Reading bot very soon.