Drive a Roomba Robot From Grasshopper Using Computer Vision

Roomba robots are amazing and we all know that. There are quite a few tutorials out there on how to hack them and millions of different ways to control them. As i work a lot with Grasshopper and Rhino I decided to build my own system for driving the robot directly from Grashopper.

In this Instructable i will show two methods(two Grasshopper definitions) for driving the robot. The first and simple one is driving a robot by giving it a single vector defining the robots orientation.

The second method requires an external camera setup. It allows to specify desired robotic position in the digital Grasshopper space and the robot would go to that point in the physical world.

Both methods send simple commands containing wheel velocity to the robot. This method could be used for other robots that employ similar steering system as Roomba does.

Step 1: For This Project You Are Going to Need...

Hardware:

- a Roomba robot. In my case i am using a Roomba 651, but this system works for most Roomba models

- a Bluetooth modem. I am using quite an old model Sparkfun BlueSMiRF-v2, but it should also work with newer ones. The updated version of the same modem is BlueSMiRF Gold.

- a breadboard

- some jumpers

- a voltage regulator ET7805

- a webcam. I am using a Microsoft LifeCam HD-3000, but it can be any camera. The higher the resolution, the bigger your scene can be.

- some fiducial markers printed out ( you can find some here: http://reactivision.sourceforge.net/data/fiducial...)

- a cable with a Mini-DIN7 connector or 4 jumper cables (I am using jumpers)

Software:

- Rhino5 with the latest version of Grasshopper (download here: http://www.grasshopper3d.com/page/download-1)

- Firefly plugin for Grasshopper (download here: http://www.food4rhino.com/project/firefly?etx)

- Reactivision software (download here: http://reactivision.sourceforge.net/)

- Python 2.7.10 (https://www.python.org/downloads/release/python-2710/) check that you have a pyserial library (http://pyserial.sourceforge.net/)

- Putty (download here: http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html)

Step 2: Preparing Hardware

Once we have all the hardware we need, it is time to wire it together (yay!).

First, in order to reach the Serial port in the Roomba, you will need to remove the top panel. It comes of fairly easy, you need to first remove the vacuum part and then just lift the top panel and it peels off. There is a great video demonstration on how to do that here: https://vimeo.com/55159816

Once you have access to the serial port, you can begin setting up the Bluetooth communication.

For this project we are only going to need 4 out of 7 pins in the Roomba data port. For the Bluetooth control module we are going to need RXD (pin3), TXD (pin4), Vpwr (pin 1 or 2) and GND (pin 6 or 7).

Connect the electronics according to the wiring diagram.

Using double sided tape or other adhesives, attach 2 fiducial markers to your Roomba in a way that they are at an equal distance from the center of the robot

Step 3: Set Up a Bluetooth Connection With the Roomba

Once all the wiring is complete, you can set up the Bluetooth connection between the Roomba and the computer.

Go to Control Panel, Devices and Printers, Add a Device. A window with all available devices pops up, you need to find your robot. When you conect it for the first time, it will most probably be called "Other". Connect the device using tits pairing code (for Sparkfun modems it is usually 0000 by default).

Once you connect to a device, be sure to check which COM port it is now allocated to. To do that right click on the device, go to Properties, Services.

To check that the robot is connected and correctly wired you can do the following:

1 - Turn the robot off

2 - Open Putty software

3 - Try connecting to the Serial port that the Bluetooth device is allocated to (in my case it is COM 10)

4 - Once the connection is established, a black window will pop up.

5 - If you turn your Roomba on now, you are supposed to see a message in your Putty window, providing basic information about the robot (see image 5)

Step 4: Setting Up the Connection Between Grasshopper and Roomba and Driving the Robot

In order to do this step you need to download the Grasshopper definition and the Python file.

1 - Open the file Bridge_Roomba_Grasshopper.py for editing (right click on the file and click on "edit with Notepad")

2 - Before you run the file you need to change a few things.

- line 41 double check that the number of the COM port matches the COM port your Roomba is attached to.

- line 55 replace 'C:\\Users\\maw\\Desktop\\' with a directory on your computer. It can be any folder.

3 - Save changes in the file, close the Notepad and run the file (double click.)

If everything is connected correctly, you should see a black window saying "Connected!", it also displays the last message it received (if there was any) and a current mode.

4 - Now you can open the Grasshopper file RoombaDriveWithAVector.gh. Here we also need to do a few changes:

- change path to a directory on your computer. Paths in GH file and in the PY file need to match.

6 - Set Roomba to a correct mode. Roomba has 3 modes: Passive, Full, Safe. We can only control the robot when it is in full mode. To do that, change the slider "mode" onto position 3. Sometimes you have to switch from position 1 to 3 a few times to make sure the robot is in the correct mode.

7 - To check if the robot is connected, you can click the Beep button. The robot is supposed to make a sound as you click.

8 - Now that everything is set up, you can finally drive the robot! Reference the Grasshopper point to a point in Rhino (right click, Set one point) Now by dragging this point around in the Rhino window you can control your Roomba!(see video)

Step 5: Driving a Robot by Giving It a Target Point

Now that we can drive the robot from Grasshopper, lets make it a little bit more complex.

For this step download a new GH file 'DrivingRoombaWithaTargetPoint.gh'

1 - Set up your camera above your working space in a way that you can see as much of the space where you are driving your robot as possible.

3 - Run Reactivision software (reactivision.exe). You are supposed to see the camera feed from your webcam in the reactivision. The fiducial markers in the feed are supposed to be represented with green marks and numbers. Make sure the lighting in the room is adequate and the markers are big enough for the software to recognize them. Sometimes it helps to print markers 2-3 times their original size. It looks funny, but that can allow you to increase your active camera space.

4 - Open the GH file 'DrivingRoombaWithaTargetPoint.gh'

5 - Same as in the previous step, change the path

6 - Reference the Grasshopper point to a point in Rhino (right click, Set one point)

7 - Set Roomba to a correct mode. Roomba has 3 modes: Passive, Full, Safe. We can only control the robot when it is in full mode. To do that, change the slider "mode" onto position 3. Sometimes you have to switch from position 1 to 3 a few times to make sure the robot is in the correct mode.

8 - To check if the robot is connected, you can click the Beep button. The robot is supposed to make a sound as you click.

9 - Now by dragging this point around in the Rhino window you can control Roomba's position in the camera's feed! (see video)