Introduction: Mimbo - a Friendly Robot

About: I'm a mechanical engineer with a passion for making things.
Mimbo is a friendly robot who mimics your emotions. He has a cardboard box body, an iPhone for a face, and a Processing sketch for a brain.

Step 1: Build the Body

Mimbo's body is made out of a single sheet of cardboard.

The best way to make the body is to first glue the pattern on the cardboard. Then, using an exacto knife, cut out the pattern. Score the fold lines lightly and valley fold such that the paper pattern is now on the inside and nice clean cardboard is on the exterior.

The files provided are designed for an iPhone 3GS but also work with an iPhone 4/4s with little fuss.

To Design Your Own:
- If you're a student, use Solidworks or Autodesk Inventor to turn your 3D model into a flat drawing with the sheet metal tools.
- Otherwise check out Pepakura

Step 2: Create the TouchOSC Interface

First, go and download the TouchOSC Editor and TouchOSC app for your smart phone (both Android and iPhone supported). You can get them both here. (The app is $5 but well worth it in my opinion)

The editor is super intuitive and quite easy to use. To add a feature just right click and select the object. You can edit the color, position, name, value range, and a number of other parameters by just selecting the object and editing the respective values.

Mimbo's face is composed of 6 features: 4 LED's (two eyes and two pupils), 1 empty label box (eyebrow/eyelid), and 1 multi-fader (mouth). All of these are changed and controlled programmatically in the processing code.

Download my interface or create your own and then just hit the "Sync" button and update the layout on your smartphone. You may need to specify the IP address of your computer but the instructions on the TouchOSC download page are pretty clear.

Step 3: Control TouchOSC Elements From Processing

To talk to TouchOSC using the Open Sound Control protocol you'll first need to download and install the oscP5 library for Processing here.

Once you figure out the basic structure of messages, OSC is a really simple and effective way of talking to all kinds of multimedia devices.

In the setup you'll need to do two things: initialize oscP5 and tell it what port to listen to for incoming messages, as well as declare a remote address (in this case the IP address of the smartphone) and the outgoing port. You can find these two values in the OSC app.

oscP5 = new OscP5(this,8000); // Start oscP5, listening for incoming messages at port 8000
myRemoteLocation = new NetAddress("10.0.1.3",9000); // IP address, outgoing port (taken from the TouchOSC app)

An OSC signal is composed of a message and a value. Let's look at the simple case of turning a TouchOSC LED on and off:

OSCMessage myMessage = new OscMessage("/1/led1"); // <--- where /1 is the first page and /led1 is led1
myMessage.add(1); // <-- assign a value to the message (in this case turn the led on)
oscP5.send(myMessage, myRemoteAddress); // <-- send the message to the remote address

To dim the LED to half of its maximum brightness you just need to send another message exactly as above but with 0.5 instead 1 in the myMessage.add(__).

The oscP5Message example pictured provides a good baseline to work from.

Step 4: Receive OSC Messages in Processing

Mimbo utilizes the accelerometer feature in TouchOSC to send it's acceleration sensor values back to the processing script.

In order to receive OSC messages, an interrupt/event handler method is required. The simplest way to debug OSC event handling is to simply printout whatever is received using code like this:

void oscEvent(OscMessage theOscMessage)
{
     println(theOscMessage.toString());
}


To get the value(s) of a message employ the following command in the oscEvent function:

float myVariable = theOscMessage.get(0).floatValue();

In the case of Mimbo, the OSC Event handler looks like this:

void oscEvent(OscMessage theOscMessage)
{
  // This runs whenever there is a new OSC message
  String addr = theOscMessage.addrPattern(); // Creates a string out of the OSC message
  if (addr.indexOf("/accxyz") != -1) {
    last_accx = accx;
    last_accy = accy;
    accx = theOscMessage.get(0).floatValue();
    accy = theOscMessage.get(1).floatValue();
    accz = theOscMessage.get(2).floatValue();
  }

Step 5: Putting Things Together

Once you've got the basics of sending and receiving OSC messages with processing and a TouchOSC interface on your phone what do you do? You put them together and make Mimbo!

TouchOSC actually allows you to control far more than just the value of the element. You can also (in most cases) control it's color, position, size, etc...

So along with sending a message to "/1/led1" with the LED brightness, we send a message to "/1/led1/position/y" to control it's y-position on the screen.

All of the controls are pretty well documented on this page.

After a whole bunch of tweaking you end up with this code and cute little robot face on your phone.

Step 6: Adding FaceTracking

To take Mimbo to the next level you need have him to track your face and mimic your facial expressions.

Fortunately a lot of the really complex facial tracking problems have been solved for us already. Yay open source! :)

FaceOSC is a pretty simple application (unfortunately OSX 10.6 only) that tracks your face using your webcam and sends OSC messages to a specified port. We can then either read-in these messages directly in processing (on a new incoming port) or re-route them through a program like Osculator to our standard incoming port in Processing. The benefit of routing them through Osculator is that you can very quickly plot whatever signal you want and figure out it's range of values.

Now it's simply a matter of mapping facial expressions (ie. mouth width, eyebrow height) to Mimbo's expressions.

ShopBot Challenge

Participated in the
ShopBot Challenge

Toy Challenge 2

Participated in the
Toy Challenge 2