Mimbo - a Friendly Robot

59,007

93

33

Posted

Introduction: Mimbo - a Friendly Robot

Mimbo is a friendly robot who mimics your emotions. He has a cardboard box body, an iPhone for a face, and a Processing sketch for a brain.
 

Step 1: Build the Body

Mimbo's body is made out of a single sheet of cardboard.

The best way to make the body is to first glue the pattern on the cardboard. Then, using an exacto knife, cut out the pattern. Score the fold lines lightly and valley fold such that the paper pattern is now on the inside and nice clean cardboard is on the exterior.

The files provided are designed for an iPhone 3GS but also work with an iPhone 4/4s with little fuss.

To Design Your Own:
- If you're a student, use Solidworks or Autodesk Inventor to turn your 3D model into a flat drawing with the sheet metal tools.
- Otherwise check out Pepakura

Step 2: Create the TouchOSC Interface

First, go and download the TouchOSC Editor and TouchOSC app for your smart phone (both Android and iPhone supported). You can get them both here. (The app is $5 but well worth it in my opinion)

The editor is super intuitive and quite easy to use. To add a feature just right click and select the object. You can edit the color, position, name, value range, and a number of other parameters by just selecting the object and editing the respective values.

Mimbo's face is composed of 6 features: 4 LED's (two eyes and two pupils), 1 empty label box (eyebrow/eyelid), and 1 multi-fader (mouth). All of these are changed and controlled programmatically in the processing code.

Download my interface or create your own and then just hit the "Sync" button and update the layout on your smartphone. You may need to specify the IP address of your computer but the instructions on the TouchOSC download page are pretty clear.

Step 3: Control TouchOSC Elements From Processing

To talk to TouchOSC using the Open Sound Control protocol you'll first need to download and install the oscP5 library for Processing here.

Once you figure out the basic structure of messages, OSC is a really simple and effective way of talking to all kinds of multimedia devices.

In the setup you'll need to do two things: initialize oscP5 and tell it what port to listen to for incoming messages, as well as declare a remote address (in this case the IP address of the smartphone) and the outgoing port. You can find these two values in the OSC app.

oscP5 = new OscP5(this,8000); // Start oscP5, listening for incoming messages at port 8000
myRemoteLocation = new NetAddress("10.0.1.3",9000); // IP address, outgoing port (taken from the TouchOSC app)

An OSC signal is composed of a message and a value. Let's look at the simple case of turning a TouchOSC LED on and off:

OSCMessage myMessage = new OscMessage("/1/led1"); // <--- where /1 is the first page and /led1 is led1
myMessage.add(1); // <-- assign a value to the message (in this case turn the led on)
oscP5.send(myMessage, myRemoteAddress); // <-- send the message to the remote address

To dim the LED to half of its maximum brightness you just need to send another message exactly as above but with 0.5 instead 1 in the myMessage.add(__).

The oscP5Message example pictured provides a good baseline to work from.

Step 4: Receive OSC Messages in Processing

Mimbo utilizes the accelerometer feature in TouchOSC to send it's acceleration sensor values back to the processing script.

In order to receive OSC messages, an interrupt/event handler method is required. The simplest way to debug OSC event handling is to simply printout whatever is received using code like this:

void oscEvent(OscMessage theOscMessage)
{
     println(theOscMessage.toString());
}


To get the value(s) of a message employ the following command in the oscEvent function:

float myVariable = theOscMessage.get(0).floatValue();

In the case of Mimbo, the OSC Event handler looks like this:

void oscEvent(OscMessage theOscMessage)
{
  // This runs whenever there is a new OSC message
  String addr = theOscMessage.addrPattern(); // Creates a string out of the OSC message
  if (addr.indexOf("/accxyz") != -1) {
    last_accx = accx;
    last_accy = accy;
    accx = theOscMessage.get(0).floatValue();
    accy = theOscMessage.get(1).floatValue();
    accz = theOscMessage.get(2).floatValue();
  }

Step 5: Putting Things Together

Once you've got the basics of sending and receiving OSC messages with processing and a TouchOSC interface on your phone what do you do? You put them together and make Mimbo!

TouchOSC actually allows you to control far more than just the value of the element. You can also (in most cases) control it's color, position, size, etc...

So along with sending a message to "/1/led1" with the LED brightness, we send a message to "/1/led1/position/y" to control it's y-position on the screen.

All of the controls are pretty well documented on this page.

After a whole bunch of tweaking you end up with this code and cute little robot face on your phone.

Step 6: Adding FaceTracking

To take Mimbo to the next level you need have him to track your face and mimic your facial expressions.

Fortunately a lot of the really complex facial tracking problems have been solved for us already. Yay open source! :)

FaceOSC is a pretty simple application (unfortunately OSX 10.6 only) that tracks your face using your webcam and sends OSC messages to a specified port. We can then either read-in these messages directly in processing (on a new incoming port) or re-route them through a program like Osculator to our standard incoming port in Processing. The benefit of routing them through Osculator is that you can very quickly plot whatever signal you want and figure out it's range of values.

Now it's simply a matter of mapping facial expressions (ie. mouth width, eyebrow height) to Mimbo's expressions.

Share

  • sewonist made it!

Recommendations

  • Remote Control Contest 2017

    Remote Control Contest 2017
  • Arduino Contest 2017

    Arduino Contest 2017
  • LED Contest 2017

    LED Contest 2017

We have a be nice policy.
Please be positive and constructive.

Tips

Questions & Answers

33 Comments

adianand

I do not have a iphone so what can i use and send all the Touch osc downloads and other downloads to me on anand_hn@yahoo.com please.i love this project and i love robots

I wish to make one, but I don't have Mac. A complete beginner, not understand the coding, but definitely will try once I understand it. Thank for sharing this. Waiting for Windows-based version too. :)

This is absolutely one way of being creative and saving the mother earth, let us make the most out of our junk, let us recycle.

This just happen to be to be one of the most useful recycling ideas I ever saw on the net. This is useful.

Hi wkl,

Inspired in your Mimbo, roy-t and I built a Windows-based version. We named it Wimbo :-)

We're currently working on an instructable for it. In the mean time: here are some pics...

it runs on a PC with two displays. Roy-t is the programming guy, he wrote the OSC alternative. I did the hardware.

Thanks for sharing!

Hi Guy

It's really nice work.
so I just follow you. I've made like this one.
You can watch the video on my blog (http://sewonist.com/?p=2776)

Cheers

Reminds me of the robot head from Lex!

What do I do with the .pde files

Open them in Processing and hit run. (You can download at http://processing.org/) Oh and be sure to change the IP address to that of your computer and phone in the code as well.