How to manipulate a physical object over the web just by using common web services and their accessible data feeds, with some added open source stuff to decode and manipulate that data, and ultimately use the data to move and control physical hardware.


Twitr_janus - a prototype web-controlled puppet

This Instructable describes Twitr_janus  - a puppet I made to see if it was possible to mash up free digital web services (Twitter, Google Spreadsheets and forms, Netvibes and Skype) with open source hardware and code (Arduino language and environment, Processing and related Processing libraries) and use them to manipulate an object over the web.

It turned out it was indeed possible!  

See how Twitr_janus works and see how you can use these ideas to build your own remotely-controlled physical objects. It was built from cheap, easily available stuff, some of it salvaged. I made a puppet, because I just like weird, creepy things. The principles it demonstrates could easily be applied to control all sorts of other objects you could build yourself.

Here's Twitr_janus in action, describing itself and how it works...

Summary of what it can do...

A puppeteer can remotely communicate over the web using Twitr_janus' data-activated head.

The puppet can:
  • speak tweets sent to its Twitter account
  • speak longer sentences that have been input as text into a field in a Google spreadsheet
  • move its jaw in time with its speech, using a car door-lock actuator (linear motor) controlled by Arduino which converts audio output into control data to trigger lip-synced movement
  • position its remote-control eyeballs with Arduino-controlled micro servos driven by data from fields in the same Google spreadsheet 
  • be commanded from a control interface hosted in a Netvibes page - created by hosting a hacked version of the standard Google input form (made by modifying the form html to restrict the data values, but riding the Google submit script.
  • be woken up remotely over the web with Skype,  to turn on sight and hearing via an HD video camera
  • use the webcam to allow the puppet operator to see what the puppet eye is looking at can see 
  • use the webcam built-in microphone to allow the puppet operator to hear what the puppet can hear
Note - this instructable is a summary of the major steps that were involved in building a working, data-driven physical object. It introduces the concepts and explains the ways its features are made to work, but does not go into minute detail.

More full detailed descriptions of each step are available in posts on my Making Weird Stuff blog 
There are lots of these - too many for an Instructable. Where relevant though, these detailed discussions are linked to on the steps here. 

A very short summary of the project is also available here:

Processing and Aruino code created to make it work is available on GitHub (as straight file downloads). For details see the steps later in this Instructable. Be warned, it's as roughly fashioned as my physical handiwork. Apologies to purist coders. It's freely shared for ideas, but contains some left over functions and snippets that were developed, but not necessarily used. Some were left in the sketches, so copying everything is not recommended. Some of it may be useful. It's built on top of other people's open source stuff so take what you can use.

This project was first shown as to demonstrate a working data-driven object prototype, at the hacking workshop:
"Slack Day" at Museum Computer Network, Seattle 2012.

I'm adding it to Instructables too, as there are loads of people here who might find at least some of it useful. Feel free to hack and modify any ideas here. I learnt a lot doing this from the various open-source communities, especially Arduino and Processing.

Step 1: What you can get from this - the basic elements that make Twitr_janus work

Twitr_janus was an attempt to see if I could hack up web stuff to make a remote control puppet - a sort of physical avatar. It is really just a puppet head, a bit like a ventriloquist dummy, but one that is wired up to the web to make it come alive!

As a bit of hackery, it worked a treat.

In this Instructable I have separated out the various elements that make it work, so you can steal any ideas that help. I have tried to keep it simple here. If anything is of potential use, the more detailed explanations are usually available in posts on my Making Weird Stuff blog. Links are provided here where relevant.

The basic building blocks that make Twitr_janus work are: 
  • The physical head, built from papier mache, and moulded plastic (hot glue in fact)
  • Input sensors for sight and hearing (using a common webcam with built-in mike)
  • servos for controlling eyball and jaw movement
  • LEDs for indicating when web data was being received
  • A connected computer running a programme for
    • listening to the web for command data to control the head (using url parsing in a Processing sketch) by communicating over a serial port with an arduino board
    • allowing text to be spoken (using a text-to-speech library in the Processing sketch)
    • running Skype to allow the head to relay what it can see and hear back to the operator
    • sending audio signals to the arduino
  • An arduino board listening for control data from the mother computer:
    • to convert it into output control data for the eyeball servos
    • to convert analogue audio signals into control signals for the jaw motor
    • to activate separate LEDs to indicate the sources of data (Google RSS and Twitter API)
  • Remote operation using
    • any web connected device capable of running Skype, Twitter and Google Doc. This was possible with an iPad or even iPhone, but was only practical with a full size computer (laptop)
    • a control form embedded in a Netvibes html page, pimped to piggy-back the back of a Google Spreadsheet form submit script
This instructable shows how, in case you ever might be inclined to nick some of the ideas.
The most useful stuff includes:
  • how to strip data from RSS feeds and APIs using Processing to repeatedly listen to the web
  • how to use expressions in Google Spreadsheets to pass special delimited strings to differentiate different data 
  • how to set up a custom html form with pre-set control values, that uses a Google spreadsheet form submit script to   the spreadsheet
  • how to detect new data, but ignore old messages that have been received already
  • how to send the data over a serial port to an Arduino
  • basic servo, relay and LED control with the Arduino
  • text-to-speech conversion using free Processing library
  • audio peak detection with Arduino to trigger a servo to operate the jaw
  • how to remote kick-start a dormant Skype contact to wake up sight and hearing
  • making a lightweight skull using papier mache over a remeable model
  • using hot-glue as a casting material to recreate a face mask model
  • lots of other silly minor details that make it all work
Explanations in this Instructable cover most things briefly. For some components, the full details on the development of specific elements may be via links to posts on my Making Weird Stuff blog. These offer much more detailed explanations.

Twitr_janus is a reflection on how we take for granted "free" web communication tools - handy services like Twitter, Google, Skype - and how when using them, we develop online personas. However these personas that represent our words are usually no more than text boxes on screen with an associated and usually rather dull 2D photo.

Twitr_janus was an attempt to make a real physical representation of online activity. The challenge was to see if I could make such a monstrosity work using the common, but actually quite advanced, data-communication channels and features that web-based services provide for free.

About This Instructable


105 favorites


Bio: I like making all sorts of stuff, out of found materials: furniture, wild food, whatever! I've learnt loads from generous people out there, so ... More »
More by rosemarybeetle: DIY blacksmithing - Converting your barbecue into a forge then using it to recycle scrap metal into tools Making a raised pork pie mould by recycling a stainless steel IKEA bin How to make a cider press with a car jack
Add instructable to: