loading

Step 11: Some more about Amazon Mechanical Turk

Background

In 2005, Amazon launched Mechanical Turk, a crowdsourcing (“Artificial Artificial Intelligence”) platform that made it possible to integrate human intelligence directly into software. The service was mostly aimed at businesses in need of quick, cheap, almost-mindless labor. It created a system for easily carrying out bulk tasks that would be difficult for a computer but very easy for a human. However, the platform raises many questions about the ethics and social impact of such a network, and the distribution of power within it. "I make $1.45 a week and I love it"

Artists have engaged with MTurk and explored the questions surrounding it. In Sheep Market, Aaron Koblin paid people $0.02 to draw a sheep facing left, and in Ten Thousand Cents, paid people $0.01 each to draw 1/10000th of a $100 dollar bill. In Social Turkers, I streamed my dates to the internet and paid people to watch and send me suggestions of what to say or do via text. Jeff Crouse’s Laborers of Love the user is directed through a series of questions about their sexual preferences. Once submitted, anonymous workers interpret each of the responses, searching the internet for the image or video they feel best matches, and presenting it to the user. Guido Segni’s Crowd Workers of the World Unite is a collection of 300+ commissioned spontaneous self portraits of cloud workers raising their middle finger.

Testing and troubleshooting workflow

MTurk is not primarily designed for real-time response, and it requires a bit of iteration and experimentation to get good quality results quickly. This blog post has some strategies for optimizing for speed and price, including being clear and explicit with prompts and questions, setting a high enough baseline pay, maintaining a consistent volume of jobs, and closely monitoring workers and responses. However, one of the main takeaways is that factors like time of day or week have much more effect than changes in price or question. Further, random variations in the system not tied to any factor generally outweigh most other factors, so some unpredictability has to be lived with. Also check out this paper from MIT CSAIL on optimizing realtime crowdsourcing, and this write-up on the research behind VizWiz, an app that allows blind people to use the crowd to identify visuals with real-time response.

<p>can you answer me please i am in eighth grade and I am making this for language arts. ASAP</p>
<p>how long did this take you</p>
<p>may price more than 10 k INR</p>
<p>What is the total price of this?</p>
<p>i really like the design. very well made! </p>
<p>ain't there a video for it? i wanna see it moving @_@ please :|</p>
Does someone have the message: &quot;Cannot read property 'HITId' of undefined&quot; when executing app.js ?
<p>Have you ever considered turning this into Wheatley from Portal 2? that is my ultimate goal for this.</p>
<p>I did some computer vision on the raspberry pi. For instance face detection and feature tracking. I am working on improving it until it is not using all of the raspi computation power, which is currently the case.</p>
<p>Great project and instructions.</p><p>Im confused on how the noodle actually processes events. What is the response time? Can I make it automated or does it have to have a real person always?</p>
<p>Response time from Mechanical Turk is around 5 minutes, but if you follow this Instructable in general and then substitute your own Turk form that posts directly you a dtabase/service you have set up, then you can get the response time down much lower (maybe 30 seconds).</p><p>Some basic things can be done automatically (like motion detection or basic speech to text) but we were focusing on the manual things.</p>
<p>Am sorry if i skipped but i found no video of your robot. A video would be really nice.</p>
<p>There isn't a video, mainly because the pictures are more representative :) The video would mostly show it playing sounds or putting pictures on the screen, or doing something more digital with the network.</p>
<p>brilliant!</p>
this is amazing! It never ceases to amaze me what people can do with a raspberry pi - you are a genius
<p>Need a video of the robot please!</p>
Oh,so intreseting
<p>Great job!</p>
<p>Amazing. You should enter in a contest. You'll be sure to win something</p>
Love it.

About This Instructable

98,456views

570favorites

License:

More by lmccart:Noodle: a crowdsourced robot 
Add instructable to: