Human Computation

1
Human Computation
Luis von Ahn
Carnegie Mellon University
2
Verification technology developed in
collaboration with Carnegie Mellon University
3

Help! Ive been waiting for 20 minutes, and
nothing happens

4
Verification technology developed in
collaboration with Carnegie Mellon University
5
QUICKVOTE
Which is the best Computer Science Grad School in
the US?
MIT
Berkeley
CMU
Washington
Stanford
Cornell
vote
(From www.slashdot.org)
6
Verification technology developed in
collaboration with Carnegie Mellon University
7
CAPTCHA Sweat Shops
Spam companies hire humans to type CAPTCHAs all
day long
2.50 per hour for each human
720 captchas per hour per human
1/3 cent per account
8
VOTE FOR LUIS
2012
9
Free Nude Photos
Type the word in the box if you want to see the
next picture
10
(No Transcript)
11
(No Transcript)
12
(No Transcript)
13
(No Transcript)
14
Roughly 200 million CAPTCHAs are typed every day
10s of human time per CAPTCHA
15
Digitizing Books
Scan
OCR
DE xi Ii ARIi..   COPBCtrAGE\, DeC. I l 1 QOn.
Tlie royal palace at Frieclericheberg took fire
last night. The li.ing himself directed the
operations fns extinguishing the lire. The whole
structure ???, ???????, ed.
16
Word that OCR cant read
Scanned book
Randomly distorted image of word
17
Free
18
(No Transcript)
19
(No Transcript)
20
(No Transcript)
21
(No Transcript)
22
(No Transcript)
23
Words Digitized Per Day
24
(No Transcript)
25
(No Transcript)
26
Two Words Instead of One?
Typing two English words takes as long as typing
6-8 random characters
27
Where do the Words Come From?
Years 1851-1980 will be completed in 2009
28
400,000,000
(6 of the worlds population)
29
CAPTCHA Sweat Shops
Mahiddin Would you help us find some Captcha
Project?
reCAPTCHA Support What kind of CAPTCHA project
are you looking for?
Mahiddin We are manually break different type of
CAPTCHA right now. We have the ability type every
day many Captcha. Tell us how we can help you.
reCAPTCHA Support How fast can you type?
Mahiddin Personally I can type more than 60 WPM.
But I have a big team. We can give you lot of
product.
reCAPTCHA Support Do you mind helping spammers?
Mahiddin No problem, you can tell me details
first.
30
reCAPTCHA Support One last question how do you
deal with IP address blocking?
Please type the following paragraph
31
Reusing Wasted Cycles
Solving CAPTCHAs (1.9s/day)
Bathroom (17m)
Electronic Games (1.1h)
Eating and Drinking (1.1h)
Household Chores (1h)
Sleeping (7.6h)
Caring for Others (1.1h)
Email and IM (1.7h)
TV (2.4h)
Other (7.6h)
32
9 Billion Human-Hours of Solitaire were Played in
2003
Empire State Building
7 million human-hours (6.8 hours of solitaire)
Panama Canal
20 million human-hours (less than a day of
solitaire)
33
Labeling Images With Words
Martha Stewart Flowers Super Evil
Still an open problem
34
Image Search On The Web
Uses filenames and html text
35
Desiderata
A method that can label all images on the web
Fast and cheap
36
Using Humans
Cleverly
The ESP game could label all images on google in
a few weeks
The ESP game
37
The ESP Game
Two-player online game
Partners dont know each other and cant
communicate
Object of the game Type the same word
Only thing in common is An image
38
(No Transcript)
39
The ESP Game is Fun
50 million agreements with 200,000 players
There are many people that play over 20 hours a
week
40
Labeling The Entire Web
5000 people playing simultaneously can label all
images on Google in 2 months!
Individual games in Yahoo! and MSN average over
5,000 players at a time
41
(No Transcript)
42
Search Results
43
Search Results
44
Search Results
45
Search Results
46
(No Transcript)
47
Guesses Over Time
younger than 40
britney
a male
hot
crazy
rehab
shaved head
48
Sample Labels
saddam mr. wilson man face moustache
bush george president man dumb yuck
49
Why do people like the ESP Game?
50
The esp game gives its players a weird and
beautiful sense of anonymous int On the one
hand, you have no idea who your partner is.
51


Strangely addictive

Its so much fun trying to guess what others
think. You have to step outside of yourself to
match

52
Blogs
53

Many fascinating things about the ESP Game
How it uses a game structure to build an index of
images on the web
What words become taboo from frequent use
How interesting some of the images pulled
randomly from the web are
54

Dear Luis von Ahn,
Your game is a fraud
I do research on ESP (extra sensory perception),
and have an online ESP telepathy test on my
website
Please refrain from using the term ESP, as it
makes us legitimate ESP researchers look like a
fraud

55
Audio ESP
SUCKS
56
Two-Player Game
Both players can see each others guesses
Players may or may not be listening to the same
song
Object of the game Tell if youre both hearing
the same thing
Edith Law
57
rock
loud electric guitar
electric guitar
rock
male singer
58
(No Transcript)
59
Crowdsourcing
Language
Translation
60
(No Transcript)
61
Input
Falls Pakistans Geschichte ein Indikator ist, so
könnte Musharrafs Entscheidung, das Kriegsrecht
zu verhängen, jener sprichwörtliche Tropfen sein,
der das Fass zum Überlaufen bringt.
Reference
If Pakistan's history is any indicator,
Musharraf's decision to impose martial law may
prove to be the proverbial straw that breaks the
camel's back.
(20c/word)
If Pakistan's history is an indicator,
Musharraf's decision to impose martial law could
be the straw that breaks the camel's back.
Our Method
62
www.recaptcha.net
www.gwap.com
63
Slide Design Luis von Ahn CAPTCHA Crew Luis von
Ahn Manuel Blum Nicholas Hopper John
Langford The ESP Game Crew Luis von Ahn Laura
Dabbish
Susan Hrishenko Takeo Kanade Mihir Kedia John
Langford Edith Law Bryant Lee Peter Lee Roy
Liu Jitendra Malik Udi Manber Ben Maurer Colin
McMillen Moni Naor Lenore Ramm Roni
Rosenfeld Push Singh
David Stork Latanya Sweeney Jenn Tam Anthony
Tomasic Shingo Uchihashi Manuela Veloso Jeannette
Wing Wardrobe Provided By Charles Spiegel This
Work Has Been Partially Supported
By Intel Microsoft Research Google, Inc The
MacArthur Foundation
Special Thanks To Aditya Akella Josh
Benaloh Lenore Blum Manuel Blum Randy
Bryant Sharon Burks Shuchi Chawla Mike
Crawford Laura Dabbish Alexei Efros Alan
Eustace Bill Gates Shiry Ginosar Severin
Hacker Nick Hopper
reCAPTCHA Crew Luis von Ahn Ben Maurer Colin
McMillen David Abraham Mike Crawford Tag a Tune
Crew Edith Law Luis von Ahn Roger
Dannenberg Speakers Entertainment Provided
By Homestarrunner.Com
64
(No Transcript)