Human%20Interface%20where%20the%20bits%20meet%20the%20flesh%20bits%20meets%20eyes%20and%20ears

1
Human Interfacewhere the bits meet the
fleshbits meets eyes and ears
Think beyond the box

• Tom Zimmerman
• IBM Almaden Research Center
• San Jose, California
• tzim_at_almaden.ibm.com
• October 2007

2
Research Theme

• Transducers Converting one thing into another
• User Interface, User Experience, First Person
• Hooking people to gadgets
• Bits Meet the Flesh
• Bits Meet Eyes Ears
• Transducer Examples
• Dance into Music (dance synthesizer, suction cup)
• Hand into Finger Spelling (DataGlove)
• Sound into Light (Singing Coach)

3
Fantasy Necessity Are the Mothers of Invention
VR Data Glove Personal Area Network
Honda Smart Seat
Violin Home Health Monitoring
Contact Microscope
4
Discrete Events

• Converting continuous input into discrete events.
• Continuous input examples
• brightness, sound level, pressure
• Discrete output 0/1, off/on, mouse click

5
Comparator- Ideal Signal
Sensor
_
Digital Output
Threshold
Threshold
SensorContinuous Input Quantized
Digitized Output
6
Comparator- Real World SignalNoise in input
causes output jitter
threshold
cross high for on cross low for off
7
Triggering Discrete Events with Hysteresis
Hysteresis, play
High threshold Low threshold
Continuous Input (e.g. brightness, sound level,
pressure) Quantized/Digitized Output
cross high for on cross low for off
8
TV Synth Interface
9
Hacking Remotes, Mice, Kbd
Mouse Keyboard IR Remote Synthesizer
T V
Sensor Signal Processing
Switch
Circuit Board
Photo Transistor
60 Hz Filter
Comparator w/ hysteresis
4066Quad CMOS Switch
10
Mouse Sensors

• Left button
• Right button
• X direction
• X increment
• Y direction
• Y increment
• 1 to 100 Hz

• Microphone Speaker Radiation Pattern
• Light Water Ripple Tank
• Ultrasonics Wavelength, Horn Radiation Pattern
• Temperature Weather Station
• Wind speed See Jet Stream, Car Aerodynamics
• Pressure
• Tilt
• Rpm
• Vibration
• Pendulum Simple Harmonic Motion
• Acceleration Seismograph, Earthquakes,
  Cars/Trucks
• Voltage
• Current
• Resistance
• Capacitance
• Magnetic field
• Electric field

11
Wireless Doorbell
12
Wireless Motion Detector
13
(No Transcript)
14
Working with Continuous Output
15
PowerGlove Protoype (VPL 1983)
16
DataGlove (Fiber Optic)
Young Harvill
sensor
source

• Fiber Optic
  Polhemus Magnetic
• Flex Sensors Position Orientation

17
Head Mounted Display(NASA 1986)
18
PowerGlove (Mattel 1987)
Computer Data Entry And
1.3 Million Sold Manipulation Apparatus And
Method 89 Retail

19
The Spirit Chair (MIT 1994)
Mediums use spirit cabinets to guide fields to
contact spirits who make sounds ...
20
Near-Field Communication (MIT/IBM 1995)
Data In
Data Out
21
Contagious Information (MIT 1995)
Fred
7/11
Jan
Bill
22
Car Seat Sensor (NEC 1998)
23
Musical Instruments
Human Actions
Musical Instrument
Pitch Amplitude Timbre Duration
24
Vocalizer (Breakaway 1981)
25
Singing Coach (Exploratorium 1990)
26
RePerformer (Zim/Wantman 1992)
27
Pen Input
28
Pen That Remembers Everything You Write (IBM
2000-2002)
Premiere UI unobtrusive, low cognitive load,
small, portable, quick, cheap
Modeless text, graphics, doodle comment on
document direct manipulation
Universal any language popular input
method classic form factor
BUT hard to index, search, archive, distribute
29
Other Pen Systems
PDA writing on plastic LCD writing
on glass Tablet invisible writing
CrossPad writing on 2 lbs.
30
Toy Pens
31
Pen Research (IBM 2002)
IRDA
Ink
CPU and Flash Memory
Battery (AAA)
CMOS Imager
Lens
Normal or dot code paper
Ultrasonic Optical
32
Electric Field Pen (IBM 1997)
33
Dynamic Signature VerificationSign and Go
(2003)

• Biometric verification using handwriting
  cadence (timing)
• A means to electronically verify customer
  signatures
• Customer signs on a digital signature capture
  station
• Software checks signature against reference
  signatures and checks for major changes in
  signature

34
Biometrics
35
Classification of Biometrics

• Physical Feature
• Iris, face, finger, hand, DNA
• Can alter presentation
• Behavior Trait
• Signature, voice, gait, keystroke
• Can change behavior

36
Identification Verification

• Identification
• Who am I?
• 1N matching (difficult, big search)
• Example 100k customers _at_ 0.1 error 100 errors
• Use Discover identity, Watch lists
• Verification
• Am I who I claim to be?
• 11 matching (easier)
• Requires identifier (not secure but unique)
• token (loyalty card)
• memory (phone number)
• Use Verify identity

37
Performance Metrics

• False Accepts (FA)
• likelihood that someone may be falsely accepted
  bad guy gets in
• False Rejects (FR)
• likelihood that a genuine user may be rejected
  good guy gets rejected
• Lower the FA and FR, the better the system
  performance
• Acceptance threshold trade-off FA to FR
• Equal Error Rate when FAFR
• Failure to enroll, Failure to acquire
• Light eyes (iris), thin skin (finger),
  simple/changing signature, noisy environment
  (voice)
• Need backup/alternative
• Example FA2 FR1
• 2 out of 100 good customers get rejected
• 1 out of 100 bad guy attempts get in
• If 1 in 1000 customers are bad guys, 1 forgery
  per 100,000 transactions

38
Biometric Comparison
Biometric EER Advantage Disadvantage
Iris lt 0.1 Most accurate, no contact Critical alignment, expensive reader
Finger 0.1-10 Wide use, standard databases, competitive market, legal precedence Special reader, criminal association, gummy attack, modest cost reader
Signature 1.7-3 Uses existing POS signature capture hardware, social acceptance, signature dual purpose (agreement biometric) Behavior (requires cooperation), name changes, ergonomics
Voice 2-10 Use with cell phones Sensitive to noisy environment, requires talking
Equal Error Rate estimates for relative
comparison only. Actual results are highly
dependent on system design, equipment, user
population and experience.
39
Tangible Programming
40
Logo Cards
35mm slide holder
41
Students Programming With LOGO Playing Cards
42
Queuing and Scanning Programs into Computer
43
Programming Robots with Playing Cards
The Robosapian robot comes with a remote control
to program robot movements. The remote control
is replaced with a deck of instruction cards and
bar code reader. Students choreograph movements
by arranging the cards, then program the robot
by scanning the cards with the bar code reader.
44
Telepresence
When you need to be there but you dont want
to travel there
45
Dummy Head Electronics (IBM 2003)
46
Digital Head Components
Frame and Handle VGA Camera MegaPixel Camera Tilt
Gimble Yaw Motor Speaker Phone Battery Tiny PC
w/WLAN
Ethernet
Ethernet AC Power
Analog Phone Line
DIGITAL HEAD BASE STATION (x-ray
view) (100 m range to head)
47
Techno Head
Binaural Mics
QVGA Screen
Digital Camera
20 W Speaker
Seat Clamp
48
Head Appearances
Standards Committee Symposium
Trade Show Booth
Breakfast Keynote Board Meeting Talk
Conference Panel
49
Shopping Buddy
Infrared Beacon
Infrared Receiver
Wireless (Bluetooth) Barcode Scanner
WLAN

• Location-aware shopping cart provide real-time
  you are here navigation map and location-based
  content-driven offers such as coupons and
  recipes.

50
Museum Interfaces
51
Sizzling Jello
52
(No Transcript)
53
Bubble Machine
54
(No Transcript)
55
Bubble Strobe Machine
56
Questions???
57
Badge Computer (IBM 1998)
58
NZ Airport Travel Card (IBM 1998)
PDA running XML Browser, linked by 802.11b WLAN
59
Multi-Function PDA (IBM 2001)
Camera with 270 deg swivel Microphones/Structu
red Light
802.11 WLAN antennas running along each side
Color Screen (no touch)
Keypad overlay Touch Pad
FM antenna along perimeter Speakers
60
BioPDA Wireless Biometric Terminal
Fingerprint Reader
InfraRed Lights
Iris Camera
Face Document Camera
Field Replaceable Biometric Module Symbol 8000
Handheld Terminal
802.11b WLAN Intel XScale Processor Numeric
Keypad
61
Using the Biometric Wireless Terminal
Lining up Iris Camera lines up and frames Face
Camera
Capturing Fingerprint
Capturing Face and Iris