Text Input to Handheld Devices for People with Physical Disabilities

1
Text Input to Handheld Devices for People with
Physical Disabilities
Human Computer Interaction Institute School of
Computer Science Carnegie Mellon University

• Brad A. Myers and Jacob O. Wobbrock
• Human Computer Interaction Institute
• School of Computer Science
• Carnegie Mellon University
• Pittsburgh, PA 15213-3891
• bam_at_cs.cmu.eduhttp//www.pebbles.hcii.cmu.edu/ass
  istive/

2
Motivation

• Provide text entry for people with physical
  disabilities
• For handheld devices like Palm Pilots
• Also called Personal Digital Assistants (PDAs)
• Also use Palm Pilots as interface to desktop
  computers

3
Part of the Pebbles Project

• Investigate use of handheld devices like Palm
  Pilots and PocketPCs, and Smartphones at the same
  time as PCs and other devices

P E B B L E S
DAs for ntry of oth ytes and ocations
from xternal ources.
http//www.pebbles.hcii.cmu.edu/
4
First Steps

• Support people with Muscular Dystrophy
• Low strength, limited motion, but high accuracy
• Tiny keyboard on handheld screens are good

5
Use Our Remote Commander

• Palm Pilot application to control PC using the
  Palm
• Extend tiny keyboard with other keys
• Word completion
• Other adaptations no auto-repeat, no auto-off
• Mouse control on same screen
• Wireless or wired

6
But what about other disabilities?

• Cerebral Palsy, Parkinsons, Spinal Injury, and
  many others
• Loose accuracy as well as strength
• Tiny buttons are exactly wrong
• Still cant do Graffiti, etc.

7
New Design EdgeWrite

• Added custom template with a square hole to input
  area
• Gain stability from edges
• Travel along edges and into corners
• Designed alphabet usinggestures inside of
  hole,along edges and intocorners

8
Alphabet Properties

• Recognizes based on sequence of corners
• Users can wiggle during diagonals
• Not speed dependent
• Easy to customize and adapt letter forms
• Even user-driven
• Letters feel normal
• Use multiple strokes along edge
• Presentation (only) uses arcs
• No capitalization mode
• Capitals end in upper left
• No special mode for numbers
• I on left, 1 on right

9
Actual Alphabet from User Studies

• Wanted high guessability
• Correlates with initial usability
• Designed a technique to let users guess the
  gestures
• Told users the rules
• Then resolve conflicts if same shape used twice
• Left multiple forms for each letter
• Result guessability improved from 51 to 80

10
Current Alphabet
11
EdgeWrite on Palm

• Novice able-bodied users
• Same speed with Graffiti and EdgeWrite
• But EdgeWrite 18 more accurate
• Handicapped users
• Too few for statistical significance, but
  dramatic
• E.g., accuracy 31 with Graffiti vs. 94
  EdgeWrite
• For example
• The dog is going fast vsThe g i gbsiangu
  fast

12
EdgeWrite on Game Controllers

• Some joysticks use a square hole
• Current techniques are veryawkward
• On-screen keyboard
• Date-stamp cycling letters at each spot

13
EdgeWrite on Wheelchair

• Look at people who use powerwheelchairs
• Custom hardware and softwareto interface to
  commercial joystick
• Informally evaluated with 7 disabled
  power-wheelchair users
• 6 with Cerebral Palsy, 1 with Multiple Sclerosis

14
Touchpad EdgeWrite

• Adaptation that uses elevated edges around a
  Synaptics touchpad
• May be easier for people with motor impairments

15
Results

• Touchpad worked better than wheelchair joystick
• Still pretty slow

16
EdgeWrite on 4 Buttons

• Only need four corners
• Implementation for any 4 keys
• Four keys on the Logitech steering wheel
• Not necessarily for text entry while driving
• Text entry especially for navigation system

17
EdgeWrite with Trackball Pointing Stick

• Newest version
• Many disabled users prefer trackballfor desktop
  pointing
• EdgeWrite for desktop
• Optimized for trackball and pointing stick
• Also works with regular mouse
• Highly parameterized
• Input window can be transparent when in use
• Has help, etc.
• Up to 20 wpm

18
Initial Reaction to Trackball Version

• One user in long-term trial
• Spinal injury, quadriplegic, with some mobility
  of one arm
• Given up on-screen keyboard in favor of EdgeWrite
• Still uses speech entry when possible
• With practice, EdgeWrite is faster
• With an on-screen keyboard, there is just too
  much visual scanning and concentration. In
  EdgeWrite, if you know the letter, you just bang
  it out by feel, and you can keep your eyes on
  your document.

19
Summary and Future Work

• EdgeWrite works across a wide range of devices
• With the same alphabet
• Learn once
• Use most comfortable device
• Future work
• Continue to refine
• Add word completion
• Further tests and deployment
• New devices (phone?)

20
Acknowledgements

• Funded by grants from NSF, General Motors and The
  NEC Foundation of America
• Equipment grants from
• Synaptics, Inc.
• A.T. Sciences
• Microsoft

http//www.edgewrite.com/
http//www.pebbles.hcii.cmu.edu/