BlindSight: EyesFree Access to Mobile Phones Kevin A' Li, Patrick Baudisch, Ken Hinckley

1
BlindSight Eyes-Free Access to Mobile
PhonesKevin A. Li,Patrick Baudisch, Ken Hinckley

• Review
• Ken Mikelinich
• University of New Hampshire
• ECE-992

2
BlindSight Eyes-Free Access to Mobile
PhonesKevin A. Li,Patrick Baudisch, Ken Hinckley

• Problems
• Mobile phone conversations are interrupted when
  needing to access other additional stored data
  using the phones visual interface.
• Moving the phone back and forth to the ear
  interferes with the conversation.

3
BlindSight Eyes-Free Access to Mobile
PhonesKevin A. Li,Patrick Baudisch, Ken Hinckley

• Goals
• Seek to replace or augment visual control
  interface with an auditory one.
• Seek a way to avoid look down on mobile phone
  usage.

4
BlindSight Eyes-Free Access to Mobile
PhonesKevin A. Li,Patrick Baudisch, Ken Hinckley

• Background
• Prior research foundation
• Why not use simpler technology?
• Headphones can be socially unacceptable, awkward
  or impact situational awareness.
• Speaker phones have privacy issues
• Audio channel feedback
• Prompt/wait/respond approach annoying
• Drove addition research
• SkipScan --- iterate, ZapZoom
• Time Compressed Utterances -- temporal
  compression
• Word removal interpolation?
• Non-speech audio more
• Tactile feedback cameo mention here
• Mobile input
• Keyboard entry via iteration or chording
• Gesture based EdgeWrite mentioned
• Back surface input LucidTouch mentioned
• Dual purpose speech small vocabularies can work
  well

5
BlindSight Eyes-Free Access to Mobile
PhonesKevin A. Li,Patrick Baudisch, Ken Hinckley

• Hypotheses
• Mobile phone control can occur while place at the
  ear
• Flip category should beat Ear category in terms
  of task time and error rate
• Is it possible to overload the auditory channel
  with feedback even though that channel is already
  in use for human-human communication.
• Experimenters would see a subjective preference
  for blindSight.

6
BlindSight Eyes-Free Access to Mobile
PhonesKevin A. Li,Patrick Baudisch, Ken Hinckley

• Approach
• Introduction of a prototype mobile phone
  application called blindSight that implements an
  eyes-free control plane using an in band auditory
  feedback interface heard only by the user.
• Initial driving survey of 9 participants
  indicated Calendaring and Add-Contact functions
  are key actions for blindSight development.
• This subsequently drove the 3X4 keyboard choice
  too.
• In house participants, 2f/7m

7
BlindSight Eyes-Free Access to Mobile
PhonesKevin A. Li,Patrick Baudisch, Ken Hinckley

• Approach
• Various hw design challenges were needed to be
  addressed
• Pilot studies determined various solutions to
  reduce error rates
• HW button spacing and edges -- discrimination
• Tactile bumps orientation
• FlipPhone -- orientates the keyboard to the back
  side

8
BlindSight Eyes-Free Access to Mobile
PhonesKevin A. Li,Patrick Baudisch, Ken Hinckley

• Approach
• Study 1 (for Hypothesis 1)
• Error rates measured against 3 categorical values
  (visual, ear, flip)
• Visual serves as the control or baseline
• 12 participants (4f, in-house, mobile phone
  users,1 known texter)
• Task enter a randomly selected 10 digit number
• Audio cues for errant entry
• 3 blocks (experiments) for each category
• Results
• ANOVA

9
BlindSight Eyes-Free Access to Mobile
PhonesKevin A. Li,Patrick Baudisch, Ken Hinckley

• Results and Explanations
• Study 1
• Error Rates and Key Press Time

Flip and Ear Not Significantly Different
10
BlindSight Eyes-Free Access to Mobile
PhonesKevin A. Li,Patrick Baudisch, Ken Hinckley

• Results and Explanation
• Mobile phone control can indeed be done while at
  the ear
• Flip did not beat Ear category.
• Perhaps more blocks needs (e.g. training needed)
• 6 of 12 participants preferred using flip over ear

11

• Approach
• Study 2 (for Hypotheses 3,4)
• Blindsight versus smartphone 2003
• A qualitative study
• Participants and experimenter placed in separate
  rooms
• Task use calendar and contact list functions
• Enter a phone number and schedule/negotiate an
  appointment
• smartPhone way -- visual ui
• blindSight way -- audible ui
• Now 8 participants --4 chose Flip, 4 chose Ear
• Two conditions (distractions) injected
• Idle and Driving
• 5 schedule and 4 calendar trials interaced
• 3 blocks per each interface for each distraction
  interface combination
• Used histogram and Likert chart for results

12

• Results and Explanations
• BlindSight vs. Smartphone

13
BlindSight Eyes-Free Access to Mobile
PhonesKevin A. Li,Patrick Baudisch, Ken Hinckley

• Results and Explanation (continued)
• BlindSight specific

14
BlindSight Eyes-Free Access to Mobile
PhonesKevin A. Li,Patrick Baudisch, Ken Hinckley

• High Level Results
• Final study 7 out of 8 participants chose
  blindSight phone over traditional smartphone
  technology

15
When design just isnt enough the unanticipated
challengesof the real world for large
collaborative displaysElaine M. Huang, Elizabeth
D. Mynatt,Jay P. Trimble

• Review
• Ken Mikelinich
• University of New Hampshire

16
When design just isnt enough the unanticipated
challengesof the real world for large
collaborative displaysElaine M. Huang, Elizabeth
D. Mynatt,Jay P. Trimble

• Problems
• People did not use the MERBoards

17
When design just isnt enough the unanticipated
challengesof the real world for large
collaborative displaysElaine M. Huang, Elizabeth
D. Mynatt,Jay P. Trimble

• Goals
• To examine the hurdles and challenges to adoption
  and integration that the MERBoards faced in
  their NASA environment (de-brief)

18
When design just isnt enough the unanticipated
challengesof the real world for large
collaborative displaysElaine M. Huang, Elizabeth
D. Mynatt,Jay P. Trimble

• Background
• Prior Research
• Perceived usefulness of IT
• Laboratory or limited term use of large
  collaborative displays
• Multi-Display environments

19
When design just isnt enough the unanticipated
challengesof the real world for large
collaborative displaysElaine M. Huang, Elizabeth
D. Mynatt,Jay P. Trimble

• Hypotheses
• perceived appropriability must also be taken into
  account in analyzing adoption,
• as availability of a shared technology must be
  negotiated in order for it to be successfully
  integrated into use by a members of a workgroup.

20
When design just isnt enough the unanticipated
challengesof the real world for large
collaborative displaysElaine M. Huang, Elizabeth
D. Mynatt,Jay P. Trimble

• Approach
• Investigate use of large collaborative displays
  used in the NASA Mars Exploration Rover (MER)
  project.
• Complex multi-display systems used in MER
• Seamlessly integrated
• MerBoard was one of these components
• 18 such boards were used in the initial (3m) MER
  mission
• 50 touch-sensitive plasma screens with a
  resolution of 1,600 900 pixels
• Standup interaction w/ finger, stylus or keyboard
• Software
• MS Office suite,
• SolTree PM package
• WhiteBoard
• CIP Portal -- collaborative share portal
• MerSpace,Merdirectory personal machine i/f for
  storage needs
• Many scientist came from other research centers
• Had pre-collaboration work
• No initial co-location work
• Work group collaboration evolved

21
When design just isnt enough the unanticipated
challengesof the real world for large
collaborative displaysElaine M. Huang, Elizabeth
D. Mynatt,Jay P. Trimble

• Approach
• 5 theme groups were identified (science depts.)
• Scientist collaborated in and among the groups
• Each group had a MERBoard
• Mission engineers were a secondary user group,
  but not studied (yet noted for their teamwork
  approach)
• Researchers became more distributed during the
  extended mission scientists went home

22
When design just isnt enough the unanticipated
challengesof the real world for large
collaborative displaysElaine M. Huang, Elizabeth
D. Mynatt,Jay P. Trimble

• Approach
• Year long field study
• 18 scientists and design team members
• On-site and telephone
• Brief visit to JPL

23
When design just isnt enough the unanticipated
challengesof the real world for large
collaborative displaysElaine M. Huang, Elizabeth
D. Mynatt,Jay P. Trimble

• Results
• Long-Term planning group used SolTree
  predominantly
• Other applications were less used such as
  WhiteBoard
• Other groups just did not use the MERBoards much
• Explanations
• The MERBoards were deemed class C nonMission
  critical. They had no direct access to live
  mission data -- perception factor not met
• Class A applications were indeed used
• Scientist were given less instruction time on
  this
• Scientist were too busy or more comfortable with
  familiar technologies
• Remote Display
• VNC connection did not live up to the intended
  use
• Firewall problems (scientist brought their own
  machines)
• Lack of wireless connection
• Unknown capability among the scientists (thought
  they had to use the kludgy MerSpace to move files

24
When design just isnt enough the unanticipated
challengesof the real world for large
collaborative displaysElaine M. Huang, Elizabeth
D. Mynatt,Jay P. Trimble

• Explanations
• Ease of appropriation (not met)
• Not easy to see if groups were using the device
• CIP Portal clock dominated the device
• Suggesting that some participants were unwilling
  to disrupt the use
• Design freeze designers could not readily
  change design during mission