Facilitator Induce Errors: The Effects of Social Facilitation and Electronic Monitoring on Usability

1
Facilitator Induce Errors The Effects of Social
Facilitation and Electronic Monitoring on
Usability Testing

• Erin Harris Jerry Weinberg
• Department of Computer Science
• Sue Thomas Danica Geaslin
• Department of Psychology

2
St. Louis Area Usability Labs
3
Electronic Performance Monitoring

• Increases levels of stress
• Decreases both the levels of productivity and
  quality of work done.

4
EPM Examples

• A study involving 762 telecommunications workers
  found that those who were monitored reported
  higher levels of stress and anxiety.
• Other studies have found that participants
  working on tasks requiring much thought and skill
  performed significantly worse under EPM while
  participants working on simple repetitive tasks
  performed better

5
Social Facilitation

• Social psychology concept that the behavior of an
  animal, including humans, is affected by the
  presence of another.
• An individuals dominant response is enhanced
  while the non-dominate response is impaired.

6
Two Social Facilitation Theories

• Mere Presence Theory
• This theory states that the presence of other
  people alone increases ones arousal
• Evaluation-Apprehension Theory
• This theory goes further to state that it is not
  the presence of another individual alone, but
  also the anticipation that the individual is
  assessing your performance that increases ones
  anxiety.

7
Social Facilitation Examples

• Cockroaches completing a maze
• Simple mazes were completed much faster in the
  presence of other cockroaches.
• Complex mazes were much more difficult to
  complete in the presence of other cockroaches.
• Pool players shot-making ability
• Good players increased their shot-making ability
• 71 to 80
• Less talented players dropped their percentage
• 36 to 25

8
Research Hypothesis

• The presence of a facilitator will increase the
  effects of social facilitation
• These effects will be magnified by viewing the
  observation lab and meeting the design team due
  to the increased salience of the evaluation
• These effects will also be magnified by being
  constantly reminded of the observation

9
SIUEs Usability Lab
10
SIUEs Usability Lab
11
Procedures

• Each subject was informed that they were going to
  be videotaped and observed over a closed circuit
  camera
• Each subject was given a series of 16 Microsoft
  Word tasks 8 simple and 8 complex
• Each task had a time limit
• The number of errors per task was recorded

12
Example Tasks

• Simple Tasks
• Center and underline the title of the document.
• Insert a page break at the end of the document
• Complex Tasks
• Add a center tab stop at the 3-inch mark from the
  left side of the screen.
• When you add a header to Microsoft Word, the
  header is added to all pages of the document.
  Setup Word so that the header of the first page
  of the document is different from the rest of the
  pages.

13
The Experimental Set-up

• Facilitator Group
• Served as control group of the study
• No Facilitator Group

14
The Experimental Set-up

• Facilitator Reminding Group
• Frequent communication with the software design
  team
• Facilitator Design Team Group
• Touring the lab and meeting the design team prior
  to the test

15
Results Mean Errors
16
Results Mean Errors
35.98
35.82
27.98
26.10
17
Results ANOVA for Errors

• Main effect for type of task
• Significantly fewer errors were made on simple
  tasks (M 9.74) than on the complex tasks (M
  53.2)
• F(1, 96) 367.34, p .0001.
• Main effect for group
• The No Facilitator and Facilitator Constant
  Reminding groups committed significantly fewer
  errors than the Facilitator and Facilitator
  Design Team groups.
• F(3, 96) 3.50, p .018.

• The interaction between group and the type of
  task did not reach statistical significance
• F(3, 96) 2.10, p .106

18
Results

• Social facilitation is present in usability
  testing
• Simple tasks were not really simple
• Simple tasks needed to be so well learned that
  they were automatic.
• Constant reminding
• Our constant reminding caused the users to
  explain their actions.
• Providing these explanations caused the users to
  think harder about what they were doing,
  improving performance
• Wright Converse (1992) had similar findings
  when instructing users to provide explanations
  for their actions

19
Putting it into Practice

• Minimize interjection into users awareness
  reduces the errors
• Will this provide a clearer picture of usability?
• Is it preferable to induce more errors?
• Subjects Remarks
• More aware of the observers when I didnt know
  what I was doing.
• Am I going to flunk?

20
Putting it into Practice

• Evaluation apprehension is subconscious
• Regardless of how much you try to put the user at
  ease, the feeling of evaluation is still present
  subconsciously
• How should we interpret results when White Coat
  Response effects performance?
• Subjects Remarks
• Out of sight, out of mind
• Didnt mind having the facilitator there, because
  she had met me, but the other observers made her
  nervous

21
Putting it into Practice

• The social-psychological context of the usability
  test can have a significant impact on a users
  performance.
• Other psychological factors that may impact
  usability results
• Different Lab Set-ups
• Gender differences between the facilitator and
  the user
• Different introductions into the process
• Our study only measured user performance
• Measure the users state anxiety