SEG 3120 Analysis and Design of User Interfaces

1
SEG 3120 Analysis and Design of User Interfaces

• Tutorial 1

2
TA Introduction

• Abdel wahab Hamam
• Email ahamam_at_site.uottawa.ca
• Masters student at the discover lab
• Site 5077, ring bell.
• Meetings by appointment only
• Include in email subject SEG3120

3
Videotaped Evaluation

• Definition A recorded (videotaped) study of
  users actively engaging a user interface,
  performing tasks that might be predefined, in a
  series of steps that might also be predefined,
  usually following the thinking-out-loud
  protocol.

4
Videotaped Evaluation

• Motivation Videotaping is an evaluation method
  that addresses
• Self-fulfilling prophecies
• Heat-of-the-moment effect
• Attention to minor details
• Advantages of repetition
• Purpose
• Minimizing Malfunctions.

5
Videotaped Evaluation

• Procedure
• Select 6 to 8 representative users per user class
• E.g. client, salesperson, manager, accounts
  receivable
• Invite them to individual sessions
• Sessions should last 30-90 minutes
• Schedule 4-6 per day
• If system involves user's clients in the
  interaction
• Have users bring important clients
• or have staff pretend to be clients

6
Videotaped Evaluation

• Select facilitators/observers and notetakers
• Prepare tasks
• Select the most commonly used tasks plus a few
  less important tasks
• Write task instructions for users
• Estimate the time it will take to complete each
  task plus extra time for discussion
• Prepare notebook or form for organizing notes

7
Videotaped Evaluation

• Set up and test equipment
• Hardware on which to run system
• Audio or video recorder (one or more?)
• Software logs
• Do a dry run (pilot study)!

8
Videotaped Evaluation

• At the Start of an Observation Session
• explain
• nature of project
• anticipated user contributions
• why user's views are important
• focus is on evaluating the user interface, not
  evaluating the user
• all notes, logs, etc., are confidential
• user can withdraw at any time
• usage of devices
• relax!
• Sign informed consent form
• very important

9
Videotaped Evaluation

• Start user verbalizing as they perform each task
  (thinking aloud)
• For co-operative evaluation, software engineer
  also verbalizes
• Appropriate questions to be posed by the
  observing software engineer

10
Videotaped Evaluation
11
Videotaped Evaluation

• Hold a wrap-up interview
• What were the most significant problems?
• What was most difficult to learn?
• Etc.
• Analyze the videotape to find malfunctions

12
Malfunction Analysis

• A disciplined approach to analyzing malfunctions
• Provides feedback into the redesign process.
• Identify Malfunction
• Answer four distinct questions
• Q1 How is the malfunction manifested?
• What do you notice and who noticed it?
• Q2. At what stage in the interaction is it
  occurring?
• Goal forming, action decision, action execution,
  interpretation of results

13
Malfunction Analysis

• Q3. At what level of the user interface is it
  occurring?
• Physical element level to task level
• Q4. Why is it occurring?
• What is its root cause
• 3. List and prioritize possible cures

14
Examples of Videotaped Evaluation

• http//www.betterdesktop.org/wiki/index.php?title
  Main

15
Assignment 1 example
16
Assignment 1 example

• Compute students final marks (numerical and
  symbolic marks).
• (for symbolic mark use U of Ottawa scale (i.e.
  90-100 A, 84-90) A, etc.)
• for the numeric mark follow the course schema
  (15 assignment 1, 5 assignment 2, 20
  assignment 3, 20 mid term exam, and 40 finial
  exam).
• Calculate the average values for the given
  numerical column.

17
Assignment 1

• Choose a software product to evaluate
• You can use any software as long as
• You are at least somewhat familiar with it.
• You can find one or two subject users who are not
  expert
• You can run the software in an experimental
  manner (i.e. not a nuclear power plant!)
• It could be something you are working with,
  developing etc.
• It must involve some data input as well as
  output (does not have to be a GUI)

18
Assignment 1 Software Aspect

• Choose an aspect of the software to focus on
• Pick just a part of a system if it is complex.
• The UI that implements one to three tasks that
  together can be done by an expert in about 20
  minutes.
• Ideally you will work with a highly used or more
  complex part of a system
• As the evaluators, you should become familiar
  with the part of the software you will evaluate
• Do not forget to sign the Informed Consent Form
  Available on the course web site

19
Assignment 1- Cooperative Evaluation

• Perform a co-operative evaluation and malfunction
  analysis
• Follow (in detail) the procedures outlined in
  module B
• Remember to do a short dry run (pilot study) so
  you become comfortable with the procedures and
  A-V equipment. The dry run must use a different
  task from the main session.
• Remember that co-operative evaluation requires
  both you and the subject user to verbalize.
• Your subject(s) should not be someone intimately
  familiar with the software (i.e. not a designer)
  however the subject should know or be taught the
  basics of the system

20
Assignment 1- Cooperative Evaluation

• Total videotaping time should be 20-30 minutes
• Videotape the session (the TA will help with
  this if needed)
• You can arrange to borrow cameras from A-V
  services, although if you do your study with the
  TA, I can take care of this for several groups at
  once.
• Do not hand in the tape with your report, but
  keep it in case the professor wants to see it.
  Erase the tape once you get your mark.

21
Assignment 1- Report

• A description of the software being evaluated in
  sufficient detail so the marker
• can understand the rest of the report (2 marks).
• A summary of the procedures you used to do the
  evaluation (3 marks)
• When, where and how did you set up the software
  to be evaluated?
• What did you ask the subject to do?
• What happened as the evaluation proceeded?
• Here you should provide sufficient detail so the
  marker can see that you followed good procedures
  and handled procedural problems well .This is not
  the protocol (transcript)
• A complete list of malfunctions that you found (1
  per line) (5 marks)
• A discussion of the four most significant
  malfunctions (5 marks). For each
• provide at least the following
• An excerpt of the protocol
• i.e. a verbatim transcript of 5-15 lines
  describing what the user did and said, what you
  said and
• what happened (around the time the malfunction
  occurred)
• You can embellish this with a picture
  illustrating the malfunction if this makes it
  clearer
• The results of malfunction analysis
• Brief recommendations for changes

22
Thank You
Questions