Results Synthesis

1
Results Synthesis

• Phuoc Lam

2
Schedule

• What is it?
• How it works
• Benefits Drawbacks
• Activity

3
What is it?

• Paper pencil based
• Stage within Heuristic Evaluation
• Heuristic Evaluation in 4 stages
• Preparing for inspection
• Inspection itself
• Results synthesis
• Communication to developers designers

4
Definition

• The process of transforming the raw inspection
  data from Heuristic Evaluation into a complete,
  concise, and coherent statement of the problems
  in the evaluated interface as well as recommended
  actions to address the problems identified.
• Cox, 1998

5

• Developed by Donald Cox in his MSc thesis, 1998
  under supervision of Dr. Saul Greenberg
• Prior to this
• Approaches were haphazard
• Problem reports produced often unhelpful

6
How it works

• Collaboratively in small groups
• Four stages
• Preparation
• Familiarizing
• Emergence
• Finalizing

7
Preparation

• Ensure raw data in suitable form
• One written per piece of paper/sticky note
• Paper easily stuck to work surface, grasped
  (singly and in small cluster), and repositioned
• Problem and heuristic stated
• Readable at distance of one meter
• Able to identify different inspectors

8
Preparation (cont)

• Create workspace suitable for activity
• Whiteboard covered with newsprint
• Table and chairs for sit-down discussion and work
• Scissors
• masking tape
• note paper
• sticky notes
• index cards
• markers
• restickable glue sticks

9
Familiarizing

• Group raw data by heuristic
• Stick on work surface in these groups
• Obvious duplicate problems stuck together
• Discussion strongly encouraged
• Review entire data set

10
Emergence

• the notion that fully formed, robust ideas and
  interpretations are not immediately obvious from
  raw data, but come about only after working with
  and discussing the data
• Cox, 1998

11
Emergence (cont)

• Goal SINGLE view of interface problems that is
  complete, coherent, and concise
• Continuously reorganizing data into groups
• Label groups and subgroups
• Through discussion and exploration
• E.g. group by interface component

12
Finalizing

• Create written problem reports
• From each group in workspace
• Good problem reports
• Detailed description in terms of users and tasks
• Description of severity
• Recommended solution

13
Problem Reports recommendations

• Problem proposed solution, but describe
  separately
• Provide justification (both problem solution)
• Assess severity
• Think about trade-offs explicitly
• Careful consideration when adding new
  functionality is proposed in solution
• flag new functionality indicate how critical it
  is for immediate release of product or whether it
  can be held off until later release
• Be aware of your evaluator/inspector bias

14
Benefits

• Produces interface problem reports that are
• Detailed
• Complete
• Coherent
• Concise
• useful to developers designers
• Takes large number of raw problems synthesizes
  into problem reports that report on underlying
  root problems in interface

15
Benefits (cont)

• Eliminates duplicates
• Is natural process only need high level
  description emergence occurs naturally
• Groups of varying sizes compositions

16
Drawbacks Problems

• spatial environment with enough room
• Duplicates in data occur can be non-trivial to
  identify -gt missed
• social process requiring privacy
• Large groups with differing backgrounds can be
  problematic -gt very time consuming

17
Activity

• Taking the raw data produced in the heuristic
  inspection done last class, we will perform a
  results synthesis. Because of time constraints,
  I have prepared the workspace prior to class. We
  will follow these steps in conducting our results
  synthesis
• Split into three groups. Each group will have a
  whiteboard as their workspace.
• Familiarize
• Group all raw data cards by heuristic and stick
  onto workspace, discussing with other group
  members. Obvious duplicates can be stuck
  together.
• Get a global understanding of entire data set,
  with group discussion.
• Emergence reorganize and regroup the data into
  groups that make sense to your group. Discuss
  ideas, explore possibilities, and move the data
  around. Try to come up with a final organization
  where all problems have been placed in a grouping
  or subgrouping.
• Interpretation write a problem report for one
  of the groups or subgroups in your workspace to
  see how this is done. Problem report should
  include
• Detailed description in terms of users and tasks
• Description of severity
• Recommended solution

18
Thank You!!