Video Logs for Reflection: Experiences with a New Approach to Teaching Usability Testing

1
Video Logs for Reflection Experiences with a New
Approach to Teaching Usability Testing

• Sharon McDonald and Tingting Zhao
• University of Sunderland

2
Project Objectives

• Create a video on the Heuristic Evaluation
  Technique.
• Create a number of video logs of students own
  usability tests
• Facilitate reflective learning through log sharing

http/evil.sunderland.ac.uk/
3
Project Involved

• Students working in small groups designing
  usability tests.
• Implementing the tests in the usability lab
  (making a video record of the test).
• Reflecting on their videos and sharing them with
  the whole class for further reflection.
• Assessed work involved a written individual
  reflection on the process of conducting their
  tests.

4
Working in the lab

• Recruited own participant
• Obtained written consent
• Produced two files
• Lab Camera
• TechSmith Screen and user image plus voice

5
Usability Lab Structure
6
Reflection

• Preceded by lots of practice
• Videos, role play, and Moons (2004) resources
• Reflective questioning
• Different methods to facilitate reflection
• Guided tutor participated as an active observer
  during the first test for each group.
• Team during and immediately after the test.
• Individual watching lab recordings in own time.
• Shared videos watched in class with reflective
  discussion

7
Evaluation

• Nominal Group Technique
• Addressed the whole module
• Questionnaire (response rate gt80)
• Focused on learning experience and use of
  reflection, mainly Likert type questions
• Repertory Grid Interviews with 12 students
• Provided Elements Teaching and Learning
  approaches
• Elicited Constructs in relation to the student
  learning experience

8
Results

• Pass rate 100 (10 firsts, 20 upper seconds, 6
  lower seconds, 5 third class).
• Students rated the learning experience as being
  extremely positive.
• Students enjoyed the shared reflection activities
  and believed them to be highly beneficial.
• Students believed the project gave them a more
  realistic insight into the world of usability
  testing.
• Reflection was a concern for students.

9
Results Attitudes to reflection (written and
class based)
Students were asked to indicate how comfortable
they felt about class room and report based
reflection.
10
Perceived Barriers to written reflection
What were your concerns about written reflection?
11
Perceived benefits of shared reflection
12
Reflection Types
How useful were the different reflection methods?
13
Staff Feedback

• Seemed to inspire and motivate the student group.
• Videos provided a convincing way to demonstrate
  testing pitfalls.
• Increased staff resource
• Watching and editing recordings from each group.
• Supervising the Usability Laboratory
• Student highlight videos tended to focus on
  similar, easier to spot aspects.
• Website based reflection negative response

14
Conclusion

• Useful Activity
• Proved to be an engaging and fun activity for the
  students
• Videos highlight the good and the bad of testing
• Reflection is daunting
• Students need to be prepared effectively
• Preparation to reflect can become a dominant
  activity