The role of eye tracking in usability evaluation of LMS in ODL context

1
The role of eye tracking in usability evaluation
of LMS in ODL context

• Mr Sam Ssemugabi
• Ms Jabulisiwe Mabila
• (Professor Helene Gelderblom)
• College of Science Engineering and Technology
• University of South Africa
• School of Computing

2

• Technology has been a part of our everyday
  environment
• for generations. It empowers us and frustrates
  us, it
• simplifies and complicates our life. It separates
  us and
• brings us closer together. But even though we
  interact with
• technology everyday, we easily forget that
  technology
• products are made by people, and that someone,
• somewhere should get the credit when technology
  works
• well for us or get the blame when it doesnt.
• (Garrett James, 2003 7)

• Introduction
• - Eye Tracking and Usability
• - LMS ODL

3
The role of eye tracking in usability evaluation
of LMS in ODL context

• Introduction
• - Eye Tracking and Usability
• - LMS ODL
• Objective of the study
• Tasks
• Results
• Conclusion

• Introduction
• Problem statement
• Methodology
• Tasks
• Results
• Conclusion

4
The role of eye tracking in usability evaluation
of LMS in ODL context
Eye tracking is the technique whereby an
individuals eye movements are measured
so that the researcher knows both where a
person is looking at, at any given time and
the sequence in which the eyes are shifting
from one location to another .
(Poole and Ball,
2006)

5
Eye tracking metrics

• Heat maps
• Fixations
• Saccades
• Eye gaze duration
• Area of interest scan path
• (AOI)

• Nielson, 2006

6
The role of eye tracking in usability evaluation
of LMS in ODL context
Usability defined as the extent to
which a product can be used by
specified users to achieve specified
goals with effectiveness, efficiency
and satisfaction in a specified
context of use.
(ISO 9241-11,
1998)

7
The ODL Context

• Diversity of student profile in ODL institutions
• Young and old
• Employed and unemployed
• Rural areas and urban areas
• Diverse student profile - widely varying ages,
  experience,
• backgrounds - language, location

8
Learning Management System (LMS)
A LMS software application includes programs for
e-learning, administration, documentation,
tracking, and reporting, training programs and
online events (Ellis, 2009).
9
Usability assessment of HCI

• Include
• Expert evaluation
• Heuristic evaluation
• Walkthrough
• Observation
• Retrospective self report
• (questionnaires, think aloud)
• 
  (Holzinger, 2005)
• Other techniques Eye tracking

10
The role of eye tracking in usability evaluation
of LMS in ODL context
Why?

• The role of eye tracking
• Usability assessment LMS
• ODL context diversity of student population

11
Participants
Who?

• First year Unisa students
• Registered for End User Computing
• (EUC131T)

12
Methodology

• Instruments
• Tobii Eye Tracker 1750
• Video recording
• Questionnaire
• Observations

Tobii 1750 eye tracker
Unisa , Pretoria School of
Computing Usability Laboratory

13
Task - Objective

• Navigation
• Reading
• Text input

)
www.unisa.ac.za

https//my.unisa.ac.za/portal/

14
The Tasks

• Task 1 Starting from myUnisa homepage, go to
  myUnisa
• (use icon or drop down menu)
• Task 2 Find Claim myLife e-mail on myUnisa
  screen
• Task 3 Set-up myLife account
• (Input of student number Surname,
  names, date of
• birth, ID number or Passport/Foreign
  ID (for foreign
• students) Click on acknowledge
  guidelines
• Task 4 Join myUnisa
• Task 5 Activate your myUnisa password

15
Discussion of Results

• Task 1 Claim myLife e-mail account
• Task completion 67
• Assistance needed 75
• Errors made 92
• Task 2 Join myUnisa
• Task completion 83
• Assistance needed 42
• Errors made 58

16
Interpretation

17
Interpretation

• Problem 1 Terminology
• Difference between Claim
  myLife e-mail
• account and Join myUnisa
  not clear
• Registration was selected
  by two users which
• gives a completely different
  screen to join
• myUnisa
• Problem 2 Process
• 2 separate processes,
  passwords mixed up
• Problem3 Instructions
• Transition difficulties,
  moving from Claim
• myLife to Join myUnisa.
• Participants did not knowing
  what to do next

18
The Tasks

• Time to complete tasks
• Participants who used computers regularly or had
• studied computing required an average of 25
  
• minutes to complete the tasks
• Participants without computing experience or who
• had not studied computing previously
  required on
• average 54 minutes to complete the tasks

19
Conclusion

• Diversity in age, experience, background, and
  location of
• ODL students necessitate rigorous
  usability evaluation
• Recommendations can be made regarding the
  interface design based on eye tracking data, and
• The interpretation of the information collected
  through eye tracking should be used together with
  other usability evaluation methods observation,
  heuristics, questionnaire