Testing And Evaluation

1
Testing AndEvaluation
2

• Evaluation

3
Ive Built My Application

• So you now have a application
• Youve followed the guidelines
• Now its time to evaluate.

4
Why Evaluate?

• To see if it suits its purpose
• To see if it is useable
• To see if the user understands the navigation
• To see if the interface works
• To see if it is acceptable for what it was aimed
  to do

5
Why is usability important?

• If they can not use it they wont use it
• If they dont use it you can loose money
• If they dont use it you will loose credibility
• If the site is an intranet, staff spend to much
  time plus employers money if unusable
• Dont design it for your self design it for the
  user
• You should conform to web standards -
  http//www.useit.com/alertbox/20040913.html
• Misconceptions - http//www.useit.com/alertbox/200
  30908.html

6
How do you evaluate

• Decide what you are evaluating
• Decide how you are going to evaluate it
• Decide who is going to evaluate
• Perform the evaluation
• Measure reactions / results
• Analyse results
• Report on results

7
Evaluating The Interface

• Colours
• Text
• Use of Icons / Use of metaphor
• Use of media
• Does it look and feel right?
• Design aspects
• Usability

8
Evaluating Media

• How many media elements are there
• Is it appropriate / Is too much media used at
  once
• Is it provided by computer or human intervention
• Can they control it
• Stop it
• Pause it
• Re-run it
• Can they skip the intro

9

• Testing

10
What do we mean by testing?

• Testing is the process of trying to discover
  every conceivable fault or weakness in a work
  product.
• /

11
Why test multimedia?

• Eliminate errors and bugs
• Compare against original specification
• Make Improvements
• Test usability

12
Testing Methods

• Black Box Testing
• White Box Testing
• Usability Testing

How functional is your system?
Functional Specification
How usable is your system?
User Specification
13
Black Box Testing

• No knowledge of internal workings
• No examination of code
• Inputs and expected outcome
• Unbiased
• Tester and Designer independent
• Tested from the users point of view
• Impossible to test every probable stream

14
White Box Testing

• Explicit knowledge of internal workings required
• Only accurate if tester knows what programs must
  do
• Knowledge of code used to examine outputs

15
Test Example
16
How usable is your system?

• Can a novice user understand it easily?
• Will I need detailed instructions to learn how to
  use it?
• How easy is it to perform a specific task?
• System needs to be evaluated

17
Example
Do you want to listen to your message?

• Press 1 to listen to your message
• Press 2 to record a message
• Press 3 . . . . . .

OR
I dont have time for this!! Arrrggghhh
Oh go on then!
18
Evaluation Methods

• Direct Observation
• Interviews, Questionnaires and Surveys
• Heuristics
• Cognitive Walkthrough

19
Usability and Evaluation
20
Purpose of Usability testing

• Ensure your application is
• Effective to use
• Efficient to use
• Safe to use
• Have good utility
• Easy to learn
• Easy to remember
• Consider HCI during development
• Heuristics Principles

21
Heuristic Principles

• Consistent placement of interface elements
• Consistent use of language
• Consistent, effective and appropriate design
• Error Prevention
• Support expert users
• Useful error messages

22
Benefits of Usability Testing

• Emphasises usability of your application
• Creates historical record
• Reduces services costs
• Increased probability of sales
• Improves future goal setting

23
Issues with Usability testing

• Cost benefit
• Time consuming
• Effectiveness of evaluation
• Can we predict usability?
• How would we know?

24
Cognitive Walkthrough

• Involves evaluators
• Walkthrough tasks
• Note problems
• Simulate a users problem solving process

25
Cognitive Walkthrough process

• Characteristics of typical user
• User Profile
• Description of interface
• Sequence of actions
• Evaluators walkthrough actions
• Record critical information
• Fix problems

26
Why evaluate multimedia?

• Ensure usability
• Eliminate errors
• Comply with principles
• Attract the correct audience

27
Approaching evaluation

• To do it correctly we need to follow a defined
  framework
• Identify issues to be measured
• Identify methods of measurement
• Go through a testing episode
• Obtain results
• Report on findings

28
Elements to consider

• Screen Design
• Menus
• Colour
• Metaphor
• Media Inclusion
• Task Structuring
• Navigation

29
Screen design

• Need a strong visible layout structure
• Must include titles, and sub-titles
• Be logical and uncluttered

30
Screen Design (continued)

• Show well grouped information
• Be distinguishable related to navigation,
  interaction, and media
• Try and observe existing multimedia convention as
  much as is possible

31
Menus

• Must be sensibly located
• On screen, or
• Drop down
• When opened
• Are they well laid out and grouped logically
• How much screen space do they take up when in
  use, do hierarchies cause interference problems

32
Colour

• How well is colour used
• Do colours contribute to give a design feel or
  atmosphere to the application
• Do they enhance the display
• Is it used to group elements
• Is colour applied consistently throughout screens
• Are real world colour conventions observed
  (especially HCI)

33
Media inclusion

• What types of media are included
• Do they combine well
• Is there a good balance (i.e. interesting)
• Is control automatic or user controlled
• Can media elements be selectively manipulated
• Are media durations appropriate
• Are special effects applied
• Are there any technical running problems

34
Task structuring

• Is the application well defined in terms of tasks
  and sub-tasks
• Logical task sectioning
• Adequate feedback for errors and completion

35
Navigation

• Is there an inherent structure to the application
  to aid the user in building up a mental model
• Can you work out where you are, where youve
  been, and where to go to easily
• Is a route recorded to aid backtracking
• Are visual mapping facilities available

36
Users

• Does the application fit the intended user type
• Address background and attributes, and provide
  functions and facilities the user wants
• Specifically does the application cater for the
  correct type of user behaviours browsers,
  planners, searchers, explorers, wanderers, the
  fearful, and fun-seekers

37
Performance

• Important for both web applications, and CD-ROMs
• Do pages take too long to load
• Once loaded, are there running issues related to
  media, e.g. staggering media clips, losing sound
  or image, or general delays, giving a feel of
  poor quality

38
Interaction

• Typically implicit or explicit interaction is
  used
• Visual objects, or defined buttons
• Are mechanisms easily identified
• Do they work?
• Can interaction activities easily be worked out
• If icons are used can
• They clearly be identified

39
Fit to Machine Specification

• Are all the facilities included usable on an
  average specification PC
• You yourself will need to identify the best
  specification for your application
• May involve testing on several machine set ups
  and specifications

40
Further Reading

• Interaction Design beyond human-computer
  interaction
• Preece, Rogers Sharp