Usability testing: A reality check for Plain Talk

1
Usability testingA reality check for Plain Talk

• A brief overview
• October 7, 2008
• Dana Howard Botka
• Manager, Customer Communications, LI
• Plain Talk Coordinator, Office of the Governor

2
What is usability testing?

• Finding out if your targeted customer can
  actually use your product, such as a
• Set of instructions
• Form
• Web site
• Software application
• Automobile
• Cell phone

3
How does it work?

• 4-6 people try out your document, web page, or
  other product one a a time.
• One facilitator conducts the try-out, asking
  open-ended questions.
• The usability participant is asked to think
  out-loud while doing tasks.
• Up to two program reps observe and take notes
  or record the session.

4
We dont ask testers what they like

• We develop a test to see if the typical user
  can actually do a task, such as
• Calculate a late fee.
• Fill out a form without making errors.
• Find the answer to a question on a web site.
• Correctly figure taxes.
• Turn on a car defroster quickly.
• Forward a cell phone call.

5
Why usability testing for Plain Talk?

• Because you discover
• Terms customers dont understand.
• Words that can mean something else.
• Basic information or context that is missing.
• Information that is unimportant distracting.
• Your customers emotional reaction to what youve
  developed.
• What you had no idea would happen!

6
How does usability work?

• Recruit 4-6 people who fit the description of
  your typical user but who are not familiar with
  the product.
• Develop 6-10 tasks that, if performed correctly,
  will let you know they can use your product.
• Write a scenario that will allow the test
  participant to understand what situation they are
  in.
• Write a series of questions that will get the
  test participant to do a task without hints.

7
LIs Quarterly Report ProjectHelping
employers self-report hours

• Challenge
• High customer error rates on this report.
• Errors often lead to late fees for employers and
  costly processing headaches for LI.
• Solution
• Make paper reporting easier for customers with
  instruction sheet and late fee worksheet.
• Encourage employers to use upgraded online report
  application.

8
Our test materials

• See the documents in your packet
• Before after examples of our test document
• Pre-test screening questionnaire
• Instructions and legal sign-off
• Scenario and test questions
• Sample thinking aloud from a test participant
• Findings and Recommendations

9
Form before revision
10
Form instructions before revision
11
Setting up the usability test. Plain Talk team
members decide what they most need to learn from
recruited test participants.
12
Form after revision
13

Form instructions after revision
14
Usability testing on a budget

• Recruit test participants from friends, family,
  agency walk-ins.
• Keep the test simple and short.
• Use results as a guide not final data.
• Consider the act of developing the test as a
  great way to focus on your project goals.

15
What about a survey or focus group?

• Not a replacement for a test of whether someone
  can do a task.
• People will often say they like a document or
  web site and not know how to use it. This is
  because they dont want to
• Admit they had trouble.
• Hurt your feelings.
• Go to the trouble of seeing if it actually
  works.

16
Invite managers and staff to observe testing!
Its a great reality check.
17
Involve frontline staff in usability testing.
They know their customers and are great at
solving problems.