Task Measurement ''' Because You Cant Manage What You Cant Measure - PowerPoint PPT Presentation

Loading...

PPT – Task Measurement ''' Because You Cant Manage What You Cant Measure PowerPoint presentation | free to view - id: d6c67-ZDc1Z



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Task Measurement ''' Because You Cant Manage What You Cant Measure

Description:

Each measurement session takes 30 to 90 minutes and comprises 10 to 20 tasks. Measurements are always done in individual one-on-one sessions. ... – PowerPoint PPT presentation

Number of Views:27
Avg rating:3.0/5.0
Slides: 48
Provided by: gerrymc
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Task Measurement ''' Because You Cant Manage What You Cant Measure


1
Task Measurement... Because You Cant Manage
What You Cant Measure
  • September 10, 2008

2
Task MeasurementTask Performance Indicator
  • Welcome
  • Opening Introduction Gerry McGovern
  • Task Measurement Rolf Molich
  • Question Answer session

3
Audio pane select audio mode. In the Audio
pane, select either Use Telephone or Use Mic
Speakers. By default, you will be joined into the
Webinar muted. If you select Use Telephone, be
sure to enter the Audio PIN noted in your Control
Panel. If you select Use Mic Speakers you can
join the Webinar via VoIP. You will need speakers
to listen to the Webinar and a microphone to
speak. Click Audio Setup to select Mic Speakers
devices. You can switch between using VoIP (Mic
Speakers) or Telephone during the Webinar
session.
4
  • Questions Answer Session
  • Use the Question pane.
  • To submit text questions and review answers.
    Broadcast messages to attendees will also show
    here.
  • Click Hand icon to raise/lower hand. If asked to
    speak you will be un-muted and you can ask you
    question.

On exiting the webinar you will be prompted to
complete a survey. We welcome your feedback.
5
Our Speakers
  • Gerry McGovern
  • Rolf Molich

www. dialogdesign.dk www.customercarewords.com
www.gerrymcgovern.com www.customercarewords.com
6
Task Measurement... Because You Cant Manage
What You Cant Measure
  • September 10, 2008

Gerry McGovern gerry_at_customercarewords.com
7
(No Transcript)
8
Sample Task
  • What are your companys rules for maternity leave
    in your country?

8
9
Search
More search options Help
Intranet Home Class B Class C About Me
About Company
Expected1st click
Homepage
10
Search
More search options Help
Intranet Home Class A Class B About
Me About Company
About Me
ABOUT ME Training Job Vacancies Travel Time and
Payroll Employee Benefits Environment (Working,
Local) Career Development
Expected 2nd click
11
Search
More search options Help
Intranet Home Class B Class C About Me
About Company
Homepage
Actual1st click
12
Search
More search options Help
Intranet Home Class A Class B About
Me About Company
About Me
ABOUT ME Training Job Vacancies Travel Time and
Payroll Employee Benefits Environment (Working,
Local) Career Development
Actual 2nd click for 6/15 users
13
Search
More search options Help
Intranet Home Class A Class B About
Me About Company
About Me
ABOUT ME Training Job Vacancies Travel Time and
Payroll Employee Benefits Environment (Working,
Local) Career Development
Only 1/15 clicked here
14
Task 7 - Employee Policies
Task What are the companys rules for maternity
leave in your country? Overall success Solved
task in first attempt - Solved task in second or
third attempt 13 Unable to solve task 1, 2, 3,
4, 5, 6, 7, 8, 9, 10, 11, 12, 14, 15 Expected
paths A. About Me gt Time Payroll 13 Alternative
paths B. About Company gt Org. gt HR gt HR
Info. 1, 15 C. About Company (somewhere) 2, 3,
4, 12 D. About Me gt Compensation Benefits 4,
7, 9, 11, 12 E. About Me gt Environment
(Working, Local) 5, 6, 7, 9, 10, 14
14
15
Task Measurement... Because You Cant Manage
What You Cant Measure
  • Presentation of Method
  • September 10, 2008

Rolf Molich rolf_at_customercarewords.com
16
Does this baby get even prettier over time?
Opinion based answer.
17
Does this baby get longer over time? Objective
answer. We can track progress, because we can
measure it.
18
Is this a usable intranet? Opinion based
answer. How long does it take to find the rules
for maternity leave? Objective answer.
19
Why Measurement?
  • You cant manage what you cant measure.
  • -- Tom DeMarco

20
Users Time is Precious
  • Websites are a product which you ask people to
    buy, using their time and attention as currency.
  • -- Tom du Pré

21
Managing a Website
  • Managing a website is about managing users tasks
  • Identify top user tasks for Internet, Intranet,
    desktop application, etc.
  • Develop performance measures and benchmarks based
    on user task performance
  • Assign responsibility for the success of each top
    user task to staff (task manager)
  • Report to senior management on task metrics
  • Set annual task performance improvement targets

22
Task MeasurementTask Performance Indicator (TPI)
  • Determine top tasks
  • Recruit representative users
  • Measure Users carry out tasks
  • Compute and report results
  • Improve task performance
  • Measure again.

23
Step 1 Determine Tasks
  • Determine 10 (or more) frequently used tasks.
  • The chosen tasks must be frequent so improving
    them significantly will have a measurable impact
    on the efficiency of the organization.

24
Step 1 Determine Tasks
  • Tasks are selected based on
  • Result of Customer Carewords study
  • Most used pages
  • Key search terms
  • Customer Carewords standard tasks (this allows
    inter-company comparisons)

25
Step 1 Determine Tasks
  • Sample Find Person Tasks
  • Where is John McGuires office?
  • Find an expert on investing in IBM.
  • Who is responsible for personal banking in
    Uruguay?

26
Step 2 Recruit Participants
  • Recruit 15-20 users who are a representative mix
    of your employees.
  • Getting the participant mix right is vital in
    ensuring that we get results that reflect the
    true nature of carrying out the identified tasks
    within the organization.

27
Step 3 Measurement Sessions
28
Step 3 Measurement Sessions
  • Ask each of the 15-20 users to carry out the
  • chosen tasks one by one as quickly as possible.
  • We are not measuring you
  • Measure time for each task.
  • Observe usability problems and successes.
  • Ask about frequency.
  • Similar to traditional usability test, except no
    think aloud and no discussions

29
Step 3 Measurement Sessions
  • An expert observes what happens.
  • Each measurement session takes 30 to 90 minutes
    and comprises 10 to 20 tasks.
  • Measurements are always done in individual
    one-on-one sessions.

30
Step 4 Compute and Report Results
31
Step 4 Compute and Report Results
  • Result Success
  • User completes task successfully.
  • We register the time it took.

32
Step 4 Compute and Report Results
  • Result Failure
  • User gives up.
  • Task is not completed within 6 minutes (for
    small tasks).
  • Technical problems prevent the user from
    completing the task.

33
Step 4 Compute and Report Results
  • Result Disaster
  • User completes task but the result is wrong.

34
Step 4 Compute and Report Results
  • Example of disaster
  • Users were asked to find out what percentage of
    customers are considered reluctant according to
    customer satisfaction studies.
  • Some users found a page that said that the
    percentage of reluctant customers was 24.
    However, this figure was outdated.
  • The right figure was 14.

35
Step 4 Compute and Report Results
  • Use all measurements for a task to compute
  • Completion rate - Effectiveness
  • Completion time - Efficiency, Median time for
    task
  • Disaster rate User finds wrong answer, but
    believes it is correct

36
Step 4 Compute and Report Results
  • Optimal time
  • Projected time for task with
  • An improved but realistic user interface
  • Realistic response times

37
Step 4 Compute and Report Results
  • Task Performance Indicator (TPI)

  • Optimal time for task
    Median measured time for task
  • The Success factor factors in the effect of
    failing and disastrous task attempts. It is 100
    if all participants manage to solve the task
    correctly within 6 minutes. It is 50 if only
    half of the participants manage to solve the
    task, etc.

TPI Success factor x
38
Step 4 Compute and Report Results
  • TPI example
  • Optimal time for task 40 secs
  • Median time for users to carry out task 100 secs
  • Optimal time / Median time 40

39
Step 4 Compute and Report Results
  • TPI example
  • Optimal time for task 40 secs
  • Median time for users to carry out task 100 secs
  • Optimal time / Median time 40
  • 9 of 15 participants complete the task 60
  • Task Performance Indicator 40 of 60 24

40
Step 5 Improve Task Performance
  • You apply the usability advice offered in
    Customer Carewords TPI report
  • General or specific usability advice,
  • Optimal solution patterns based on best practice.

41
Step 6 Measure Again
  • Measure again, for example 1 year later.
  • Compute improvement in completion rate,
    completion time and disaster rate.

Measure
Improve
Analyse
42
We expected that most participants would be able
to complete the key tasks we gave them so we
could get valid measurements.
43
We see again and again that a significant number
of key task attempts fail completely. Users give
up and find other means to get the answer.
44
Main Reasons for Failures, Disasters
  • Search
  • Navigation
  • Freshness

45
Potential Productivity Gains
Fact Existing Customers have identified
potential annual savings to management ranging
from 15,000,000 to 35,000,000. (20M-50M US
Dollars).
Potential productivity gains with 36,000
employees
46
Task Measurement Highlights
  • Management tool Which task improvement
    investments will pay off?
  • Focus on task performance rather than tool
    performance or system performance
  • Systematic treatment of task failures and
    disasters
  • All-in-one Task Performance Indicator (TPI)
  • Suggested optimal task flow defines clear goal

47
Questions Answers
  • There are two ways to submit a question
  • Type your question in the Questions panel.
  • Press Raise your Hand and you will be invited
    to speak to the panel directly.

Would you like to know more? Visit our website
www.customercarewords.com or Email
andrew_at_customercarewords.com Customer Carewords
has a network of partners across North America,
Europe and Asia/Pacific. Our partners have deep
experience with creating customer-centric,
task-managed websites utilizing Customer
Carewords. Contact us to talk to a partner.
About PowerShow.com