Methods for assessing the outcomes of rehabilitation: theory and practice

1

• Methods for assessing the outcomes of
  rehabilitation theory and practice
• Harvey Dillon
• National Acoustic Laboratories (NAL)
• CRC for Cochlear Implant and Hearing Aid
  Innovation
• Hearing International
• Pattaya, January, 2002

2
Talk outline

• Applications of outcome measures
• Types of outcomes measures
• When to measure outcomes
• Talk can be downloaded from www.nal.gov.au (from
  next week)

3
Definition of Outcomes

• A measure of the changes in a clients life as a
  result of delivering a service and/or fitting
  devices
• Outcomes versus inputs
• Inputs what the service does to or for a
  client.
• Outcomes what changes in the life of the
  client.

4
Need for outcome measures
Systematically study device effectiveness
Determine when to finish the program
Know what to teach
Which device for which clients
Develop realistic expectations and goals
Know which devices to pay more for
Know what procedures to teach
Know which devices to pay more for
Know what procedures to pay for
Know what to spend time doing
Benefit from choosing an effective provider
Benefit from time well spent
Verify effectiveness of new procedures
Government/insurance money spent most
effectively
Monitor improvements / deterioration in average
outcome quality
Get best value for money
5
Types of outcome measures
6
Types of outcome measures

• Broad type Objective versus self-report
• Quantity measured Benefit, satisfaction, aid
  usage, generic quality-of-life
• Calculation Change versus state
• Format Standardized versus individualized
• Administration Clinician versus anonymous

7
Broad types of measures

• Speech Tests (before and after)
• Objective
• Time consuming
• Depend on conditions
• (Totally speech oriented)
• Observation of clients
• Impractical
• Client Report
• From clients perspective
• All inclusive

8
Broad types of measures

• Speech Tests (before and after)
• Objective
• Time consuming
• Depend on conditions
• Totally speech oriented (awareness,
  localization)
• Observation of clients
• Impractical
• Client Report
• From clients perspective
• Specific or all inclusive

Not Insertion Gain
9
Quantity measured

• Benefit
• Impairment
• Disability (or activity limitation)

Impairment
10
HAPI / SHAPIEWalden, Demorest Hepler,
1984Dillon, 1994

• 1. You are watching TV and
• there are distracting noises
• such as others talking. ____ ___
  ___ ___ ___
  
• 2. You are at home in
• conversation with a member
• of your family who is in
• another room ___
  ___ ___ ___ ___

11
Quantity measured

• Benefit
• Impairment
• Disability (or activity limitation)
• Handicap (or participation limitation)

12
Hearing Handicap Inventory for the Elderly
(HHIE)Ventry Weinstein, 1982

• YES SOME- NO
• (or TIMES (or
• usually) rarely)
• 1. Does a hearing problem cause you
  ____ ____ ____
• to use the phone less often than you
• would like?
• 2. Does a hearing problem cause
  you ____ ____ ____
• to feel embarrassed when meeting
• new people?
• 3. Does a hearing problem cause you
  ____ ____ ____
• to avoid groups of people?

13
Quantity measured

• Benefit
• Impairment
• Disability (or activity limitation)
• Handicap (or participation limitation)
• Satisfaction

14
Measuring overall satisfaction

• 1. Hearing Aid
• Please mark, on the scale below, how satisfied
  you are overall with your hearing aid.
• Zero would mean that you are not at all
  satisfied
• 100 would mean that you are totally satisfied.
• ----------------------------------------
  -----
• 0 20 40 60
  80 100

15
Measuring overall satisfaction

• 2. Service provided
• Overall, how satisfied you with the
  rehabilitation service you have received ?
• Extremely satisfied
• Satisfied
• Neither satisfied nor dissatisfied
• Dissatisfied
• Extremely dissatisfied

16
Measuring satisfaction in detail
Satisfaction with Amplification in Daily Life
(The SADL)
17
Quantity measured

• Benefit
• Impairment
• Disability (or activity limitation)
• Handicap (or participation limitation)
• Satisfaction
• Device usage

18
Device usage

• On average, how many hours per day do you wear
  your hearing aids
• More than 8 hours
• 4 to 8 hours
• 1 to 4 hours
• lt 1 hour
• Never
• For what proportion of the time that you need
  hearing aids do you actually wear them?

19
Quantity measured

• Benefit
• Impairment
• Disability (or activity limitation)
• Handicap (or participation limitation)
• Satisfaction
• Device usage
• Quality of life
• SF 36
• Sickness Impact Profile

20
Calculation method

• Change measures
• How much better do you hear when aided?

A more accurate estimate of the difference

• State measures
• How well do you hear when unaided?
• How well do you hear when aided?

Tells us about the starting and finishing states
as well as the difference
21
Calculation method

• Change method e.g. HAPI / SHAPIE

4 3 2 1 -
You are watching TV and there are distracting
noises such as others talking.
____ ___ ___ ___ ___

22
Calculation method

• State method e.g. APHAB (Cox Alexander, 1995)

1 2 3 4 5 6 7
I have difficulty hearing a conversation when
Im with one of my family at home.
___ ___ __ ___ ___ ___ ___

23
Calculation method

• Change versus state method
• Glasgow Hearing Aid Benefit Profile (GHABP)
• (Gatehouse, 1997)
• Change measure more sensitive than state measure
• But change alone can be inadequate
• starting or finishing position.

24
Format standardized versus individualized

• Questionnaires usually standardized in format
• same questions for everyone
• easy to score
• easy to compare results across people
• easy to combine results across people
• can be tedious
• can contain irrelevant items
• difficult to integrate with rehab process
• Questionnaires can be individualized e.g. the
  Client Oriented Scale of Improvement (COSI)
• individualized concept from Stephens (1980)

25
COSI

• The NAL
• Client Oriented Scale of Improvement

26
Client Oriented Scale of Improvement (COSI)

• At interview
• Where / when would you like better hearing?
• At final appointment, for each situation
• How much improvement have you noticed?
• How well can you now hear?

27
(No Transcript)
28
(No Transcript)
29
Things clinicians like about COSI
30
Things clinicians dislike about COSI
31
Some Anecdotal Observations

• The COSI helps build rapport
• Clients feel they are being listened to, and that
  clinician remains interested in their problems
• Goal/needs gives direction regarding aid fitting
  and other services
• The COSI clarifies clients needs to themselves
  and requires acknowledgment of disability
• Simple documentation helps case continuity

32
Administration

• Administered by clinician
• Pre-measures may influence rehab program
• Can act on results immediately
• Responses may be less honest
• Self administered
• Less clinician time needed
• More chance of mis-interpretation (e.g. HHIE
  critical differences doubling)

33
When to measure outcomes

• Three to four weeks after fitting
• Six to seven weeks after fitting
• Three months after fitting
• Six months after fitting
• One year after fitting

34

• The importance of a good fitting . and of
  checking

HAUQ questionnaire
35
Problems reported at 3 months
36
Conclusions

• Do measure outcomes - replace anecdotes with
  systematic observations, biases with supportable
  facts.
• Include dimensions of benefit, satisfaction, and
  usage
• e.g. International Outcomes Inventory - Hearing
  Aids (IOI-HA)
• Assess problems as well
• Use a measure that also contributes to the
  outcome
• e.g. Client Oriented Scale of Improvement (COSI)
• Be ready to argue about outcomes for
  hearing-impaired people with people who want to
  argue about dollars and cents (or Bahts)

37
For further information

• 1. Download presentation www.nal.gov.au
• 2. Chapter 13 Hearing Aids (Dillon, 2001
  Boomerang Press / Thieme Publishing).
• 3. Entire book Noble, W (1998).
  Self-assessment of hearing and related functions.
  Whurr.
• 4. Special issue Ear Hearing, Vol 21, Number
  4 (2000), edited by Robyn Cox.
• 5. International Journal of Audiology First
  issue, just out.

38
The end