Outcome measurement Train the trainer workshop Day 2

1
Outcome measurement Train the trainer workshop
Day 2
Aged Persons Mental Health Services
2
Housekeeping

• Toilets
• Morning and afternoon tea
• Lunch
• About mobile phones
• Evaluation

3
National objectives

• To build an informed mental health system
  where information is available to guide decisions
  at all levels to
• support clinicians in their treatment decisions
• inform consumers about services they receive
• help managers manage
• inform policy makers in planning and paying for
  services

4
National governance and advisory structure
AHMAC Mental Health
Working Group Australian Health Ministers
Advisory Council
NMHWG National Mental Health Working Group
National Mental Health
Information Strategy
Committee - ISC
National Minimum Data Set Sub-committee
Technical Specifications Drafting Group
OPMHOEG Older Persons Mental Health Outcomes
Expert Group
5
Episodes, cases and collection occasions
Period of no care
Intake to community
Collection occasions
6
Outcome measure protocol for older persons mental
health services - Victoria

7
Your local protocol

• Is it completed?
• Well on the way?
• A distant vision?

8
Criteria for instrument selection

• Acceptability
• Compatibility
• Economy
• Feasibility
• Adequate coverage
• Continuity
• Integration
• Aggregation

9
Diagnosis

• Principal diagnosis
• The principal diagnosis is the diagnosis
  established after study to be chiefly responsible
  for occasioning the patient or clients care
  during the preceding period of care

10
Mental health legal status

• Was the person treated on an involuntary basis
  (under the relevant mental health legislation) at
  some point during the preceding period of care?

11
The suite for APMHS

• Health of the Nation Outcome Scales for Older
  Persons (HoNOS 65)
• Abbreviated Life Skills Profile (LSP-16)
• Resource Utilisation Groups Activities of Daily
  Living (RUG-ADL)
• Focus of Care (FOC)
• Behaviour and Symptom Identification Scale
  (BASIS-32)

12
Missing data

• As a general rule of thumb
• there should be
• NO MISSING DATA
• from the
• clinician-completed outcome measures
• Missing items (scored as 9) are assigned a
  value of zero in the Wellbeing Reporting Tool.

13
Health of the Nation Outcomes Scales for Older
Persons - HoNOS 65

• 12-item measure
• Rating made by mental health professional
• Not diagnosis-specific
• Each rating point defined in glossary
• Scores 0-4 (or 9 for not known)
• Takes into account all available information
• Is a good indicator of severity
• Outcomes can be derived by comparing ratings over
  time
• Quick to rate once familiar with the glossary
• OM rather than assessment tool clinical depth
  traded off for ease of comparison
• Timeframe the most serious problem in last two
  weeks
• (excludes discharge in acute inpatient units
  three days)

14
HoNOS 65 scale structure

• Behavioural disturbance
• Non-accidental self-injury
• Problem drinking or drug-taking
• Cognitive problems
• Physical illness or disability problems
• Problems associated with hallucinations and
  delusions
• Problems with depressive symptoms
• Other mental and behavioural problems
• Problems with relationships
• Problems with activities of daily living
• Problems with living conditions
• Problems associated with occupation and activities

15
1.Behavioural disturbance

• Exclude
• Bizarre behaviours rate in Scale 6 -
• Problems with hallucinations and delusions

• Include
• Overactive, aggressive, disruptive, agitated,
  uncooperative, resistive behaviours due to any
  cause.

Causes dementia, alcohol, drugs, psychosis,
delirium, depression
16
2 Non-accidental self-injury

• Exclude
• Accidental self-injury due to
• Dementia
• Severe learning disability

• Exclude
• Illness or injury as a direct result of
  drug/alcohol -

Why? Cognitive problems are rated in Scale
4 Injury is rated in Scale 5
Why? Lung cancer related to smoking or injury
from drink driving is rated at Scale 5
17
3 Problem drinking or drug taking

• Exclude
• Aggressive or destructive behaviour due to
  alcohol/substances

• Exclude
• Illness or injury as a direct result of
  drug/alcohol

Why? AIDS from poor needle exchange or injury
from drink driving is rated at Scale 5
Why? Aggressive behaviours are rated in Scale 1
18
4. Cognitive problems

• Include
• Problems of orientation, memory, language
  associated with any disorder

• Exclude
• Temporary problems, hangovers associated with
  alcohol/substance use
• rated in Scale 3

Types of disorders dementia, learning
disability, delirium, depression, schizophrenia
19
5. Physical illness or disability problems

• Include
• Side effects from medications, substance or
  alcohol use, physical disabilities resulting from
  accidents or self-harm associated with cognitive
  problems, drink driving accident

• Include
• Illness or disability from any cause that limits
  mobility, hearing, sight, or interferes with
  personal functioning

Types of disorders Rate pain here

• Exclude
• Mental/memory problems rate in Scale 4

20
6. Problems associated with hallucinations and
delusions

• Include
• Hallucinations and delusions or false beliefs
  irrespective of diagnosis

• Include
• Odd and bizarre behaviour behaviour associated
  with hallucinations or delusions, false beliefs

• Exclude
• Aggression, destruction, overactive behaviours
  attributed to hallucinations, delusion, false
  beliefs Rate in Scale 1

21
7. Problems with depressive symptoms

• Exclude
• Over activity or agitation

• Exclude
• Suicidal ideation or attempts

WHY? Aggressive, agitated behaviours are rated
in Scale 1
WHY? Self-harm is rated in Scale 2

• Exclude
• Delusions or hallucinations

Why? Delusions and hallucinations are rated in
Scale 6
22
8. Other mental and behavioural problems
Rate only the most severe clinical problem that
is not considered at scales 6 and 7

• Somatoform
• Eating
• Sleep
• Sexual
• Other

• Phobic
• Anxiety
• Obsessive-compulsive
• Stress
• Dissociative

23
9. Problems with relationships
Problems associated with social
relationships Can be identified by Patient,
client, carers, family, others

• Rate the most severe problem associated with
• Active/passive withdrawal from
• Attempts to dominate
• Destructive
• Self-damaging
• Non-supportive relationships

24
10. Problems with activities of daily living

• Exclude
• Lack of opportunities fro exercising intact
  abilities/skills

• Include
• Lack of motivation for using self-help
  opportunities

Why? Rated in Scales 11 12
Why? This contributes to overall level of
functioning
Rate the overall level of functioning in ADLs
Problems BASIC activities of self-care eating,
dressing, toileting, showering etc COMPLEX
activities budgeting, public transport use,
budgeting, bill paying, use of oven/microwave
25
11. Problems with living conditions
Rate the older persons usual accommodation

• Rate
• The overall severity of problems with
• -Quality of living conditions/accommodation
• -Daily domestic routine
• Take into account the persons preferences/degree
  of satisfaction

Are the BASIC necessities met? YES! THEN - Does
the environment contribute to maximising
independence, minimise RISK, provide choice and
opportunities to develop new skills and maintain
old ones?

• Exclude
• The level of disability rated in Scale 10

26
12. Problems associated with occupation and
activities
Rate the older persons usual situation

• Rate
• The overall problems with quality of daytime
  environment
• Help to cope with disabilities? Any stigma, lack
  of qualified staff, supportive facilities
• Access to staff, day centres, elderly community
  groups/activities, equipment, bowls, day trips
  etc.

• Exclude
• The level of disability rated in Scale 10

27
HoNOS 65 rating rules

• Rate each item in order from 1 to 12
• Do not include information rated in an earlier
  item, minimal item overlap
• Rate the most severe problem that has occurred
  over the previous two weeks
• three days discharge acute inpatient unit
• Consider both the impact on behaviour and/or the
  degree of distress it causes
• When in doubt, read the glossary

28
HoNOS 65 scoring

• Each item is scored
• 0 no problem
• 1 sub-clinical problem
• 2 mild problem
• 3 moderate problem
• 4 severe problem
• 9 not known 0
• Only use 9 when you are genuinely unable to
  make a definitive rating

29
Important variations in rating guides
30
Sources of information

• The measures are not clinical interviews.
  Information should be gathered from
• The consumer
• Direct observation
• Information in the medical record
• Information provided by other staff
• Information provided by family and friends
• Information provided by other agencies including
  general practitioner, housing, police and
  ambulance staff

31
From an alternative perspective
32
(No Transcript)
33
Measuring outcomes in aged persons MHS
Training Vignette 1 Mr Nguyen
Mr Nguyen is a 69-year-old man who arrived in
Australia as a refugee following the Vietnam War.
His wife died four years ago and he has
remained living in their family home in inner
Melbourne with his 19-year-old grandson Tran, who
works full-time.
34
Training Vignette 1 Mr Nguyen (continued)

• Tran contacted his grandfathers GP after the
  neighbours expressed concern about Mr Nguyens
  behaviour.
• The GP hadnt seen Mr Nguyen for 18 months and
  had assumed he had moved.
• Based on what Tran has told him, the GP has
  referred Mr Nguyen to the Aged Psychiatry
  Community Team for assessment.
• Mr Nguyen understands very little English and
  speaks none.
• From Tran you learn Mr Nguyen hasnt left his
  house for 3-4 months, with food being provided by
  family members and some close neighbours.
• The family had arranged for meals on wheels but
  Mr Nguyen just left them uneaten.
• Tran does the banking, which consists of going to
  the bank each second Thursday, and withdrawing
  almost the total pension cheque in cash.

35

• Mr Nguyen looks after the money and gives Tran
  the accounts that need to be paid, with
  directions and cash.
• Mr Nguyen also insists on Tran buying the
  following items each week rice, soap and
  tealeaves.
• Tran is concerned because Mr Nguyen doesnt use
  any of these items but stacks them in the sitting
  room.
• There is so much of it that Tran has had to take
  some of the furniture into other rooms.
• Mr Nguyen keeps the house really clean and tidy
  and he stacks all the papers, newspapers, cartons
  and containers in the spare room.
• Tran has tried to throw some of it out but Mr
  Nguyen becomes very distressed and angry and then
  yells at Tran.

36

• Mr Nguyen no longer sleeps in his bedroom but has
  taken to sitting on a stool in the passageway of
  the house each night.
• Tran says he talks to himself a lot and wont
  turn the lights out. This has been happening for
  the last two weeks.
• Tran doesnt want to be a part of any assessment
  because Mr Nguyen threatens to make him leave if
  Tran tries to interfere.
• Tran is worried Mr Nguyen wont talk to any
  interpreter that he doesnt know.
• Tran reports Mr Nguyen hasnt lost any weight,
  but he has stopped drinking water out of the
  taps.
• Instead, he takes water from the rusted-out water
  tank at the back of the house.

37
Measuring outcomes in aged persons MHS
Training Vignette 2 Mrs Tilly
38
Case scenario

• Mrs Tilly is an 82-year-old woman living in a
  community-based hostel in outer Melbourne.
• Mrs Tilly has become increasingly irritable over
  the last two months and has hit several staff
  members, usually in the shower.
• Two days ago she threw her glass and cutlery at
  another resident and cut his face.

39

• Mrs Tilly has lived in the hostel for seven years
  and has always felt at home.
• Four weeks ago Mrs Tilly had a turn as
  described by the staff and was seen by her GP who
  found little, except for an elevated BP and some
  increased peripheral oedema, for which he
  increased her Atenolol.
• Mrs Tilly denied any headaches or memory
  problems.
• Over the past three weeks the staff report a
  decline in Mrs Tillys self-care, with her
  requiring almost full assist with washing,
  changing her clothes and general activities.
• There is a note in the file about Mrs Tilly
  smelling of urine but there appears to be no
  follow-up.

40

• Mrs Tilly has stopped going to her bowls, bingo
  and knitting groups and just wanders around the
  hostel.
• The other residents report Mrs Tilly is up most
  of the night either banging about in her room or
  wandering up and down the passageway looking for
  her room.
• Mrs Tilly is difficult to engage on assessment
  and appears vague, disorganised and her speech is
  somewhat slurred.
• Her thoughts jump all over the place and she is
  very easily distracted.
• It is difficult to assess her memory, as she
  cannot concentrate to complete a mini mental
  state examination.

41

• Review of Mrs Tillys history reveals she has
  been seen by the Aged Care Assessment Team and
  given a high level approval for transfer to a
  nursing home called Happy Horizons in northern
  Victoria.
• It appears that Mrs Tillys daughter Peg and
  family moved to northern Victoria some five years
  before and Peg has been trying to get Mrs Tilly
  to move there without success.
• Mrs Tilly has been adamant about seeing out her
  days where she is close to her husbands grave.
• The Aged Care Assessment Team requested the GP do
  some follow up investigations given the short
  timeframe of decline in Mrs Tilly.
• You obtain the results and it is apparent that
  Mrs Tilly has a urinary tract infection and is
  dehydrated.

42
Six months later

• Now living in northern Victoria (five weeks).
• Physically back to normal, sleeping all night.
• Very occasionally ends up in someone elses room.
• MMSE 28/30.
• Is attending a great day program and knitting,
  playing bowls.
• Will not see or speak to Peg verbally threatens
  to cut her out of the will but sees the rest of
  the family.
• What score now?

43
Life Skills Profile-16
44
LSP - A non-technical instrument - originally
designed to require little or no training

• Key measure of function and disability in people
  with mental illness.
• Complements the problem-based HoNOS.
• Originally a 39-item scale reduced to 16 items.
• Brief five minutes to rate.
• Good inter-rater reliability.
• Sensitive to change.
• Focuses on the person's general functioning over
  the last three months social relationships,
  day-to-day tasks etc
• The LSP-16 is not a clinical interview
• Take into account age, social and cultural
  context.
• Do not rate the crisis or when the client was
  becoming ill.

45
Example of item structure

• 1) Does this person generally have any difficulty
  with initiating and responding to conversation?
• 0 No difficulty with conversation
• 1 Slight difficulty with conversation
• 2 Moderate difficulty with conversation
• 3 Extreme difficulty with conversation
• 2) Does this person generally withdraw from
  social contact?
• 0 Does not withdraw at all
• 1 Withdraws slightly
• 2 Withdraws moderately
• 3 Withdraws totally or near totally

46
Specific LSP-16 items

• Item 6 - neglect their physical health? Not
  receiving treatment for a health condition, lead
  a generally healthy life style
• Item 10 - behaviour related to medication
  adherence
• Item 11 - attitude towards medication
• Item 12 - cooperate with health services
• Item 15 - deliberate intention

47
The LSP-16 subscales

• Withdrawal
• Antisocial behaviour
• Self-care
• Compliance

48
Focus of Care
49
Focus of Care

• A 1-item tick box requiring the clinician to make
  a retrospective judgement about each consumers
  primary goal of care.
• There are 4 choices Acute, Functional Gain,
  Intensive Extended and Maintenance.
• The FOC informs interpretation of OM data and
  contributes to casemix classification.
• In APMHS, the FOC is collected in inpatient
  services (at discharge) and by community teams
  (at review and discharge).

50
Rating the Focus of Care

• Single rating item to identify the main focus of
  care.
• Assesses the primary goal of care.
• Based on concept of phase of illness in people
  with mental health disorders.
• Rate main focus of care over whole episode - is
  therefore a retrospective measure
• Measures categories - not rankings

51
(No Transcript)
52
RUG-ADL
53
RUG-ADL

• Abbreviated version of the US measure of nursing
  dependency
• Four items covering bed mobility, toileting,
  transfer and eating
• Used only in inpatient services in APMHS (at
  admission and review)
• Required for casemix classification rather than
  outcome measurement

54

• Early loss dressing, shaving legs, makeup, hair
  style etc
• Late loss
• Independent eating
• Bed mobility
• Toileting
• Transfer

55
Outcome measurementaged persons MHS

• Training vignette Mr Alloy

56
Case scenario

• Mr Alloy is a 77-year old wheat farmer living in
  western Victoria. He has a seven-year history of
  vascular dementia and at least five documented
  collapses.
• Mr Alloy lives in a small, purpose-built cottage
  on his and his sons large property 36 kilometres
  out of town. His son Thom and daughter-in-law
  Shedra and their three daughters live 800 metres
  away in the main house.
• Mr Alloy has always been up early and a
  seven-day-a-week farmer. He has a collection of
  guns, having established the local gun club and
  has used the guns to shoot the massive flocks of
  cockatoos that can destroy the fields of wheat.

57
Case scenario

• Thom has removed most of the ammunition from the
  cottage but is reluctant to take Mr Alloys
  beloved gun collection.
• Shedra cooks, cleans and takes Mr Alloy to all
  the necessary appointments which she makes sure
  are early in the mornings because as she
  describes, Mr Alloy is too damn irritable after
  lunch.
• Shedra had arranged for home help through the
  local council but Mr Alloy chased the woman out
  of the cottage threatening to shoot her.

58
Case scenario

• Shedra is afraid of Mr Alloy and always makes one
  of her daughters accompany her when she has to
  deal with him. Mr Alloy has often mistaken her
  for his deceased wife and tried to kiss and
  cuddle her and there are other times when he
  becomes uncontrollably angry and threatens to
  shoot everyone he sees.
• There are times when Shedra is convinced Mr Alloy
  knows exactly what is happening, who she is and
  she knows he is putting it all on.
• She has had enough and has given Thom an
  ultimatum of either making Mr Alloy have home
  help and have all the guns removed from the house
  or she will make sure his is locked up in an old
  folks home.

59
Case scenario

• It has placed their relationship under a lot of
  strain.
• Currently Thom showers his father three times a
  week.
• Mr Alloy is physically fit except for a very high
  BP. He is on an antihypertensive and some
  aspirin. His GP had prescribed Haloperidol but
  Thom threw it out after Mr Alloy had had trouble
  walking and eating.
• Mr Alloy naps throughout the day but as far as
  Thom and Shedra know, he sleeps through the night

60
Part 2 three weeks on

• Thom does not remove all of the guns from the
  cottage as promised to the Aged Persons Mental
  Health Team Case Manager and Mr Alloy fires a
  weapon at Shedra.
• Mr Alloy was admitted to the Acute Admission Unit
  for assessment three weeks ago.
• Mr. Alloy is now disorientated in the unit and
  becomes lost easily.

61
Case scenario

• He sleeps most of the night and naps through the
  day.
• Mr. Alloy also requires a soft diet as his
  dentures are causing him pain since the injury to
  his face.
• He does not appear to recognise Shedra, Thom or
  the grandchildren.
• Thom is very worried about his father being
  unwell enough to make it back home and Shedra is
  blaming herself for the situation as she knows if
  she had left well enough alone and the psychiatry
  team hadnt interfered, things would have gone
  along the same way and none of this would have
  happened.

62
Case scenario

• He is placid and compliant but he refuses to
  shower, although can be coaxed into a wash. He
  has been urinating in the hand basin.
• His BP is very high and he has fallen twice. He
  has skin tears to his left arm and a severely
  bruised face.
• Since his last fall, Mr Alloy is fearful of
  walking and will not move without full assist.

63
Group rate

• HoNOS 65
• Focus of Care
• RUG-ADL
• Group feedback

64
BASIS-32
65
BASIS-32

• Current national protocol leaves choice of
  self-rating measure up to the States
• BASIS-32 selected by Victoria as interim
  self-rating measure in 2000
• 32-item consumer self-rating measure
• Mental Health Branch of DHS coordinating a
  national review of consumer self-rating
  instruments

66
BASIS-32 Subscales

• Relation to self and others
• Daily living and role functioning
• Depression and anxiety
• Impulsive and addictive behaviour
• Psychosis

67
Offering the consumer self-report measure

• General rule always offer the consumer
  self-report measure. Exclusions
• Complements the clinician-rated measures.
• Consumer self-rating information is subject to
  the same rules of confidentiality and privacy as
  all the other information held in their file.
• Explain why is it important that I complete this
  questionnaire.
• Non-completion will not have any detrimental
  effect on treatment.
• Encourage them to answer all the questions but
  accept partial completions.
• Explain who is going to use the information.
• Explain what is the information going to be used
  for.

68
When not to offer the selfreport measure

• General contra-indications
• The consumers cognitive functioning is
  insufficient to enable understanding of the task
  as a result of an organic mental disorder or an
  intellectual disability
• or
•  Cultural or language issues make offering the
  BASIS-32 inappropriate.
• It is anticipated that the BASIS-32 will be made
  available
• in translation in a range of community languages.
•  

69
When not to offer the selfreport measure

• Temporary contra-indications
• Where the consumers current clinical state is of
  sufficient severity to make it unlikely that
  their responses to a BASIS-32 questionnaire could
  be obtained or, if their responses were obtained,
  it would be unlikely that they were a reasonable
  indication of the persons feeling and thoughts
  about their current emotional and behavioural
  problems and wellbeing.
•  Where an invitation to complete the BASIS-32 is
  likely to be experienced as distressing or
  requires a level of concentration and effort the
  person feels unable to give.
• Consumers who meet any of the above exclusion
  criteria should be offered an opportunity to
  complete a BASIS-32 at the future time when they
  do meet the criteria.
• Record the reason for non-completion in the CMI
  Wellbeing Module

70
Opportunity to self-report

• At all other times, a client should be given an
  opportunity to participate in outcome measurement
  by completing a BASIS-32.
• Conversely, consumers who request a copy of the
  BASIS-32 should generally be given the
  opportunity to complete one, even if the data
  collection occasion or the service setting does
  not require the BASIS-32 to be offered routinely.
  
• Bear in mind that the NOCC protocol specifies
  minimum requirements and that it is acceptable
  for either the consumers or the clinician to make
  and enter additional ratings.

71
Completion of the BASIS-32 is strictly
voluntary

• Consumers can be offered completion by
  themselves, with assistance or as an interview
  with the clinician or case manager.
• Consumers should be encouraged to seek assistance
  of their carer or a staff member if they need
  help to complete it.
• Under most conditions, the BASIS-32 takes only 10
  to 15 minutes to complete.

72
Missing data

• Missing data are not included in the calculation
  of the BASIS-32 subscale or overall mean scores.
• If more than five items have been omitted, the
  entire instrument will be considered missing
  data and will not be counted as a valid record
  in the Wellbeing Reporting Tool.

73
Making it worthwhile

• Consumer participation means more than just
  asking
• consumers to complete self-report
  questionnaires.
• It means
• involving consumers in ongoing mental health
  education
• having a genuine interest in consumers views
• incorporating consumers perceptions into
  individual care plans
• sharing the knowledge gained from the clinician
  ratings with the consumer
• following up differences between consumer and
  clinician ratings
• integrating the results into individual care plans

74
To deliver training know

• Your local protocol
• Why were implementing OM
• Where the OM initiative fits with other projects
  quality, information management, incentive
  strategies, PCP etc
• Key milestones for implementation in your service
• How important the training of staff is

75
Training materials

• Training manual includes
• Background and rationale
• Training vignettes and recommended ratings
• Paperwork necessary for training
• LCD vignettes/presentations
• CD-ROM copy
• Clinicians reference guide
• Always ask for assistance if you need it !

76
What help is available?

• Train the trainer is provided
• Training materials
• Website includes everything
• Rater guides
• Consumer Information and How To
• DHS project implementation team
• Local implementation group and Outcomes
  Coordinator
• Helpline on Wellbeing Reporting Tool
• Subject to availability of experts, some support
  for you in delivering training

77
Eight keys to effective training

• See yourself as the facilitator
• Promote a positive workshop environment
• Learning is more than most of us think
• Approaches to learning and training
• Feedback and evaluation
• Prepare for the workshop
• Pitch the material at the targeted audience
• Dont try and answer questions you are not sure
  of the answer to

78
Questions to think about

• Who do I need to provide training to?
• Do different people need different information,
  skills?
• Can I make assumptions about prior knowledge?
• Does our service have training priorities?
• How will I recruit people?
• Manager direction, volunteers, flyers?
• Why will people want to attend?
• Costs venues, materials, backfill, travel?
• How will I know the training has worked?

79

• How will we trainers be able to get feedback on
  the compliance from your training?
• Is the training in step with the resources needed
  or is there an extended lag time?
• How do we sustain interest for initial 3-6
  months?
• Will we provide ongoing and refresher training?
• Will there be an established program or ad-hoc?
• Who do we need to meet with to promote and
  sustain the training?
• Clinically who will buy into the use of the OM
  data at clinical forums/discussions?

80
What now?

• Know what is expected from you
• Know your material
• Work out how you are going to deliver the
  training to the raters

81
Group activity

• Divide into Service-Specific Groups
• List at least five issues you have to address
  before you start rater training.
• Select two issues and design how to
  address/resolve them.
• Feedback and discussion

82

• GOOD LUCK