Implementing HoNOS in the Waikato: II Practitioners Perspective'

1
Implementing HoNOS in the Waikato II
Practitioners Perspective.

• John Fitzgerald, PhD.
• Karma Galyer, PhD.
• Philippa Thomas
• The Psychology Centre, Hamilton

2
Acknowledgements

• All participants (Waikato DHB staff) who
  completed the survey, attended focus groups, and
  generously shared their insights, ideas, and
  advice.
• Chris Harris, David Ireland, and Waikato DHB
  staff who assisted in accessing data

3
Project Aims

• To determine how Health Waikato mental health
  staff were collecting the Health of the Nation
  Outcome Survey (HoNOS)
• Usefulness of training
• Suitability and use of collection processes
• Completeness of data sets
• Assess organisational and/or personal barriers to
  collecting and using the HoNOS
• Staff perceptions and attitudes towards the
  collection of outcome data
• Views on data collection processes
• Impact of data collection and its application to
  practice
• Provide recommendations regarding training,
  system support, and processes which may enhance
  compliance with collection protocols, accuracy
  and utilisation of these data

4
Milnes eight stage approach to implementing
HoNOS (2001)

• Needs Assessment (why are you doing this? clarify
  organizational expectations.)
• Planning (what to measure, stakeholder
  collaboration)
• Working the organisation (address resistance)
• Pre-testing the instrument/processes (fine tune
  processes)
• Training and initiation
• Maintenance and management (support use, timely
  feedback, troubleshooting)
• Analysis and interpretation (consult, feedback)
• Re-engineering (transparent learning, data
  ownership, review, revision)

5
Data sources

• Attendance at Health Waikato HoNOS training
• Analysis of archival data
• Staff Survey
• Focus Groups

6
Summary of available archival data (July 2005
February 2007)
7
Reason for completion of forms (July 2005
February 2007)
8
Matched-Pairs, by client (July 2005 February
2007)
9
Data Collection Staff Survey

• 142 survey forms were returned (N550 fte, 650
  staff)
• Each team visited, multiple response
  opportunities.
• 72 of respondents were community staff (Adult
  Mental Health-52, MHSOP-11, CAMHS-9).
• 28 were inpatient services staff (Adult Mental
  Health-24, MHSOP-4).
• The majority of staff who returned the survey
  were nurses (56).
• The survey covered four areas,
• general information
• training
• completion
• use of the HoNOS

10
Data Collection Focus Groups

• Three focus groups were held Adult Community
  Mental Health (n 6), Community Services (7),
  and Inpatient Services (6).
• The focus groups discussed,
• HoNOS training
• completion of the HoNOS
• common barriers to completing questionnaires
• clinical utility of HoNOS
• broader service utility of the HoNOS

11
General comments
CAUTION - All suggesting inconsistent usage

• Usage
• 59 said HoNOS routinely used in their service
• 30 said HoNOS sometimes used
• Specific roles within teams were reported to have
  an influence on completion rates.
• Staff were unsure about some aspects of the HoNOS
  completion protocols, e.g., when transferring a
  client between services or when assessing and
  discharging a client with a short admission time
  (e.g., 24 - 48 hours).
• The current reminder system was reported as
  being problematic for many staff.
• Archived client HoNOS data was perceived as being
  inaccurate/inconsistent.

12
Difficulties/Barriers in completing the HoNOS

• Availability of help not a barrier
• Computer vs. paper submission
• preference for on-line completion
• inadequate computer resources
• poor software design
• Lack of clarity of the HoNOS questions, rating
  scales and manual
• unclear questionnaire layout (paper form)
• codes vague, scoring uncertain
• Feedback reports were not available/used
• rated as problematic, but generally not used
• ?clinical utility
• Forms/manual not readily available
• Who completes the HoNOS?
• passing the buck
• Difficulties getting enough information
• but not the data
• discharge forms easier to complete than intake
  more data available
• Competing paperwork and work demands
• HoNOS not prioritised

13
Clinical application of the HoNOS

• Accuracy
• HoNOS seen as clumsy, blunt, black and
  white
• The suitability of the HoNOS to the clinicians
  client group
• HoNOS seen as lacking sensitivity to youth,
  community out-patients
• No mention of cultural acceptability
• Felt like someone elses research project
• Time frame of information collected
• Two week window is not long enough
• Quality of information obtained
• Forced choice seen as unhelpful
• Deficits in scales that yield numeric summary
  scores only
• Clinical application in assessment
• 46 said HoNOS had no clinical assessment use,
  i.e., second-rate summary only
• Clinical application in treatment and monitoring
• Some saw value in monitoring progress with HoNOS

14
Recommendations

• Training
• Include practical tasks in training
• Expand information of the development and purpose
  of the HoNOS
• Review potential clinical applications
• Ensure initial training within 3 months of
  start-date
• Train on all relevant versions
• Consolidation of training (updates, support
  integration into practice)
• Seen as duplicating information training needs
  to cover rationale for collecting
  monitoring/outcome measures

15
Recommendations

• Data collection
• Usefulness Many clinicians found the HoNOS a
  useful monitoring tool
• Buy-in Information alone is unlikely to persuade
  staff that the HoNOS is worthwhile
• Data integrity Few matched-pair, without these
  the HoNOS (or any instrument) has limited utility
  as an outcome measure
• Collection format Clear trend to on-line
  completion
• Speed of input, error trapping
• Interactive analysis (if available)

16
Recommendations

• Resources
• Increased access to computer resources
• Allow time for HoNOS to be completed - resource
• Process
• Revise paper format
• Develop robust reminder system
• Protocol to specify who should complete HoNOS
• Consultation at point of transfer
• Integrate HoNOS completion into clinical case
  management and review processes (Use it or loose
  it)

17
The final word
the routine use of HoNOS will require
continued effort from staff and their managers
(e.g. refresher training courses) and alternative
or additional outcome measures may be necessary
to provide useful clinical information. (Milne,
Reichelt, Wood, 2001)