Quality Network for Inpatient CAMHS QNIC ROUTINE OUTCOME MEASUREMENT SERVICE

1
Quality Network for In-patient
CAMHS(QNIC)ROUTINE OUTCOME MEASUREMENT
SERVICE
The Royal College of Psychiatrists Centre for
Quality Improvement Charity registration number
228636

• Farah Khalid
• fkhalid_at_cru.rcpsych.ac.uk

2
Aims of QNIC ROM

• To develop, implement and support a model of
  routine outcome evaluation specifically for
  inpatient CAMHS
• Work in collaboration with CORC
• To collate and analyse (anonymised) information
  from all ROM members in order to share findings
  within and between services 
• To assist in allowing the information obtained to
  inform service users, providers and commissioners
• To collaborate with ROM members in using outcome
  information to inform and develop service
  evaluation and good practice

3
Aims of Today

• Outline and discuss practical ways to begin and
  continue collecting information
• Communicate how process can be smooth and
  streamlined as possible for you unit
• Your Unit please interrupt and join in the
  discussion
• Feedback from pilot study has led to many
  amendments but still an evolving service
• Take home message - We can do this!

4
Course of the day

• 10.45 - 11.00 Welcome and Introductions Peter
  Thompson
• 11.00 - 11.30 Introduction to QNIC ROM Farah
  Khalid
• 11.30 - 11.45 Tea / Coffee
• 11.45 - 12.45 Principles of ROM / How to make the
  most of ROM Professor Simon Gowers
• 12.45 - 1.30 Lunch
• 1.30 - 1.50 Talk through online system Farah
  Khalid
• 1.50 - 3.00 Workshop Entering data and
  submitting online questionnaires
• 3.00 - 3.15 Tea/Coffee
• 3.15 - 3.30 Review day / QA
• 3.30 Depart

5
QNIC ROM Process

• Information to be collected at 2 time points
• Admission and 4 months (or discharge depending on
  what comes first)
• Information to be collected from multiple
  perspectives to provide robust measures
• Clinician
• Self (young person)
• Parent/carer

6
Core Measures

• Health of the Nation Outcome Scales for Children
  and Adolescents (HoNOSCA)
• Clinician
• Self
• Parent
• The Childrens Global Assessment Scale (CGAS)
• Clinician rated only

7
Core Measures

• Strengths and Difficulties Questionnaire (SDQ)
• Self
• Parent
• Teacher
• CAMHS Satisfaction Survey (CAMHSSS)
• Self
• Parent

8
Measures

• Baseline and follow up variables
• Demographics
• Diagnosis
• Severity of condition
• Length of stay
• References on reliability and validity of the
  core measures are included in the handbook

9
Current Optional Measures

• EDE eating disorders examination gold star
• EDE-Q self rated version
• Others?

10
Measures

• All measures, score sheets, further information
  including links to external websites are
  available on the QNIC ROM web pages.

11
Measures - Training

• QNIC does not offer training on specific measures
• HoNOSCA http//www.liv.ac.uk/honosca
• CGAS rating guide available - website/manual
• SDQ no training needed
• EDE workshop offered by University of Oxford

12
Submitting data

• Information is submitted online through the QNIC
  web pages
• Each unit is provided with their own username and
  password to access the secure site
• SNAP Surveys Software used

13
ADMISSION QUESTIONNAIRE Required
processQuestionnaire to be completed and
submitted within 2 weeks of admission

• Go to QNIC ROM
• webpage and login
• using allocated unit
• username
• and password to
• access
• a new (or saved)
• questionnaire

Clinician rated tools complete on or just after
assessment Baseline variables inc. demographics,
diagnosis HoNOSCA CGAS
Young people / parents / teachers to
complete Self-rating HoNOSCA / SDQ Parent
rating HoNOSCA / SDQ Teacher rating SDQ
Print off completed questionnaires and place in
young persons file UPDATE RECORD SHEET Press
SUBMIT
QNIC to analyse results
Collate all information from various sources
and enter online (SNAP system)
14
FOLLOW UP QUESTIONNAIRE Required
processQuestionnaire to be completed at 6 months
or discharge
Young people / parents / teachers to
complete Self-rating HoNOSCA / SDQ / CAMHSSS
Parent ratings HoNOSCA / SDQ / CAMHSSS Teacher
rating SDQ

• Go to QNIC ROM webpage
• and login using allocated
• unit username and
• password to access a new
• (or saved) questionnaire
• Re-enter unique code of
• young person allocated at
• admission

Clinician rated tools Demographics, followup
variables HoNOSCA CGAS
Print off completed questionnaires and place in
young persons file UPDATE RECORD SHEET Press
SUBMIT
Receive electronic reports from QNIC with
analyses of information
Collate all information from various sources
and enter online (SNAP system)
15
Alternative Returns

• All information can now also be sent via
• Post
• Fax
• Email (scan the documents)
• QNIC will enter the information online
• There is a 50 charge per young person
• This includes admission, discharge and all
  intermittent information

16
Information Received

• What happens with the raw data?
• Analysis frequencies (socio-demographic data,
  bar charts outcomes)
• The more information we receive the more
  meaningful it will be
• Mean of one unit can be compared with mean of all
  units.

17

• Reporting Outcome Data

18
HoNOSCA - clinician rated

• (for the 9 young people with ratings at both
  admission and discharge)

Scores at discharge were significantly lower than
at admission (t4.581, df8, p0.002). Lower
HoNOSCA scores denote less severe problems.
19
HoNOSCA self rated

• (for the 7 young people with ratings at both
  admission and discharge)

Scores at discharge were not significantly
different from those at admission (t1.812, df6,
p0.120).
20
HoNOSCA parent rated

• (for the 7 young people with ratings at both
  admission and discharge)

Scores at discharge were significantly lower than
at admission (t2.750, df6, p0.003). Lower
HoNOSCA scores denote less severe problems.
21
CGAS-Admission and Discharge Scores

• (for the 9 young people with ratings at both
  admission and discharge)

Scores at discharge were significantly higher
than at admission (t-3.668, df7, p0.008).
Higher CGAS scores denote less severe symptoms.
22
Feedback

• Regular feedback is essential
• Regular individual reports - your reports to
  share and feedback to all team members, users and
  commissioners
• Feedback needs to be a two way process with QNIC
  too

23
Key Findings from Pilot Study

• Each unit must have a designated person to lead
  outcome measurement.
• Under this lead an assistant psychologist /
  psychologist / administrator etc must be in place
  to ensure clinicians, young people and parents
  complete the measures and then take
  responsibility for collating the information for
  submission to QNIC.
• Those who found the pilot most challenging were
  those who tried to complete the entire process on
  their own. ROM needs to be integrated into the
  MDT. Ward round is one idea when to complete
  measures.
• Young people should enter data straight onto the
  computer instead of completing paper
  questionnaires (except SDQ). Direct them to the
  correct webpage and allow them to complete
  online.

24
Key Findings from Pilot Study

• Parents should be given the option of either
  entering data straight onto a computer (except
  SDQ) or completing paper forms. Ask parents to
  come in 15 20 minutes earlier to complete
  measures.
• A handbook has been produced for units to refer
  back to.
• The online questionnaires have been amended to be
  more streamlined and user friendly to assist in
  submitting data.
• A RECORD SHEET has been created for you to keep
  track of what has and has not been completed.

25
How to get it wrong!

• Do not designate a lead for outcome measurement.
• Do not have a worker to ensure measures are
  completed and data are submitted.
• Try and complete the entire process on your own
  and do not integrate it in the MDT.

26
Contacts and On-going Support

• Main ROM contact - Farah Khalid, Research Worker
  (fkhalid_at_cru.rcpsych.ac.uk)
• QNIC Programme Manager Peter Thompson
  (pthompson_at_cru.rcpsych.ac.uk)
• Unit visits will be made by myself if necessary
  to provide support and assistance
• ROM will be incorporated into QNIC Review Days