Title: The Urgent Consultation Clinic of the Ottawa HospitalGeneral Campus: Implementation and Outcome Eval
1The Urgent Consultation Clinic of the Ottawa
Hospital-General Campus Implementation and
Outcome Evaluation Frameworks J. Robert
Swenson, M.D., Colleen MacPhee, M.H.A.,John
Kowal, Ph.D., Mary-Clare Royle, M.D., Katharine
Gillis, M.D., Tim Aubry, Ph.D.University of
Ottawa Centre for Research on Community
ServicesJune 2007 - National SHARED Care
Conference
2Objectives
- Framework of our program evaluation
- Orientation to the outcome evaluation of the UCC
- Evaluation design
- Clinician rating measures
- Outcome Questionnaire
- Other self-report measures
- Procedures
- Preliminary Results
3Program Definition
- A more or less coherent system of inputs,
activities, outputs, and outcomes that have been
organized to address specific needs and resolve
particular problems of a given population - e.g., Urgent Care Clinic
4Program Evaluation Definition
- The systematic collection of information
about the activities, characteristics, and
outcomes of programs to make judgements about the
program, to improve program effectiveness, or to
inform decisions about future programming
5Make-Up of a Program
PROGRAM MISSION
PROGRAM PROCESSES
PROGRAM OUTCOMES
NEEDS
INPUTS
INITIAL OUTCOMES
INTERMEDIATE OUTCOMES
GOALS OBJECTIVES
ACTIVITIES
VALUES PRINCIPLES
OUTPUTS
LONGER-TERM OUTCOMES
RELATIONSHIP TO THE THREE MAIN TYPES OF PROGRAM
EVALUATION
NEEDS ASSESSMENT
IMPLEMENTATION / PROCESS EVALUATION
IMPACT / OUTCOME EVALUATION
6Purposes of Program Evaluation(Posavac Carey,
1993)
- Develop and improve programs
- Identify program outcomes
- Report to funders
- Help qualify for accreditation
- Respond to requests for information
7The Ottawa Hospital Urgent Consultation Clinic
(UCC)
- Interdisciplinary program providing short-term
outpatient mental health services. - Patients referred from Psychiatric Emergency
Services and Medical/Surgical Services and
Psychiatric Inpatient Services. - UCC team Psychiatry, Psychology, Nursing, and
Social Work - Services provided psychodiagnostic assessments,
medication management, short-term psychotherapy,
and liaison with community mental health
services.
8 The Ottawa Hospital Urgent Consultation Clinic
Program Logic Model
ASSUMPTION An interdisciplinary Urgent
Consultation Clinic (UCC) mental health team as
part of a comprehensive range of intensive-level
psychiatric services at The Ottawa Hospital (TOH)
will result in improved access to, and quality
of, psychiatric care for outpatients with mental
health problems
GOAL To demonstrate that this Urgent
Consultation Clinic model improves quality of
mental health care.
- ACTIVITIES
- Referrals are screened for acuity and
appropriate level of intervention to determine
suitable services - Psychiatry
- Psychodiagnostic assessment
- Medication review and management
- Short-term psychotherapy and psycho-education
- Resident training
- Psychiatric Nursing
- Assessment
- Management of patients with co-morbid mental
health and physical problems - Symptom and medication monitoring
- Counselling and psycho-education
- Psychiatric Social Work
- Assessment
- Individual and family therapy
- OBJECTIVES
- To provide short-term outpatient mental health
services for patients referred by - Psychiatric emergency services (PES)
- Medical/surgical services
- Emergency services
- Inpatient psychiatry
- To improve patients mental health and quality
of life
- OUTPUTS
- of patients referred
- of patient assessments
- of patients seen in follow-up counseling or
therapy - of follow-up counseling or therapy sessions
- of patients seen for medication management
- of patients referred to the Acute Partial
Hospitalization Program - of patients referred to a community-based
mental health professionals - of patients discharged to family physician
- SHORT-TERM OUTCOMES
- Patients
- Increased patient access to mental health
programs and services - Increased knowledge of mental health issues
- Improved participation in patient-centered
treatment plan - Increased psycho-social support
- Increased knowledge of adaptive coping skills
and strategies - Program
- Improved triage system for incoming referrals
- INPUTS
- UCC Team
- Psychiatry (2.5
- FTE)
- Residents (1.3 FTE)
- Social work (0.5 FTE)
- Nursing (2.0 FTE)
- Psychology (0.3 FTE)
- Resident (0.1 FTE)
- Support staff (1.5 FTE)
- Office space and equipment
- LONG-TERM OUTCOMES
- Patients
- Decreased severity of mental health problems
- Improved functioning
- Increased coping skills and strategies
- Improved quality of life
- Increased satisfaction with acute mental health
services - Mental Health Practitioners
- Enhanced training for psychiatry and psychology
residents in the provision of acute mental health
services
9UCC Outcome Evaluation Questions
- Do patients experience a decrease in the severity
of their mental health problems over the course
of receiving UCC services? - Do patients experience an improvement in
functioning over the course of receiving UCC
services? - Do patients experience an improvement in their
quality of life over the course of receiving UCC
services? - Are patients satisfied with UCC services?
10Clinician Rating Measures
11Threshold Assessment Grid(Slade, Powell, Rosen
Strathdee, 2000)
- Assesses the severity of mental health problems
- Ratings obtained on 7 domains in 3 areas
- Safety (intentional unintentional)
- Risk (from to others)
- Needs Disabilities (survival, psychological
social difficulties) - Response alternatives range from none (0) to very
severe (4) - Total scores obtained by summing scores across
domains - Cut-off total score of 5 or more or at least 2
moderates - Good psychometric properties
12Global Assessment of Functioning(Spitzer,
Gibbon, Williams Endicott, 1994)
- Overall Psychosocial Functioning
- psychological symptoms, occupational social
functioning - 100-point single-item rating scale
- Values range from 1 (poorest functioning) to 100
(highest functioning) - Cut-off points
- Reasonably good psychometric properties
13Patient Self-Report Measures
14Outcome Questionnaire(Lambert, 1996)
- 45 item version most commonly used
- Educational level sixth grade
- Symptom distress, interpersonal relations, social
role - Response alternatives - Never (0) to Almost
Always (4) - OQ-45 5 minutes
- Total score ranges from 0 to 180 on the OQ-45
15Outcome Questionnaire Items (Lambert, 1996)
- Symptom Distress
- My heart pounds too much
- Interpersonal Relations
- I am satisfied with my relationships with
others - Social Role
- I feel stressed at work / school
16Outcome Questionnaire (Lambert et al., 1996)
Strengths
- Brief and simple
- Good psychometric properties
- Sensitive to changes
- Economical
- Measures multiple dimensions of well- being
- Identifies difficulties found in wide range of
mental disorders - Useful for clinicians and administrators
17Outcome Questionnaire(Lambert, 1996)
Scoring Interpretation
- Total (addition of items)
- Critical items (8,11,26,32,44)
- Cut-off scores OQ-45 64
- Clinical change OQ-45 14 or more
18(No Transcript)
19Possible Outcomes Based on OQ-45 Scores
64 ----------------------------------------------
--------------------------------------------------
------
20Comparison of Client Scores on the OQ-45F at
Intake to the Scores of Normative Samples(OQ
Measures, LLC, 2004)
- Undergraduate students (n 235)
42.15 - Community residents (n 815)
45.19 - EAP clients (n 441)
73.61 - University counselling centre (n 486)
75.16 - Medical Clinic (n166)
80.92 - CLSC II (n70)
81.80 - Outpatient clinics (n 342) 83.09
- Community mental health centre (n100)
86.07 - CLSC I (n174)
87.41 - Inpatient clinics (n 207)
88.80 - Psychiatric Day Hospital Programme (n 158)
104.15
21Patient Health Questionnaire (PHQ)(Spitzer,
Williams, Kroenke, et al., 1999)
- Based on the PRIME-MD
- Screen for most common groups of disorders (i.e.,
depressive, anxiety, alcohol, somatoform, and
eating disorders) - Additional categories problems with
menstruation, pregnancy, childbirth, psychosocial
stressors - 16 sub-sections questions range from past
2-weeks to past 6-months - presence of distress (Y/N), intensity (not
bothered, a little, a lot), frequency (not at
all, several days, more than half of days, nearly
every day), social/occupational impairment - very good psychometric properties
22SF-12(Ware,1996)
- Generic Health Measure
- 12-items
- Two scales (physical and mental health overall
health-related quality of life) - Total score ranging from 0 (lowest level of
health) to 100 (highest level of health) - Physical Mental Health Composite Scores
- Good psychometric properties
23SF-12
- Flow Chart for Scoring
- Enter data
- Recode out-of-range item values as missing
- Reverse score and/or recalibrate scores for four
items - Compute raw scale scores
- Transform raw scale scores to 0-100 scale
24Satisfaction with Life Scale (Diener, Emmons,
Larsen, Griffin, 1985)
- Global measure of subjective well-being
- Cognitive appraisal of life situation
- 5-item self-report measure
- 7 response alternatives
- Strongly Disagree to Strongly Agree
- Total score ranging from 5 to 35
- Very good psychometric properties
25Client Satisfaction Questionnaire
- Measures overall level of satisfaction with
services - Use of 8-item version in study
- Widely used allowing for comparison to other
similar settings - Possible 4 response alternatives
- 2 reflecting satisfaction
- 2 reflecting disatisfaction
- Very good psychometric properties
26UCC Evaluation Methods
27UCC Outcome Evaluation Design
28UCC Outcome Evaluation Design
29UCC Evaluation
- Preliminary Results
- (n31)
30Baseline Characteristics(n31)
- Age (yrs.)
- Mean SD 36.9 12.1
- Range 20 - 62
- Sex
- Female 11 (36)
- Male 20 (64)
- Language
- English 29 (94)
- French 2 (6)
- Educational level1
- Secondary 15 (56)
- College 6 (22)
- University 6 (22)
- Employment status2
- Full- or part-time 15 (52)
- Unemployed 14 (48)
- _________________________________________________
- 1Date are missing for 4 participants.
- 2Date are missing for 2 participants.
31Referral Source and Follow-up
- Participants
- Variable n 31
- _________________________________________________
- Referral source
- Psychiatric emergency services 5 (16)
- Emergency department 7 (23)
- Inpatient psychiatry 1 (3)
- Medical/surgical services 18 (58)
- Follow-up planned
- Yes 9 (29)
- No 22 (71)
- _________________________________________________
- seen on a single occasion and, referred to other
mental health services
32Baseline Clinical Characteristics(n31)
- Psychiatric diagnosis
- Axis I
- Mood disorder 18 (58)
- Adjustment disorder 1 (3)
- Delerium 1 (3)
- Schizophrenia 2 (6)
- Axis II
- Schizoid PD 1 (3)
- Antisocial PD 3 (10)
- Borderline PD 3 (10)
- Narcissistic PD 2 (6)
- Dependent PD 3 (10)
- Concurrent condition 11 (36)
- Psychological measures (mean SD)
- Satisfaction with life scale 16.0 7.8
- GAF 59.9 8.9
- TAG 4.2 2.6
33Future Directions
- Recruit 150 participants
- Examine outcomes over time (e.g., change in
clinical status, quality of life, symptoms, etc.)
based on patient and clinician ratings - Ascertain to what extent the UCC is being
implemented as intended (e.g., time from referral
to initial clinic visit, number of sessions,
types of services provided, etc.) - Improve the quality of clinical services by
integrating study results into clinical practice - Disseminate study findings to relevant
stakeholders