Introducing the COJAC Screener: A Short Screening Instrument for COD and Trauma

1
Introducing the COJAC Screener A Short Screening
Instrument for COD and Trauma
Vivian B. Brown, Ph.D. PROTOTYPES Ken Bachrach,
Ph.D. Tarzana Treatment Center Lisa Melchior,
Ph.D. The Measurement Group
2
What is COJAC?

• In the summer of 2005, the State Co-Occurring
  Disorders Workgroup/COD Policy Academy members,
  along with representatives from the County
  Alcohol and Drug Program Administrators
  Association of California (CADPAAC) and the
  California Mental Health Directors Association
  (CMHDA), formed the Co-Occurring Joint Action
  Council (COJAC) to develop and implement the
  States COD Action Plan

3
The Screening Committee of COJAC

• One of the major objectives of the COJAC State
  Action Plan was to identify screening protocols
  designed to meet the needs of a variety of
  populations served by both AOD and Mental Health
  Systems, including adolescents, women with
  children, adults, and transition age youth with
  trauma
• The Screening Committee was established chair of
  the committee is Dr. Vivian Brown

4
The Screening Committee of COJAC

• The Committee was charged with identifying the
  best screening tool(s) for COD
• The Screening Committee identified all
  instruments being utilized across the country we
  found that the most widely used instruments were
  those designed either for identification of
  substance abuse or identification of mental
  illness

5
The Screening Committee of COJAC

• We, therefore, decided to design a California
  screening tool that not only would identify COD,
  but would be short enough to not burden clients
  nor staff, and simple enough to be utilized in a
  wide range of community service sites (including
  emergency rooms)

6
What is the COJAC Screener?

• The Co-Occurring Disorders Screening Instrument
  is composed of 9 questions
• 3 questions on mental health
• 3 questions on alcohol and drug use
• 3 questions on trauma
• These questions were adapted from the
  Collaborative Care Project, Canada, and the
  Co-Morbidity Screen from the Boston Consortium

7
(No Transcript)
8
Pilot Testing

• Pilot testing of the screener was implemented by
  PROTOTYPES and Tarzana Treatment Center in May
  2007 both agencies have Community Assessment
  Service Centers (CASCs), AOD programs, and mental
  health programs
• The Screening Committee set the cut point for
  pilot testing low at 1 point, in order to test
  the COJAC Screener with the GAIN Short Screener
  (SS) and the Addiction Severity Index (ASI)

9
Pilot Testing

• Tarzana did not use the GAIN only the ASI
  Tarzana piloted the COJAC Screener on 1,386
  clients, including 51 AOD clients, 12 CASC, 31
  primary care, 5 ER, and 2 high school students
• PROTOTYPES piloted on 365 CASC clients over 10
  had COJAC Screener and GAIN SS 90 had COJAC and
  ASI

10
Select Data from PROTOTYPES
Type of COD with Positive Screen Type of COD with Positive Screen Type of COD with Positive Screen Type of COD with Positive Screen
Frequency Valid Percent Cumulative Percent
None 42 11.5 11.5
MH Only 21 5.8 17.3
AOD Only 95 26.0 43.3
Trauma Only 6 1.6 44.9
MH AOD 55 15.1 60.0
MH Trauma 20 5.5 65.5
AOD Trauma 37 10.1 75.6
MH AOD Trauma 89 24.4 100.0
Total 365 100
11
Select Data from Tarzana
Yes Responses to Mental Health Questions Yes Responses to Mental Health Questions Yes Responses to Mental Health Questions Yes Responses to Mental Health Questions Yes Responses to Mental Health Questions Yes Responses to Mental Health Questions
Primary Care CASC School AOD ER (Medical) OV CASC
15 24 42 47 63 70
Yes Responses to Alcohol and Drug Use Questions Yes Responses to Alcohol and Drug Use Questions Yes Responses to Alcohol and Drug Use Questions Yes Responses to Alcohol and Drug Use Questions Yes Responses to Alcohol and Drug Use Questions Yes Responses to Alcohol and Drug Use Questions
Primary Care CASC School AOD ER (Medical) OV CASC
14 45 46 72 80 92
Yes Responses to Trauma/Domestic Violence Questions Yes Responses to Trauma/Domestic Violence Questions Yes Responses to Trauma/Domestic Violence Questions Yes Responses to Trauma/Domestic Violence Questions Yes Responses to Trauma/Domestic Violence Questions Yes Responses to Trauma/Domestic Violence Questions
Primary Care CASC School AOD ER (Medical) OV CASC
10 13 25 30 30 30
12
Pilot Testing Results

• What the Screening Committee was attempting to
  answer with this first pilot was
• Will this short screener pick up potential COD
  and will it be correlated with longer screening
  instruments?
• Will this screener be easy to administer not
  burden client nor staff?
• Are the results good enough to begin
  implementation?

13
Question 1 Picking up COD

• PROTOTYPES Sample
• More than half of those screened (55.1),
  screened positive in at least 2 of the 3 domains
• 30.7 screened positive in 2 domains
• 24.4 screened positive in all 3 domains
• Only 11.5 did not identify problems in any of
  the 3 domains

14
Question 1 Picking up COD

• Tarzana Sample
• We get an interesting picture of responses across
  6 groups across groups, 76 responded with at
  least one yes response
• Primary Care 4-18 responded positive to all
  questions 23 have been worried about MH
• ER 52 worried about thinking, etc. (MH)
• Olive View CASC 79 positive for MH, 90-95
  positive for AOD 53 positive for partner DV
• High School 40 worried about MH 31 harmed
  self or thought of harming self 42 AOD, 42
  partner abuse 35 physical abuse

15
Question 1 Correlated with Other Screeners

• PROTOTYPES Sample
• The 3 MH screening questions appear to have a
  strong relationship with the GAIN MH measure
  (both internal and external) and the ASI
  psychiatric problem severity
• The 3 AOD questions appear to have a strong
  relationship with the GAIN substance disorder
  measure, but minimal to the ASI AOD severity
  measures
• The 3 Trauma questions appear strongly related to
  the GAIN crime/violence measure (more than the MH
  measure) and also appear to relate strongly with
  the ASI psychiatric problem severity

16
Question 1 Correlated with Other Screeners

• Tarzana Sample
• Tarzana data looked at item by item
• Responses to MH COJAC and ASI were in the same
  direction and chi-square tests were strong
• For AOD, COJAC and ASI, drugs were in the same
  direction, but alcohol questions were not
• There was only one ASI question to compare COJAC
  and ASI trauma responses were in same direction,
  but not significant

17
Question 2 Comfort/Not Burdensome

• In discussions with PROTOTYPES staff and Tarzana
  staff, it appeared that neither staff nor clients
  were burdened by the COJAC Screener

18
Question 3 Results Good Enough to Go to Next
Steps

• From both pilot sites, the answer appears to be
  yes

19
Limitations of Pilot Testing

• While we set the cut point low at 1 yes, we do
  not have data analyzed for negatives i.e., those
  people who had zero on COJAC and GAIN data
  would GAIN have picked up other problems?
• We did not analyze by gender and this could
  explain some issues of the trauma questions

20
Next Steps

• A meeting was held with all CASC directors to
  discuss the positive results of the pilot testing
  and possible implementation by all CASCs
• Los Angeles and other Counties have decided to
  begin implementation of the COJAC Screener, with
  the cut point raised to a minimum of two yes
  responses one in MH and one in AOD or one in
  either MH and AOD and one in trauma
• State ADP is implementing an expanded pilot test
  of the Screener

21
The COJAC Screening Committee

• Vivian B. Brown, Prototypes Chair
• Carmen Delgado, ADP
• Terry Robinson, ADPI
• Tom Metcalf
• Karen Streich, LA County DMH
• Lisa Melchior, The Measurement Group
• Sandy Mills, LA County DMH
• John Sheehe, LA County DMH

22
The COJAC ScreenerAlbert Senella, Ken Bachrach,
Ph.D. Clarita Lantican, Ph.D.Tarzana
Treatment CentersSixth Annual Conference on
Co-Occurring DisordersLong Beach, CAFebruary 7,
2008
23
Survey Timeframe Sites

• Data collected the entire month of May 2007
• Tarzana Treatment Center sites
• Inpatient detox
• 3 adult residential programs in Tarzana, Long
  Beach and Lancaster
• 1 youth residential program in Lancaster
• 2 outpatient programs in Tarzana and Lancaster
• 2 substance abuse assessment centers in Tarzana
  and Lancaster
• Northridge Hospital Medical Center ER
• Olive View Hospital ER

24
Survey Participant Groups

• AOD TTC patients admitted to Detox, Residential
  and Outpatient programs at all sites.
• Primary Care TTC Family Clinic patients in
  Tarzana and Lancaster.
• CASC Clients referred to Community Assessment
  Service Centers in Tarzana and Lancaster for
  substance abuse assessment.
• ER (Medical) Emergency Room patients at
  Northridge Hospital and Olive View Medical
  Center.
• Olive View CASC Clients referred to CASC for
  psychiatric assessment.
• School High School students in Lancaster
  participating in a substance abuse and HIV
  prevention project.

25
Survey Participants

• A total of 1,386 patients/clients participated in
  the survey
• 51 AOD patients
• 31 primary care clinic patients
• 12 Community Assessment Service Center (CASC)
  clients
• 5 ER patients for medical psychiatric care
• 2 high school students

26
Positive Responses for MH, AOD and Trauma /
Domestic Violence Based on COJAC Screen
  MH AOD Trauma / Violence
  Percent Percent Percent
Primary Care 15 14 10
CASC 24 45 13
School 42 46 25
AOD 47 72 30
ER(Medical) 63 80 30
OV-CASC 70 92 30

27
Comparison of AOD patient responses to COJAC and
ASI for similar items
Mental Heath COJAC (n 697) ASI (n 201)
Have you ever harmed yourself or thought about harming yourself? (COJAC) 75
Serious thoughts of suicide lifetime (ASI) 80
AOD
Have you ever had any problem related to your use of alcohol or other drugs? (COJAC) 78
Troubled or bothered by drug problems in the past 30 days (ASI) 77
28
Findings from Pilot Test

• The comparison between COJAC and ASI of MH and SA
  questions are statistically significant.
  Overall, the responses are in the same direction.
  
• The comparison between COJAC and ASI questions
  for trauma and domestic violence are not
  statistically significant. This can be explained
  by the fact that the ASI does not have a question
  that is a good match for COJAC questions.

29
Conclusions

• The findings of the survey provide valuable
  insights on the history of TTC patients/clients
  concerning mental health, AOD and trauma/domestic
  violence issues.
• The findings provide TTC the capability to
  identify the needs of patients/clients as part of
  substance abuse treatment.
• The findings provide insights to prioritize the
  patients/clients in addressing their needs.
• More importantly, the findings show the validity
  of the COJAC tool as a screening tool.

30
COJAC Co-Occurring Disorders Screening
Instrument Pilot Test

• Lisa A. Melchior, Ph.D.The Measurement
  GroupCulver City, California

In collaboration with Vivian B. Brown, Ph.D. and
G. J. Huba, Ph.D. with additional contributions
from Aaron Griffith, MA and Eva Sofia Mendoza.
Pilot study data collection protocols were
designed by the COJAC COD Screener Subcommittee.
31
About these Pilot Test Data

• PROTOTYPES collected pilot test data for the
  COJAC Co-Occurring Disorders Screening Instrument
  April May 2007
• Data were collected from 365 individuals at the
  PROTOTYPES SPA 3 CASC locations in El Monte,
  Pomona, and Pasadena
• n 323
• n 268 with data from the COD screening
  instrument, Addiction Severity Index (ASI)
  composite scores, and ASI severity ratings
• n 34 with data from the COD screening
  instrument and the GAIN Short Screener (GAIN SS)

32
Preliminary Validity Evidence

• The COD screening items and composites are
  moderately correlated with ASI and GAIN measures
  of comparable constructs

33
(No Transcript)
34
Mental Health Screening

• The three mental health screening items on the
  COD screening instrument appear to have a strong
  relationship with ASI measures of psychiatric
  problem severity and GAIN mental health measures
  of internalizing and externalizing disorders
• These appear to work well as screening items for
  mental health issues

35
AOD Screening

• The three alcohol and drug use items on the COD
  screening instrument appear to relate minimally
  to ASI alcohol/drug problem severity measures
• There are stronger relationships between the
  alcohol and drug use COD screening items and the
  GAIN SS substance disorders measure

36
Trauma/DV Screening

• Similar to the mental health items, the three
  trauma/domestic violence items on the COD
  screening instrument also appear to relate
  strongly to ASI measures of psychiatric problem
  severity
• However, they differentiate with respect to the
  GAIN SS
• The COD trauma screening items have stronger
  relationships with the GAIN SS Crime/Violence
  measure than with the mental health measures (for
  internalizing and externalizing disorders)
• This is an important distinction that is
  consistent with constructs of trauma/domestic
  violence

37
Screening Composite Scores

• Composite scores were formed for each of the
  content areas in the COJAC COD screener
• Count of number of items answered yes within
  each domain
• Mental Health (0-3)
• Alcohol/Drug Use (0-3)
• Trauma/Domestic Violence (0-3)
• Plus total score across all nine items (0-9)

38
COD Screener Summary Scores and ASI Composite
Scores
n 268
Mental Health Items Alcohol Drug Use Items Trauma/DV Items Total Score
Medical .23 .00 .14 .18
Employment .20 .09 .18 .22
Alcohol .15 .14 -.04 .12
Drug .02 .16 .03 .09
Legal .05 .01 .10 .08
Family-Social .16 .03 .17 .17
Psychiatric .56 .11 .33 .47
39
COD Screener Summary Scores and ASI Severity
Ratings
n 268
Mental Health Items Alcohol Drug Use Items Trauma/DV Items Total Score
Medical .18 -.03 .08 .11
Employment -.08 -.18 -.01 -.12
Alcohol .08 -.04 -.08 -.02
Drug .05 -.08 .07 .02
Legal .02 -.13 .12 .01
Family-Social .10 -.13 .11 .03
Psychiatric .56 .03 .37 .46
40
COD Screener Summary Scores and GAIN Short
Screener
n 34
Mental Health Items Alcohol Drug Use Items Trauma/DV Items Total Score
Internalizing Disorders .54 -.03 .22 .34
Externalizing Disorders .30 -.09 .09 .14
Substance Disorders .04 .60 .22 .42
Crime-Violence .47 .48 .45 .66
Total Disorders .47 .39 .36 .57
41
Mental Health Composite

• Overall, the Mental Health COD screening
  composite works well, correlating strongly with
  ASI and GAIN SS measures of mental health problems

42
AOD Use Composite

• The Alcohol/Drug Use COD screening composite
  relates well to the GAIN SS measure of substance
  disorder severity but not with the ASI alcohol
  and drug problems measures

43
Trauma/DV Composite

• As was the case for the individual
  trauma/domestic violence screening items, the
  Trauma/Domestic Violence COD screening composite
  related moderately to the ASI mental health
  measures and strongly with the GAIN SS
  crime/violence measure

44
Total COD Composite

• The Total COD screening composite correlated with
  ASI measures of psychiatric problems and GAIN SS
  measures of mental health, substance disorders,
  crime/violence, and total disorders
• It appears this measure shows promise for
  screening for co-occurring disorders

45
Pilot Study Limitation

• Because the instructions specifically indicated
  the longer screening measures (i.e., ASI, GAIN
  SS) were only to be administered if one or more
  answer to the nine COD screening items was
  endorsed, there is a restricted range of
  responses in the data available to predict
  scores on the longer (more established) criterion
  measures
• By design, it was not possible to examine whether
  negative screening data (i.e., cases where all
  nine items are answered no) predicts the
  absence of problems as measured by the ASI and/or
  GAIN SS
• These data do illustrate the degree to which
  persons screened as having a possible substance
  abuse, mental health, and/or trauma issue are
  likely to have treatment needs as measured by the
  ASI and GAIN SS measures
• That is, among persons already identified as
  having screened positive for one or more of
  these issues, endorsement of COD screening items
  and composites is related to the severity of
  substance abuse and mental health problems