Alcohol and Other Drug Treatment Effectiveness Study

1
Alcohol and Other Drug Treatment Effectiveness
Study

• A Review of Individual State Outcome Studies

March 2001 Prepared by the National
Association of State Alcohol and Drug Abuse
Directors, Inc. (NASADAD) For The Center for
Substance Abuse Treatment (CSAT)
2
Overview

• NASADAD conducted a review of Alcohol and Other
  Drug (AOD) Treatment Outcome Studies conducted by
  or in the States.
• A Qualitative Meta-analysis was performed to
  review, compare, and synthesize the results of
  individual State outcome studies from 1994-1999.
• The treatment effectiveness indicator areas
  employed were identified along with operational
  definitions of specific performance measures.
• Results were positive and consistently
  demonstrated that treatment is effective.

3
Objectives

• To collect, compile, and compare State treatment
  outcome data in common indicator areas across
  varied treatment populations and study
  methodologies.
• To determine trends and commonalities of State
  study results based on a review of study results
  under the four treatment effectiveness indicator
  areas reached through a CSAT/NASADAD consensus
  process and included in the FY 2000 SAPT Block
  Grant Applications (AOD Use, Employment Status,
  Criminal Justice Involvement, and Living
  Arrangement.)

4
Data Collection

• All State AOD Agency Directors were asked to
  provide the following information on treatment
  outcome studies conducted between 1994 through
  1999
• study design
• client population
• type of services
• data sources
• measures and data collection points
• key findings and outcomes

5
Qualitative Meta-Analysis

• A Meta-analysis was conducted using a qualitative
  approach to analyze the State outcome study
  results.
• Use of this technique provides a systematic way
  to identify important trends and the direction of
  those trends among study results when variables
  and measures are not uniform and directly
  comparable.
• In this study, the term Meta-analysis means an
  overview and comparison of the results from
  multiple studies rather than a statistical
  approach.

6
Qualitative Meta-Analysis

• Criteria for including study results in the
  analysis were based on an assessment of study
  components and study status.
• Factors considered were
• relevant findings
• appropriate methodology
• study completion
• inclusion of measures of treatment effectiveness

7
Qualitative Meta-Analysis

• Fifty-three (53) out of sixty-six (66) individual
  studies from 24 States were selected for
  inclusion in the analysis, based on the soundness
  of their scientific method.
• Study results were reviewed, compiled,
  synthesized, and a comparison was made based on
  outcome measures falling under the four
  CSAT/NASADAD treatment effectiveness indicator
  areas, two additional indicator areas, and other
  relevant variables.

8
Results - Overview

• Study results were compared along the following
  dimensions
• treatment effectiveness indicator areas
• performance measures and operational definitions
  under treatment indicator areas
• treatment evaluation periods (follow-up)
• important variables that impact treatment

9
Results - Indicator Areas

• CSAT/NASADAD Indicator areas
• AOD Use
• Employment Status
• Criminal Justice Involvement
• Living Arrangement
• Two Additional Indicator areas
• Physical Health
• Mental Health and Family/ Social Functioning

10
Results - Operational Definitions

• AOD Use Measures Used In State Studies
• abstinence within past 30 days
• abstinence at discharge
• alcohol or drug use at discharge
• substance use in 6 months following treatment
• frequency of substance use
• number of days of alcohol, cocaine, and marijuana
  use in prior 30 days
• Addiction Severity Index (ASI) items from
  Drug/Alcohol Use Scale
• prevalence of use - five times or more
• peak density - percent of days drug used in a
  month

11
Results - Operational Definitions

• Employment Status Measures Used In State Studies
• currently employed or unemployed
• number of days working in the prior 30 days
• Addiction Severity Index (ASI) items from
  Employment/Support Status Scale
• part or full time employment
• days missed in last year
• Treatment Episode Data Set (TEDS) employment item

12
Results - Operational Definitions

• Criminal Justice Involvement Measures Used In
  State Studies
• number of days participated in illegal activities
  during prior 30 days
• Addiction Severity Index (ASI) items from Legal
  Status Scale
• arrested in last 3 months or jailed in last 24
  months
• number of legal problems 24 months prior to
  treatment and 6 months following treatment
• number of days arrested and incarcerated in the 6
  months prior to admission and discharge
• reported arrests in year before treatment and in
  year after treatment
• recidivism rate of clients versus comparison group

13
Results - Operational Definitions

• Living Arrangement Measures Used In State
  Studies
• no fixed address or shelter or fixed residence
• living alone or with others
• Addiction Severity Index (ASI) items from
  Family/Social Relationships Scale
• number reporting homelessness/total number
• living alone, living with a parent, a spouse, a
  partner, a boy or girlfriend, other relatives,
  friends, other recovering people

14
Results -Operational Definitions

• Physical Health Measures Used In State Studies
• average number of hospitalizations
• number of days in inpatient detox
• number of days in hospital
• medical overnight stays
• no medical overnight stays
• rates of inpatient hospitalization
• number of emergency room visits
• changes in reported general health status

15
Results -Operational Definitions

• Mental Health and Family/Social Functioning
  Measures Used in State Studies
• psychiatric overnight stays
• use of mental health services
• mental health symptoms change
• family interaction skills change
• number of days experienced conflict with family
  members
• client reports of changes in relationships with
  spouse and children
• client reports of changes in ability to deal with
  stress

16
Results Evaluation Periods

• Follow up points
• 6 and 12 month post admission
• 2, 3, 4, and 5 years post admission
• 10 years post admission
• 3, 6, and 12 month post discharge
• 2 and 3 years post discharge
• 14 month post intake by phone
• none or unspecified

17
Generalizability of Results

• In general, positive results across the State
  studies revealed a clear trend that treatment is
  effective.
• In general, treatment is effective regardless of
  the measures used, the treatment effectiveness
  indicator areas examined, or the duration of the
  study or follow-up period.

18
Generalizability of Results (cont)

• Although trends were consistently positive, this
  study recognizes limitations in its finding as a
  result of
• the numerous and different operational
  definitions which, in turn, led to different
  measures under each indicator area
• the variability of evaluation periods (follow-up
  points)
• the lack of similar study designs
• differences in study populations
• differences in study objectives, e.g. program
  evaluation vs. determination of treatment
  effectiveness

19
Summary of ResultsCSAT/NASADAD Indicator Areas

• Analysis of performance measures revealed that
  after treatment, AOD use decreased.
• Total number of studies 39
• 100 percent showed decrease in AOD use

20
Summary of ResultsCSAT/NASADAD Indicator Areas

• Analysis of performance measures revealed that
  after treatment, employment increased.
• Total number of studies 33
• 100 percent showed increase in employment.

21
Summary of ResultsCSAT/NASADAD Indicator Areas

• Analysis of performance measures revealed that
  after treatment, criminal justice involvement
  decreased.
• Total number of studies 31
• 100 percent showed decrease in criminal justice
  involvement

22
Summary of ResultsCSAT/NADADAD Indicator Areas

• Analysis of performance measures under the Living
  Arrangement Indicator revealed that the area was
  infrequently employed and the measures used were
  extremely varied.
• The results of those few studies that examined
  living status were not definitive.
• Total number of studies 7
• 72 percent showed improvement in living
  arrangement
• 28 percent were neutral

23
Summary of Results - Other Effectiveness
Indicator Areas

• Measures for two additional treatment
  effectiveness indicator areas were reviewed and
  analyzed.
• Analysis of performance measures revealed that
  after treatment, physical health improved.
• Total number of studies 19
• 95 percent showed improvement in physical health
• 5 percent were neutral

24
Summary of Results - Other Effectiveness
Indicator Areas

• Analysis of performance measures revealed that
  after treatment, mental health and family/social
  functioning improved.
• Total number of studies 14
• 86 percent showed improvement in functioning
• 7 percent were neutral
• 7 percent were negative

25
Summary of Results Additional Findings

• Treatment is effective
• treatment effectiveness improves as duration of
  engagement in the treatment process increases
• participation in aftercare or peer support groups
  enhances treatment effectiveness
• Treatment is effective for two target populations
  
• the criminal justice system
• youth residential programs (but less dramatically
  when gauged by the same indicator areas as adult
  treatment)
• Treatment is consistently effective in different
  types of programs
• outpatient programs (stronger effect)
• residential programs
• methadone programs

26
Summary of Results Additional Studies

• Decrease in costs and increase in savings in
  other areas
• Findings consistent with State Outcome Studies
• Improve Statewide treatment program
  accountability and support Statewide data
  collection efforts on treatment effectiveness

• Cost Offset Studies
• Individual Provider Studies
• TOPPS I Pilot Studies

27
Summary of Results Additional Studies (cont)

• TOPPS II Studies

• Developing standardized approach for
  systematically measuring program performance and
  treatment outcomes

28
Conclusions

• Meta-analysis results are positive and indicate
  that treatment is effective.
• In general, AOD treatment is effective regardless
  of the measure used under various indicator areas
  or the duration of the evaluation period.
• The CSAT/NASADAD treatment effectiveness
  indicator areas and associated operational
  definitions and measures provide a strong
  framework for examining treatment effectiveness.

29
Conclusions (cont)

• Other indicators areas, variables, and related
  studies provide information needed to improve the
  treatment process and for the informed the
  targeting of treatment resources.

30
Recommendations

• Continue to develop and refine appropriate
  performance measures that can accurately reflect
  AOD treatment effectiveness and efficiency.
• Continue to develop common processes for the
  collection and reporting of performance
  measurement data.
• Support outcome research and performance
  measurement infrastructure development activities
  that will lead to the establishment of a minimum
  set of state data/research capabilities.