An EvidenceBased Investment Strategy

1
An Evidence-Based Investment Strategy Identifyin
g Implementing
Evidence-Based Programs and Policies to
Improve Public Outcomes (at Less
Cost) The Washington State Approach Belfast,
Northern Ireland June 22, 2009
Steve Aos Assistant Director Washington State
Institute for Public Policy Phone (360)
586-2740 E-mail saos_at_wsipp.wa.gov Institute
Publications www.wsipp.wa.gov
1 of 19
2
Overview
Implement
E.g. Prevention
E.g. Crime
Approach
Proposition An Investment Model Can Help

• improve public policies by focusing decisions on
  evidence-based and cost-beneficial options
• identify gaps in current knowledge and
• put prevention and intervention on an
  equal analytical and policy-setting playing
  field.

Topics for Today

• Our general analytical approach
• Our experience in Washington State supplying the
  legislature with investment information
• Examples in 2 policy areas crime and prevention
• How the legislature has used the information

2 of 19
3
Overview
Implement
E.g. Prevention
E.g. Crime
Approach
the other Washington
the real Washington
3 of 19
4
Overview
Implement
E.g. Prevention
E.g. Crime
Approach
Washington State Institute for Public Policy

• Created by the 1983 Legislature
• Board of Directors
• Senator Karen Fraser
• Senator Jeanne Kohl-Welles
• Senator Pam Roach
• Senator Mark Schoesler
• Representative Glenn Anderson
• Representative Mary Lou Dickerson
• Representative Phyllis Kenney
• Representative Skip Priest
• Ken Conte, House Staff
• Richard Rodger, Senate Staff
• Robin Arnold-Williams, Gov. Policy Advisor
• Victor Moore, State Budget Director
• Sandra Archibald, University of WA
• Andrew Bodman, Western WA University
• Les Purce, The Evergreen State College
• Robert Rosenman, WA State University

Olympia, Washington
4 of 19
5
Overview
Implement
E.g. Prevention
E.g. Crime
Approach
UK (Northern Ireland)
Washington State
United States
5 of 19
6
Approach
Implement
E.g. Prevention
E.g. Crime
Overview

• Recent Directions to WSIPP from the Legislature
• What Works, What are the Costs Benefits?
• The legislature has asked WSIPP to examine
  options affecting
• Crime (1994, 1999, 2003, 2005, 2009),
• Education, Early Education (2003, 2006, 2007,
  2009),
• Child Abuse Neglect (2003, 2007, 2009,
• Substance Abuse (2003, 2005, 2009),
• Mental Health (2005, 2009),
• Developmental Disabilities (2008),
• Employment (2009),
• Public Assistance (2009),
• Public Health (2009), and
• Housing (2009)
• What Difference Could an Evidence-Based Strategy
  Make to Washington?
• Can a investment portfolio of evidencebased
  options affect statewide outcomes and save
  taxpayers money?

Our topics today
6 of 19
7
Approach
Implement
E.g. Prevention
E.g. Crime
Overview
What Works? Definition of Evidence.

• Standards of Evidence
• Petri dish experiments
• Real-world random assignment experiments
• Real-world natural experiments
• Statistically well-controlled comparisons
• Not so well-controlled comparisons
• Anecdotes

1. WSIPP researcher gathers all the studies she
can locate on a topic.
2. She applies standards of evidence to
identify the high quality studies.
3. She meta- analyzes all high quality studies to
estimate an average effect.
7 of 19
8
Approach
Implement
E.g. Prevention
E.g. Crime
Overview
If Something Works, What is it Worth to Achieve
It?
8 of 19
9
Approach
Implement
E.g. Prevention
E.g. Crime
Overview
Our Consumer Reports Lists

• What Works?
• What Doesnt?
• What Dont We Know?
• What Are the Benefits Costs?

Reports available for downloading www.wsipp.wa.gov
9 of 19
10
E.g. Crime
Implement
E.g. Prevention
Approach
Overview
The Big Picture Crime Rates Taxpayer Costs
1980 to 2007
In 1980, taxpayers spent 557 per household on
the criminal justice system. Today they
spend 1,223 per year. A 120 increase.
In 2007, crime rates were 39 lower than they
were in 1980.
10 of 19
11
E.g. Crime
Implement
E.g. Prevention
Approach
Overview
Fighting Crime Pinching Pennies Are There
Evidence-Based Policy Options That Reduce Crime,
but at Less Cost? Our 3-Step Research Approach

• What works what doesnt?
• We locate rigorous (comparison group), real world
  evaluations of adult and juvenile corrections
  programs, prevention. (571 studies to date)

• What are the economics of each option?
• We estimate the taxpayer and crime victim
  benefits and costs to people in Washington

• Statewide, how would alternative portfolios
  affect prison demand, public spending, crime?
  

11 of 19
12
Restorative Justice (low risk) -8.7 (21) 7,067
Reducing Crime Some Findings
E.g. Crime
Implement
E.g. Prevention
Approach
Overview
Change In Crime ( of EB Studies)
Benefits - Costs (per-person, life cycle)
Adult Offenders
Adult Drug Courts -8.0 (57) 4,767
Education Prgs., Prison -7.0 (17) 10,669
Drug Tx in Prison (TC or out-patient) -5.7
(20) 7,835
Cog-Behavioral Treatment -6.3 (25) 10,299
ISP surveillance -0.0 (23) -3,747
ISP treatment -17.1 (11) 11,563

• We located and analyzed 57 rigorous drug court
  outcome evaluations conducted in the United
  States.
• On average they reduced recidivism rates 8
  percent.
• Without drug court, an offender has a 58 chance
  of being reconvicted for a new felony or
  misdemeanor after 13 years
• With drug court, the odds drop to about 54.
• The reduced recidivism generates a NET gain of
  4,767 per drug court participant.
• We estimate drug courts cost 4,333 more per
  person than regular court processing (court
  costs, treatment) benefits of reduced recidivism
  total 9,100 to taxpayers (lower criminal justice
  costs) and crime victims (reduced victimization).

Juvenile Offenders
Functional Family Thpy. -15.9 (7) 31,821
Multisystemic Therapy -10.5 (10) 18,213
Aggression Repl. Trng. -7.3 (4) 14,660
Family Int. Transitions -13.0 (1) 40,545
MDT Foster Care -22.0 (3) 77,798
Prevention
Pre-School (low income) -14.2 (8) 12,196
Nurse Family Partnership -36.3 (2) 18,052
12 of 19
Why focus on juveniles if our focus is
prison? 73 of adults in Washingtons prisons
have been in Washingtons juvenile justice system
13
E.g. Crime
Implement
E.g. Prevention
Approach
Overview
The Crime Example Washington State Legislative
Response

• Started funding juvenile justice EBPs in late
  1990s.
• 2007 Legislature made a substantial change by
  funding a portfolio of evidence-based adult and
  juvenile corrections, and prevention.
• One planned prison taken off the drawing board.
• Juvenile facilities closed, another adult
  facility to be closed.
• 2009 Legislature, even amidst other budget cuts,
  continued to fund the evidence-based portfolio.

13 of 19
14
E.g. Prevention
Implement
E.g. Crime
Approach
Overview
What Can Early Childhood Education Achieve?We
conducted a systematic review of every rigorous
evaluation in the United States since 1965
Number of Studies
Outcomes for LOW INCOME 3 4 year olds
Average Result
On-time graduation increases from 62 to 68
High School Graduation
10
Percent repeating a grade drops from 13 to 8.
K-12 Grade Repetition
24
Percent using special ed drops from 12 to 8
K-12 Special Education
23
2 to 4 percentage point increase in pass rates
Standardized Test Scores
27
Pct. with conviction by age 30 drops from 24 to
17
Crime
8
Public Assistance
No significant effect
3
Teen Births (under 18)
4
No significant effect
Pct. with a substantiated case drops from 12 to
7
Child Abuse and Neglect
1
14 of 19
15
E.g. Prevention
Implement
E.g. Crime
Approach
Overview
The Economic Question Is Early Childhood
Education for Low-Income 3 and 4 Year Olds a
Good Investment?(2005 US dollars, life-cycle
present value)
Main Source of Benefits
Benefits
Increased earnings
Increased high school graduation
9,966
Lower K12 Costs
Reduced K12 grade repetition
206
Reduced K12 special education
135
Lower K12 Costs
Lower CJS Victim
Reduced crime
5,068
Reduced child abuse neglect
1,919
Lower CWS Victim
Reduced alcohol and drug abuse
278
Increased earnings
Offset child care costs
1,897
Lower CC Costs
19,469
Total Benefits Per Youth
7,709
Cost Per Youth
Benefits Per Dollar of Cost
15 of 19
16
E.g. Prevention
Implement
E.g. Crime
Approach
Overview
Example Program Nurse Family Partnership (NFP)
Decrease
K-12 Grade Repetition
16 of 19
17
E.g. Prevention
Implement
E.g. Crime
Approach
Overview
Economic Bottom Line for Nurse Family Partnership
Benefits Per Youth
Main Source of Benefits
26,986
Total Benefits Per Youth
8,931
Cost Per Youth
( 9 ROI)
Benefits Per Dollar of Cost
17 of 19
18
Implement
Early Education
Crime
Evidence
Overview
Actions Taken or Underway in Washington

• MORE COMPHENSIVE RESOURCE PORTFOLIOS
• 2007 and 2009 Legislatures have funded longrun
  crime portfolio adult juvenile offender
  programs, and prevention.
• Legislature added early childhood education to
  the states basic education definition.
• Other funding changes in prevention/intervention
  programs
• OTHER IMPLEMENTATION ACTIONS
• For EB Programs A state-adopted evidence-based
  list, coupled with local government choice (from
  the list).
• For New Ideas A research development funding
  level set by the state for local initiatives
  state evaluation.
• Formal statewide assessment tools
  (to align the programs
  and the participants).
• Program quality assurance provided by state.

18 of 19
3 of 3
19
Implement
E.g. Prevention
E.g. Crime
Approach
Overview
Summing Up

• To Improve Public Outcomes, Develop An
  Investment Strategy that Uses
• Evidence
• Economics, and
• Portfolios
• In Time, This Will Help Deliver a Better Return
  on Investment to Taxpayers

19 of 19
3 of 3
20
(No Transcript)
21
E.g. Crime
Implement
E.g. Prevention
Approach
Overview
Adult Prison Incarceration Rates 1930 to 2005

11 of 19
22
Example Nurse Family Partnership
(NFP) Monetization of NFPs CAN outcome on High
School Graduation Step 1 The Effect of the NFP
Program on CAN outcome -.883 NFP Unadjusted
Effect Size on CAN (Olds et al., 1997, p value
.000) -.441 NFP Adjusted Effect Size (Adjusted
by WSIPP) on CAN .256 Base CAN Rate Without
NFP .113 Change in CAN rate given the program
effect via dcox transformation Step 2 The
Effect of CAN on High School Graduation -.215
CAN Adjusted Effect Size (by WSIPP) on High
School Graduation .700 Base High School
Graduation Rate -.0793 Change in High School
Grad Rate given CAN via dcox transformation Ste
p 3 The Value of High School Graduation 166,139
Expected Value of HS Graduation, Present Valued
to age 0 Step 4 Expected Value of NFP on
HSchool Grad, via Effect of NFP on
CAN 1489 Value of NFP on CAN from High School
Grad .113 X .0793 X 166,139
Appendix
23
An Evidence-Based Investment Strategy Workshop
on Cost-Benefit Approach Used by the Washington
State Institute for Public Policy The Washington
State Approach Belfast, Northern Ireland June
22, 2009
Steve Aos Assistant Director Washington State
Institute for Public Policy Phone (360)
586-2740 E-mail saos_at_wsipp.wa.gov Institute
Publications www.wsipp.wa.gov
1 of 19
24
Investment Model
Implications
WA Examples
Overview
The Building Blocks of Washingtons Investment
Model
Individual Policy and Program Level
Portfolio Level
Analytical Step 1
Analytical Step 2
Analytical Step 3

• Producing Effect Sizes Two Types
  
• What works inventory. What does not work?
  What dont we know?
• e.g., do home visitation programs affect child
  abuse rates?
• Blueprint findings when possible and
  meta-analysis
• Linking effect sizes.
• Do measured outcomes link to other outcomes of
  interest?
• e.g., does child abuse affect later schooling
  outcomes, criminality, substance abuse, etc.?

• Statewide Effects of an Evidence-Based Strategy
• What is the size of the market for
  evidence-based options?
• Size of statewide population that could benefit
  and effective prg.
• Proportion currently untreated
• Realistic expansion potential
  diminishing returns
• Bottom Line Estimates
• Meaningful state-level outcomes
• e.g., change in crime rate, child abuse rate,
  prison construction , net fiscal gain, etc.

• Turning Effect Sizes into Dollars
  (ROI)
• What costs are avoided (or benefits gained)?
• Information needed
• Population characteristics
• e.g., child abuse prevalence
• System responses (cases costs)
• e.g. costs of crime, child welfare
• Economic information
• e.g. labor market data
• Costs of Programs
• e.g. cost of home visitation
• Consistent calculations of cash flows ROI and
  risk.

Consumer Reports like lists of ranked,
ready-to-go, individual options, and an RD
strategy
Forecast of state-level outcomes accountability
15 of 17
25
Investment Model
Implications
WA Examples
Overview
Example Nurse Family Partnership
(NFP) Monetization of NFPs CAN outcome on High
School Graduation Step 1 The Effect of the NFP
Program on CAN outcome -.883 NFP Unadjusted
Effect Size on CAN (Olds et al., 1997, p value
.000) -.441 NFP Adjusted Effect Size (Adjusted
by WSIPP) on CAN .256 Base CAN Rate Without
NFP .113 Change in CAN rate given the program
effect via dcox transformation Step 2 The
Effect of CAN on High School Graduation -.215
CAN Adjusted Effect Size (by WSIPP) on High
School Graduation .700 Base High School
Graduation Rate -.0793 Change in High School
Grad Rate given CAN via dcox transformation Ste
p 3 The Value of High School Graduation 166,139
Expected Value of HS Graduation, Present Valued
to age 0 Step 4 Expected Value of NFP on
HSchool Grad, via Effect of NFP on
CAN 1489 Value of NFP on CAN from High School
Grad .113 X .0793 X 166,139
15 of 17