Success and CBAS: An Economic Evaluation of the Incredible Years

1
Success and CBAS An Economic Evaluation of the
Incredible Years

• Biglan, Taylor, Meltzer, Rusby, and others
• Oregon Research Institute

Foster Central PA Office Penn State
2

• www.personal.psu.edu/emf10
• Recent In-Press or Working Papers on
• economic issues in prevention forthcoming in
  Applied Developmental Science
• public costs of improved mental health services
• How Economists Think about Family Resources and
  Child Development forthcoming in Child
  Development
• dosage that uses propensity score methods
• other topics
• Paper using FT data on screening
• Paper using FT data on non-ignorable non-response
  and multiple imputation
• Stay tuned social costs of conduct disorder

3
There are many economic questions one might
consider

• What is the most cost-effective research design?
• What does an intervention cost (efficacy)?
• What does an intervention cost II
  (effectiveness)?
• Inputs (personnel, space)?
• Scale?
• How do those costs vary?

4
There are many economic (cont)

• Is an intervention cost-effective?
• Does an intervention reduce the costs of illness?

5
Lets talk about two grants at ORISpecific Aims

• Evaluate the indicated and universal components
  of a multi-component intervention
• Estimate the full costs of the intervention
• Estimate the
• Impact on net social costs (a circumscribed
  benefit-cost analysis)
• Relative cost-effectiveness of different levels
  of intervention

6
Outline

• DesignIntervention
• DesignEvaluation
• Overview of the economic evaluation
• Estimating the full costs of the intervention
• Impact on net social costs
• Cost-effectiveness analysis
• Hypothetical Illustration
• Problems and Challenges

7
1) Design--Intervention

• Three components
• Videotape-mediated training of teachers in
  behavior management
• Videotape-mediated social problem-solving skills
  training for elementary school children
• Empirically based parent training

8
2) DesignEvaluationSuccess

• Evaluation of targeted components
• Longitudinal study of 162 children
• (3 children per class X 2 classrooms X 27
  schools)
• Participants identified as at risk in K using
  teacher reports

9
2) DesignEvaluationSuccess

• Random assignment of high-risk children within
  schools
• Data collection through
• Annual parent interview two phone interviews
• Through grades 2-3-4 (depending on the cohort)

10
2) DesignEvaluationCBAS

• Track 648 additional children
• (12 children per class X 2 classrooms X 27
  schools)
• Extends data collection two years
• Adds economic analysis of entire project

11
3) Overview of economic evaluation

• Estimating the full costs of the intervention
• Impact on net social costs (circumscribed
  benefit-cost analysis)
• Cost-effectiveness analysis

12
NOTE were only capturing a portion of the
economic impact.
13
3a) Estimating the full costs of the intervention

• Allocating explicit costs
• Divide labor costs into intervention and
  evaluation
• Identify and measure other variable costs (e.g.,
  materials)

14
3) Estimating the . (cont.)

• Allocating explicit costs (cont.)
• Allocated other facility and fixed costs
• Allocate total costs to intervention components
• Aside how to handle training costs

15
3) Estimating the . (cont.)

• Allocating implicit costs
• Space costs
• Teacher time
• Parent time
• Lost instruction
• Aside implications for parent interview

16
3b) Impact on net social costs

• Identify the relevant outcomes
• Use of alternative services, such as school,
  mental health and social services
• Care giver employment (including impact on
  transfers)
• Measure treatment impact

17
3b) Impact on net social costs

• Express those behaviors in dollar terms
• School budgets
• Billing records
• Estimating net costs
• Discounting
• Sensitivity Analysis
• Precision

18
Aside What should a services instrument look
like?

• Whats the right time period?
• How many modules (or service types) do we
  include?
• Do we ask about expenditures?
• What level of other detail do we specify (e.g.,
  visits)?

19
Aside Why medical and administrative records?

• Advantages
• Parents are inaccurate
• Parents are not knowledgeable
• Reduces respondent burden
• Creates new and more mysterious forms of missing
  data

20
Aside Why medical (cont).

• Disadvantages
• Privacy concerns
• IRB issues
• Agency and provider burden
• Health providers HIPPA
• Public agencies
• Legal issues
• formatting of authorization
• Whose authority

21
3c) Cost-effectiveness analysis

• Reports of childs behavior
• Academic performance
• Care giver strain
• Health-related quality of life

22
4) A Hypothetical Example
Well replace these figures with real ones from
our current study.
23
What Does IY Cost?

• Costs as disseminated under reasonable
  circumstances
• Agency would run program in two classes with 14
  children
• Agency would need parent group leader (who would
  have to be trained)
• Group and classroom sizes consistent with
  experience

24
What does IY Cost? (cont)

• Included appropriate materials
• Other costs included (such as meals and sitters)
• Included parent and teacher time (including
  preparation time)
• Includes training costs

25
What does IY Cost? (cont)

• Reasonable assumptions
• number of parent groups a trainer leads during
  tenure
• Salaries
• Value of parental time (?)
• Excludes
• costs of development (e.g., tapes)
• space costs
• opportunity costs for classroom time
• additional administrative costs (such as
  receptionists)

26
Bottom Line
27
Breakdown of Costs
28
(No Transcript)
29
Costs
600
500
400
To pick the most cost-effective parent group
size, we need the outcome side.
300
200
100
0
10
12
15
17
20
22
25
Size of P Groups
30
5) Problems and Challenges

• Design
• Power
• Methodology
• Combining record review and self-report data
• Fully capturing the future (without waiting for
  it to arrive)
• Dealing with highly influential cases

31
5) Problems (cont)

• Administrative data
• All the usual problems
• Combining data from multiple sources
• Measurement
• Overall well-being
• Broader conceptual issues
• Efficacy v. effectiveness