Phaedra Corso, Ph'D'

1
Program Evaluation from an Economic Perspective
Phaedra Corso, Ph.D. Associate
Professor College of Public Health University of
Georgia
2
Why Care About Economics in the Context of
Prevention?

• Maximizing outcomes is important.
• Minimizing costs is important too.
• Resources are limited, so hard (resource
  allocation) decisions must be made.
• Demonstrates the value provided from the
  resources expended (return on investment).

3
The PH Model for Prevention Economics?
Problem Identification
Risk and Protective Factor Identification
Program and Policy Development
Economic Impact - COI
Program and Policy Evaluation
Implementation and Dissemination
Economic Evaluation
4
EE Methods

• Partial evaluation costs only
• Cost of illness (COI) analysis
• Cost analysis (program costs)
• Full evaluation costs and outcomes
• Cost-benefit analysis (CBA)
• Cost-utility analysis (CUA)
• Cost-effectiveness analysis (CEA)

5
COI Analysis

• Estimates total costs incurred because of a
  disease or condition
• (i.e., medical costs, non-medical costs,
  productivity losses)
• Generally reported as
• annual total cost
• average per person lifetime cost
• Used to show potential benefits of prevention
  efforts

6
Costs of Violence in the United States

• Corso et al., AJPM 2007

7
Cost Parameters

• Direct costs
• Medical care
• ED visits
• Hospitalizations
• Ambulance/paramedic
• MD visits
• Dental
• Physical Therapy
• Prescription Drugs
• Mental health care
• Productivity losses
• Work losses
• Household productivity losses

8
Incidence

• 2.2. million medically-treated injuries
  associated with violence occurred in 2000
• 17,000 homicides, 30,000 suicides
• People aged 15 to 44 years comprise 44 percent of
  the population, but account for nearly 75 percent
  of violent injuries

9
Costs

• The total cost associated with nonfatal injuries
  and deaths due to violence in 2000 was more than
  70 billion.
• 37 billion for interpersonal violence
• 33 billion for self-inflicted violence
• The average cost per homicide was 1.3 million in
  lost productivity and 4,906 in medical costs.
• The average cost per case for a non-fatal assault
  resulting in hospitalization was 57,209 in lost
  productivity and 24,353 in medical costs.
• The average cost per case of suicide is 1
  million lost productivity and 2,596 in medical
  costs.
• The average cost for a non-fatal self inflicted
  injury was 9,726 in lost productivity and 7,234
  in medical costs.

10
So What?

• The incidence and economic burden of injuries in
  the US is substantial
• This information can be used to lobby for more
  prevention resources
• Implementation of effective interventions could
  reduce this burden
• The cost to implement effective interventions
  less the economic burden prevented represents
  the potential returns on investment for
  prevention.
• This is what economic evaluation
  is all about!!

11
Cost Analysis (CA)

• Estimates total costs of running a program
• Costs are the value of the resources (people,
  building, equipment and supplies) used to produce
  a good or a service
• Important for realizing costs from varying
  perspectives
• e.g., incurred by program, incurred by
  participant
• Includes not just financial, but also economic
  costs.
• Important for budget justification, decision
  making, and forecasting.
• Also called cost consequence or cost
  identification analysis
• Provides the first step of a full economic
  evaluation

12
Cost Analysis of a national replication of a
child maltreatment program

• Corso et al., CDC, OCAN (in progress)

13
Define Cost Categories
14
Preliminary Results at end of Year 1

• The average cost per family referral ranged from
  2,319 to 8,906.
• The average cost per family receiving services
  ranged from 4,238 to 33,742.
• At the end of the first year of implementation,
  pre-implementation costs as a percentage of total
  costs ranged from 23 to 42 of the total costs
  of the program.

15
So What?

• Provides information for Agency X who might want
  to implement the program in the future.
• Provides the cost component of a full EE.
• Lessons learned on how to conduct a programmatic
  CA
• Prospective data collection
• Input from site implementers
• Technical assistance throughout data collection
• Revisions of cost collection templates along the
  way

Reference Applying Cost Analysis to Public
Health Programs (at www.phf.org)
16
What is Economic Evaluation (EE)?

• Applied analytic methods to
• Identify,
• Measure,
• Value, and
• Compare
• the costs and consequences of
• treatment and prevention strategies.

Done a lot Done not so much
17
Cost-benefit Analysis - CBA

• A method used to compare costs and benefits of an
  intervention
• where all the costs and benefits are standardized
  or valued in monetary terms.
• Provides a list of all costs and benefits over
  time
• Can have different time lines
• Can have different amounts at different times
• Provides a single value
• Net Benefits NB (Benefits Costs)

18
When is CBA Used?

• To decide whether to implement specific programs
• If NB gt 0, implement
• When choosing among competing options
• Implement program with highest NB
• For setting priorities on options given resource
  constraints

19
Quantify Benefits - CBA

• Cost-of-Illness (COI) approach
• Willingness-to-Pay (WTP) or Contingent-valuation
  surveys
• (e.g., how much is society willing to pay to
  reduce the annual morbidity and mortality risk
  associated with a disease or injury)

20
Corso Survey (Fall 2007, Georgia)

• Based on national data, 2 out of every 100,000
  children annually, or an average of 4 children
  every day are killed as a result of child
  maltreatment by parents or caretakers.

Now imagine we had a nationally-sponsored child
maltreatment prevention program that was
available to your state and this this program was
proven to reduce the risk of a child being killed
due to child maltreatment by 50. This means
that the number of children killed on average
every day in the U.S. by child maltreatment is
reduced from 4 per day to 2 per day.
If this program were available to your state,
would you be willing to pay 150 in extra taxes
per year to sponsor this program? YES Would
you be willing to pay 225? NO Would you be
willing to pay 75?
21
Cost-utility Analysis - CUA

• A method used to compare costs and benefits of
  interventions where benefits are expressed as the
  number of life years saved adjusted to account
  for loss of quality.
• Combines
• Length of life (survival), and
• Quality of life
• Compares disparate outcomes in terms of utility
• Quality-adjusted life years (QALYs)
• Disability-adjusted life years (DALYs)
• Derives a ratio of cost per health outcome
• /QALY or /DALY

22
When is CUA Used?

• When quality of life is the important outcome.
• When the program affects both morbidity and
  mortality.
• When the programs being compared have a wide
  range of different outcomes.
• When the program is being compared with a program
  that has already been evaluated using CUA.

23
Quantify Benefits - CUA

• Utilities are
• A preference-based measure of health, that
  relies on choice and uncertainty to elicit
  preferences
• Typically based on a 0 (death) to 1 (perfect
  health) scale

24
Example of Tool to Elicit Utilities Time
Trade-Off (TTO)
Which life do you prefer?
Quality of Life
Short and fun
Long and dull
Length of Life
25
Example TTO
healthy
Utility
U(healthy) 1.0
blind both eyes
U (blind both eyes) ?
Dead
Years
0
20
12
26
Combining Quality of Life with Length of Life
Utility
without prevention
with prevention
1.0
0.7
Years
0
75
30
QALYS (with prevention) 1.075 75 QALYS
(without prevention) 0.730 21
27
Cost-effectiveness Analysis - CEA

• Estimates costs and outcomes of interventions
• Expresses outcomes in natural units
• e.g., cases prevented, lives saved
• Compares results with other interventions
  affecting the same outcome
• Summary measure cost-effectiveness ratio
• Cost per some outcome achieved
• e.g., cost per case prevented, cost per life
  saved

28
When is CEA Used?

• Used to identify
• most cost-effective strategy from options that
  produce a common effect
• practices that are not worth their costs
• Used for empirical support for under-funded
• programs

29
Quantify Outcomes CEA of parenting intervention

• Intermediate outcomes
• Increased child self-esteem and mental health
  status
• Increased family cohesiveness/coping skills
• Decreased depression in parents
• Final outcomes
• QOL improvements in parents and children
• Child maltreatment cases prevented
• Lives or life years saved

30
CEA Caveat

• Outcomes cannot be combined, so one or two of the
  most important effectiveness measures should be
  considered (separately) for the CEA.
• The number of summary measures depends on the
  number of outcomes chosen.
• If 2 outcomes, A and B, are considered the most
  important for evaluation, then
• Cost/outcome A
• Cost/outcome B
• This makes translation for policy makers
  difficult!!

31
Example CM Prevention ProgramAverage CE Ratios
for depressive symptoms
Compared to baseline
32
Final Comments

• Economic evaluation (EE) is valuable to decision
  making and for setting health policy.
• For new researchers in PH, this is an important
  specialization to consider because the demand
  for these skills is growing.

33
Coming soon.. Center for Economic
Evaluation Institute for Behavioral Research
and College of Public Health pcorso_at_uga.edu