Title: Issues and Challenges in the Economics of Prevention and Treatment: Five Studies
1Issues and Challenges in the Economics of
Prevention and Treatment Five Studies
- E. Michael Foster and numerous collaborators
- 4th Biennial Niagara Conference
- July, 2005
www.personal.psu.edu/emf10/ foster_at_pop.psu.edu
2Five Studies
- Family Bereavement Project
- Success Through IY
- Fast Track Intervention
- Incredible Years
- MTA
3There are many economic questions one might
consider
- What is the most cost-effective research design?
- What does an intervention cost (efficacy)?
- What does an intervention cost II
(effectiveness)? - Inputs (personnel, space)?
- Scale?
- How do those costs vary?
4There are many economic (cont)
- Is an intervention cost-effective?
- Does an intervention reduce the costs of illness?
5Isnt this obvious? NO
- Costs are experienced now, but benefits
experienced in the future - (aside Olds NHV)
- Prevention involves treating some individuals who
will not experience the disorder - (aside low-income women in NHV)
- Too many programs are cadillacs
61. Estimating the Costs of an Intervention
- Family Bereavement Program
- Success through the Incredible Years
7Content of Presentation
- Steps in Estimating Costs
- Principles Shaping Estimation of Costs
- Overview of the Incredible Years
- Scope of study
- Sources of data
- Challenges in developing a cost framework
- Results for teacher and parent training
8Steps in Estimating Costs
- Establish the perspective and scope of study
- Identify the program inputs
- Measure the use of these resources
- Value the resources in terms of marginal,
opportunity costs - Conduct sensitivity analyses
9Principles of Sound Estimation
- Perspective and scope of the costs analyses
should be clearly stated - Costs should be presented for each of several
perspectives, including the social perspective - Costs should be estimated for the program as
delivered, but - Sensitivity analyses could include an attempt to
estimate real-world costs
10Principles of Sound Estimation of Costs
- Relevant costs are marginal costs
- Costs should not be double-counted
- Costs of intervention development and of
implementation should be tracked separately - Costs should be described in sufficient detail
such that the reader can understand key
assumptions and limitations.
11Overarching Questions
- What do high-quality interventions cost?
- How large are implicit (or time) costs?
- What would these programs cost in the real world?
- How large are those costs relative to
- the costs of clinical services?
- the costs of illness?
- the programs benefits?
121. Estimating the Costs of a Behavioral
Intervention The Family Bereavement Program
- E. Michael Foster, Michele Porter, Tim S. Ayers,
Debra Kaplan, Irwin N. Sandler
Research Supported by NIMH Grant P30- MH39246 for
a Prevention Research Center and NIMH Grant R0-
MH49155 to evaluate a preventive intervention for
bereaved children
13Family Bereavement Program as an Example
- Plan is to
- Provide a brief overview of our efforts to
estimate costs of FBP - Describe methods used in calculating the costs of
FBP - Develop estimates of the costs of program
development - Share cost estimates for delivering this
preventive intervention in an efficacy trial
14Family Bereavement Program as an Example
- Goal is to
- improve the practice and reporting of cost
estimates of prevention programs - highlight issues to consider in designing and
conducting efficacy trials that might improve the
transition to effectiveness trials
15Design of Efficacy Trial
- Random assignment to
- Psychoeducational group intervention
- Comparison condition was a guided reading program
(bibliotherapy) - Recruitment through schools, bereavement support
agencies, mental health agencies, hospices,
victim assistance programs etc. - Two clinical pilots were conducted prior to start
of efficacy trial.
16Design of Efficacy Trial
- Recruitment occurred over course of three years
- Six cohorts of intervention/comparison
- 3 assessments pre, post and 11 month follow-up
17Family Bereavement Program
- Small group, psychoeducational format
- Separate groups for parents, children and
adolescents - 12 group sessions
- 2 individual sessions for each participant
- Delivered over 3 month period
18Facilitators and Training
- Intervention groups co-lead by two MA level
clinicians who served as facilitators - Received 40 hrs of training prior to start of
program - Received 2 hrs of training and 1.5 hrs of group
supervision per week
19Perspective of FBP Cost Analysis
- Perspective of a public agency or funding source
adopted - Focus on explicit costs personnel, consultants,
benefits, intervention direct expenses, travel,
miscellaneous supplies and equipment - Social perspective includes some implicit costs.
- E.g. intervention space, volunteered time by
faculty, clinical staff for supervision and
project management purposes, plus the associated
fringe benefits - Interest in estimating real world costs so these
were important to calculate
20Perspective of FBP Cost Analysis
- Participant time was not calculated which would
typically be included in a societal perspective - Debated this decision to include participant
time ultimately did not have good data on time
spent outside of group in intervention activities
21Calculation of FBP Intervention Costs
- Budget spread sheets used to enumerate project
inputs and payments - Available for September 1994 through August 2000.
- Enumerated project inputs
22Calculation of FBP Intervention Costs
- All FBP costs were separated into development,
intervention and research - Research expenses were tracked and removed.
- Each budget year was divided into both explicit
and implicit categories
23Explicit and Implicit Costs
- Explicit (examples)
- Salaries
- Fringes
- Implicit
- Volunteer clinical supervisors time
- Faculty time (2 PIs) that was not charged to the
grant - University space and overhead
24Explicit Expenses
- Research only expenses e.g. travel for
interviewers, subject fees, professional books,
statistical software costs. - Intervention only expenses e.g. printing
handouts and manuals, catering of meals,
counseling referral services, transportation to
group, snacks and incentives.
25Study Periods Delineated
June 94
Dec 95
Jan 96
Jan 95
Dec 98
Jan 97
Jan 98
Development Period
Intervention Period
Two Pilots
Six Cohorts
- Study periods were delineated, and costs were
calculated for each period
26Further Segmenting of Time Periods
- On Time Periods high in intervention
activities - Off Time Periods intervention not delivered
and low in intervention activities.
Cohort 1
Cohort 6
Intervention Period
27More Time Distinctions
- Divided into
- Pre Intervention recruitment, interviews, and
training - Intervention delivery of the psychoeducational
group - Post Intervention follow-up and debriefing
28Intervention vs. Research Costs
- Expenses were further delineated by determining
ratio of time to development, pilot delivery and
intervention delivery. - Division of supplies and materials were fairly
straight forward to divide into research vs.
intervention related. - Dividing personnel time into research vs.
intervention was more difficult for staff
involved in both. - Project Director, PI, Field Manager and Group
Facilitators
29Proportioning Personnel Time
- Time diaries and time sheets recorded job tasks
related to intervention and research - Project Director kept an electronic diary of time
spent on FBP
30Proportioning Group Facilitators Time
- Group facilitators were required to fill out
weekly timesheets in which they recorded time
spent in 9 separate categories of activities - This information allowed us to accurately
estimate the proportion of their time engaged in
different intervention related tasks - We were able to examine these proportions for the
- pre-intervention,
- intervention and
- post intervention segments
- of the On Time Periods and across the six
cohorts - All activities except time completing evaluation
materials, e.g. process evaluations were coded as
intervention related. - These proportions were applied to the actual
payments (salary and fringe benefits) made to
group facilitators
In advance, we tracked group facilitator time
carefully across activities. This information
will help us in estimating the costs of
delivering the program in the real world.
31Implicit Costs
- Although not actually expended, do reflect value
of resources used in the delivery of the
intervention - Within FBP the following reflected implicit costs
- PIs and Co-investigators time
- Volunteered clinical supervision
- Cost of intervention space and overhead
32Calculation of Implicit Costs
- PIs Co-Investigators time spent more time
on project than was reflected in their
salaries/benefits - Volunteered Clinical Supervision
- Two cohorts had unpaid clinical supervision
- Valued at the standard rate and applied as
implicit costs to these cohorts
33Intervention Space and Overhead
- The site at which the FBP was delivered was
rotated between a local school and the
university. - No charges, except custodial fees at the local
schools were assigned. - Used standard university rental rate to value the
use of this space - Local real estate analysis helped to value the
cost of space within the schools - Overhead was calculated using salaries of those
with offices. - A standard space/overhead -age based on local
real estate analysis and the proportion staff
members spent in intervention vs research
activities was used to calculate overhead costs.
34Estimated Costs of Intervention
- Costs were calculated at 3 levels
- Per family
- Per person
- Per hour of contact
- Calculated from a
- Public agency perspective
- Societal perspective
35Intervention Costs A Public Agency Perspective
Costs varied across cohorts. A driver was cohort
size, but costs still varied for cohorts of the
same size (e.g., c4 v. c5). Surprisingly, the
intervention costs per family were not much
higher for the pilot cohorts.
36Intervention Costs A Societal Perspective
37(No Transcript)
381. Estimating the Costs of an Intervention
Success Through the Incredible Years
- Deb Johnson-Shelton, E. Michael Foster and Ted
Taylor
39Expand on two themes
- Magnitude of time costs
- Sensitivity analyses
40Overview of The Incredible Years
- School level
- 6 teacher workshops for 7 hours each session
- Classroom management
- Dinosaur School (child social skill training)
- Family level
- 12 parent training group sessions for 2 hours
each session (including some self-administered
sessions through video-taped curriculum) - Child-centered play, encouragement, good
directions, and positive discipline
41Scope of Study
- Effort at low cost approach to implementing
intervention - Modest level of supervision
- Only one group leader
- Experienced, trained facilitators
- Use of existing school, community facilities
- Time-limited intervention
- Resources focused on teacher component
42Evaluation of Success
- Evaluation of targeted components
- Longitudinal study of 162 children
- (3 children per class X 2 classrooms X 27
schools) - Participants identified as at risk in K using
teacher reports
43Evaluation of Success (cont)
- Random assignment of high-risk children within
schools - Data collection through
- Annual parent interview two phone interviews
- Through grades 2-3-4 (depending on the cohort)
44Evaluation of Success (cont)
- Track 648 additional children
- (12 children per class X 2 classrooms X 27
schools) - Extends data collection two years
- Adds economic analysis of entire project
45Challenges in Cost Assessment Valuation of
Staff, Teacher, and Parent Time
We calculated an rate (plus appropriate fringes)
per minute averaged across all persons in a
position. For example Child care providers
mean 9.9 an hour Group leaders mean 22.17
an hour Interventionists 47.04 an hour
Support staff 16.48 an hour Teachers
39.12 an hour (40,000 annual salary/170 days/8
hours) Administrator 33.20 an hour
46(No Transcript)
47(No Transcript)
48(No Transcript)
49(No Transcript)
502. What Would These Programs Cost In The Real
World?
Ted K. Taylor Oregon Research Institute E.
Michael Foster
51What Assumptions Differ from These Settings to
the Real World
- The previous presentations highlight tracking the
cost of implementing the intervention in research
settings. - How would delivery differ in the real world?
521. Training Supervision Costs
- Costs may be higher or lower depending on
- How often training is conducted
- How steep or flat is the learning curve
532. Supervision Efforts to Ensure Fidelity
- Costs may be higher or lower depending on
- Amount of supervision and efforts to ensure
fidelity - Training level of staff
543. Benefits of Specialization
- Costs may be higher or lower depending on
- Level of specialization and how it affects staff
performance.
554. Differences in Staffing
- Costs may be higher or lower depending on
- Terms of interventionists contracts (e.g., hourly
v. salary) - In two of the previous examples, the therapist
time is considered hourly. If groups are
delayed, staff arent paid. This is possible with
part-time staff, or staff that can fulfill other
research responsibilities during that time. - Whether therapists are get paid during slow
periods and/or agency can smooth work flow to
keep staff busy. - Range of activities and responsibilities for
interventionists and administrative staff. - Availability of low-cost labor.
56Summary The Real World May Cost More, Not Less
- My own belief is that doing these interventions
in the real world may cost MORE, not less, than
in research trials. - However, implementing other interventions which
have no evidence is even more expensive to
society. - The challenge will be to disseminate
interventions in ways that maximize fidelity and
adherence while containing costs. - There will be important implications for the
effect of the intervention.
57The Answer to these Questions
- Likely depends on a variety of contextual factors
- Culture of the agency
- Economies of scale
- Economies of scope (range of activities)
- Nature of the local labor market
58Costs in the Natural Service Delivery Settings
- Two different approaches
- Simply use different input costs based on
standard rates in these natural service
delivery settings - Calculate costs based on the assumption youd
modify practice i.e. less clinical supervision,
training etc.
59Modifying Practice
Approx
- Informal survey with agencies that provide
bereavement related services - Relative to efficacy trial we ran training is
considered a one time cost. They would not
provide training prior to each group - Supervision would occur within the context of
peer supervision within the agency and does not
come close to approximating an hour for hour
supervision to direct contact ratio. - Preparation for the intervention is considered to
be the responsibility of the therapist.
Therapists in order to pay themselves for
training try to ensure that their group size is
large enough to reimburse themselves for the time
spent in preparation. - Assume that time spent in group intervention,
doing clinical notes and all other direct contact
would happen at the same rate.
27 4
1 --
4
11 4 2
-- 53
60Adjustments to costs
- Not only would Group Facilitator time need to be
reduced - Time spent by faculty in training and supervision
would also need to be reduced - 33 of certain faculty members time was spent in
doing this additional training and supervision - Re-calculation of costs based on the adjustments
to both Group Facilitator and Facultys time
results in the following costs based on a public
agency perspective - originally 59 per hour of contact
613. How Large Are These Costs Relative To Costs
Of Illness (COI)?
- Damon Jones, Michael Foster and Elizabeth Gifford
62Why Examine Real Costs?
- We cannot hope to measure all benefits of an
intervention. - However, a key outcome involves the behaviors
that are costly to public sectors and society
more generally. - Measuring these costs requires additional sources
of data.
63Measuring Direct Costs
- Measuring frequency of service use
- Through self-report
- Through examination of service records
- Applying cost estimates
- From research
- From examination of records
64Methodological Issues
- Need cooperation to examine information
- From participants
- From facilities
- Need to understand/decipher information
- Understand cost arrangements
- Understand what records are involved
- Need to have missing data strategy (multiple
imputation combining sources) - Incorporating an oversample included in the data
using weights
65Cost of Illness Example
- Using Fast Track sample
- Service categories
- Mental health facilities
- General health facilities
- Juvenile justice facilities
- School services (IEP, counseling)
- Medications (for emotional/behavioral issues)
66Research Questions
- How large are (a subset of) the COI for
conduct-disordered children? - Are those costs lower for children with lower
(sub-clinical) levels of problems?
67Characteristics of Sample
- Data available for many service areas
- Data available longitudinally (grades 6-12)
- Data for over 600 children (not including Tx
participants) - Cost data available from multiple sources
- Diagnostic grouping
68Used Best Estimates
- Used costs from literature review
- Sound estimates in the literature
- JJ (detention facility, cost of arrest) from Fass
Pi (2000) - Medication costs per child Zuvekas (2001)
- Not enough RR data to have great estimates (Day
Treatment Center DA clinic) - Used costs from the record review data
- Likely better estimates due to regional data
(family doctor ER) - Lack of good information in literature (RTC
Group Home services)
69Characteristics of Sample (contd)
- Used the DISC (Diagnostic Interview Schedule for
Children) - Children diagnosed with/without conduct disorder
- Also examined children who were on the border for
being diagnosed, or who were diagnosed with ODD
70(No Transcript)
71TOTAL Average Costs Per Child(by CD grouping)
Time trends reflect development and non-MAR
attrition.
72Average JJ Costs Per Child(by CD grouping)
73Average Inpatient MH costs per child(by CD
grouping)
74Average Medications Costs Per Child(by CD
grouping)
753. How Large Are Those Costs Relative To The
Program Benefits?
76Are More Complex Treatments Cost-Effective for
More Complex Cases?
- E. Michael Foster
- Professor of Health Policy and Demography
- Pennsylvania State University
77- Cost-Effectiveness of ADHD Treatments Findings
from the MTA Study - Forthcoming in the American Journal of Psychiatry
- Peter S. Jensen, M.D.,
- Joe Albert Garcia, Ph.D.,
- Sherry Glied, Ph.D.,
- Maura Crowe, B.A.,
- E. Michael Foster, Ph.D.,
- Michael Schlander, M.D.,
- Stephen Hinshaw, Ph.D.,
- Benedetto Vitiello, M.D.,
- L. Eugene Arnold, M.D.,
- Glen Elliott, M.D.,
- Lily Hechtman, M.D.,
- Jeffrey H. Newcorn, M.D.,
- William E. Pelham, Ph.D.,
- James Swanson, Ph.D.,
- Karen Wells, Ph.D.
- The Cost-Effectiveness of Treatment for ADHD
Variation by Comorbidity - Under review at Health Services Research.
- Foster, Jensen among others
78Outline
- Brief Review of MTA
- Study Design
- Prior Findings
- CEAround 1
- CEACa better alternative
- CEACround 2
- What should a policy makers WTP be?
79Brief Review of MTA--Design
- 6 sites
- 579 Children ages 7-10
- Randomized to four arms
- Co-morbidities common
80Presence of Comorbidities
81Brief Review of MTA--Design
- 14 months of treatment
- Wide range of outcomes across multiple domains
and respondents - Four arms
- Community care (CC)
- Two-thirds received medications
- Medication management (MM)
- Careful titration
- Monthly med checks with pharmacotherapist
- Intensive behavioral treatment (Beh)
- Parent training
- Child training (STP)
- Biweekly teacher consultation trained classroom
aide - Combination (Comb)
82Brief Review of MTAInitial Findings
- Participants in all groups improved
- Core symptoms Comb, MM gt Beh or CC
- Broader outcomes Comb gt MM
- Significant moderation by comorbidity status
83TABLE 1. Rates of Normalization
84Rates of Normalization
85Brief Review of MTAInitial Findings
- Participants in all groups improved
- Core symptoms Comb, MM gt Beh or CC
- Broader outcomes Comb gt MM
- Significant moderation by comorbidity status
86CEA Round 1
- Costs were measured according to economic
principles - Research costs were excluded
- All costs included regardless of payor
- Wholesale prices of medications were extracted
from the National Database of Drug Formularies
(NDDF)
87CEA Round 1
- The costs of personnel were calculated using
hourly wages and fringes taken from national
databases (such as the AMA) - STP costs were labor only
88TABLE 2 Costs Per Subject Per Treatment Arm
89TABLE 3 Costs Per Subject Per Treatment Arm for
Comorbidity
90Skip to CEAC
91X
15,993
55,253
68,128
360
92Incremental Cost Effectiveness of Alternative Tx
Relative to Control
Combination
5000
Behavioral
Therapy
Incremental Costs
3000
1000
Community
Medication
Care
Management
-1000
-.1
0
.1
.2
.3
.4
.5
.6
Incremental Benefits
93Results from IYP
- Stacking intervention components
- Move away from ICER presentationkind of obscure
94CEA of Program Outcomes
- Three Treatment Components
- Child Training
- Parent Training
- Teacher Training
- Ecological Treatment Approach
- Key Question Is Stacking Intervention
Components Cost-Effective? - Trade-Off Treatment Intensity vs. Number of
Children Treated
95Results Treatment Costs
- Control 0.00 (n21)
- CT only 1,164 (n54)
- CTTT 1,454 (n11)
- PT only 1,579 (n292)
- PTTT 1,868 (n24)
- CTPT 2,713 (n38)
- CTPTTT 3,003 (n19)
96(No Transcript)
97(No Transcript)
98Cost-Effectiveness Analysis of Fast Track
- E. Michael Foster
- Damon Jones and
- Conduct Problems Prevention Research Group
99Fast Track Sample
- Change in sample
- Add back the treatment group
- Remove the lower-risk youth
- Three outcomes
- Diagnosis of conduct disorder
- Act of interpersonal violence
- Index offense
100(No Transcript)
101(No Transcript)
102(No Transcript)
103Are More Complex Treatments Cost-Effective for
More Complex Cases?
So, whats the answer?
104(No Transcript)
105(No Transcript)
106But, we should note
- Comorbid children differ in other ways, and
- Other factors may serve as moderators, but
supplemental analyses reveal that the effect of
comorbidity is distinct.
107Economic Analysis of Family Bereavement Project
108(No Transcript)
109Conclusion
- Economic analysis takes substantial resources and
planning. - This task involves difficult conceptual issues
above and beyond nuts -n- bolts, budgeting
issues. - You will benefit from knowing where your money
really goes. - Were making progress but have much to learn,
especially about effectiveness studies.