Getting Programmes into Systems? Three replication RCTs of evidence-based programmes in a local authority setting

1
Getting Programmes into Systems? Three
replication RCTs of evidence-based programmes in
a local authority setting

• Tracey Bywater, Nick Axford, Michael Little,
  Louise Morpeth, Sarah Blower Daniel Perkins
• The Social Research Unit

2
Overview

• Brighter Futures Strategy
• The programmes are we looking at and why?
• How do we integrate the programmes into systems
  and get systems ready for inclusion of
  programmes?
• Design
• Results (TBC)

3
Brighter FuturesFor Birmingham

• places the emotional health of the young
  population on the same level of importance as
  literacy and numeracy
• unemployment is twice the national average
• four times as many one-parent households
• 37 per cent of Birminghams population are under
  25 years old and the average age is continuing to
  fall.
• half of children (250,000) are from minority
  ethnic groups and not getting support needed
• 27 of 8,000 14-16 year olds agreed with the
  statement I often feel depressed

4
The process to develop the strategy is unique in
England

• It is distinctive in five ways
• driven by high-quality evidence on the wellbeing
  of our children
• Prepared by a multi-disciplinary leadership team
  of 35 people, supported by 200 practitioners
  across the citys children's organisations
• Perspectives of stakeholders have been taken
  seriously
• Strategic advantage is made of government
  initiatives (eg Every Child Matters) but future
  policy is not driven only by Westminster
• a commitment to evaluating the strategy and
  measuring impact on our children's wellbeing
  year-on-year.

5
Common Language approach

• four concepts
• Outcomes (epidemiology 6000 children - what
  works evidence)
• Activities (perspectives of key stakeholders,
  evaluation planning)
• Inputs (Investments)
• Outputs (support from key constituencies,
  national policy)

6
Childhood Antisocial Behavioura growing
political issue

• increasing in numbers, about 10 of children in
  Britain and USA but as many as 35 in high risk
  disadvantaged areas
• resistant to intervention if not treated early
• if unresolved can predict delinquency, adult
  mental health problems and/or crime
• costly to society - health, education social
  service costs

7
Recent Government initiatives to support high
risk, hard to engage families

• 2001 - Sure Start in England
• The Welsh Assembly Government Parenting Action
  Plan (2005), Flying Start, Genesis
• Reaching Out the Action Plan on Social Exclusion
  published Sept 2006
• The Respect Agenda
• The Pathfinder project (DfES) 18 services to
  deliver and evaluate three parent programmes
• The Family Intervention Project DfES (50
  Authorities)
• Nurse Family Partnership DoH (Olds - 10
  Authorities)
• The National Parenting Academy

8
As part of Brighter Futures Strategy

• Birmingham City Council has funded the evaluation
  by RCT of three evidence-based programmes
• The purpose of the studies is to establish the
  impact and cost-effectiveness of these programmes

9
The Programmes

• The PATHS Programme (Providing Alternative
  Thinking Strategies)
• Incredible Years Triple P

10

• Key specific components of effective parenting
  interventions
• new parenting skills must be modelled and
  rehearsed
• home-based practice or homework
• parenting programmes should should be
  collaborative emphasise principles rather than
  prescribe techniques
• (non-violent) sanctions for negative behaviour
  and relationship building, praise and rewards
• must address difficulties in adult relationships
  or other family problems
• interventions work best when delivered early

11
The PATHS Programme (Providing Alternative
Thinking Strategies)

• facilitates the development of self-control,
  emotional awareness, and interpersonal
  problem-solving skills.
• enhances the social competence and social
  understanding of children
• facilitates educational processes in the
  classroom.

12
Making programmes system ready

• Adapting content of PATHS programme
• Developing implementation manuals

13
Making the system programme ready

• Staff training and ongoing support
• Practising for roll-out
• Service design training and Brighter Future
  events
• PATHS head teachers reference group
• Regular events
• Benefit realisation exercise

14
Components of implementation fidelity

• is the programme delivered as designed?
• are all the core components present?
• to the right population?
• with appropriately trained staff?
• using the right protocols, techniques
  materials?
• in the right context?

15
Evaluation Questions

• Do the programmes meet the needs and improve
  outcomes for children and families who
  participate?
• 2. Are the programmes implemented efficiently and
  effectively with fidelity?
• 3. Do the programmes offer value for money and
  net benefit to the Council, Government, society,
  children and families?

16
Design

• IY 162, 3-4 year-olds, 21 randomisation,
  parent-child is unit of randomisation, stratified
  by age sex, 9 Children Centres, 3 data
  collection points
• TP 144 x 2, 4-6 9-11 year-olds, 11
  randomisation, parent-child is unit of
  randomisation, stratified by age sex, 6 areas,
  3 data collection points
• PATHS 60 schools cluster randomised trial,
  stratified by size FSM, 11, Reception Year
  1 classes, 3 data main collection points (Oct,
  June 2010, June 2011)

17
Measures

• IY TP
• SDQ (screening primary outcome) social,
  emotional
• ECBI child behaviour
• Arnold OLeary parenting practices
• Service use questionnaire
• Demographics
• Facilitator completed cost diaries attendance
  sheets
• PATHS
• SDQ child behaviour
• Paths Teacher Rating Scale child behaviour
• Teacher background school climate
• Attendance records teacher pupils
• Academic grades
• Observations in class (Oct April both academic
  years)

18
Analyses for behaviour outcomes

• Parent Programes (see BMJ BJP papers)
• All families will be included in the analysis
  irrespective of uptake of intervention. We will
  carry out 2 analyses an intention to treat
  analysis and analyses with completers, ie
  complete data sets.
• We will present the differences between the
  intervention and control conditions on follow-up
  scores from analysis of covariance (ANCOVA) of
  the response, taking account of area, treatment,
  and baseline response value, age and sex. Effect
  sizes will be calculated with Cohens guidelines
• (LT analysis) A linear mixed model command in
  SPSS was used to perform repeated measures
  analysis of variance (ANOVA) over time with no
  assumption of equal variances, using an
  unstructured covariance matrix, within subjects
  for time. All time points were compared to scores
  at follow-up 1
• School Based Programme
• Under discussion,

19
Cost data analysis

• Parent Programmes (See companion paper in BMJ
  BJP)
• Using cost (group costs) clinical data we will
  compute how much it will cost per point drop on
  SDQ
• Service use frequencies costs
• Modeling to establish any future savings to
  Birmingham Council
• School based Programme
• Under discussion

20
Results. We have none!

• Hutchings, Bywater, Daley et al., (2007). A
  Pragmatic Randomised Controlled Trial of a
  Parenting Intervention in Sure Start Services for
  Children at Risk of Developing Conduct Disorder,
  BMJ. doi10.1136/bmj.39126.620799.55
• Bywater, Hutchings, Daley et al., (2009).
  Long-Term Effectiveness of a Parenting
  Intervention in Sure Start Services in Wales for
  Children at Risk of Developing Conduct Disorder,
  BJP. Doi10.1192/bjp.bp.108.056531
• Edwards, R.T., Ó Céilleachair, A., Bywater, T.,
  Hughes, D.A., Hutchings, J. (2007). Parenting
  Programme for Parents of Children at Risk of
  Developing Conduct Disorder Cost-Effective
  Analysis. BMJ, doi10.1136/bmj.39126.699421.55.
• Sanders, M. (2008). Triple P-Positive Parenting
  Program as a Public Health Approach to
  Strengthening Parenting. Journal of Family
  Psychology.
• Berlin, L. J., Ziv, Y., Amaya-Jackson, L.
  Greenberg, M. T. (Eds.). (2005). Enhancing early
  attachments Theory, research, intervention, and
  policy. New York Guilford. Duke Series in Child
  Development and Public Policy.

21
Summary

• To make systems ready for programmes
• To make Programmes ready for systems
• To employ RCT with evidenced based programmes to
  reduce unwanted (risk) behaviour and increase
  positive (protective) behaviour
• To inform decision makers
• To take to scale and support growing numbers of
  children in Birmingham

22
What its really about.

• I wanted to write to you two to say thankyou,
  you have taughet me to appriciat my little girl
  and the time we have together Since doing this
  course working with Zoe her speach has come on
  She knows and rembers things I think thats were
  praise come,s in. I love and respect my daughter
  more thankyou for all your help, your support and
  your praise It makes me feel Im Finaly doing a
  good Job
• Thankyou

23

• Thank you
• t.bywater_at_bangor.ac.uk
• naxford_at_dartington.org.uk
• Read Prevention Actionhttp//www.preventionaction.
  org
• Daily news on prevention science its
  application to policy and practice
• t 44-1803 762400
• f 44-1803 762983