Cluster Randomised Trials Of Schools Based Health Interventions - PowerPoint PPT Presentation


PPT – Cluster Randomised Trials Of Schools Based Health Interventions PowerPoint presentation | free to view - id: 459cd-NWJmM


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation

Cluster Randomised Trials Of Schools Based Health Interventions


Cluster Randomised Trials Of Schools Based Health Interventions ... Laurence Moore. Cardiff Institute of Society, Health and Ethics. Email: ... – PowerPoint PPT presentation

Number of Views:45
Avg rating:3.0/5.0
Slides: 31
Provided by: SSO68


Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Cluster Randomised Trials Of Schools Based Health Interventions

Cluster Randomised Trials Of Schools Based Health
  • What are the barriers to greater use of RCTs in
    educational research?
  • Possibilities for progress or Mission impossible?
  • Demonstrate by example that RCTs of complex
    educational interventions are feasible

  • Trial of fruit tuck shops in primary schools
  • Trial of emergency contraception lessons (NHS
  • ASSIST Trial of peer-led intervention to reduce
    adolescent smoking (MRC)
  • Free breakfast initiative in primary schools in
    Wales (Welsh Assembly Government)

  • What are the barriers to greater use of RCTs in
    educational research?

Challenges in applying RCTs to evaluation of
educational interventions
  • Ethical concerns
  • Randomisation
  • Recruitment and retention
  • Scale and Cost
  • Variability in delivery
  • Context dependent
  • Generalisability

Ethical concerns
  • Often thought unethical to deprive one group of
    people of the innovative intervention, which is
    believed or assumed to be beneficial
  • Contrast to medicine where exposure to untested
    new treatments often considered unethical
  • Very common in education, despite
  • In medicine, target audience is sick and the
    moral imperative to do something (must be better
    then nothing) is great. eg. AIDS, cancer
  • Frequent examples of new interventions being
    ineffective or even harmful

Ethical concerns (2)
  • Is randomisation less fair / ethical than
    postcode lottery or local policy / bid success?
  • Only if we are certain that the intervention can
    do no harm should we
  • Implement without strong evidence of effect
  • Begin to think that randomisation might be
  • How do we define harm?
  • Cost / opportunity cost
  • Raised expectations

Ethical concerns (3)
  • Not wise / moral / prudent / ethical to conduct a
    trial unless one has good reason to believe that
    the intervention may be effective
  • Theory
  • Formative evaluation
  • Principle of equipoise remains

  • Often impossible / impractical to randomly assign
    individuals to intervention / control groups
  • Within one cluster, control subjects liable to be
    contaminated by exposure to some/all
    intervention activities
  • Many interventions act explicitly at the cluster
    level (e.g. class, school)
  • Randomisation to intervention / control may be
    undertaken at group level (cluster randomisation)
  • Usually stratified randomisation or minimisation
    to ensure reasonable baseline balance

Cluster randomised trials
  • ASSIST Peer-led smoking intervention
  • 59 schools randomised
  • Fruit tuck shops
  • 43 schools randomised
  • Free Breakfast Initiative
  • 57 schools randomised
  • Emergency contraception
  • 25 schools randomised

Recruitment and retention
  • Those recruited to trial should be representative
    of target population
  • Participants need to consent to having their
    treatment determined by randomisation
  • Thought to be particularly difficult (unethical)
    in cluster randomised trials
  • In some cluster trials, those randomised to
    control may then not maintain their commitment to
  • Major threat of differential drop-out

Recruitment and retention
  • Recruit all schools on basis of equal probability
    of being in intervention or control group
  • Clear, honest detailed description of research
  • School research contract
  • Offer equal reward to both groups
  • eg. Control schools given cash buy-out time
  • Control schools offered intervention at end of
    measurement period
  • Maintain motivation
  • briefings, personal contact
  • newsletters, prize draws

Experience with recruitment and retention of
  • Recruitment
  • School recruitment easier than anticipated
  • Refusal to participate more often due to strong
    preference regarding intervention than objection
    to randomisation or data collection requirements
  • Retention
  • 5 school cluster randomised trials
  • 196 schools
  • 1 3 years fieldwork duration
  • No school drop-outs
  • 2 closures

Scale and Cost
  • Co-ordination and timeliness
  • Major challenge in large scale trials
  • Requirement for
  • Communication between researchers and
  • Research networks
  • Natural experiments
  • Innovations in policy / practice introduced in an
    experimental manner, ideally through randomised

Scale and Cost (2)
  • Trials, particularly cluster randomised trials,
    can be large and expensive
  • Intervention costs
  • Outcome data collection costs
  • Natural experiment no extra intervention costs
  • e.g. Free Breakfast Initiative
  • Use of routinely collected outcome data
  • Education has unexploited resource
  • Frequently, trials can be very low-cost

Variability in delivery
  • RCTs traditionally require that interventions are
    standardised and uniformly delivered
  • (efficacy trial)
  • Educational interventions highly dependent on
    quality of delivery
  • Value of efficacy trials limited
  • eg. school smoking education
  • Results of efficacy trials involving enthused
    teachers not replicated in roll-out

Efficacy and effectiveness
  • Efficacy trial
  • To test whether the treatment does more good than
    harm when delivered under optimal conditions
  • Effectiveness trial
  • To test whether the treatment does more good than
    harm when delivered via a real-world program in
    realistic conditions
  • Pragmatic, allowing variability in delivery as
    would be experienced in real world

Context dependent
  • Educational interventions often highly dependent
    on the context within which they are delivered
  • Argued therefore that RCTs not suited to their
  • However, RCT design has the advantage that
    randomisation process ensures that systematic
    differences in external influences between groups
    do not occur
  • Will achieve unbiased estimate of average effect

  • Efficacy trials may demonstrate that intervention
    has active ingredients that work
  • Effect unlikely to be reproduced in real world
  • Attenuated by context and implementation
  • Generalisability of small trials with one
    educator in one school will be limited

  • Possibilities for progress or Mission Impossible?

Public Health Improvement Evidence base conundrum
  • Good quality trials successfully conducted,
    evaluating weak interventions. Small or zero
    effect sizes.
  • Good quality complex interventions evaluated
    using weak research designs. Biased effect

When do we do RCTs?
  • In medicine, there are distinct phases in the
    development evaluation of new interventions
    (eg. drugs)
  • Basic research (eg. molecular, genetic)
  • Applied research development (eg.
  • Trials to determine efficacy
  • Trials to determine effectiveness
  • Post-marketing surveillance

Phases of RCTs of complex interventions MRC
April 2000
MRC Assist TrialPeer-led smoking intervention
  • Theory based (Diffusion of innovations)
  • Developed from similar approach used in sex
  • Extensively piloted
  • Feasibility trial conducted in 6 schools
  • Funding for main trial (59 schools) sought and
    obtained from MRC

Effectiveness trials with embedded process
  • Effectiveness trials, implementing interventions
    in a manner reproducible in real world
  • Crucial to conduct a comprehensive process
    evaluation (largely qualitative) within such a
  • Monitor variability in context and delivery
  • Identify barriers / facilitators
  • Relate variability in these factors to
    variability in intervention impact

Fruit tuck shop trial
  • Minimisation used to ensure balance in terms of
  • School size
  • School policy on snacks
  • Schools given minimal support in setting up tuck
    shops, with wide variability in detailed
  • Detailed process evaluation
  • Environment of school and locality
  • Operation of fruit tuck shops
  • Detailed case studies of 8 selected schools
  • Observation, interview, focus groups

  • Intervention led by specialists, as would be the
    case if rolled out in the real world
  • Not to be implemented by untrained, unmotivated
  • Process evaluation in all 30 intervention
    schools, with parallel measures in the 29 control
  • In-depth process evaluation in sub-sample
  • Observations, field notes, diaries, records,
    interviews with pupils, teachers, staff

Free Breakfast Initiative Trial
  • 111 schools across 9 LEAs
  • Variable models of staffing and delivery
  • Trial powered to identify overall mean effect on
    dietary and behavioural outcomes
  • Process evaluation to monitor variation in
    delivery and identify strengths and weaknesses

A role for RCTs in evaluating health education
  • RCTs not always possible!
  • Difficult to do well, and can be expensive
  • Take opportunity of natural experiments
  • Theory-driven development, formative evaluation
    and feasibility studies essential prerequisites
    prior to trial
  • Get the intervention right

Research design
  • Cluster randomised design
  • Pragmatic, effectiveness trials
  • Unbiased estimate of overall intervention effect
  • Additional qualitative and quantitative data
    collection to measure variation in context,
    process, delivery and outcome
  • Identifies issues for further development of
    intervention / further testing of its (variable)
  • Hypothesis generation, not testing

The end.
  • Stanley (1957)
  • Expert opinions, pooled judgements, brilliant
    intuitions and shrewd hunches are frequently
  • MacIntyre Petticrew (2000)
  • Good intentions and received wisdom are not

Laurence Moore Cardiff Institute of Society,
Health and Ethics Email Tel
02920 875387