Cluster Randomised Trials Of Schools Based Health Interventions - PowerPoint PPT Presentation

Loading...

PPT – Cluster Randomised Trials Of Schools Based Health Interventions PowerPoint presentation | free to view - id: 459cd-NWJmM



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Cluster Randomised Trials Of Schools Based Health Interventions

Description:

Cluster Randomised Trials Of Schools Based Health Interventions ... Laurence Moore. Cardiff Institute of Society, Health and Ethics. Email: MooreL1_at_cf.ac.uk ... – PowerPoint PPT presentation

Number of Views:45
Avg rating:3.0/5.0
Slides: 31
Provided by: SSO68
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Cluster Randomised Trials Of Schools Based Health Interventions


1
Cluster Randomised Trials Of Schools Based Health
Interventions
  • What are the barriers to greater use of RCTs in
    educational research?
  • Possibilities for progress or Mission impossible?
  • Demonstrate by example that RCTs of complex
    educational interventions are feasible

2
Projects
  • Trial of fruit tuck shops in primary schools
    (FSA)
  • Trial of emergency contraception lessons (NHS
    RD)
  • ASSIST Trial of peer-led intervention to reduce
    adolescent smoking (MRC)
  • Free breakfast initiative in primary schools in
    Wales (Welsh Assembly Government)

3
  • What are the barriers to greater use of RCTs in
    educational research?

4
Challenges in applying RCTs to evaluation of
educational interventions
  • Ethical concerns
  • Randomisation
  • Recruitment and retention
  • Scale and Cost
  • Variability in delivery
  • Context dependent
  • Generalisability

5
Ethical concerns
  • Often thought unethical to deprive one group of
    people of the innovative intervention, which is
    believed or assumed to be beneficial
  • Contrast to medicine where exposure to untested
    new treatments often considered unethical
  • Very common in education, despite
  • In medicine, target audience is sick and the
    moral imperative to do something (must be better
    then nothing) is great. eg. AIDS, cancer
  • Frequent examples of new interventions being
    ineffective or even harmful

6
Ethical concerns (2)
  • Is randomisation less fair / ethical than
    postcode lottery or local policy / bid success?
  • Only if we are certain that the intervention can
    do no harm should we
  • Implement without strong evidence of effect
  • Begin to think that randomisation might be
    unethical
  • How do we define harm?
  • Cost / opportunity cost
  • Raised expectations

7
Ethical concerns (3)
  • Not wise / moral / prudent / ethical to conduct a
    trial unless one has good reason to believe that
    the intervention may be effective
  • Theory
  • Formative evaluation
  • Principle of equipoise remains

8
Randomisation
  • Often impossible / impractical to randomly assign
    individuals to intervention / control groups
  • Within one cluster, control subjects liable to be
    contaminated by exposure to some/all
    intervention activities
  • Many interventions act explicitly at the cluster
    level (e.g. class, school)
  • Randomisation to intervention / control may be
    undertaken at group level (cluster randomisation)
  • Usually stratified randomisation or minimisation
    to ensure reasonable baseline balance

9
Cluster randomised trials
  • ASSIST Peer-led smoking intervention
  • 59 schools randomised
  • Fruit tuck shops
  • 43 schools randomised
  • Free Breakfast Initiative
  • 57 schools randomised
  • Emergency contraception
  • 25 schools randomised

10
Recruitment and retention
  • Those recruited to trial should be representative
    of target population
  • Participants need to consent to having their
    treatment determined by randomisation
  • Thought to be particularly difficult (unethical)
    in cluster randomised trials
  • In some cluster trials, those randomised to
    control may then not maintain their commitment to
    study
  • Major threat of differential drop-out

11
Recruitment and retention
  • Recruit all schools on basis of equal probability
    of being in intervention or control group
  • Clear, honest detailed description of research
    activities
  • School research contract
  • Offer equal reward to both groups
  • eg. Control schools given cash buy-out time
  • Control schools offered intervention at end of
    measurement period
  • Maintain motivation
  • briefings, personal contact
  • newsletters, prize draws

12
Experience with recruitment and retention of
schools
  • Recruitment
  • School recruitment easier than anticipated
  • Refusal to participate more often due to strong
    preference regarding intervention than objection
    to randomisation or data collection requirements
  • Retention
  • 5 school cluster randomised trials
  • 196 schools
  • 1 3 years fieldwork duration
  • No school drop-outs
  • 2 closures

13
Scale and Cost
  • Co-ordination and timeliness
  • Major challenge in large scale trials
  • Requirement for
  • Communication between researchers and
    policy/practice
  • Research networks
  • Natural experiments
  • Innovations in policy / practice introduced in an
    experimental manner, ideally through randomised
    roll-out

14
Scale and Cost (2)
  • Trials, particularly cluster randomised trials,
    can be large and expensive
  • Intervention costs
  • Outcome data collection costs
  • Natural experiment no extra intervention costs
  • e.g. Free Breakfast Initiative
  • Use of routinely collected outcome data
  • Education has unexploited resource
  • Frequently, trials can be very low-cost

15
Variability in delivery
  • RCTs traditionally require that interventions are
    standardised and uniformly delivered
  • (efficacy trial)
  • Educational interventions highly dependent on
    quality of delivery
  • Value of efficacy trials limited
  • eg. school smoking education
  • Results of efficacy trials involving enthused
    teachers not replicated in roll-out

16
Efficacy and effectiveness
  • Efficacy trial
  • To test whether the treatment does more good than
    harm when delivered under optimal conditions
  • Effectiveness trial
  • To test whether the treatment does more good than
    harm when delivered via a real-world program in
    realistic conditions
  • Pragmatic, allowing variability in delivery as
    would be experienced in real world

17
Context dependent
  • Educational interventions often highly dependent
    on the context within which they are delivered
  • Argued therefore that RCTs not suited to their
    evaluation
  • However, RCT design has the advantage that
    randomisation process ensures that systematic
    differences in external influences between groups
    do not occur
  • Will achieve unbiased estimate of average effect

18
Generalisability
  • Efficacy trials may demonstrate that intervention
    has active ingredients that work
  • Effect unlikely to be reproduced in real world
  • Attenuated by context and implementation
  • Generalisability of small trials with one
    educator in one school will be limited

19
  • Possibilities for progress or Mission Impossible?

20
Public Health Improvement Evidence base conundrum
  • Good quality trials successfully conducted,
    evaluating weak interventions. Small or zero
    effect sizes.
  • Good quality complex interventions evaluated
    using weak research designs. Biased effect
    estimates.

21
When do we do RCTs?
  • In medicine, there are distinct phases in the
    development evaluation of new interventions
    (eg. drugs)
  • Basic research (eg. molecular, genetic)
  • Applied research development (eg.
    pharmacological)
  • Trials to determine efficacy
  • Trials to determine effectiveness
  • Post-marketing surveillance

22
Phases of RCTs of complex interventions MRC
April 2000
23
MRC Assist TrialPeer-led smoking intervention
  • Theory based (Diffusion of innovations)
  • Developed from similar approach used in sex
    education
  • Extensively piloted
  • Feasibility trial conducted in 6 schools
  • Funding for main trial (59 schools) sought and
    obtained from MRC

24
Effectiveness trials with embedded process
evaluation
  • Effectiveness trials, implementing interventions
    in a manner reproducible in real world
  • Crucial to conduct a comprehensive process
    evaluation (largely qualitative) within such a
    trial
  • Monitor variability in context and delivery
  • Identify barriers / facilitators
  • Relate variability in these factors to
    variability in intervention impact

25
Fruit tuck shop trial
  • Minimisation used to ensure balance in terms of
  • School size
  • School policy on snacks
  • Schools given minimal support in setting up tuck
    shops, with wide variability in detailed
    operation
  • Detailed process evaluation
  • Environment of school and locality
  • Operation of fruit tuck shops
  • Detailed case studies of 8 selected schools
  • Observation, interview, focus groups

26
ASSIST Trial
  • Intervention led by specialists, as would be the
    case if rolled out in the real world
  • Not to be implemented by untrained, unmotivated
    teachers
  • Process evaluation in all 30 intervention
    schools, with parallel measures in the 29 control
    schools
  • In-depth process evaluation in sub-sample
  • Observations, field notes, diaries, records,
    interviews with pupils, teachers, staff

27
Free Breakfast Initiative Trial
  • 111 schools across 9 LEAs
  • Variable models of staffing and delivery
  • Trial powered to identify overall mean effect on
    dietary and behavioural outcomes
  • Process evaluation to monitor variation in
    delivery and identify strengths and weaknesses

28
A role for RCTs in evaluating health education
interventions?
  • RCTs not always possible!
  • Difficult to do well, and can be expensive
  • Take opportunity of natural experiments
  • Theory-driven development, formative evaluation
    and feasibility studies essential prerequisites
    prior to trial
  • Get the intervention right

29
Research design
  • Cluster randomised design
  • Pragmatic, effectiveness trials
  • Unbiased estimate of overall intervention effect
  • Additional qualitative and quantitative data
    collection to measure variation in context,
    process, delivery and outcome
  • Identifies issues for further development of
    intervention / further testing of its (variable)
    effect
  • Hypothesis generation, not testing

30
The end.
  • Stanley (1957)
  • Expert opinions, pooled judgements, brilliant
    intuitions and shrewd hunches are frequently
    misleading
  • MacIntyre Petticrew (2000)
  • Good intentions and received wisdom are not
    enough

Laurence Moore Cardiff Institute of Society,
Health and Ethics Email MooreL1_at_cf.ac.uk Tel
02920 875387
About PowerShow.com