The Analysis Of Clinical Trials With Clustering Effects Due To Treatment - PowerPoint PPT Presentation

1 / 62
About This Presentation
Title:

The Analysis Of Clinical Trials With Clustering Effects Due To Treatment

Description:

Participants: 262 patients presenting with depression or mixed anxiety depression. ... Primary outcome measures: Beck Depression Inventory scores at 4 and 12 months. ... – PowerPoint PPT presentation

Number of Views:423
Avg rating:3.0/5.0
Slides: 63
Provided by: chrisr88
Category:

less

Transcript and Presenter's Notes

Title: The Analysis Of Clinical Trials With Clustering Effects Due To Treatment


1
The Analysis Of Clinical Trials With Clustering
Effects Due To Treatment
  • University of Manchester, UK
  • Email chris.roberts_at_manchester.ac.uk

2
Therapist Effect and Clinical Trials
  • Outcome for patient treated by the same therapist
    may be more similar than outcome for patients
    treated by different therapists.
  • Therapist characteristics
  • Experience
  • Competence
  • Empathy
  • Patients clustered by therapist.

3
Designs of Trials of Therapies
  • Hierarchical Therapist Design.
  • Therapist - Non Therapist Design.
  • Stratified Therapist Design.

4
Hierarchical Therapist Design
  • Therapists nested within treatment
  • Patients clustered by therapists
  • Cluster Randomised Trial Therapist ? cluster
  • e.g non-directive counselling (GP counsellors)
    or cognitive-behaviour therapy (clinical
    psychologist)

Treatment Effect
Therapist Variation
Covariates
Patient Variation
5
Therapist - Non Therapist Design
Example No Drug vs Drug Therapy Similar to
comparison of group therapy with individual
therapy.
Treatment Effect
Therapist Variation
Patient Variation
Covariates
  • Subjects in control arm are treated as clusters
    of size 1

6
Comparison of Trials With Clustering Due to
Randomisation or Treatment
7
Comparison of Trials With Clustering Due to
Randomisation or Treatment
8
Comparison of Trials With Clustering Due to
Randomisation or Treatment
9
Comparison of Trials With Clustering Due to
Randomisation or Treatment
10
Heteroscedasticity in a Hierarchical Therapist
Design
  • In cluster randomised trial between arm
    heteroscedasticity generally ignored.

Treatment Effect
Therapist Variation
Covariates
Patient Variation
Does heteroscedasticity matter or can one use a
random intercept model or equivalent.
11
An Example RCT Of General Practitioners And
Nurse Practitioners In Primary Care
  • Health professionals
  • 20 nurse practitioners seeing a mean of 30
    patients (range 30 to 36)
  • 71 general practitioners seeing a mean of 8
    patients (range 2 to 29)
  • Patients Sample size
  • 1280 patients approx 64 per Practice in 20
    practices
  • Outcome measures
  • Patient satisfaction
  • Consultation process (length of consultation,
    examinations, prescriptions, referrals),
  • Health service costs

12
NP-GP TrialTreatment Effect- Patient Satisfaction
Choice of model effects treatment effect standard
errors
13
Total Variance Equal In Both Arms
14
Conclusions- Heterogeneity
  • Fitting a random intercept model may lead to
    under or over estimation of the intra-class
    correlation coefficient if cluster size differ
    systematically between arms
  • Test size for the treatment effect for a random
    intercept model may therefore be biased .
  • Ideally, fit a heteroscedastic model.
  • But this may be difficult in small samples sizes.
  • If you fit the wrong model,
  • you can expect to get the wrong answer.

15
Conclusions- Heterogeneity
  • Sample size for cluster randomised trials based
    on a summary measures t-test
  • Heterogeneity in intra-class correlation
    coefficients cluster size leads to
    heteroscedasticity at the summary level.
  • Base sample size on Satterthwaites t-test
  • Implemented in NQuery adviser
  • Stata routine

16
Unequal Randomization Group vs Individual Therapy
Moderate Effect Size 0.5 80 Power, 5 sig.
level ?? 0.05
17
Cluster Membership
18
Trial Non-directive Counselling or
Cognitive-Behaviour Therapy
  • Participants 262 patients presenting with
    depression or mixed anxiety depression.
  • Interventions Up to 12 sessions of therapy
  • cognitive-behaviour therapy (12 psychologist)
  • non-directive counselling (14 counsellors)
  • Primary outcome measures Beck Depression
    Inventory scores at 4 and 12 months.
  • Original published analysis ignored clustering by
    therapist

19
Numbers of Patients per Therapist
67 of patients treated by 5 therapists
13 of patients have no identified therapist
20
Issues for Design Analysis
  • No therapist information for some patients.
  • Non-compliance?
  • Grossly unequal cluster size.

21
Analysis of Therapist Effects with Non-compliance
  • Intended Therapist (IT)
  • Subjects analysed according to the intended
    therapist.
  • If the patient does not receive therapy, do they
    have between therapist variation?
  • IT method mis-specifies the therapist random
    effect.
  • Intended therapist may not be known.
  • Actual Therapist (AT)
  • Subjects analysed according to the actual
    therapist (AT).
  • Non-compliant subjects assumed to have no
    therapist random effect.

22
Simulations Study of Non-compliance with
Treatment Related Clustering Variation
  • Objectives Compare performance of Intended
    Therapist (IT) and Actual Therapist (AT) analysis
    strategies.
  • Assess bias in
  • Treatment effect.
  • Clustering effect of therapist.
  • Data simulated for a trial with comparing
    therapists with control.

23
Simulations Study with All or Nothing Compliance
24
Analysis Methods Compared
  • Ignore Clustering.
  • Using either Intended Therapist (IT) or Actual
    Therapist (AT) Clustering
  • Random Effects ML
  • GEE
  • Robust Standard Error / Sandwich Estimators
  • Analysis carried out in STATA

25
Treatment Effect
26
Treatment Effect Intended and Actual Therapist
27
ITT with No Clustering
28
Clustering- ML Estimation
29
Maximum likehood Estimatorfor Actual Therapist
Analysis
Likelihood for therapists
Approximation for AT treatment effect estimate
30
Treatment Effect ML GEE
31
Treatment Effect Robust Estimators
32
Intra-class Correlation for Therapist
33
Estimated ICC Random Effects ML
34
Estimated ICC
35
Estimated ICC Random Effects Robust
36
Summary
  • Ignoring therapist will be non-conservative.
  • Actual Therapist (AT) method does not perform
    well for ML and GEE.
  • Intended Therapist (IT) method gives unbiased
    estimates of treatment effect for ML GEE and
    Robust estimation.
  • IT method can be non-conservative.

37
Causal Analysis Methods
  • Ignoring Therapist Clustering
  • Maximum Likelihood Latent Class
  • Instrumental variables regression
  • Using either Intended or Actual Therapist
  • Instrumental variables regression with Robust
    Standard Error / Sandwich Estimators.
  • Using either Actual Therapist
  • Maximum Likelihood Latent Class with random
    effect.

38
CACE Random Effects Mixture Model
Latent Class / Normal Mixture Model fitted to
control subjects
Therapist arm
- compliant
- non compliant
Control arm
39
CACE Mixture Model
40
Estimated ICC from CACE estimates
41
CACE Random Effects Mixture Model
  • Fitted using ML in Stata
  • Covariates could be added for compliance
    probability
  • What if compliance probability varies between
    therapist?
  • This require a combined IT model for compliance
    probability and AT model of outcome.

42
Summary
  • Ignoring clustering is non-conservative.
  • Robust instrumental variable methods are unbiased
    but may be conservative for both IT and AT
    methods.
  • Mixture model methods performed well for AT
    method.
  • Problems with convergence for mixture model with
    IT method.

43
Conclusions
  • Therapist trials more complex than cluster
    randomised trials if clustering is considered as
    cluster size and intra-cluster correlation may
    differ systematically between treatment arms.
  • Therapist effects must be considered in trial
    design.

44
Comparison of Trials With Clustering Due to
Randomisation or Treatment
45
Additional/Left Over Slides
46
Trial Non-directive Counselling or
Cognitive-Behaviour Therapy
Problems due to wide variation in cluster size
47
Problems of Large Variation in Cluster Size
  • ICC values for therapist likely to be close to
    zero
  • Maximum cluster size places lower limit on ICC
    -1/(max cluster size -1) causing non-convergence
  • Negative ICC (GEE or ML were negative variances
    are allowed) and design effect (Robust) leading
    to downwardly biased treatment effect standard
    errors.

48
References
  • Heteroscedasticity and sample size discussed in
  • Roberts C Roberts SA (2005) Design and analysis
    of clinical trials with clustering effects due to
    treatment Clinical Trials, 2152-162.

49
Acknowledgements
  • Steve Roberts Help with simulation work.
  • Pete Bower Providing data from Trial of
    Non-Directive Counselling or Cognitive-Behaviour
    Therapy.

50
Design Effect
Above 1 is conservative
51
Design Effect
Above 1 is conservative
52
Stratified Therapist Design
  • Same health professional delivers both
    interventions
  • Example CBT delivered face-to-face or by phone
  • Similar to
  • Multi-centre studyTherapist ? Centre/Study
  • Meta-analysis Therapist ? Study
  • Match pairs cluster randomised Therapist ? Match
    pair

53
Stratified Therapist Design
Patient Variation
Treatment Effect
Therapist Variation
Covariates
Therapist-Treatment Interaction
Patient Variation
Treatment Effect
Therapist Variation
Covariates
Treatment within Therapist
54
IV Random Effects Model
55
IV Random Effects Model
56
Likelihood Equation For Random Intercept Model
Replace red bits by expected values for
different cluster size and heteroscedasticity
then maximize
57
Modelling NP vs GPAdjusting for Age and Sex
58
Comparison of observed with approximation
59
Simulation Study
60
Total Variance Equal In Both Arms
1000 simulation ML REML in R Solid line are
Guesstimates
61
Total variance 25 smaller for smaller clusters
62
Total variance 25 larger for smaller clusters
Write a Comment
User Comments (0)
About PowerShow.com