Comparison of methods for the analysis of pairmatched cluster randomised trials: A simulation study

1
Comparison of methods for the analysis of
pair-matched cluster randomised trials A
simulation study
Patty Chondros Prof John B Carlin Dr Obioha C
Ukoumunne Prof Jane M Gunn
2
Overview

• Pair-matched cluster randomised trials (CRTs)
  analysis issues
• Methods for analysing continuous outcomes for
  pair-matched CRTs
• Simulation design and results
• Summary

3
Why use pair-matched CRTs?

• When number of clusters is small ...
• Minimise risk of imbalance on known prognostic
  factors
• Increase precision of the estimated intervention
  effect
• Add credibility to the study
• Clusters are matched on risk factors associated
  with the outcome
• For each pair, one cluster is randomly allocated
  to intervention arm one to the control arm
• If matching is effective, pair-matched CRTs will
  be more efficient than completely randomised CRTs
• Matching correlation
• Correlation between cluster level outcome means
  within pairs (strata)

4
Analysis issues of pair-matched CRTs

• Analysis methods for pair-matched CRTs are more
  limited than completely randomised CRTs
• There is no replication of clusters within each
  stratum and trial arm combination
• Between-cluster variance cannot be estimated
• variation within each trial arm is confounded
  with variation between-strata
• variation within each stratum is confounded with
  the intervention effect
• Intra-cluster correlation (ICC) can not be
  estimated
• measures the positive correlation of individuals
  within the same cluster
• proportion of the total variance due to
  between-cluster variance
• For the analysis of pair-matched CRTs
• Standard error of the intervention effect is
  derived from between-stratum information

5
Cluster level methods

• Paired t-test on cluster means
• Two sample t-test on cluster means
• Ignore the matching
• Cluster level random effects regression (CL/RE)
• Random effects meta-analysis approach on cluster
  means (Thompson et al, 1997)
• Each stratum is treated as an individual study
• Effects of strata are treated as random effects
• Intervention effect is a weighted average of
  differences between cluster means within strata
  (pair of clusters)
• Use t-based confidence intervals with dfk-1,
  where k is the number of strata

6
Individual level regression-based methods

• Marginal models using Generalised Estimating
  Equations (GEEs)
• Extension of GLM that allows for correlated
  outcomes
• Information sandwich (robust) standard errors
• gives consistent SEs even when correlation
  structure is misspecified
• Exchangeable working correlation
• Use t-based CIs
• Two approaches
• Adjust for the clustering effect, ignoring the
  matching (df2k-2)
• Treat strata (pairs of clusters) as clusters
  (dfk-1)
• where knumber of strata
• Random effects (RE) model
• Random effect of the cluster (or stratum) is
  modelled explicitly

7
For the analysis of pair-matched CRTs.

• Martin et al (1993) Diehr et al (1995) showed
  that using cluster level two sample t-test
  provides more efficient estimates than cluster
  level paired t-test when number of stratalt10
  matching is not effective
• any gains in precision using matched analysis is
  offset by loss in degrees of freedom
• We extend this work to individual level
  regression methods and meta-analysis approach for
  the analysis of pair-matched CRTs

8
Aim

• Evaluate the performance of the statistical
  analysis methods that can be applied to CRTs with
  a pair-matched design for continuous outcomes
• Methods for analysing CRTs
• Cluster level analysis methods
• Individual level analysis methods

9
Simulation design Parameter values
Continuous outcome Intervention effect 0
10
Simulation design

• 2000 datasets generated for each combination of
  design parameter values
• Correlated data generated from random effects
  model
• Analysis methods applied to each data set
• For each data set saved the
• estimated intervention effect
• standard error of the estimate
• 95 confidence interval of the intervention
  effect

11
Model for generating correlated continuous outcome
Strata
Clusters
Individuals
12
Model for generating correlated continuous outcome
Matching correlation
Intra-cluster correlation
13
Simulation design Evaluation

• Coverage of 95 Confidence Intervals
• Percentage of confidence intervals that include
  the true intervention effect
• Coverage estimated with a margin of error of 1
• Mean width of confidence intervals
• Given two analysis methods with similar coverage,
  method with narrower CIs is preferred as it
  indicates greater precision

14
Adjust for clustering effect Matching
correlation0.1Coverage
15
Adjust for clustering effect Matching
correlation0.1 Mean width of 95 CIs
16
Adjust for clustering effect Matching
correlation0.5Coverage
17
Adjust for clustering effect Matching
correlation0.5 Mean width of 95 CIs
18
Summary Analysis methods that ignore matching

• Cluster level 2 sample t-test provided good
  coverage and had narrower CIs than cluster level
  paired t-test when matching correlation was 0.1
  number of strata was lt10
• GEEs adjusting for clustering effect
• generally provided good coverage when matching
  correlation was 0.1 and number of strata lt10
• similar mean width of CI with cluster level 2
  sample t-test
• Generally, RE model adjusting for clustering
  effect only, had poor coverage when number of
  strata lt10
• All methods above provided good coverage when
  number of strata was large and matching
  correlation was 0.1 or less
• mean width of CIs similar to CL paired t-test
  (Results not shown)

19
Methods that allow for matching Matching
correlation0.5Coverage
20
Methods allowing for matching Matching
correlation0.5Coverage
21
Summary Analysis methods that allow for matching

• Random effects meta-analysis approach performed
  poorly when number of strata lt10
• GEEs treating strata as clusters had identical
  results to cluster level paired t-test
• Individual level random effects model with random
  effects for strata gave very poor coverage as ICC
  increased and/or cluster size increased
• design effect increases

22
Further work

• Unbalanced cluster size
• Binary outcomes
• Loss of cluster from a pair
• Adjustment of covariates (Donner, Taljaard, Klar,
  2007)

23
References

• Martin DC, Diehr P, Perrin EB, Koepsell TD
  The Effect of Matching on the Power of Randomized
  Community Intervention Studies. Statistics in
  Medicine 1993, 12(3-4)
• Diehr P, Martin DC, Koepsell T, Cheadle A
  Breaking the matches in a paired t-test for
  community interventions when the number of pairs
  is small. Statistics in Medicine 1995, 14(13)
• Thompson SG, Pyke SDM., Hardy RJ The design
  and analysis of paired cluster randomized trials
  an application of meta-analysis techniques.
  Statistics in Medicine 1997, 16(18)
• Klar N, Donner A The merits of matching in
  community intervention trials A cautionary tale.
  Statistics in Medicine 1997, 16(15)
• Donner A, Taljaard M, Klar N The merits of
  breaking the matches a cautionary tale.
  Statistics in Medicine 2007, 26(13)

24
Variance components

• If we let,