Title: Randomisationbased efficacy estimators in randomised trials: when are they useful
1Randomisation-based efficacy estimators in
randomised trials when are they useful?
- Ian White
- MRC Biostatistics Unit, Cambridge
- RSS Manchester local group, 20th April 2005
2Setting for todays talk
- Randomised controlled trial to evaluate an
intervention - Departures from randomised intervention
- non-compliance
- changes in prescribed treatment
- Want to infer causal effect of treatment
- Ill discuss medical applications, but the
methods apply in any (quasi-)experiment
3Types of departure from randomised intervention
- Switches to other trial treatment
- Or changes to non-trial treatment
- regard nothing as a treatment!
- Departures may be
- yes/no or quantitative
- constant or time-dependent
? FOCUS ON SWITCHES
4Plan of talk
- Methods
- Intention-to-treat analysis
- Per-protocol analysis
- Randomisation-based efficacy estimators (RBEEs)
- Examples where RBEEs are useful
- Patient information
- Treatment-time interactions
- Treatment-covariate interactions
- Cost-effectiveness analysis
- Obstacles to wider use of RBEEs
5Intention-to-treat analysis(ITT)
6Intention-to-treat analysis
- Compare groups as randomised, ignoring any
departures - Now the standard analysis and rightly so
- Advantages
- respects randomisation
- avoids selection bias
- answers an important pragmatic question e.g. what
is public health impact of prescribing X? - Disadvantages
- may answer the wrong question
7Disadvantage of ITT
- Doctor doctor, how much will taking this tablet
reduce my risk of heart disease? - I dont know, but me prescribing it will reduce
your disease risk by 10 - on average
- thats on average over whether you take it or not
8Alternatives to ITT
- Per-protocol analysisexclude any data collected
after a departure from randomised treatment - Randomisation-based efficacy estimators (RBEEs)
9Simple example
- Experimental vs. Standard treatment
- Interested in mean outcome
- E arm some patients get E, some immediately
switch to S - all-or-nothing switches
- S arm no switches
10Observed data
if rand to E
Define compliers as those who would get E if
randomised to E.
11Model
12Three contrasts
13Is per-protocol analysis reasonable?
- Not reasonable to assume random non-compliance
- if it were, then we wouldnt do RCTs at all!
- Less unreasonable
- if we condition on covariates
- in double-blind trial
14Randomisation-based efficacy estimators (RBEEs)
15Randomisation-based efficacy estimators
- Estimate causal effect of treatment in trials
with treatment switches - Based entirely on comparisons of groups as
randomised - No assumptions of comparability between groups as
treated
16A simple RBEE (binary outcome)
nNE
if nE nS
17General techniques Potential outcomes framework
- Model outcome given the pair of potential
treatments - actual treatment if randomised to control
- actual treatment if randomised to treatment
- Maximum likelihood or Bayesian estimation
- Mostly for all-or-nothing switches
- Causal model relating actual outcome to potential
outcome if untreated - fit e.g. by G-estimation
- works for complex switch patterns
18Likelihood techniques
- Complier, non-complier with prob. wC, wN
- Density
- fCE(yq) for compliers in E
- fCS(yq) for compliers in S
- fN(yq) for noncompliers
- Likelihood (e.g. Imbens and Rubin, 1997)
- wC fCE (yq) for compliers in E
- wN fN(yq) for noncompliers in E
- wC fCS (yq) wN fN(yq) for all in S
19G-estimation techniques
- Define a causal model relating
- outcome Y(z) if treatment z
- to potential outcome Y(0) if no treatment
- e.g. Y(z) Y(0) bz
- Given b, deduce Y(0) for each individual
- Estimate b using Y(0) - R
- e.g. Robins, 1994
- Basic analysis consistent with ITT P-value
- but covariates can increase power
20Four examples of situations where RBEEs are useful
21Example 1.Patient information
22MASS trial
- Abdominal aortic aneurysms are often fatal if
they rupture - May be repaired if detected before rupture
- Reliably detected by ultrasound screening
- MASS trial 67,800 men were randomised to
invitation to screening or control - ITT analysis invitation to screening reduced
aneurysm-related death (Lancet, 2002) - hazard ratio 0.58 (95 CI, 0.42 to 0.78), P0.002
23Non-attendance in MASS
- 20 of invited group didnt attend for screening
- ITT measures the average benefit of screening in
invitees - What is the benefit of screening in attenders?
24MASS model
- Method of Loeys Goetghebeur (2003)
- Stata implementation by Kim White (2004)
- Survivor functions
- SCE(t) in attenders randomised to E,
- SNE(t) in non- attenders randomised to E
- SS(t) in all randomised to S
- P(non-attender) a
- Hazard ratio in attenders y
- Assume exclusion restriction
- So (1-a)SCE(t)1/y a SNE(t) SS(t)
25MASS estimation
- Kaplan-Meier estimates of SCE(t), SNE(t), SS(t)
- Estimate P(non- attender) by a
- Find y where (1-a)SCE(t)1/y a SNE(t) balances
SS(t) - a form of G-estimation
26MASS results
- Effect of screening in attenders (CACE) HR
0.47 (95 CI 0.36 to 0.70), P0.002 - Effect of invitation to screening (ITT) HR
0.58 (95 CI, 0.42 to 0.78), P0.002
27Comment
- In the MASS trial, the P-value is unchanged
because causal and ITT null hypotheses are the
same (under the exclusion restriction) - causal NH screening has no effect in those
screened - ITT NH screening has no effect overall
- The CACE is simply an interpretation of the ITT
result - In the next 2 examples, the causal and ITT null
hypotheses differ
28Example 2.Treatment-time interactions
29Treatment-time interactions
- Switches accumulating over time affect the
treatment-time interaction - Constant causal effect ? declining ITT effect
- Transient causal effect may ? reversed ITT effect
30Simple case
- Causal effect of E is
- b1 at time 1 after receiving E
- b2 at time 2 after receiving E
- Switches occur just after time 1
- a fraction a of the S arm get E
- but all the E arm get E at the start
- Then the ITT difference at time 2 is b2 - a b1
- e.g. b110, b2 2, a 0.3 ? ITT2 -1
- ITT can get wrong sign with treatment-time
interaction
31Estimation
- Solve equations like ITT2 b2 - a b1
- This is a special case of a structural nested
mean model (Robins, 1994) - More generally
- add covariates that predict switch or outcome
- model covariance
- allow for missing data
32TARGET trial
- Treatment of glue ear in children
- 376 children randomised to 3 arms
- insertion of ventilation tubes (VT) (grommets)
- insertion of ventilation tubes adenoidectomy
(AD) - medical management (MM)
- Outcome hearing loss measured at 5 visits
- Approx 50 of MM arm eventually got VT
- Some of VT arm had re-insertion
33Sorry!
- The talk as presented contained numerical results
from TARGET - These cant yet be presented on the WWW
- Broadly, RBEEs demonstrated a longer-lasting
benefit of VTs
34Example 3.Treatment-covariate interactions
35Treatment-covariate interactions
- Suppose the switch rate depends on covariates
- Then ITT effect will be more attenuated in
subgroups with more switches - Common causal effect ? different ITT effects
36Example
- Suppose in subgroup j
- causal effect of E is bj
- fraction aj of S arm get E
- then ITT effect of E is bj(1-aj)
- e.g. by baseline severity
- less severe b18, a10.1 ? ITT1 7.2
- more severe b2 10, a20.5 ? ITT2 5
- Treatment works better in more severe subgroup,
but appears to work worse
37TARGET again
- Covariate baseline hearing loss (dichotomised
at median) - Clinically plausible that VT works better when
hearing loss is greater not confirmed by ITT
analyses - Modify RBEE to incorporate covariate and
interaction
38Sorry again!
- Again, the numerical results from TARGET cant
yet be presented on the WWW - Broadly, RBEEs demonstrated a pattern of
interactions that was more in line with clinical
plausibility
39Example 4.Cost-effectiveness analysis
40Cost-effectiveness analysis
- In a RCT, explore difference in costs as well as
difference in effects - A pragmatic analysis cost-effectiveness among
compliers is not of interest - But compliance in practice is probably less than
in an RCT - Need to allow for this in cost-effectiveness
analysis
41Two cases
- If its reasonable to assume
- effect of intervention ? compliance
- cost of intervention ? compliance
- then cost-effectiveness is independent of
compliance - But cost of intervention is more likely to
comprise - a constant part (e.g. cost of prescribing drug)
- a part ? compliance (e.g. cost of repeat
prescriptions) - Here, cost-effectiveness does depend on
compliance.
42Obstacles to wider use of RBEEs
43What happens in practice?
- Most trials report an ITT analysis
- or at least claim to do so
- Many trials additionally report a per-protocol
analysis - RBEEs are rarely used
- Why?
44Changes to non-trial treatment
- E.g. in a trial of A vs. placebo, some patients
get B - Awkward to adjust for receipt of B, because we
need to know/estimate effect of B - estimate it observationally within trial?
- sensitivity analysis?
- use knowledge from other trials?
- Similar problem arises with changes to no
treatment in an equivalence trial
45How well developed are RBEE methods?
- Plenty of examples for all-or-nothing compliance
- pitfalls are known
- Few examples for more complex compliance
- Methods are complex not very general
- Not much software available
- Stata strbee, snmm, stcomply (BSU web site)
- Can implement in Mplus, gllamm, WinBUGS
46How useable are RBEE methods?
- Methods are unfamiliar to most statisticians
- often wrongly seen as a variant of per-protocol
analysis - Need ways to explain them to non-statisticians
- may be usefully described as interpreting ITT
analysis
47Conclusions
- ITT analysis is still essential
- Per-protocol analysis should be avoided
- Statisticians should learn to recognise
situations in which RBEEs can give clinically
useful information - those in which ITT is sufficient
- At present this is largely restricted to
situations with switches
48- Thanks to
- Mark Haggard and the TARGET investigators
- Lois Kim
49References
- Imbens GW, Rubin DB. Bayesian inference for
causal effects in randomized experiments with
noncompliance. Annals of Statistics 1997 25
305327. - Robins JM. Correcting for non-compliance in
randomized trials using structural nested mean
models. Communications in Statistics Theory
and Methods 1994 23 23792412. - The Multicentre Aneurysm Screening Study Group.
The multicentre aneurysm screening study (MASS)
into the effect of abdominal aortic aneurysm
screening on mortality in men a randomised
controlled trial. Lancet 2002 360 15311539. - Loeys T, Goetghebeur E. A causal proportional
hazards estimator for the effect of treatment
actually received in a randomized trial with
all-or-nothing compliance. Biometrics 2003 59
100105. - Kim LG, White IR. Compliance-adjusted treatment
effects in survival data. STATA journal 2004 4
257264. - White IR. Uses and limitations of
randomisation-based efficacy estimators.
Statistical Methods in Medical Research, to
appear. - ian.white_at_mrc-bsu.cam.ac.uk
- http//www.mrc-bsu.cam.ac.uk/pub/software/stata/