Joel Singer, Programme Head, Methodology and Statistics, CIHR Canadian HIV Trials Network

1
Joel Singer, Programme Head, Methodology and
Statistics,CIHR Canadian HIV Trials Network
2
What is a Non-inferiority Trial?

• A trial to show that a particular therapeutic
  agent or policy is no worse than some other
  standard.

3
What is a Non-inferiority Trial?

• Type 1 Both agents have been shown superior to
  placebo/no treatment
• Type 2 Only one agent has been shown superior to
  placebo/no treatment

4
What is a Non-inferiority Trial?

• The standard (STD) should have clinical efficacy
  (i) that is of substantial magnitude (ii) is
  precisely estimated (iii) estimates that are
  relevant to the setting in which the trial is
  being conducted.

5
ICH Guidelines

• A suitable active comparator could be a widely
  used therapy whose efficacy in the relevant
  indication has been clearly established
• and

6
ICH Constancy Assumption

• quantified in well-designed and well documented
  superiority trials and which can be reliably
  expected to have similar efficacy in the
  contemplated active control trial.

7
Example of non-Constancy

• Suppose vancomycin, has been shown to provide a
  large improvement, compared to no treatment, in
  cure rate in patients with various infections.

8
Example of non-Constancy

• Suppose a high prevalence of vancomycin-resistant
  enterococci (VRE) has developed. NI trial
  comparing an experimental antibiotic (EXP) with
  vancomycin is conducted where VRE is prevalent

9
Example of non-Constancy

• A biased estimate of the efficacy of EXP would be
  obtained if the constancy assumption is falsely
  presumed to hold in the NI study.

10
What is a Non-inferiority Trial?

• Because proof of exact equality is impossible, a
  prestated margin of noninferiority for the
  treatment effect in a primary patient outcome is
  defined.

11
What is a Non-inferiority Trial?

• Non-inferiority trials are intended to
• show whether a new treatment has at least as
  much efficacy as the standard or is worse by an
  amount less than ?,often on the premise that it
  has some other advantage

12
What is a Non-inferiority Trial?

• Both participants and outcome measures in a
  noninferiority or equivalence trial should be
  similar to those in trial(s) that established the
  efficacy of the reference treatment.

13
What is a Non-inferiority Trial?

• The required sample size is calculated using the
  confidence interval (CI) approach, considering
  where the CI for the treatment effect lies with
  respect to both the margin of noninferiority and
  a null effect.

14
What is a Non-inferiority Trial?

• Sample size depends on the level of confidence
  chosen, the risk of type II error (or desired
  power), and ?.

15
What is a Non-inferiority Trial?

• A prestated margin of noninferiority can be
  specified as a difference in means or proportions
  or the logarithm of an odds ratio, risk ratio, or
  hazard ratio.

16
What is a Non-inferiority Trial?

• The meaning and clinical importance of measures
  such as relative risk, odds ratio and relative
  hazard will be dependent on the base rate of
  outcome

17
What is a Non-inferiority Trial?

• A prestated margin of noninferiority is often
  chosen as the smallest value that would be a
  clinically important effect.

18
What is a Non-inferiority Trial?

• The required size of non-inferiority trials is
  therefore usually larger than that for
  superiority trials,
• But
• The actual calculation is the same as in
  superiority trials

19
What is a Non-inferiority Trial?

• One should avoid features that might dilute true
  differences between treatments, thereby enhancing
  the risk of erroneously concluding non-inferiority

20
What is a Non-inferiority Trial?

• For superiority trials, intention-to-treat (ITT)
  analysis (analyzing all patients within their
  randomized groups, regardless of whether they
  completed allocated treatment) is recommended

21
What is a Non-inferiority Trial?

• In non-inferiority trials, ITT analysis will
  often increase the risk of falsely claiming
  non-inferiority (type I error), although not
  always

22
What is a Non-inferiority Trial?

• In non-inferiority and equivalence trials,
  non-ITT analyses might be desirable as a
  protection from ITTs increase of type I error
  risk (falsely concluding non-inferiority).

23
What is a Non-inferiority Trial?

• Interpreting a non-inferiority trials results
  depends on where the CI for the treatment effect
  lies relative to both the margin of
  non-inferiority and a null effect. The observed
  treatment effect is not sufficiently informative.

24
What is a Non-inferiority Trial?

• For superiority trials, intention-to-treat (ITT)
  analysis (analyzing all patients within their
  randomized groups, regardless of whether they
  completed allocated treatment) is recommended

25
Major Methodological Issues for
Equivalence/Non-inferiority Trials

• 1. Difficult to get consensus on difference
  considered equivalence or non-inferior.
• 2. In superiority trials, lack of rigor in the
  conduct of a trial will tend to lead to
  equivalence between the comparators and will
  cause a conservative bias.

26

• 3. Lack of rigor or anything which
  causes background noise in an
  equivalence/noninferiority trial
  will bias in favour of the
  hypothesis of interest
  (no difference).

27
Important Variables for Defining
Equivalence/non-inferiority

• 1. Outcome.
• 2. Prevalence of the disease.
• 3. Other cost, invasiveness, ease of use.

28
Trial Design
Nucleoside backbone d4T and 3TC
Inclusion criteria pVL gt 5000 copies/mL any
CD4 cell count any stage of CDC-classification
29
Outcome Measures

• of patients with treatment failure at week 48
• less than 1log10 decline in pVL in first 12 weeks
• 2 consecutive pVL gt 50 copies/mL from week 24
  onwards
• new CDC-C event or death
• change of allocated treatment

30
Methods

• All analyses Intention-to-Treat (unless stated
  otherwise)
• all randomised patients
• pVL data missingfailure
• Four pre-defined pairwise comparisons
• NVP-bd vs EFV (primary)
• NVP-od vs NVP-bd
• NVP-od vs NVPEFV
• EFV vs NVPEFV

31

• Given that the 2NN trial was conducted as a
  non-inferiority trial, how does one go about
  interpreting the results?
• The goal of an inferiority trial is to be able
  to rule out differences greater than the
  minimally important clinical difference (MICD).

32

• Translated into a statistical statement, this
  means we can say with confidence (based on the
  95 confidence interval) that the true
  difference is unlikely to exceed the difference
  deemed clinically important.

33
Methodological Concerns With the 2NN Trial

• 1. Choice of definition of outcome.
• 2. Background noise.

34
Concerns in 2NN Study re non-inferiority

• 1. patients who never received treatment (20 in
  EFV, 10 in NVP-BD).
• 2. early compliance failures.
• 3. Question inclusion of changes in d4T or 3TC as
  evidence of failure.

35
What Were the Primary Results of the 2NN Trial?
36
Treatment Success and Failure
of patients
Success only significant difference EFV vs
NVPEFV, plt 0.001
37

• There was no statistical difference for the
  primary comparison (NVP vs EFV) so one cannot say
  definitively that one treatment is better than
  the other.

38

• The 95 confidence interval (1 in favour of
  nevirapine to 13 in favour of efavirenz) does
  not allow one to rule out a clinically important
  difference (at least as defined by the
  investigators) on the primary efficacy variable.

39
Conclusions

• NVP and EFV have comparable potency in
  suppressing HIV-1 replication
• NVP-od and NVP-bd show comparable efficacy
• Co-administration of NVP and EFV results in
  higher treatment failure due to increased
  toxicity

40
Comments

• Conclusion slide does not address primary outcome
  (combination of disease progression, virologic
  failure and change of therapy)
• Does not address issue of proof of
  non-inferiority

41
Some Technical Sample Size issues

• Typically, one wants the sample size to be large
  enough so that the bounds of the confidence
  interval will exclude values outside the
  non-inferiority range 80-90 of the time

42
Some Technical Sample Size issues

• If the outcome is continuous, and one assumes
  homogeneity of variance, then the sample size
  required for non-inferiority under the null
  hypothesis is the same as the sample size to
  detect the indicated difference under the
  alternative hypothesis

43
Some Technical Sample Size issues

• If the outcome is dichotomous, then the sample
  size required for non-inferiority under the null
  hypothesis is not the same as the sample size to
  detect the indicated difference under the
  alternative hypothesis because the variability is
  a function of p1 and p2

44
Some Technical Sample Size issues

• In a non-inferiority trial, the hypotheses are
  inverted.
• The null hypothesis is that there is a difference
  gt m, and the alternative is the null hypothesis

45
Some Technical Sample Size issues

• Generally, the clinical world is concerned about
  absolute differences between treatments, but
  depending on the statistical model being used,
  the null and alternative may be stated in other
  terms eg hazard ratios

46
Some Technical Sample Size issues

• The choice of what is considered non-inferior in
  terms of a hazard ratio is typically driven by
  what one assumes the baseline rates are
• Eg if the hazard rates are relatively low, a
  somewhat large hazard rate may translate into a
  relatively low absolute difference

47
Summary of 2NN Study
48

• 1. Demonstrated a non-significant 6 advantage in
  the primary outcome in favour of efavirenz over
  nevirapine bid on intent-to-treat analysis.
• 2. Confidence interval around obtained difference
  between efavirenz and nevirapine bid include
  confidence bounds to 13 in favour of efavirenz.
  Cannot exclude non-inferiority on primary outcome
  set out by study criteria.