Issues of Simultaneous Tests for Non-Inferiority and Superiority

1
Issues of Simultaneous Tests for Non-Inferiority
and Superiority

• Tie-Hua Ng, Ph. D.
• U.S. Food and Drug Administration
• Ng_at_cber.fda.gov
• Presented at
• MCP 2002
• August 5-7, 2002
• Bethesda, Maryland
• _______
• The views expressed in this presentation are
  not necessarily of the U.S. Food and Drug
  Administration.

2
Simultaneous Tests for Non-Inferiority and
Superiority

• Multiplicity adjustment is not necessary
• Intersection-union principle (IU)
• Dunnett and Gent (1996)
• Closed testing procedure (CTP)
• Morikawa and Yoshida (1995)
• Indisputable

3
A Big Question

• Is Multiplicity Adjustment Necessary?

4

• Is
• Multiplicity
• Adjustment
• Necessary?

5
Outline

• Assumptions and Notations
• Switching between Superiority and Non-Inferiority
• Is Simultaneous Testing Acceptable?
• Use of Confidence Interval in Hypothesis Testing
  --- Pitfall
• Problems of Simultaneous Testing
• Conclusion

6
Assumptions/Notations

• Normality and larger is better
• T Test/Experimental treatment (?t)
• S Standard therapy/Active control (?s)
• ? Non-Inferiority Margin (gt 0)
• For a given d (real number), define
• Null H0(d) T ? S - d
• Alternative H1(d) T gt S - d
• Non-Inferiority d ?
• Superiority d 0

7
Non-Inferiority (d ?)

• H0(?) T ? S - ? against H1(?) T gt S - ?

H1(?)
H0(?)
?
S
Worse
Better
Mean Response
8
Superiority (d 0)

• H0 (0) T ? S against H1 (0) T gt S

H1(0)
H0(0)


S
T
Worse
Better
Boundary
Mean Response
9
Switching between Superiority and
Non-Inferiority

• CPMP (Committee for Proprietary Medicinal
  Products), European Agency for the Evaluation of
  Medicinal Products
• Points to Consider on Switching Between
  Superiority and Non-Inferiority, 2000.
• http//www.emea.eu.int/htms/human/ewp/ewpptc.htm

10
Switching between Superiority and
Non-Inferiority (2)

• Non-Inferiority Trial
• If H0(?) is rejected, proceed to test H0(0)
• No multiplicity issue, closed testing procedure
• Superiority Trial
• Fail to reject H0(0), proceed to test H0(?)
• No multiplicity issue
• Post hoc specification of ?

11
Switching between Superiority and
Non-Inferiority (3)

• Non-inferiority Trial
• Intention-to-treat (ITT)
• Per protocol (PP)
• Superiority Trial
• Primary Intention-to-treat (ITT)
• Supportive Per protocol (PP)
• Assume ITT PP

12
Simultaneous Testing

• One-sided 100(1 - ?) lower Confidence Interval
  for T - S

Superiority
Non-inferiority
Neither
-?
0
Test is worse
Test is better
Mean Difference (T S)
13
Simultaneous Testing (2)

• Multiplicity adjustment is not necessary
• Dunnett and Gent (1996)
• Intersection-Union (IU)
• Superiority Both H0(?) and H0(0) are rejected
• Morikawa and Yoshida (1995)
• Closed Testing Procedure (CTP)
• Test H0(0) when H0(?)?H0(0) is rejected

14
Simultaneous Testing (3)

• Discussion Forum (October 1998)
• London
• PSI (Statisticians in Pharmaceutical Industry)
• Is Simultaneous Testing of Equivalence
  Non-Inferiority and Superiority Acceptable?
• Superiority trial
• Fail to reject H0 (0)
• No equivalence/non-inferiority claim
• Ok Morikawa and Yoshida (1995)
• Ref Phillips et al (2000), DIJ

15
Is Simultaneous TestingAcceptable?
16
Use of Confidence Interval in Hypothesis Testing

• H0(d) T ? S - d (at significance level ?)
• One-sided 100(1-?) lower CI for T-S
• Reject H0(d) if and only if the CI excludes -d

Reject H0(d)
Do not reject H0(d)
-d
Test is worse
Test is better
Mean Difference (T S)
17
Use of Confidence Interval in Hypothesis Testing
(2)

• If CI (L, ?), then H0(d) will be rejected for
  all -d lt L.
• A Tricky Question
• Suppose CI (-1.999, ?), L -1.999
• H0(2) T ? S - 2 is rejected (d2) since -d lt L
• Can we conclude that T gt S - 2?
• Yes, if H0(2) is prespecified.
• No, otherwise.

18
Use of Confidence Interval in Hypothesis Testing
(3)

• Post hoc specification of
• H0(d)
• is a
• No No

19
Simultaneous Testing Problems

• H0(d1) and H0(d2), for d1 gt d2
• One-sided (1 - ?)100 lower CI for T - S

Reject H0(d2)
Reject H0(d1)
Neither
-d2
-d1
Test is worse
Test is better
Mean Difference (T S)
20
Simultaneous Testing Problems (2)

• H0(d1), H0(d2) and H0(d3), for d1 gt d2 gt d3
• One-sided (1 - ?)100 lower CI for T - S

Reject H0(d3)
Reject H0(d2)
Reject H0(d1)
None
-d3
-d1
-d2
Test is worse
Test is better
Mean Difference (T S)
21
Simultaneous Testing Problems (3)

• H0(d1), H0(d2),, H0(dk), for d1 gt d2 gt gt dk
• One-sided (1 - ?)100 lower CI for T - S

Reject H0(dk)
. . .
Reject H0(d2)

Reject H0(d1)
None
-dk
-d1
-d2

-d3
Test is worse
Test is better
Mean Difference (T S)
22
Simultaneous Testing Problems (4)

• Choose k large enough
• Pr-d1 lt Lower limit lt -dk close to 1
• Max dk - dk-1 lt a given small number
• Simultaneous testing of H0(di), i 1,, k
• ? Post hoc specification of H0(d)

23
Simultaneous Testing Problems (5)

24
Simultaneous Testing Problems (6)

• What is wrong with IU and CTP?
• Nothing
• PrRejecting at least one true null ? ?
• What kind of problems?

25
Simultaneous Testing Problems (7)

• Post hoc specification of H0(d)
• Let -d0 100(1 - ?) lower limit - ?
• Reject H0(d0), since -d0 lt lower limit
• Repeat the same trial independently
• PrRejecting H0(d0) 0.5

26
Simultaneous Testing Problems (8)

• Simultaneous testing of many H0(d)
• Repeat the same trial independently
• Low probability of confirming the finding
• 1st trial Reject H0(dj) but not H0(dj1)
• 2nd trial PrRejecting H0(dj) is low (e.g.,
  0.5)

27
Simultaneous Testing Problems (9)

• Simultaneous testing of H0(?) and H0(0)?
• Confirm the finding
• ? 2
• Known variance
• Let ? ? T - S
• Significance level ? 0.025
• 80 power for H0(?) (at ? 0)

28
Simultaneous Testing Problems (10)
f?(?) PrRejecting H0(?) ? f0(?)
PrRejecting H0(0) ?
29
Simultaneous Testing Problems (11)

• Test one null hypothesis H0(?)
• Suppose that H0(?) is rejected
• Repeat the same trial independently
• PrRejecting H0(?) again f?(?)

30
Simultaneous Testing Problems (12)

• Test H0(?) and H0(0) simultaneously
• Suppose that H0(?) or H0(0) is rejected
• Repeat the same trial independently
• PrRejecting the same null hypothesis again 1
  - w(?) f?(?) w(?) f0(?)
• f?(?) - f0(?) 1 f0(?)/f?(?),
• where w(?) f0(?)/f?(?)

31
Simultaneous Testing Problems (13)
1 - w(?) f?(?) w(?) f0(?) where w(?)
f0(?)/f?(?)
Simultaneous tests in the 2nd trial
32
Simultaneous Testing Problems (14)

• Ratio 1 f0(?)/f?(?) 1 f0(?)/f?(?)
• Ratio may be as low as 0.75

33
Conclusion

• Many H0(d) Problematic
• Not type I error rate
• H0(?) and H0(0) Acceptable?
• If zero tolerance policy No
• If 25 reduction cannot be tolerated No
• If 25 reduction can be tolerated Yes

34
Is Simultaneous Testing of H0(?) and H0(0)
Acceptable?
35
You be the judge
36
References

• Dunnett and Gent (1976), Statistics in Medicine,
  15, 1729-1738.
• Committee for Proprietary Medicinal Products
  (CPMP 2002). Points to Consider on Switching
  Between Superiority and Non-Inferiority.
  http//www.emea.eu.int/htms/human/ewp/ewpptc.htm
• Morikawa T, Yoshida M. (1995), Journal of
  Biopharmaceutical Statistics, 5297-306.
• Phillips et al., (2000), Drug Information
  Journal, 34337-348. 

37
God Bless America