Multiplicity Adjustment For Clinical Trials with Two Doses of an Active Treatment and Multiple Primary and Secondary Endpoints

1
Multiplicity Adjustment For Clinical Trials
with Two Doses of an Active Treatment and
Multiple Primary and Secondary Endpoints

• Hui Quan1, Tom Capizzi1, Ji Zhang2
• 1Merck Research Laboratories
• 2Sanofi-Synthelabo Research
• FDA/Industry Statistics Workshop
• September 21-23, 2004

2
Outline

• Background and clinical trial setting
• Procedures for a two-dimensional problem
• Procedures for a three-dimensional problem
• Numerical examples for comparing powers
• Real data example
• Procedures without overall strong control
• Discussion

3
Background

• Multiple primary and secondary endpoints are
  often simultaneously considered in Phase III
  trials.
• Positive findings on secondary endpoints could be
  for
• 1 Supporting primary results or Hypotheses
  generating
• 2 Additional claims or description in drug
  label
• For category 2 ? recent regulatory guidances and
  publications emphasized to have Type I error rate
  controlled for secondary endpoints.

4
Background (2)

• Different approaches for interpreting trial
  results
• Conventional standard for positive trial
  positive effect on primary endpoints.
• Others a trial as positive even when effect is
  positive only on a secondary endpoint ? secondary
  endpoints must be considered in multiplicity
  adjustment.
• For example Moyé (2000) proposed a PAAS for
  controlling the FWE rate aF with aP for the
  primary endpoints and aF-aPaS for the secondary
  endpoints.

5
Trial Setting

• Trial examples
• Parathyroid hormone fracture trial (Neer et al.
  2000) 20 and 40 µg vs. placebo on vertebral
  fracture and other fractures
• Rofecoxib RA trial (Geunese, et al. 2002) 25 and
  50 mg vs. placebo on four primary and other
  secondary endpoints
• Estrogen Alzheimer trial (Mulnard, et al. 2000)
  0.625 and 1.25 mg vs. placebo on CGIC and other
  scores
• Metformin and rosiglitazone combination trial
  (Fonseca, et al. 2000) Rosiglitazone 4 and 8 mg
  vs. placebo on glucose and lipid endpoints

6
Trial Setting (2)

• Three-dimensional multiplicity adjustment
  problem
• Two doses
• Two priority categories of endpoints primary
  secondary
• Multiple endpoints in each priority category
• Higher dose-the primary dose, lower dose-the
  secondary dose (no monotonic order on parameter
  space for dose response is assumed for any
  endpoint)
• Trial positive only if positive on one or more
  primary endpoints except when PAAS is used for
  multiplicity adjustment
• No further priority order for endpoints within
  each category

7
Trial Setting (3)

• Traditional procedures may be lack of power.
• For example for a trial with two primary
  and two secondary endpoints-a total of 8 active
  to control comparisons, if a Hochberg procedure
  is used and if the lower dose shows no effect on
  the two secondary endpoints, the higher dose on
  the primary endpoints will be assessed at most at
  level of ?/3.
• Since no monotonic order in dose response is
  assumed, stepdown trend tests are not applicable.
  
• More specific gatekeeping procedures may have
  higher power.

8
Procedures for a Two-dimensional Problem Two
doses and multiple endpoints

• Hij is the null hypothesis for comparing the i
  th dose to the control on the j th endpoint, and
  pij is the corresponding p-value, i1, 2 and j1,
  2, , g. (Dose 1 the higher and primary dose)
• Modified Bonferroni-Closed Procedure (2.1)
• Step1. Reject all Hij if all pij ?, i1,2,
  j1,, g.
• Otherwise,
• Step 2. Reject H1j if p1j ?/g
• Reject H2j if p1j ?/g and p2j
  ?/g

9
Modified Hochberg-Bonferroni Procedure (2.2)

• Step 1. Reject all Hij if all pij ?, i1,2,
  j1,, g.
• Otherwise,
• Step 2. Use Hochberg procedure for the higher
  dose at ?2?/3.
• Step 3. Let l of endpoints of no effect for
  the higher dose. If l0, use
• a Hochberg procedure for the lower
  dose at level ?. Otherwise,
• Step 4
• a) Use Hochberg procedure for the lower dose
  at ? with all endpoints
• b) Assume no effect on the l (gt0) endpoints
  for the lower dose
• c) Use Hochberg procedure for the lower dose
  on the other (g-l)
• endpoints at level ?/3 then
• d) Claim the effect for the lower dose on an
  endpoint if the
• effect is significant at both a) and c).

10
Weighted Modified Bonferroni-Closed Procedure
(2.3)

• Step 1. Reject all Hij if all pij?
• Otherwise,
• Step 2. Reject H1j if p1j ?wj
• reject H2j if p1j ?wj and p2j
  ?wj .
• where wj 0 and Swj1
• Applicable to a three-dimensional problem by
  assigning more weights on the primary and less
  weights on the secondary endpoints. However,
  different from PAAS, no need to split the
  significance level if all pij ?.

11
Properties of Procedures (2.n)

• Reject all the hypotheses if all p-values are ?.
• Step down from the higher dose to the lower dose
  to form closed procedures.
• Theoretically or based on simulation, provide
  strong control on the FWE rate under appropriate
  conditions.
• for (2.1) and (2.2) the condition for the
  Simes test
• for (2.3) positive regression dependency
  condition for
• the weighted Simes test.

12
Procedures for the Three-dimensional Problem

• Hij is the null hypothesis for comparing the i th
  dose to the control on the j th primary endpoint,
  i1, 2 and j1,, gp
• Sik is the null hypothesis for comparing the i th
  dose to the control on the k th secondary
  endpoint, i1, 2 and k1,, gs.
• Following Westfall and Krishen (2001),
• Gatekeeping Procedure (3.1) for gp1 and gs1
• (a) H11 ? (b) H21, S11 ? (c) S21
• level a for each .

13
Generalized Gatekeeping Procedure (3.2)

• (a) H1j (b) H2j,
  S1k One-dimensional
• (c) Sik
  Two-dimensional
• One-dimensional procedure at (a) and (b), and
  two-dimensional procedure at (c).
• Reject H1j (H2j) if it should be rejected at
  (a) ((b))
• Reject S1k if it should be rejected at both (b)
  and (c).
• Reject S2k if all H2js have been rejected, S1k
  has been
• rejected at (b) and (c), and S2k should be
  rejected at (c).

14
Other Procedures for the Three-dimensional Problem

• PAAS Hij two-dimensional at level ap and
• Sik two-dimensional at level as
  (apas a)
• Primary-Secondary Gatekeeping
• (a) Hij two-dimensional
• ?
• (b) Sik two-dimensional
• each at level a
• Procedure (2.2) or others for the
  two-dimensional Problems

15
Numerical Examples(RRejection, NRNo
Rejection)

• 4 Procedures are compared to the regular Hochberg
  Procedure
• P1PAAS with P(2.2) for primary and secondary
  endpoints
• P2P(2.3) with weight wp for primary and ws for
  secondary
• P3P(3.2) with P(2.1) for Step (c)
• P4P(3.2) with P(2.2) for Step (c)

16
Numerical Example 1

• gp1, gs3, ap.0375, as.0125, wp3/6 and
  ws1/6

H11 H12 S11 S12 S13 S21 S22 S23
P-value .005 .018 .006 .014 .070 .012 .018 .100
P1 R R NR NR NR NR NR NR
P2 R R R NR NR NR NR NR
P3 R R R R NR R NR NR
P4 R R R R NR NR NR NR
HB R NR R NR NR NR NR NR
17
Numerical Example 2

• gp2, gs2, ap.0375, as.0125, wp3/8 and
  ws1/8

H11 H12 H21 H22 S11 S12 S21 S22
P-value .001 .014 .012 .045 .006 .033 .009 .130
P1 R R R NR NR NR NR NR
P2 R R R NR R NR NR NR
P3 R R R R R NR R NR
P4 R R R R R R R NR
HB R NR NR NR R NR NR NR
18
Real Data Example

• A randomized trial to compare mitoxantrone 12
  mg/m2 (n64) or 5 mg/m2 (n64) to placebo (n60)
  on patients with worsening relapsing remitting or
  secondary progressive multiple sclerosis (Hartung
  et al. 2002).
• Here, two primary and two secondary endpoints for
  which p-values for both doses are available
  EDSS, ambulation index, with relapses and
  admitted to hospital.

19
Real Data Example (2)

• gp2, gs2, ap.040, as.010, wp4/10 and
  ws1/10

H11 H12 H21 H22 S11 S12 S21 S22
P-value .0194 .0306 .0100 gt.050 .0206 .0024 .7150 .2031
P1 NR NR NR NR NR R NR NR
P2 R NR R NR NR R NR NR
P3 R R R NR R R NR NR
P4 R R R NR R R NR NR
HB NR NR NR NR NR R NR NR
20
Procedures without Overall Strong Control

• Strong control is theoretically appealing.
  However, the procedures are generally
  conservative.
• Strong control in certain key sub-families of
  hypotheses may be sufficient.
• Procedure ()
• (a) H1j H2j two-dimensional at
  level a
• ? ?
• (b) S1k S2k two-dimensional at
  level a
• Procedure () strongly controls the FWE rate
  in the sub-families of individual doses,
  individual endpoints, within the families of
  primary endpoints and secondary endpoints. Also,
  it weakly controls the overall FWE rate.

21
Procedures without Overall Strong Control (2)

• Procedure () does not strongly control the
  overall FWE rate.
• When gp1 and gs1, Procedure () becomes
• H11 ? H21
• ? ?
  each at level a
• S11 ? and ? S21
• (pij is the p-value for Hij and qik is the
  p-value for Sik)
• The probability of rejecting H21S11 when H21S11
  is true is
• Sup Pr (p11 a)(p21 a) or (p11 a)(q11 a)
  H21S11
• Sup Pr(p21a) or (q11a) H21S112a-a2
  

22
Other Procedures without Strong Control

• For example Step-down Closed Procedure
• Step a. Apply a procedure for each dose vs. the
  control separately at level ? for all primary and
  secondary endpoints.
• Step b. Reject H1j (or S1k) if it should be
  rejected at Step a. Reject H2j (or S2k) if both
  H1j (or S1k) H2j (or S2k) should be rejected at
  Step a.
• It controls the FWE rates in the strong sense
  within sub-families of individual doses and
  endpoints, but not the family of the primary
  endpoints nor the family of the secondary
  endpoints. It controls the overall FWE rate in
  the weak sense.

23
Discussion

• After Phase IIb dose finding study, Phase III
  confirmatory trials may still have two doses of
  the active treatment. The setting considered here
  should have very broad application.
• As shown by numerical examples (here and in our
  manuscript), no procedure is uniformly more
  powerful than the others.
• All procedures need only individual p-values --
  no need to calculate the adjusted p-values based
  on different configurations of intersection
  hypotheses.
• One should think hard whether the overall strong
  control is necessary for a particular trial.

24
Major References

• Benjamini Y and Hochberg T. Multiple hypotheses
  testing with weights. Scandinavian Journal of
  Statistics 1997 24 407-418.
• CPMP Points to Consider on Multiplicity Issues in
  Clinical Trials. 2002.
• DAgostino RB. Controlling alpha in a clinical
  trial the case for secondary endpoints.
  Statistics in Medicine 2000 19, 763-766.
• Dmitrienko A, Offen WW and Westfall PH.
  Getekeeping strategies for clinical trials that
  do not require all primary effects to be
  significant. Statistics in Medicine 2003 22
  2387-2400.
• Hartung HP, Gonsette R, Konig N, Kwiecinski H,
  Guseo A, Morrissey SP, Krapf H, Zwingers T and
  the Mitoxantrone in Multiple Sclerosis Study
  Group. Mitoxantrone in progressive multiple
  sclerosis a placebo-controlled, double-blind,
  randomized, multicentre trial. Lancet 2002 360,
  2018-2025.
• Hochberg Y. A sharper Bonferroni procedure for
  multiple tests of significance. Biometrika 1988
  75, 800-802.
• Macus R, Peritz E and Gabriel KR. On closed
  testing procedures with special reference to
  ordered analysis of variance. Biometrika 1976
  64, 655-660.
• Moye LA. Alpha calculus in clinical trials
  considerations and commentary for the new
  millennium. Statistics in Medicine 2000 19,
  767-779.
• Neer RM, Arnaud CD, Zanchetta JR, Prince R, Gaich
  GA, Reginster JY, Hodsman AB, Eriksen EF,
  Ish-Shalom S, Genant HK, Wang O and Mitlak BH.
  Effect of parathyroid hormone (1-34) on fractures
  and bone mineral density in postmenopausal women
  with osteoporosis. The New England Journal of
  Medicine 2001 344, 19, 1434-1441.
• Quan H, Luo E, Capizzi T. Xun Chen, Lynn Wei, and
  Bruce Binkowitz, Multiplicity adjustment for
  multiple endpoints in clinical trials with
  multiple doses of an active control. JSM
  Proceedings 2003.
• Samuel-Cahn, E. Is the Simes improved Bonferroni
  procedure conservative? Biometrika 1996 83,
  928-933.
• Simes RJ. An improved Bonferroni procedure for
  multiple tests of significance. Biometrika, 1986
  63 655-660.
• Treanor JJ, Hayden FG, Vrooman PS, Barbarash R,
  Bettis R, Riff D, Singh S, Kinnersley N, Ward P
  and Mills RG. Efficacy and safety of the oral
  neuraminidase inhibitor oseltamivir in treating
  acute influenza. JAMA 2000 283, 8, 1016-1024.
• Westfall PH and Krishen A. Optimally weighted,
  fixed sequence and gatekeeper multiple testing
  procedures. Journal of Statistical Planning and
  Inference 2001 99, 25-40.