Optimal MultiStage Phase II Design With Sequential Testing of Hypotheses Within Each Stage

1
Optimal Multi-Stage Phase II Design With
Sequential Testing of Hypotheses Within Each
Stage

• S. Poulopoulou1, 2, U. Dafni1, M. Kateri2, CT.
  Yiannoutsos3, D. Karlis4
• 1University of Athens, Athens, Greece
• 2University of Piraeus, Piraeus, Greece
• 3Indiana University, Indiana, USA
• 4University of Economics and Business, Athens,
  Greece
• Email spoulopo_at_unipi.gr

2
Phase II Trials

• The objective of a Phase II trial is to decide if
  a particular therapeutic regimen is effective
  enough to warrant further study (Phase III).
• The hypothesis to be tested is
• ?O p ? pO versus ?A p ? pA,
  pAgtpO
• The conditions to be satisfied
• Preject HO HO true Ptype I error ? a
  (false positive)
• Preject HA HA true Ptype II error ? ß
  (false negative)

3
Simons Two Stage Design (1989)

• A two stage design, testing two complementary
  events is proposed.
• The following hypotheses are tested
• the null hypothesis ?O p ? pO in the first stage
  and if rejected
• the alternative hypothesis ?A p ? pA, (pAgtpO) in
  the second and last stage
• Conclusion on the alternative hypothesis requires
  additional patients (going to the next stage)

4
Multi-stage Designs

• In a Multi-stage Design we consider
• i1,..,k stages, N total patients
• n1,,nk, where n1nkN
• Si the sum of responses observed up to i-th
  stage
• ai acceptance point in i-th stage
• ri rejection point in i-th stage
• Decision rule
• If Si ? ai , stop and reject ?A p ? pA
• If Si ? ri ,stop and reject ?O p ? pO
• If ai ? Si ? ri , continue to the stage i1
• In the last stage rk ak 1
• Average Sample Number
• ASN(p)

5
Multi-stage Designs (cont.)

• According to Schultz et al (1973)
• The probability that Sim is given by
• Ci(m,p)
• where ?max(ai - 11, m-ni) and Fmin(ri - 1,m).
• POi P(accept HO at i-th stage)
• PAi P( reject HO at i-th stage)
• Fleming (1982) proposed a method of determining
  appropriate acceptance and rejection points based
  on the asymptotic normal distribution and the
  general formulation by Schultz.

6
Multi-stage Designs Points of Interest

• The hypothesis of a multi-stage design consists
  of two non-complementary events.
• The conclusion that a particular therapy is
  effective enough to warrant further study is
  based only on rejecting the hypothesis p?pO (and
  not on accepting the hypothesis p?pA)
• (Schultz et al (1973), Fleming (1982), Simon
  (1989) and Chen (1997) etc)
• Ambiguity pointed out by Storer (1992)
• the evaluation of type I and II errors and
• the choice of the appropriate practical decision
  at the end of the study.

7
Three Outcome Design Storer (1992), Sargent et
al(2001)

• To overcome these problems a three outcome
  design was proposed
• Decision rule
• If Si ? ai ,stop sampling and reject ?A p ? pA
• If Si ? ri ,stop sampling and reject ?O p ? pO
• If ai ? Si ? ri ,continue to stage i1
• In the last stage k
• If Sk ? ak , stop sampling and reject ?A p ? pA
• If Sk ? rk ,stop sampling and reject ?O p ? pO
• If ak ? Sk ? rk , reject neither ?O nor ?A and
  decide that the trial has been inconclusive.
• The additional constraint in the three outcome
  design increases the sample size.
• Probability calculations are based on the
  asymptotic normal distribution.

8
Proposal Sequential Testing of Two Hypotheses
within the Same Stage

• In order to address the testing of
  non-complementary events at the same stage, we
  propose a class of designs in which we test two
  hypotheses sequentially at each stage (i1,,k)
• First hypothesis
• ?O1 p ? pO versus ?A1 p gt pO
• If we reject ?O1, then we test the second
  hypothesis
• ?O2 p ? pA versus ?A2 p lt pA

9
Characteristics of the Proposed Optimal
Sequential Testing (OST) Procedure

• Two distinct hypotheses tested sequentially
  within the same stage
• Use of Exact Binomial distribution instead of
  asymptotic Normal Distribution
• Optimal design minimizing ASN with respect to
  conditions on a and ß levels
• Confidence interval estimation for p if
  inconclusive at the last stage
• Sample size chosen to satisfy predetermined
  precision for the confidence interval of p

10
Decision Rule

• For each stage (except the last), i1,,k -1
• If Si ? ai, stop sampling and conclude the
  therapy is harmful
• If Si gt ai, proceed to test the second hypothesis
  
• ?o2 p ? pA
• If Si ? ri, stop sampling and conclude the
  therapy is effective
• If Si lt ri, continue to stage i1

11
Decision Rule (cont.)

• For the last stage k, stop sampling and
• If Sk ? ak, conclude the therapy is harmful
• If Sk gt ak, proceed to test the hypothesis ?02 p
  ? pA
• If Sk ? rk, conclude the therapy is effective
• If Sk lt rk, conclude the therapy is not harmful
  but also not as effective as originally targeted
  (polt p ltpA) and
• Estimate Confidence interval for p

12
First Hypothesis HO1

• For testing the hypothesis
• ?O1 p ? po versus ?A1 p gt pO
• We have two complementary events
• Probability of rejecting ?A1
• A1(p)
  
• Probability of rejecting ?O1
• R1(p) 1-A1(p)1-
• Error Probabilities
• a1 sup P(reject HO1 p pO) R1(pO)
• ß1 P(reject HA1 p gt pO) A1(pA) i.e., at ppA

13
Second Hypothesis HO2

• For testing the hypothesis
• ?O2 p ? pA versus ?A2 p lt pA
• We have two complementary events
• Probability of rejecting ?A2

14
Second Hypothesis Ho2 (cont.)

• Probability of rejecting ?O2 R2(p) R1(p) -
  A2(p)
• Error Probabilities
• a2 sup P(reject HO2 p pA) R2(pA) R1(pA)
  - A2(pA)
• ß2 P(reject HA2 p lt pA) A2(po) , i.e., at
  ppo
• Average Sample Number for the total procedure
• ASN(p)

15
Optimal Sequential Testing procedure

• Following Simon, we define an Optimal Sequential
  Testing procedure that satisfies the constraint
  parameters for the errors and minimizes the
  Average Sample Number under Ho (ASN(po)).
• For finding the Optimal Sequential Testing design
  we use the Simulated Annealing Method.

16
Algorithm Outline Simulation Annealing Method

• For given n1,, nk, pO and pA
• Step 1 - initialization For prob0 and
  temperature1, choose a start value for ai and ri
  that satisfy the constraints and calculate the
  ASN(pO).
• Step 2 For i1 to 15000
• Step 2.1 Generate random numbers xi and yi from
  discrete uniform distribution U-1,1.
• Step 2.2 Calculate new critical points
  ai(new) ai xi and ri(new) ri yi
• Step 2.3 If new critical points satisfy the
  constrains find the difference
  diffASN(po)new-ASN(po).
• If difflt0 then ai ai(new) and riri(new)
• If diffgt0 then generate random number u from
  U(0,1) and calculate probexp(-diff/temperature)
• If ultprob then ai ai(new) and riri(new)

17
Design Parameters for Two- and Three-stage
Optimal Sequential Testing (OST) Procedure

• For pO0.05 , pA0.20 with a2lt0.05 and ß1, ß2lt0.20

18
Comparison of Flemings Design with the Optimal
Sequential Testing Procedure

• For pO0.05 , pA0.20 with a2lt0.05 and ß1,
  ß2lt0.20

19
Comparison of Storers Three Outcome Design with
the Optimal Sequential Testing Procedure
20
Required Sample Size for a 10 Precision for a
95 Confidence Interval for p

• If the conclusion at the last stage is that (polt
  p ltpA)
• the therapy is not harmful
• but also not as effective as originally targeted
  
• i.e., inconclusive region

21
Summary of Results for Proposed Design

• Comparable ai and ri to Flemings procedure
• Optimal ASN(pO), comparable or lower than
  Flemings procedure
• Ability to control the error rates when reaching
  conclusions
• reject a therapy as harmful when it has response
  probability p at most p? (type II error ß1)
• accept a therapy as effective when it has
  response probability p at least pA, (type II
  error ß2)
• reject both ??1 and ??2, i.e., p?ltpltpA (type I
  error a2)
• Confidence interval estimation for p within 10
  precision feasible with proposed sample sizes

22
References

• Schultz, JR., Nichol, FR., Elfring, GL. and Weed,
  SD. (1973). Multiple stage procedures for drug
  screening. Biometrics 29, 293-300.
• Fleming, TR. (1982). One-Sample Multiple Testing
  Procedure for Phase II Clinical Trials.
  Biometrics 38, 143-151.
• Storer, BE. (1992). A class of phase II designs
  with three possible outcomes. Biometrics 48,
  5560.
• Simon, R. (1989). Optimal two-stage designs for
  phase II clinical trials. Controlled Clinical
  Trials 10, 1-10.
• Chen, TT. (1997). Optimal three-stage designs for
  phase II cancer clinical trials. Statistics in
  Medicine 16, 2701-2711.
• Sargent, DJ., Chan, V. and Goldberg, MD. (2001) A
  three-outcome desugn for phase II clinical
  trials. Controlled Clinical Trials 22, 117-125.