WHY%20BAYES?%20INNOVATIONS%20IN%20CLINICAL%20TRIAL%20DESIGN%20

1
WHY BAYES?INNOVATIONS IN CLINICAL TRIAL DESIGN
ANALYSIS

• Donald A. Berry
• dberry_at_mdanderson.org

2
Conclusion These data add to the growing
evidence that supports the regular use of aspirin
and other NSAIDs as effective chemopreventive
agents for breast cancer.
3
Results Ever use of aspirin or other NSAIDs
was reported in 301 cases (20.9) and 345
controls (24.3) (odds ratio 0.80, 95 CI
0.66-0.97).
4
Bayesian analysis?

• Naïve Bayesian analysis of Results is wrong
• Gives Bayesians a bad name
• Any naïve frequentist analysis is also wrong

5
What is Bayesian analysis?

• Bayes' theorem
• ?'( q? X ) ? ?(q) f( X q )
• Assess prior ? (subjective, include available
  evidence)
• Construct model f for data

6
Implication The Likelihood Principle

• Where X is observed data, the likelihood
  function
• LX(?) f( X ? )
• contains all the information in an experiment
  relevant for inferences about ?

7

• Short version of LP Take data at face value
• Data
• Among cases 301/1442
• Among controls 345/1420
• But Data is deceptive
• These are not the full data

8
The data

• Methods
• Population-based case-control study of breast
  cancer
• Study design published previously
• Aspirin/NSAIDs? (2.25-hr ?naire)
• Includes superficial data
• Among cases 301/1442
• Among controls 345/1420
• Other studies ( fact published!!)

9
Silent multiplicities

• Are the most difficult problems in statistical
  inference
• Can render what we do irrelevant
• and wrong!

?
10
Which city is furthest north?

• Portland, OR
• Portland, ME
• Milan, Italy
• Vladivostok, Russia

11
Beating a dead horse . . .

• Piattelli-Palmarini (inevitable illusions) asks
  I have just tossed a coin 7 times. Which did I
  get?
• 1 THHTHTT
• 2 TTTTTTT
• Most people say 1. But the probabilities are
  totally even
• Most people are right hes totally wrong!
• Data He presented us with 1 2!

• Piattelli-Palmarini (inevitable illusions) asks
  I have just tossed a coin 7 times. Which did I
  get?
• 1 THHTHTT
• 2 TTTTTTT
• Most people say 1. But the probabilities are
  totally even
• Most people are right hes totally wrong!
• Data He presented us with 1 2!

12
THHTHTT or TTTTTTT?

• LR Bayes factor of 1 over 2
• P(Wrote 12 Got 1)
• P(Wrote 12 Got 2)

• LR gt 1 ? P(Got 1 Wrote 12) gt 1/2
• Eg LR (1/2)/(1/42) 21 ?
• P(Got 1 Wrote 12) 21/22 95
• Probs totally even if a coin was used to
  generate the alternative sequence

13
Marker/dose interaction Marker negative
Marker positive
14
Proportional hazards model

• Variable Comp RelRisk P
• PosNodes 10/1 2.7 lt0.001
• MenoStatus pre/post 1.5 0.05
• TumorSize T2/T1 2.6 lt0.001
• Dose NS
• Marker 50/0 4.0 lt0.001
• MarkerxDose lt0.001

This analysis is wrong!
15
Data at face value?

• How identified?
• Why am I showing you these results?
• What am I not showing you?
• What related studies show?

16
Solutions?

• Short answer I dont know!
• A solution
• Supervise experiment yourself
• Become an expert on substance
• Partial solution
• Supervise supervisors
• Learn as much substance as you can
• Danger You risk projecting yourself as uniquely
  scientific

17
A consequence

• Statisticians come to believe
• NOTHING!!

18
OUTLINE

• Silent multiplicities
• Bayes and predictive probabilities
• Bayes as a frequentist tool
• Adaptive designs
• Adaptive randomization
• Investigating many phase II drugs
• Seamless Phase II/III trial
• Adaptive dose-response
• Extraim analysis
• Trial design as decision analysis

19
Bayes in pharma and FDA
20
http//www.cfsan.fda.gov/frf/bayesdl.html
http//www.prous.com/bayesian2004/
21
(No Transcript)
22
(No Transcript)
23
BAYES AND PREDICTIVE PROBABILITY

• Critical component of experimental design
• In monitoring trials

24
Example calculation

• Data 13 A's and 4 B's
• Likelihood ? p13 (1p)4

25
Posterior density of p for uniform prior
Beta(14,5)
26
Laplaces rule of succession
P(A wins next pair data) EP(A wins next pair
data, p) E(p data) mean of Beta(14, 5)
14/19
27
Updating w/next observation
28
Suppose 17 more observations

• P(A wins x of 17 data)
• EP(A wins x data, p)
• E px(1p)17x data, p

( )
17 x
?
29
Best fitting binomial vs. predictive probabilities
Binomial, p14/19
Predictive, p beta(14,5)
30
Comparison of predictive with posterior
31
Example Baxters DCLHb predictive probabilities

• Diaspirin Cross-Linked Hemoglobin
• Blood substitute emergency trauma
• Randomized controlled trial (1996)
• Treatment DCLHb
• Control saline
• N 850 ( 2x425)
• Endpoint death

32

• Waiver of informed consent
• Data Monitoring Committee
• First DMC meeting
• DCLHb Saline
• Dead 21 (43) 8 (20)
• Alive 28 33
• Total 49 41
• P-value? No formal interim analysis

33
Predictive probability of future results (after n
850)

• Probability of significant survival benefit for
  DCLHb after 850 patients 0.00045
• DMC paused trial Covariates?
• No imbalance
• DMC stopped trial

34
OUTLINE

• Silent multiplicities
• Bayes and predictive probabilities
• Bayes as a frequentist tool
• Adaptive designs
• Adaptive randomization
• Investigating many phase II drugs
• Seamless Phase II/III trial
• Adaptive dose-response
• Extraim analysis
• Trial design as decision analysis

35
BAYES AS A FREQUENTIST TOOL

• Design a Bayesian trial
• Check operating characteristics
• Adjust design to get ? 0.05
• ? frequentist design
• Thats fine!
• We have 50 such trials at MDACC

36
OUTLINE

• Silent multiplicities
• Bayes and predictive probabilities
• Bayes as a frequentist tool
• Adaptive designs
• Adaptive randomization
• Investigating many phase II drugs
• Seamless Phase II/III trial
• Adaptive dose-response
• Extraim analysis
• Trial design as decision analysis

37
ADAPTIVE DESIGN

• Look at accumulating data without blushing
• Update probabilities
• Find predictive probabilities
• Modify future course of trial
• Give details in protocol
• Simulate to find operating characteristics

38
OUTLINE

• Silent multiplicities
• Bayes and predictive probabilities
• Bayes as a frequentist tool
• Adaptive designs
• Adaptive randomization
• Investigating many phase II drugs
• Seamless Phase II/III trial
• Adaptive dose-response
• Extraim analysis
• Trial design as decision analysis

39
Giles, et al JCO (2003)

• Troxacitabine (T) in acute myeloid leukemia (AML)
  when combined with cytarabine (A) or idarubicin
  (I)
• Adaptive randomization to IA vs TA vs TI
• Max n 75
• End point CR (time to CR lt 50 days)

40
Randomization

• Adaptive
• Assign 1/3 to IA (standard) throughout (unless
  only 2 arms)
• Adaptive to TA and TI based on current results
• Final results ?

41
(No Transcript)
42
Drop TI
Compare n 75
43
Summary of results

• CR rates
• IA 10/18 56
• TA 3/11 27
• TI 0/5 0
• Criticisms . . .

44
OUTLINE

• Silent multiplicities
• Bayes and predictive probabilities
• Bayes as a frequentist tool
• Adaptive designs
• Adaptive randomization
• Investigating many phase II drugs
• Seamless Phase II/III trial
• Adaptive dose-response
• Extraim analysis
• Trial design as decision analysis

45
Example Adaptive allocation of therapies

• Design for phase II Many drugs
• Advanced breast cancer (MDA) endpoint is tumor
  response
• Goals
• Treat effectively
• Learn quickly

46
Comparison Standard designs

• One drug (or dose) at a time no drug/dose
  comparisons
• Typical comparison by null hypothesis response
  rate 20
• Progress is slow!

47
Standard designs

• One stage, 14 patients
• If 0 responses then stop
• If 1 response then phase III
• Two stages, first stage 20 patients
• If 4 or 9 responses then stop
• Else second set of 20 patients

48
An adaptive allocation

• When assigning next patient, find r P(rate
  20data) for each drug
• Or, r P(drug is bestdata)
• Assign drugs in proportion to r
• Add drugs as become available
• Drop drugs that have small r
• Drugs with large r ? phase III

49
Suppose 10 drugs, 200 patients

• 9 drugs have mix of response rates 20 40, 1
  (nugget) has 60
• Standard 2-stage design finds nugget with
  probability lt 70 (After 110 patients on average)
• Adaptive design finds nugget with probability gt
  99 (After about 50 patients on average)
• Adaptive also better at finding 40

50
Suppose 100 drugs, 2000 patients

• 99 drugs have mix of response rates 20 40, 1
  (nugget) has 60
• Standard 2-stage design finds nugget with
  probability lt 70 (After 1100 patients on
  average)
• Adaptive design finds nugget with probability gt
  99 (After about 500 patients on average)
• Adaptive also better at finding 40

51
Consequences

• Recall goals
• (1) Treat effectively
• (2) Learn quickly
• Attractive to patients, in and out of the trial
• Better drugs identified faster move through
  faster

52
OUTLINE

• Silent multiplicities
• Bayes and predictive probabilities
• Bayes as a frequentist tool
• Adaptive designs
• Adaptive randomization
• Investigating many phase II drugs
• Seamless Phase II/III trial
• Adaptive dose-response
• Extraim analysis
• Trial design as decision analysis

53
Example Seamless phase II/III

• Drug vs placebo, randomized
• Local control (or biomarker, etc) early endpoint
  related to survival?
• May depend on treatment

Inoue et al (2002 Biometrics)
54
Conventional drug development
Survivaladvantage
Market
Local control
No survivaladvantage
Not
No local control
Stop
Phase III
Phase II
gt 2 yrs
6 mos
9-12 mos
Seamless phase II/III
lt 2 yrs (usually)
55
Seamless phases

• Phase II Two centers 10 pts/mo. drug vs
  placebo. If predictive probabilities look good,
  expand to
• Phase III Many centers 40 pts/mo.(Initial
  centers accrue during set-up)
• Max sample size 900
• Single trial survival data from both phases
  combined in final analysis

56
Early stopping

• Use predictive probs of stat. signif.
• Frequent analyses (total of 18) using predictive
  probabilities
• To switch to Phase III
• To stop accrual
• For futility
• For efficacy
• To submit NDA

57
Comparisons

• Conventional Phase III designs Conv4 Conv18,
  max N 900
• (same power as adaptive design)

58
Expected N under H0
59
Expected N under H1
60
Benefits

• Duration of drug development is greatly shortened
  under adaptive design
• Fewer patients in trial
• No hiatus for setting up phase III
• Use all patients to assess phase III endpoint and
  relationship between local control and survival

61
Possibility of large N

• N seldom near 900
• When it is, its necessary!
• This possibility gives Bayesian design its edge
• Other reason for edge is modeling local
  control/survival

62
OUTLINE

• Silent multiplicities
• Bayes and predictive probabilities
• Bayes as a frequentist tool
• Adaptive designs
• Adaptive randomization
• Investigating many phase II drugs
• Seamless Phase II/III trial
• Adaptive dose-response
• Extraim analysis
• Trial design as decision analysis

63

Berry, et al. Case Studies in Bayesian
Statistics 2001
64
Example Stroke and adaptive dose-response

• Adaptive doses in Phase II setting learn
  efficiently and rapidly about dose-response
  relationship
• Pfizer trial of a neutrofil inhibitory factor
  results recently announced
• Endpoint stroke scale at week 13
• Early endpoints weekly stroke scale

65
Standard Parallel Group Design
Equal sample sizes at each of k doses.
Doses
66
True dose-response curve (unknown)
Response
Doses
67
Observe responses (with error) at chosen doses
Response
Doses
68
Dose at which 95 max effect
Response
True ED95
Doses
69
Uncertainty about ED95
Response
True ED95
?
Dose
70
Uncertainty about ED95
Response
?
Dose
71
Solution Increase number of doses
ED95
Response
Doses
72
But, enormous sample size, and . . . wasted dose
assignmentsalways!
ED95
Response
Doses
73
Our adaptive approach

• Observe data continuously
• Select next dose to maximize information about
  ED95, given available evidence
• Stop dose-ranging trial when know ED95 response
  at ED95 sufficiently well

74
Our approach (contd)

• Info accrues gradually about each patient
  prediction using longitudinal model

Longitudinal Model Copenhagen Stroke Database
Difference from baseline in SSS week 12
-40
-30
-20
-10
0
10
20
30
40
50
Difference from baseline in SSS week 3
75
Our approach (contd)

• Model dose-response (borrow strength from
  neighboring doses)
• Many doses (logistical issues)

76
Possible decisions each day

• Stop trial and drugs development
• Stop and set up confirmatory trial
• Continue dose-finding (what dose?)
• Size of confirmatory trial based on info from
  dose-ranging phase
• Choices by decision analysis (Human safeguard
  DSMB)

77
Dose-response trial

• Learn efficiently and rapidly about
  dose-response if go to Phase III
• Assign dose to maximize info about dose-response
  parameters given current info
• Use predictive probabilities, based on early
  endpoints
• Doses in continuum, or preset grid

78
Dose-response trial (contd)

• Learn about SD on-line
• Halt dose-ranging when know dose sufficiently
  well
• Seamless switch from dose-ranging to confirmatory
  trial2 trials in 1!

79
Advantages over standard design

• Fewer patients (generally) faster more
  effective learning
• Better at finding ED95
• Tends to treat patients in trial more effectively
• Drops duds early

actual trial!
80
Dose-assignment simulation

• Assumes particular dose-response curve
• Assumes SD 12
• Shows weekly results, several patients at a time
  (green circles)

81
Prior
82
(No Transcript)
83
(No Transcript)
84
(No Transcript)
85
(No Transcript)
86
(No Transcript)
87
(No Transcript)
88
(No Transcript)
89
(No Transcript)
90
(No Transcript)
91
(No Transcript)
92
(No Transcript)
93
(No Transcript)
94
(No Transcript)
95
(No Transcript)
96
(No Transcript)
97
(No Transcript)
98
(No Transcript)
99
(No Transcript)
100
(No Transcript)
101
(No Transcript)
102
(No Transcript)
103
(No Transcript)
104
(No Transcript)
105
(No Transcript)
106
(No Transcript)
107
(No Transcript)
108
(No Transcript)
109
(No Transcript)
110
(No Transcript)
111
(No Transcript)
112
(No Transcript)
113
(No Transcript)
114
(No Transcript)
115
(No Transcript)
116
(No Transcript)
117
(No Transcript)
118
(No Transcript)
119
(No Transcript)
120
(No Transcript)
121
(No Transcript)
122
(No Transcript)
123
(No Transcript)
124
(No Transcript)
125
(No Transcript)
126
(No Transcript)
127
(No Transcript)
128
(No Transcript)
129
(No Transcript)
130
(No Transcript)
131
(No Transcript)
132
(No Transcript)
133
Estimated ED95
Confirmatory
134
0.0 0.5
1.0 1.5
DOSE
DOSE
135
(No Transcript)
136
(No Transcript)
137
(no dose effect)
138
(No Transcript)
139
Consequences of Using Bayesian Adaptive Approach

• Fundamental change in the way we do medical
  research
• More rapid progress
• Well get the dose right!
• Better treatment of patients
• . . . at less cost

140
Reactions

• FDA Positive. Makes coming to work worthwhile.
  In five years all trials may be seamless.
• Pfizer management Enthusiastic
• Other companies Cautious

141
OUTLINE

• Silent multiplicities
• Bayes and predictive probabilities
• Bayes as a frequentist tool
• Adaptive designs
• Adaptive randomization
• Investigating many phase II drugs
• Seamless Phase II/III trial
• Adaptive dose-response
• Extraim analysis
• Trial design as decision analysis

142
Example Extraim analysis

• Endpoint CR (detect 0.42 vs 0.32)
• 80 power N 800
• Two extraim analyses, one at 800
• Another after up to 300 added pts
• Maximum n 1400 (only rarely)
• Accrual 70/month
• Delay in assessing response

143

• After 800 patients, have response info on 450
• Find predictive probability of stat significance
  when full info on 800
• Also when full info on 1400
• Continue if . . .
• Stop if . . .
• If continue, n via predictive power
• Repeat at second extraim analysis

144
vs 0.80
145
OUTLINE

• Silent multiplicities
• Bayes and predictive probabilities
• Bayes as a frequentist tool
• Adaptive designs
• Adaptive randomization
• Investigating many phase II drugs
• Seamless Phase II/III trial
• Adaptive dose-response
• Extraim analysis
• Trial design as decision analysis

146
Decision-analytic approach

• For each trial design
• List possible results
• Calculate their predictive probabilities
• Evaluate their utilities
• Average utilities by probabilities to give
  utility of trial with that design
• Compare utilities of various designs
• Choose design with high utility

147
Choosing sample size

• Special case of above
• One utility Effective overall treatment of
  patients, both those
• after the trial
• in the trial
• Example, dichotomous endpointMaximize expected
  number of successes over all patients

Cheng et al (2003 Biometrika)
148
Compare Joffe/Weeks JNCI Dec 18, 2002

• Many respondents viewed the main societal
  purpose of clinical trials as benefiting the
  participants rather than as creating
  generalizable knowledge to advance future
  therapy. This view, which was more prevalent
  among specialists such as pediatric oncologists
  that enrolled greater proportions of patients in
  trials, conflicts with established principles of
  research ethics.

149
Maximize effective treatment overall

• What is overall?
• All patients who will be treated with therapies
  assessed in trial
• Call it N, patient horizon
• Enough to know mean of N
• Enough to know magnitude of N100? 1000?
  1,000,000?

150

• Goal maximize expected number of successes in N
• Either one- or two-armed trial
• Suppose n 1000 is right for N 1,000,000
• Then for other Ns use n

151
Optimal allocations in a two-armed trial
152
Knowledge about success rate r
153
OUTLINE

• Silent multiplicities
• Bayes and predictive probabilities
• Bayes as a frequentist tool
• Adaptive designs
• Adaptive randomization
• Investigating many phase II drugs
• Seamless Phase II/III trial
• Adaptive dose-response
• Extraim analysis
• Trial design as decision analysis