New Approaches to Clinical Trial Design Development of New Drugs

1
New Approaches to Clinical Trial Design
Development of New Drugs Predictive Biomarkers

• Richard Simon, D.Sc.
• Chief, Biometric Research Branch
• National Cancer Institute

2
http//linus.nci.nih.gov

• Powerpoint presentations
• Reprints
• BRB-ArrayTools software
• BRB-ArrayTools Data Archive
• Sample Size Planning
• Targeted Clinical Trials
• Phase II Trials
• Developing Predictive Classifiers

3
Biomarkers

• Surrogate endpoints
• Of treatment effect
• Of patient benefit
• Prognostic marker
• Pre-treatment measurement correlated with
  long-term outcome
• Predictive classifier
• A measurement made before treatment to predict
  whether a particular treatment is likely to be
  beneficial

4
Surrogate Endpoints

• It is extremely difficult to properly validate a
  biomarker as a surrogate for clinical benefit.
• It requires a series of randomized trials with
  both the candidate biomarker and clinical outcome
  measured
• Biomarkers can be useful in phase I/II studies as
  indicators of treatment effect and need not be
  validated as surrogates for clinical benefit.

5
Predictive Classifiers

• Most cancer treatments benefit only a minority of
  patients to whom they are administered
• Particularly true for molecularly targeted drugs
• Being able to predict which patients are likely
  to benefit would
• save patients from unnecessary toxicity, and
  enhance their chance of receiving a drug that
  helps them
• Help control medical costs
• Improve the efficiency of clinical drug
  development

6
If new refrigerators hurt 7 of customers and
failed to work for another one-third of them,
customers would expect refunds.BJ Evans, DA
Flockhart, EM Meslin Nature Med 101289, 2004
7
Oncology Needs Predictive Markers not Prognostic
Factors
8
Pusztai et al. The Oncologist 8252-8, 2003

• 939 articles on prognostic markers or
  prognostic factors in breast cancer in past 20
  years
• ASCO guidelines only recommend routine testing
  for ER, PR and HER-2 in breast cancer
• With the exception of ER or progesterone
  receptor expression and HER-2 gene amplification,
  there are no clinically useful molecular
  predictors of response to any form of anticancer
  therapy.

9
Oncology Needs Predictive Markers not Prognostic
Factors

• Many prognostic factor studies use a convenience
  sample of patients for whom tissue is available.
  Generally the patients are too heterogeneous to
  support therapeutically relevant conclusions

10

• Targeted clinical trials can be much more
  efficient than untargeted clinical trials, if we
  know who to target

11

• In new drug development, the role of a predictive
  classifier is to select a target population for
  treatment
• The focus should be on evaluating the new drug in
  a population defined by a predictive classifier,
  not on validating the classifier

12
Developmental Strategy (I)

• Develop a diagnostic classifier that identifies
  the patients likely to benefit from the new drug
• Develop a reproducible assay for the classifier
• Use the diagnostic to restrict eligibility to a
  prospectively planned evaluation of the new drug
• Demonstrate that the new drug is effective in the
  prospectively defined set of patients determined
  by the diagnostic

13
Develop Predictor of Response to New Drug
Using phase II data, develop predictor of
response to new drug
Patient Predicted Responsive
Patient Predicted Non-Responsive
Off Study
New Drug
Control
14
Applicability of Design I

• Primarily for settings where there is a
  substantial biological basis for restricting
  development to classifier positive patients
• eg HER2 expression with Herceptin
• With substantial biological basis for the
  classifier, it may be ethically unacceptable to
  expose classifier negative patients to the new
  drug

15
Evaluating the Efficiency of Strategy (I)

• Simon R and Maitnourim A. Evaluating the
  efficiency of targeted designs for randomized
  clinical trials. Clinical Cancer Research
  106759-63, 2004.
• Maitnourim A and Simon R. On the efficiency of
  targeted clinical trials. Statistics in Medicine
  24329-339, 2005.

16

• Efficiency relative to trial of unselected
  patients depends on proportion of patients test
  positive, and effectiveness of drug (compared to
  control) for test negative patients
• When less than half of patients are test positive
  and the drug has little or no benefit for test
  negative patients, the targeted design requires
  dramatically fewer randomized patients

17
No treatment Benefit for Assay - Patientsnstd /
ntargeted
Proportion Assay Positive Randomized Screened
0.75 1.78 1.33
0.5 4 2
0.25 16 4
18
Treatment Benefit for Assay Pts Half that of
Assay Pts nstd / ntargeted
Proportion Assay Positive Randomized Screened
0.75 1.31 0.98
0.5 1.78 0.89
0.25 2.56 0.64
19
Trastuzumab

• Metastatic breast cancer
• 234 randomized patients per arm
• 90 power for 13.5 improvement in 1-year
  survival over 67 baseline at 2-sided .05 level
• If benefit were limited to the 25 assay
  patients, overall improvement in survival would
  have been 3.375
• 4025 patients/arm would have been required
• If assay patients benefited half as much, 627
  patients per arm would have been required

20
Interactive Software for Evaluating a Targeted
Design

• http//linus.nci.nih.gov

21
(No Transcript)
22
(No Transcript)
23
(No Transcript)
24
Developmental Strategy (II)
25
Developmental Strategy (II)

• Do not use the diagnostic to restrict
  eligibility, but to structure a prospective
  analysis plan.
• Compare the new drug to the control overall for
  all patients ignoring the classifier.
• If poverall? 0.04 claim effectiveness for the
  eligible population as a whole
• Otherwise perform a single subset analysis
  evaluating the new drug in the classifier
  patients
• If psubset? 0.01 claim effectiveness for the
  classifier patients.

26
Developmental Strategy (IIb)

• Do not use the diagnostic to restrict
  eligibility, but to structure a prospective
  analysis plan.
• Compare the new drug to the control for
  classifier positive patients
• If pgt0.05 make no claim of effectiveness
• If p? 0.05 claim effectiveness for the
  classifier positive patients and
• Continue accrual of classifier negative patients
  and eventually test treatment effect at 0.05
  level

27
Key Features of Design (II)

• The purpose of the RCT is to evaluate treatment T
  vs C overall and for the pre-defined subset not
  to re-evaluate the components of the classifier,
  or to modify or refine the classifier

28
The Roadmap

1. Develop a completely specified genomic classifier
   of the patients likely to benefit from a new drug
2. Establish reproducibility of measurement of the
   classifier
3. Use the completely specified classifier to design
   and analyze a new clinical trial to evaluate
   effectiveness of the new treatment with a
   pre-defined analysis plan.

29
Guiding Principle

• The data used to develop the classifier must be
  distinct from the data used to test hypotheses
  about treatment effect in subsets determined by
  the classifier
• Developmental studies are exploratory
• Studies on which treatment effectiveness claims
  are to be based should be definitive studies that
  test a treatment hypothesis in a patient
  population completely pre-specified by the
  classifier

30
Use of Archived Samples

• From a non-targeted negative clinical trial to
  develop a binary classifier of a subset thought
  to benefit from treatment
• Test that subset hypothesis in a separate
  clinical trial
• Prospective targeted type (I) trial
• Using archived specimens from a second previously
  conducted clinical trial

31
Development of Genomic Classifiers

• Single gene or protein based on knowledge of
  therapeutic target
• Single gene or protein culled from set of
  candidate genes identified based on imperfect
  knowledge of therapeutic target
• Empirically determined based on correlating gene
  expression to patient outcome after treatment

32
Use of DNA Microarray Expression Profiling

• For settings where you dont know how to identify
  the patients likely to be responsive to the new
  treatment based on its mechanism of action
• Only pre-treatment specimens are needed

33
Development of Genomic Classifiers

• During phase II development or
• After failed phase III trial using archived
  specimens.
• Adaptively during early portion of phase III
  trial.

34
Development of Empirical Gene Expression Based
Classifier

• 20-30 phase II responders are needed to compare
  to non-responders in order to develop signature
  for predicting response
• Dobbin KK, Simon RM. Sample size planning for
  developing classifiers using high dimensional DNA
  microarray data, Biostatistics 8101-117,2007

35
Adaptive Signature Design An adaptive design for
generating and prospectively testing a gene
expression signature for sensitive patients

• Boris Freidlin and Richard Simon
• Clinical Cancer Research 117872-8, 2005

36
Adaptive Signature DesignEnd of Trial Analysis

• Compare E to C for all patients at significance
  level 0.04
• If overall H0 is rejected, then claim
  effectiveness of E for eligible patients
• Otherwise

37

• Otherwise
• Using only the first half of patients accrued
  during the trial, develop a binary classifier
  that predicts the subset of patients most likely
  to benefit from the new treatment E compared to
  control C
• Compare E to C for patients accrued in second
  stage who are predicted responsive to E based on
  classifier
• Perform test at significance level 0.01
• If H0 is rejected, claim effectiveness of E for
  subset defined by classifier

38
Biomarker Adaptive Threshold Design

• Wenyu Jiang, Boris Freidlin Richard Simon
• JNCI 991036-43, 2007

39
Biomarker Adaptive Threshold Design

• Randomized phase III trial comparing new
  treatment E to control C
• Survival or DFS endpoint

40
Biomarker Adaptive Threshold Design

• Have identified a predictive index B thought to
  be predictive of patients likely to benefit from
  E relative to C
• Eligibility not restricted by biomarker
• No threshold for biomarker determined

41

• S(b)log likelihood ratio statistic for treatment
  versus control comparison in subset of patients
  with B?b
• Compute S(b) for all possible threshold values
• Determine TmaxS(b)
• Compute null distribution of T by permuting
  treatment labels
• Permute the labels of which patients are in which
  treatment group
• Re-analyze to determine T for permuted data
• Repeat for 10,000 permutations

42

• If the data value of T is significant at 0.05
  level using the permutation null distribution of
  T, then reject null hypothesis that E is
  ineffective
• Compute point and interval estimates of the
  threshold b

43
(No Transcript)
44
Validation of Predictive Classifiers for Use with
Available Treatments

• Should establish that the classifier is
  reproducibly measurable and has clinical utility

45
Developmental vs Validation Studies

• Developmental studies should select patients
  sufficiently homogeneous for addressing a
  therapeutically relevant question
• Developmental studies should develop a completely
  specified classifier
• Developmental studies should provide an unbiased
  estimate of predictive accuracy

46
Limitations to Developmental Studies

• Sample handling and assay conduct are performed
  under controlled conditions that do not
  incorporate real world sources of variability
• Small study size limits precision of estimates of
  predictive accuracy
• Predictive accuracy may not reflect clinical
  utility

47
Types of Clinical Utility

• Identify patients whose prognosis is sufficiently
  good without cytotoxic chemotherapy
• Identify patients who are likely or unlikely to
  benefit from a specific therapy

48
Prognosis Good Without Chemotherapy

• Develop prognostic classifier for patients not
  receiving cytotoxic chemotherapy
• Identify patients for whom
• current practice standards imply chemotherapy
• Classifier indicates very good prognosis without
  chemotherapy
• Withhold chemotherapy to test predictions

49
Prospectively Planned Validation Using Archived
MaterialsOncotype-Dx

• Fully specified classifier developed using data
  from NSABP B20 applied prospectively to frozen
  specimens from NSABP B14 patients who received
  Tamoxifen for 5 years
• Good risk patients had very good relapse-free
  survival

50
B-14 ResultsRelapse-Free Survival
Paik et al, SABCS 2003
51
Prospective ValidationUS Intergroup Study

• OncotypeDx risk score lt15
• Tam alone
• OncotypeDx risk score gt30
• Tam Chemo
• OncotypeDx risk score 15-30
• Randomize to Tam vs Tam Chemo

52
(No Transcript)
53

• Measure classifier for all patients and randomize
  only those for whom classifier determined therapy
  differs from standard of care

54
(No Transcript)
55
Types of Clinical Utility

• Identify patients whose prognosis is sufficiently
  good without cytotoxic chemotherapy
• Identify patients whose prognosis is so good on
  standard therapy S that they do not need
  additional treatment T
• Identify patients who are likely or unlikely to
  benefit from a specific systemic therapy

56
Developmental Strategy (II)EGFR Inhibitor in
NSCLC
57
Validation Study for Identifying Patients Who Do
Not Benefit from Standard Adjuvant Regimen

• Standard adjuvant treatment S
• Classifier based on previous data for identifying
  patients who do not benefit from S relative to
  previous control C
• RCT of S vs C for patients predicted not to
  benefit from S
• Its difficult to go back
• Alternatively, RCT comparing S vs new treatment T
  for patients predicted not to benefit from S

58
BRB-ArrayTools

• Contains analysis tools that I have selected as
  valid and useful
• Targeted to biomedical scientists with analysis
  wizard and numerous help screens
• Imports data from all platforms and major
  databases
• Extensive built-in gene annotation and linkage to
  gene annotation websites
• Extensive gene-set enrichment tools for
  integrating gene expression with pathways,
  transcription factor targets, microRNA targets,
  protein domains and other biological information

59
Predictive Classifiers in BRB-ArrayTools

• Classifiers
• Diagonal linear discriminant
• Compound covariate
• Bayesian compound covariate
• Support vector machine with inner product kernel
• K-nearest neighbor
• Nearest centroid
• Shrunken centroid (PAM)
• Random forrest
• Tree of binary classifiers for k-classes
• Survival risk-group
• Supervised pcs

• Feature selection options
• Univariate t/F statistic
• Hierarchical variance option
• Restricted by fold effect
• Univariate classification power
• Recursive feature elimination
• Top-scoring pairs
• Validation methods
• Split-sample
• LOOCV
• Repeated k-fold CV
• .632 bootstrap

60
BRB-ArrayTools

• Publicly available for non-commercial use
• http//linus.nci.nih.gov/brb

61
Conclusions

• New technology makes it increasingly feasible to
  identify which patients are likely or unlikely to
  benefit from a specified treatment
• Targeting treatment can greatly improve the
  therapeutic ratio of benefit to adverse effects
• Smaller clinical trials needed
• Treated patients benefit
• Economic benefit

62
Conclusions

• Some of the conventional wisdom about how to
  develop predictive classifiers and how to use
  them in clinical trial design is flawed
• Prospectively specified analysis plans for phase
  III studies are essential to achieve reliable
  results
• Biomarker analysis does not mean exploratory
  analysis except in developmental studies

63
Conclusions

• Achieving the potential of new technology
  requires paradigm changes in correlative
  science.
• Effective interdisciplinary research requires
  increased emphasis on cross education of
  laboratory, clinical and statistical/computational
  scientists

64
Acknowledgements

• Kevin Dobbin
• Boris Freidlin
• Wenyu Jiang
• Aboubakar Maitnourim