Biomarkers and surrogate end points -- the challenges of statistical validation

1
Biomarkers and surrogate end points -- the
challenges of statistical validation

• Daniel Sargent, PhD
• Mayo Clinic, Rochester MN
• April 18, 2012

2
Biomarkers Possible Uses

• Screening for disease
• Predicting prognosis
• Identifying which patients to treat
• Monitoring for disease recurrence
• As an efficacy endpoint (surrogate)

Disease Identification
Prognosis, Prediction
Screening
Monitoring
Endpoint
3
Prognostic Marker

• Single trait or signature of traits that
  separates different populations with respect to
  the risk of an outcome of interest in absence of
  treatment or despite non targeted standard
  treatment

Prognostic
No treatment or Standard, non targeted treatment
Marker Marker
4
Predictive Marker

• Single trait or signature of traits that
  separates different populations with respect to
  the outcome of interest in response to a
  particular (targeted) treatment

Predictive
No treatment Or Standard
Targeted Treatment
Marker Marker
5
Surrogate Endpoints

• An endpoint obtained sooner, at less cost, or
  less invasively than the true endpoint of
  interest.
• When using a potential surrogate endpoint, one
  would like to make the same inference as if one
  had observed a true endpoint.
• Few surrogate endpoints have been validated
• Any validation applies only to agents with same
  mechanism of action

6
1 Challenge for Biomarker Validation -
Multiplicity

• Single marker
• Multiple assay methodologies
• Multiple cut-points
• Multiple endpoints
• Combination of markers
• Multiple combinations
• Combination of clinical and marker information
• Solution Pre-specification, replication
• Challenge This is not the laboratory culture

7
Prognostic Markers
8
Prognostic Marker

• In theory, can be validated on observational
  sample, provided
• Uniform treatment
• Complete, uniform follow-up
• Unbiased case selection
• Complete ascertainment of possible confounders
• In practice, these rarely hold, and clinical
  trial cohorts may be preferable

9
Development and Validation of a 21-Gene Assay
for N, ER, Tam Patients
YEAR
Develop real-time RT-PCR method for paraffin block
2001
Select candidate genes (250 genes)
2002
Model building studies (N 447, including 233
from NSABP B-20)
2002
Commit to a single 21-gene assay
2003
Validation studies in NSABP B-14 and Kaiser
Permanente
2003
Paik et al. N Engl J Med. 20043512817-2826.
10
Predictive Markers
11
Randomized Controlled Trial (RCT) for predictive
marker validation

• Goal Determine which treatment will work for
  which patient
• Vital Patients treated with treatment choices
  in question must be comparable
• Only true assurance Patients randomized between
  treatments in question

12
Design strategies for predictive marker
validation

• Retrospective Validation
• Prospective Validation
• Enrichment Designs
• Hybrid Designs
• All-comers or Unselected Designs
• Adaptive Designs

13
Retrospective Validation

• Test a marker by treatment interaction effect
  utilizing data collected from previously
  conducted randomized controlled trial (RCT)
  comparing therapies for which a marker is
  proposed to be predictive
• Reasonable when
• a prospective RCT is ethically impossible based
  on results from previous trials, and/or
• a prospective RCT is not logistically feasible
  (large trial and long time to complete).
• Feasible and timely

Simon et al JNCI 2009
14
Retrospective Validation

• Samples must be available on a large majority of
  patients to avoid selection bias in the patients
  that have or do not have the samples
• Hypotheses, analyses techniques, patient
  population, and precise algorithm for assay
  techniques must be stated prospectively
• All marker subgroup analyses have to be stated
  upfront, with appropriate sample size
  justification
• Replication critical, as many markers may be
  tested in this manner

Simon et al JNCI 2009
15
Panitumumab in last line CRCPFS by Treatment
Median In Weeks
Mean In Weeks
1
.
0
Events/N ()
0
.
9
191/208 (92)
8.0
15.4
Pmab BSC
9.6
BSC Alone
209/219 (95)
7.3
0
.
8
0
.
7
HR 0.59 (95 CI 0.480.72)
0
.
6
Proportion with PFS
0
.
5
0
.
4
0
.
3
0
.
2
0
.
1
0
.
0
0
2
4
6
8
1
0
1
2
1
4
1
6
1
8
2
0
2
2
2
4
2
6
2
8
3
0
3
2
3
4
3
6
3
8
4
0
4
2
4
4
4
6
4
8
5
0
5
2
Weeks
Patients at Risk
1
9
7
1
8
8
1
7
8
1
0
6
7
9
7
1
6
4
5
5
5
0
4
9
4
9
3
7
2
9
2
5
2
4
1
9
1
5
1
5
1
5
1
2
9
9
7
6
6
2
0
8
Pmab BSC
2
0
0
1
6
8
1
4
2
7
5
4
2
3
4
2
5
2
3
1
9
1
6
1
4
1
4
1
0
1
0
1
0
1
0
9
8
6
6
5
4
4
4
3
2
1
9
BSC Alone
Amado, JCO 2008
16
Mutant KRAS SubgroupPFS by Treatment
Median In Weeks
Mean In Weeks
1.0
Events/N ()
0.9
76/84 (90)
7.4
9.9
Pmab BSC
0.8
10.2
BSC Alone
95/100 (95)
7.3
0.7
HR 0.99 (95 CI 0.731.36)
0.6
Proportion with PFS
0.5
0.4
0.3
0.2
0.1
0.0
0
2
4
6
8
10
12
14
16
18
20
22
24
26
28
30
32
34
36
38
40
42
44
46
48
50
52
Weeks
Patients at Risk
Pmab BSC
78
76
72
26
10
8
6
5
5
5
5
4
4
4
4
2
2
2
2
2
2
2
1
1
1
84
91
77
61
37
22
19
10
9
8
6
5
5
4
4
4
4
4
4
3
3
3
2
2
2
2
100
BSC Alone
Amado, JCO 2009
17
Wild-type KRAS Subgroup PFS by Treatment
p lt 0.0001 for quantitative-interaction test
comparing PFS log-HR (pmab/BSC) between KRAS
groups
Amado, JCO 2009
18
OncotypeDX Benefit of Chemo by Risk Score
LOW RS
INT RS
28 absolute benefit from tam chemo
High RS
Paik et al. J Clin Oncol. 2006.
19
Schema TAILORx
Node Neg, ER Pos Breast Cancer
Register Specimen banking
Oncotype DX Assay
RS 11-25 Randomize Hormone Rx vs. Chemotherapy
Hormone Rx
RS lt10 Hormone Therapy Registry
RS gt25 Chemotherapy Hormone Rx
Primary study group
20
Phase III Trial Designs

• Retrospective Validation
• Prospective Validation
• Enrichment Designs
• Hybrid Designs
• All-comers or Unselected Designs
• Adaptive Analysis Designs

21
Using markers to restrict trial eligibility
success Her 2 Breast Cancer
Romond et al, NEJM 2005
22
Using markers to restrict trial eligibility
beware

• What about Herceptin in Her2- breast cancer?
• New Data No difference in benefit based on
  strength of HER2
• After 10 years, may need new study of Herceptin
  in Her2- patients

Paik et al, NEJM 2008
23
Unselected Design Upfront Stratification by
Marker status
Treatment A
Marker Level (-)
Randomize
Treatment B
Register
Test Marker
Treatment A
Marker Level ()
Randomize
Power trial separately within marker groups
Treatment B
Sargent et al., JCO 2005
24
MARVEL - Marker Validation for Erlotinib in Lung
Cancer
Primary aim To evaluate whether there are
differences in PFS between erlotinib and
pemetrexed within the FISH () and FISH (-)
subgroups
25
Adaptive Design

• Randomize between at least 2 arms within
  biomarker-defined strata
• Different signatures, different allowed drugs
• Evaluate success in an ongoing manner
• Alter randomization ratio?
• Drop poor performers
• Graduate good performers to phase III trials
• Ongoing trials ISPY-2 (Breast), BATTLE (NSCLC)

Zhou, Clinical Trials 2008
26
ISPY-2 Adaptive DesignLearn, Drop, Graduate, and
Replace Agents Over Time

• Investigational agent may be used in place

27
Surrogate Endpoint Evaluation

• Single trial settings Proportional Treatment
  Effect explained by surrogate endpoint (PTE)1
• Hypothesis testing Prentice Criteria2
• Meta-analytic Methods and Surrogacy estimation3
• Challenge Must convince both statistical and
  clinical communities

1Freedman et al (Stat Med 1992) 2Prentice (Stat
Med 1989) 3Buyse et al (Biostatistics 2000)
28
Meta-Analytic Surrogacy Evaluation

• Key question Does the treatment effect on a
  surrogate reliably predict the treatment effect
  on the true endpoint?
• To answer Need data from multiple trials
• Challenges
• Validation always disease, setting, and treatment
  class specific
• Meta-analytic methods suffer poor power when
  trials small (lt 10)

29
Meta-Analytic Validation

• Multi-level model
• First level within trial (individual level R2)
• Second level between trials (Trial level R2)

30
Meta-analysis Simulation study
Shi et al (Computational Statistics and Data
Analysis 2011)
31
Response Assessment Biomarkers Positron
Emission Tomography (PET)
32
PET as a biomarker

• Prognostic Does survival for patients with
  hot PET exams differ from those with cold PET
  exams?
• Surrogate endpoints PET may provide early
  signal for long-term treatment efficacy
• Patient management (predictive) PET results may
  be used to alter therapy
• Randomized trial needed

33
Value for patient managementExample of design
Continue
Hot PET
R
Survival Difference?
Switch
3 cycles of therapy
Cold PET
Continue
34
Conclusions

• Biomarkers highly active research area
• Existing methods focus mainly on single markers
• Statisticians must play a leading role, and
  emphasize fundamental principles
• Pre-specification
• Randomization
• Replication
• New statistical methods clearly needed