A Statistician

1
A Statisticians Perspective on Biomarkers in
Drug Development

• PSI Biomarkers Special Interest Group
• Martin Jenkins and Chris Harbron, AstraZeneca
• Co-authors Aiden Flynn, Trevor Smart, Chris
  Harbron, Tony Sabin, Jayantha Ratnayake, Paul
  Delmar, Athula Herath, Philip Jarvis and James
  Matcham
• Volume 10, Issue 6, Pages 494-507,
  November/December 2011

2
What is a biomarker?
Biological marker (biomarker) A characteristic
that is objectively measured and evaluated as an
indicator of normal biological processes,
pathogenic processes, or pharmacologic responses
to a therapeutic intervention.
NIH biomarker definitions working group, 2001

• Many endpoints could be considered as biomarkers
• Commonly thought of in terms of biological /
  tissue samples (eg blood), imaging techniques or
  even examinations
• Many technology platforms (proteomics,
  histopathology etc)
• Does not define one purpose and so a clear
  objective and terminology can be helpful
• Appear at many stages throughout the development
  program

3
Types of biomarker
4
Biomarker Examples
Biomarker Current Use Classification Qualification
HER2, EGFR, K-RAS mutations Directing treatment in oncology Predictive biomarker Defines indication in label, diagnostic development
UGT1A1, TMPT polymorphisms Predisposition to certain toxicities Predictive biomarker Appear in label as risk factor suggesting dose adjustment
AB1-42 Diagnose prodromal Alzheimers Disease Prognostic biomarker Enrich trial populations. Example of qualification
Gene signature chips Prognosis prediction in oncology Prognostic biomarker Diagnostic qualification process applies
CRP, IL-6, TNFa in blood samples Proof of principle in inflammatory diseases Pharmacodynamic biomarker Fit for purpose assay validation
FDG-PET imaging Proof of concept (eg Tumour metabolism) Pharmacodynamic biomarker Collaborative opportunities for qualification
HbA1c Represents glycemic control in diabetics Surrogate Endpoint Primary evidence for label. Qualification burden
5
What characteristics are we interested in?

• It is generally advisable to learn as much as
  possible about a biomarker prior to its
  application for decision-making
• The science behind the choice of biomarker should
  be supported prior to initiating studies
• The practical feasibility of using the marker
  should be examined
• The statistical properties of the biomarker are
  of interest
• Statistical properties of interest include
• Variability estimates (and components of
  variability, within and between subjects)
• Effect sizes (using a positive control, challenge
  model or other data sources) and dynamic range
• Distributions and scaling approaches required
• Appropriate analysis methods considered

6
Methodology studies

• Feasibility / Methodology studies can be used to
  learn the characteristics of an endpoint, but
  also the practicalities
• Feasibility of multiple assessments
• Acceptability to the patient, degree of dropout
• The same design and analysis methodology should
  be used as is envisaged for the subsequent
  clinical trial
• Information learned can aid in designing or
  sizing this trial
• Work carried out should be fit-for-purpose enough
  to ensure that there is sufficient confidence
  that decisions can be made based upon the
  biomarker, given its intended use. Further
  studies, reviews or meta-analyses can aid in this
  if required.

7
Sources of variation

• Much biomarker work is accompanied by similar
  practical challenges around controlling sources
  of variation
• Patient-related factors
• Individual biological factors genetics, race,
  age, comorbidities
• Social and habitual factors smoking, physical
  fitness, diet
• Natural biological variation eg diurnal effects
• External factors
• Pre-analytical variation sample collection
  and fixation practices
• Analytical variation between laboratories,
  readers and batches,
  technical precision of
  the assay

8
Sources of bias

• Bias can arise, especially in open-label studies,
  for several reasons
• Verification bias because of the choice of
  locally available methods
• Patient selection bias, for example, patients
  with breast cancer family history, may not
  consent to BRCA DNA testing
• Treatment allocation bias, if treatment
  assignment is based on a subjective assessment,
  as in some histo-pathological endpoints
• Being asked the same question repeatedly, for
  example in a pain challenge model, could induce
  false differences
• If a scoring method is subjective, then measures
  should be taken to minimize bias, for example,
  with the use of scripted questions, blinding or
  standard operating procedures.
• Conclusions may be subject to many caveats if
  they are not based upon complete and
  representative data-sets

9
Sources of missing data

• Biomarker studies can suffer from missing data
  issues, especially in exploratory situations
• Inaccessibility of tissues (e.g. lung cancer)
• Low consent rates for optional samples
• Lack of residual tissues in complete responders
• Poor quality of fixation in archival samples
• Patient drop-outs (lack of response or toxicity)
• Poor data handling procedures cf. clinical data
• Likely sources of missing data should be
  considered in advance so as to minimise potential
  implications and make appropriate assumptions

10
Multiplicity when considering many markers

• Range of new high-dimensional technologies
• e.g. genetics, genomics, proteomics,
  metabonomics, NGS
• Allow understanding in detail at a molecular
  level the processes of disease and response to
  treatment and identify biomarkers that can
  identify or predict these changes.
• Multiplicity becomes a concern and requires new
  approaches to be adopted
• e.g. False Discovery Rate, Permutation testing,
  Significance Analysis of Microarrays (SAM)
• Pre-analysis filtering of variables can help
• Want to emerge with both statistical significance
  scientifically relevant
  and meaningful effect.
• 2D FDR

11
Composites and Multivariate Analyses

• Visualisation key
• Principal Components Analysis (PCA), Clustering
• Wide variety of supervised multivariate
  predictive modeling techniques are available
• Regression-based approaches e.g. Partial least
  Squares (PLS), Elastic Nets
• Proximity-based methods e.g. Nearest Neighbours
• Tree-based methods e.g. Random Forests, Gradient
  Boosting
• Distance-based approaches e.g. Support Vector
  Machines (SVM)
• Not just Black-Box, Interpretation important
• Importance scores
• Study design is critical, as is visualization,
  understanding data quality and its impact on
  subsequent analyses.
• MAQC-II

12
Biomarkers for Personalised Healthcare

• Personalized health care offers the potential to
  identify patients more likely to derive benefit
  from treatment and as such is of great interest
  to Pharmaceutical companies, Regulatory
  Authorities and Health care Providers
• Predictive markers marker by treatment
  interaction
• Gives rise to new set of challenges eg. Low
  power to robustly identify biomarkers in
  exploratory studies
• Developing from a Biomarker to a Companion
  Diagnostic
• Understanding and quantifying the factors that
  may impact the performance of the diagnostic
• Identifying an optimal cut-off
• Additional regulatory process

13
Study design options

• All-comers design
• Subjects enrolled into groups, and
  retrospectively measured
• Stratification design
• Biomarker status at screening used as a
  stratification factor in randomisation to ensure
  balance

• Targeted design
• Only biomarker positive patients recruited into
  study
• Enriched design
• Hybrid, recruiting a limited number of biomarker
  negative patients with a greater representation
  of biomarker positives
• Adaptive design
• Many options allowing testing and refinement of a
  biomarker

14
Safety biomarkers preclinical translation

• Predictive safety biomarkers allow early
  detection of toxicity and assessment of human
  risk
• Preclinical qualification including mechanistic
  understanding of the relationship between the
  biomarker and organ damage
• Use known organ toxicants
• Determination of organ toxicity in rat by
  qualified toxicological pathologist
• Understanding properties of translation to man
• Animal Model Framework (AMF) project is a
  collaborative effort combining pre-clinical and
  clinical data to determine operating
  characteristics and refine thresholds of Safety
  Pharmacology models

15
Safety biomarkers - clinical

• Clinical qualification less straightforward
  because generally verification of organ toxicity
  not possible
• Utilise methods that can assess performance in
  absence of gold standard including Bayesian
  approaches
• Currently lab parameters used to indicate kidney
  and liver injury, but poor performance
• e.g. by time serum creatinine indicates drug
  induced kidney injury, a high degree of kidney
  function loss has already occurred
• Public-private precompetitive partnerships
  established for searching for, validating and
  qualifying safety biomarkers for predicting
  drug-induced organ injury
• Predictive Safety Testing Consortium (C-Path),
  SAFE-T (IMI)

16
Biomarker qualification

• Formal biomarker qualification process have been
  introduced
• FDA (qualification process for drug development
  tools)
• EMA (qualification of novel methodologies and
  biomarkers)
• Examples on EMA and FDA websites
• These work alongside usual scientific advice
  routes and are a way to seek regulatory opinion
  on acceptability of a marker for a given use
• This is not mandatory and so is usually not
  required, but may be an advantageous step,
  particularly for a marker with wide applicability
  (For example for collaborative groups seeking to
  develop a new endpoint, particularly for
  toxicity)
• Diagnostic biomarkers would need to follow the
  in-vitro diagnostic regulatory processes

17
Surrogate endpoints

• Those outcomes for which treatment effects
  correlate well with those for an accepted
  clinical outcome at an individual and group level
  could potentially substitute for a recognized
  clinical endpoint
• Should lead to the same decision being made
  as if the
  clinical outcome had been used
• Examples exist (eg in diabetes or HIV), but
  a large
  body of evidence is required and this
  is not required for
  many purposes
• Surrogacy is often an unrealistic target.
• Statistical methods have been developed to
  examine the association between endpoints, but
  these can also be applied to other situations,
  for example when considering assurance when
  endpoints differ between studies

18
Future challenges
Issues to address in the future
i How can the traditional clinical development program be reengineered, so allow for greater learning around a biomarker without delaying the drug program?
Ii What study designs can be used to translate markers of drug toxicity into man given that the pre-clinical studies used for validation are not feasible to conduct in humans?
iii Many technologies suffer from a batch effect. What pre-processing techniques can be developed to ensure we are measuring true biology rather than sample quality?
iv New techniques have moved the degree of dimensionality by several orders of magnitude. How, both practically and analytically, can we cope with this deluge of data?
v Effective ways, possibly Bayesian, could be developed to build existing pathway knowledge into data analyses.
vi When identifying biomarkers for PHC, interaction analyses typically have low power, especially with many potential markers. How can such markers be reliably identified?
vii As PHC becomes the processes for gaining approval of a drug, diagnostic and biomarker qualification may become more burdensome. How can this evidence be streamlined?
19
Conclusions Recommendations
Key considerations for statisticians in biomarker development
i Define clear objectives shaped by the potential future use of the marker This will clarify what can reasonably be claimed based upon the research Validation work and study designs should be fit-for-purpose relative to these aims
ii Prospectively plan studies even if they are exploratory - Defining what represents success in advance will aid objective decision-making
iii Consult expert colleagues to understand and shape the endpoints definitions Learn about distributions and pre-processing Influence scoring methods so as to get maximum information
iv Gather prior knowledge on sources of variability and causes of missing data Plan ahead to avoid sample and data management issues rather than being reactive Adjust missing data assumptions accordingly
v Learn about a new marker using methodology studies or translational science - Estimates of variability or effect size can be valuable if reflective of situation of use
20
Conclusions Recommendations
Key considerations for statisticians in biomarker development
vi Analyse a biomarker using the appropriate methodology for the given endpoint type - As with other endpoints there is no universal approach
vii Be alert when using large numbers of candidate biomarkers - Particular attention should be paid to issues of multiplicity and model validation
viii Maintain a sceptical and questioning point of view to manage expectations - Consider results in relation to prior knowledge and study aims, avoiding extrapolation
ix Consider how to add confidence to findings using other sources of information - Integrate biological information as well as numerical results
x Remember that there is nothing mysterious about biomarkers - A consideration of key statistical principles can be instructive in most situations
21
PSI Biomarker Special Interest Group

• Please get in touch if you would like to get
  involved!
• Join the mailing list or linked-in group in
• Help on the SIG committee
• Review papers, training, discussion groups
• Attend one of our free meetings /or our
  conference sessions
• Go to www.psiweb.org and click on committees and
  sigs in the menu to find our webpage with more
  information

Biomarker Validation - Case Studies and
Approaches  MedImmune, Cambridge, 10 October 2012