Can Consultancy Provide the Bridge Between Research and the Applied Statistician? Professor Andy Grieve Department of Public Health Sciences King

1
Can Consultancy Provide the Bridge Between
Research and the Applied Statistician?Professor
Andy GrieveDepartment of Public Health
SciencesKings College London
Andy.Grieve_at_kcl.ac.uk
2
Outline

• An Old Idea Resurrected (1930s)
• Taxonomies of Designs
• Examples from the literature
• Up-and-Down Designs (1940s)
• Play-The-Winner Designs (1960s)
• Randomized Play-The-Winner Designs (1970s)
• Ethice in Clinical Trials
• Clinical CRM Designs (1990s)
• Adaptive Interim Analyses (1990s)
• Sample Size Re-estimation (1990s)
• Seamless Phase II/Phase III (2000s)
• Discussion

3
Statistical Research Consulting Centre Mission

• To maximize statistical value to the projects
  through strategic customer service from
  discovery, clinical development, and regulatory
  interactions to product defense and lifecycle
  support
• To accomplish that we will
• Develop and champion techniques based on
  state-of-the-art statistical research to increase
  shared understanding of problems and seek
  creative solutions by interacting with
  thought-leaders
• Establish effective statistical policies to
  provide clear guidance to project teams for
  efficient execution through a well-trained
  statistical staff worldwide

4
Vision and Mission

• The goal is to be achieved by
• Consulting with the projects clinical and
  non-clinical
• Business-led statistical research
• Education both of statisticians and
  non-statisticians
• Influencing the internal environment
• Influencing the external environment academic
  and regulatory

5
Consultancy-Based Statisticians
Donald W. Marquardt The Importance of
Statisticians ASA Presidential Address, 1987

• entrepreneurial
• customer focused
• aimed at novel, innovative uses of statistics
• profoundly involved in improvement (TQM, CI)
• involved in change
• is a team member

6
The Ideal Consultation

• Working together
• Familiarity with Languages
• Appreciation of Contribution
• Challenging Statistical Problem
• Design, Experiment, Analysis
• Multiple Publications
• Further Experimentation
• Significant Medical Advance
• NOBEL PRIZE

7
Nobel Prize the Undervaluation of Statistics
Norman E. Breslow Are Statistical Contributions
to Medicine Undervalued? Biometrics Society
Presidential Address, 2002

• Statisticians/epidemiologists have made major
  contributions to medicine
• Case-control studies
• Analysis of incomplete data
• Causal Inference
• BUT no Nobel Prize for medicine despite numerous
  nominations for Austin Bradford-Hill and Richard
  Doll
• Ronald Ross Nobel Prize in 1902
  malaria/mosquito
• developed a mathematical theory of
  epidemics
• However In economics McFadden / Heckman
  (2000)
• 
  Daniel Kahneman (2002)
• Engel / Granger (2003)

8
Most Important

• Build Relationships

9
Customer Stereotypes (Lyon Hyams, Biometrics
1971)

• The Probabilist
• Only interested in a significant p-value
• The lower, the better
• Two types
• Leaves the choice of test upto the statistician
• Insists on a particular test, invariably
  inappropriate
• The Numbers Collector
• Has huge amounts of data and wants the nuggets
  found in it
• If you find it, the customer takes the credit
• Find nothing youre to blame

• The Sporadic Leech
• Corridor meeting have you got 5 minutes can
  you help with this data ?
• If you respond they never let go.
• The Amateur Statistician
• He knows statistics and only wants your help
  because he doesnt have the 50 minutes required
  to solve the problem
• The Long Distance Runner
• Has always runs a competitor
• Needs you to keep him ahead of the field

10
Statistician Stereotypes (Lyon Hyams, Biometrics
1971)

• The Model Builder
• Fits every problem with his favourite model
• The Hunter
• The counterpart of the Numbers Collector If
  there is a significant p-value hell find it
  report it to 17 figures
• The Gong
• Starts every consult by drawing a normal curve
• The Traditionalist
• Is convinced nothing worthwhile has been done in
  stats since RA Fisher

• The Randomphiliac
• Believes it doesnt matter what else you do so
  long as you randomise
• The Quantophreniac
• Believes in hard measurements
• The More Data Yeller
• Clear
• The Nit Picker
• Always focusses on the unimportant, but debatable

11
Most Important

• Build Relationships
• Speak using plain language not mathematics
• Never be afraid to ask questions
• Question established practice
• Look for practical solutions
• Be inclusive
• Solve the real problem, not only the statistical
  ones which may interest us
• Collect examples from familiar projects

12
Collaboration not Consultancy
Joseph John Fox The province of
statistics Journal of the Statistical Society of
London, 1860
Statistics has no facts of its own in so far
as it is a science, it belongs to the domain of
mathematics. Its great and inestimable value is
that it is a method for the prosecution of the
other sciences Statistics a parasitic science
? a symbiotic science
Two Examples
13
Example 1 Lethal Dose
Estimation

• 1979-1987 Switzerland
• Central Research Function providing support to 2
  Toxiclogy departments Pharma Agro
• Routine work computerized
• Consulting for non-routine problems
• In 1983 Swiss Poison Regulation Introduced to
  Classify Chemicals

14
Motivating Example
Dose (mg/kg) of Animals of Deaths
500 5 1
1000 5 2
2500 5 3
5000 5 2
Based on these data we wish to
determine the LD50 to classify
the drug according to the following
classification scheme (Swiss Poison Regs.)
Toxicity Class 1 2 3 4 5
Range of LD50 (mg/kg) lt 5 5-50 50-500 500-2000 2000-5000
15
Model

• Data triplets
• di , ni , ri i1,..,k
• Probabilities of response
• pi i1,,k
• Logistic Model
• log pi / (1-pi) a blog(di)
• Median Lethal dose (LD50)
• log(LD50) -a/b m

Probit Model pi F(a blog(di))
16
Fiellers Theorem

• MLE Estimates
• Asymptotic 2nd Moments
• Fiellers Approximate Result

MLE 4049 mg / kg Fiellers Interval
(-? , ?)
17
?2-Squared Statistic vs log(LD50)
3
2.5
Asymptote
3
?2-Squared
1.5
Max likelihood Estimate
1
0.5
0
2
4
6
8
10
12
Log(LD50)
18
Bayesian Solutions

• Likelihood Function
• Prior distribution - p(a,b) (b gt 0 )
• Define m-a/b - log(LD50)
• Inference

19
Likelihood Contours - Motivating Example
Dose (mg/kg) of Animals of Deaths
500 5 1
1000 5 2
2500 5 3
5000 5 2
20
Likelihood Function - Hypothetical Example
Dose (mg/kg) of Animals of Deaths
100 3 1
1000 3 2
Alpha
Normal Analytic Approximation
0
1
2
-1
Beta
21
Where Does the Prior Come From?
Dose (mg/kg) of Animals of Deaths
500 5 1
1000 5 2
2500 5 3
5000 5 2
Experienced toxicologists
will know that they need
to span the LD50 with
the doses they choose. The choice of doses
contains information concerning the
toxicologists beliefs about the likely value of
the LD50.
22
Choice of Prior 1) Tsutakawa (1975) logit

• Choose doses d1 d2
• s.t. P(d1ltLD50ltd2)0.5

• Implies p1 and p2 uniform over the half square

• p(a,b) logit ? n.c.p.
• probit ? BN (truncated)
• (Grieve , 1988)

• Implies knowledge of pi ? i ?1,2

23
Choice of Prior 2) Tsutakawa (1975) - logit

• Choose doses d1 d2

• Assume n.c.p. for p1 and p2

• p(a,b) logit ? n.c.p.
• probit ? BN (truncated)
• 
  (Grieve, 1988)

• Implies knowledge of pi ? i ?1,2

24
Choice of Prior 3) Grieve (1988) - probit
Toxicity Class 1 2 3 4 5
Probability 0.05 0.15 0.40 0.30 0.05

• Can the parameters be determined ?
• Not uniquely !!!
• The c.d.f. of a/b depends only on

• Implying any 4 probabilities are sufficient to
  determine c1,c2,c3 and c4
• Any one of the 5 parameters is also needed
• Modal slope ? How about median ?
• Feedback

25
Fiducial Failure

• Ross, Bailey, Gower, Cox (1986)
• Failure of fiducial approach
• Invalidity of normal approximation for small
  numbers of animals
• Likelihood Ratio / Profile Likelihood

26
Profile Likelihood

• Definition - two parameter likelihood -
  L(q,jX)
• Profile likelihood for q -
• where is the solution of
• Interval estimate

27
Profile Likelihood - Cont.

• Applied to LD50
• Reparameterise a , b -gt m , b
• logqi/(1-qi) b(log(di)-m)

28
Log(Profile Likelihood) - Inhalation Data
29
Profile Likelihood

• the modest shortfall in the true confidence
  probability of (the) LR interval may be judged as
  a small price to pay for an interval that can
  always be determined
• Williams, Biometrics, 1986

30
Profile Likelihood - General Property

• Theorem Grieve, 1995
• The profile likelihood for log(LD50) has a
  minimum given by

31
Log(Profile Likelihood) - Motivating Example
32
Profile Likelihood

• the maximum relative likelihood does not take
  account of the uncertainty due to lack of
  knowledge of b and so can be misleading in terms
  of both precision and location.
• Kalbfleisch and Sprott, J. Roy. Stat. Soc. Ser.
  B, 1986

33
Modified Profile Likelihood

• Cox Reid, JRSSB, 1987 Barndorff-Nielsen,
  Biometrika 1983
• Conditional inference
• reduce the impact of nuisance parameters
• Modified Profile Likelihood for q is
• where is the observed
  information for f given fixed q

34
Modified Profile Likelihood - Cont.

• the effect of the second term is to penalise
  values of q for which information about f is
  relatively large.
• Cox and Reid (1987)
• ORTHOGONAL PARAMETERS !!

35
Likelihood Contours - Inhalation Data
36
Connection to Bayes

• Prior density for q and f - p(fq)p(q)
• Laplace Approximation - Tierney Kadane - JASA,
  1986

37
Logit Model - Bugs

for (i in 1N)
logit(pi) lt- alpha betalog(di)
ri dbin(pi,ni)

alpha dnorm(0.0,1.0E-6) intercept
beta dnorm(0.0,1.0E-6) slope
mu lt- -alpha/beta

38
Probit Model - Data AugmentationAlbert and Chib
- JASA (1993)

• Introduce latent variables
• Zj N(ab log(dj), 1) , j1,,N
• Define Yj 1 if Zj gt 0
• Yj 0 otherwise
• Implies Yj are ind. Bernoulli
• with P(Yj 1)F(ab log(dj)
• Albert and Chib

39
Probit Model - Data AugmentationAlbert and Chib
- JASA (1993)

• Given Zs can be estimated using
  linear regression a and b can be generated from
  the appropriate posterior distribution centred at
  
• Given b the Zs can be generated as follows
• Zjy,b N(ab log(dj), 1)
• truncated at the left by 0 if yj1
• Zjy,b N(ab log(dj), 1)
• truncated at the right by 0 if yj0

40
Gibbs Sampling / Data Augmentation
20
10
log(LD50)
0
-10
0
2
-1
1
Slope
41
Posterior Sample
80
60
Percent
40
-ve slope
ve slope
20
0
-10
-5
0
5
10
15
20
Midpoints - Log(LD50)
42
The Virtuous Circle

• A Simple Problem
• Applied Bayesian methods
• Eliciting Prior distributions
• Profile likelihoods
• Conditional Profile Likelihood Methods
• Latent Variable Simulation
• Consulation -gt Research -gt Better consultant

43
A Sporadic Leech
44
Background

• Approached by a clinical colleague on a stroke
  program (neuroprotection)
• He believed that many stroke programs had failed
• Because they had not established the dose before
  phase III
• Bad designs
• Can we develop a better design for establish the
  dose ?

45
Background

• Protect against damage by Neutrophils
• NIF (Neutrophil inhibitory factor) is a
  recombinant 41-kDA glycoprotein, originally
  derived from the canine hookworm

46
Dose Selection Standard design

• Placebo 4 doses available where do I put them ?

Response
Dose
47
Issues in Dose Selection Increase
Number of Doses????????????????
48
Improvements to Standard Design

• Increase number of doses (placebo a large
  number - 15)
• Learn about dose-response and adapt
• Prevent allocating patients to ineffective doses
  (ETHICAL)
• Model dose response nor pairwise comparison
• Futility analysis / early decision making

49
ASTIN Study - Design ProcessGrieve and Krams,
Clin. Trials 2005 2 340351
Predictive Model
Randomizer
Data Interface
...Ongoing... Update patient data
Model predicts final outcomes
Surrogate/ Early Response
Dose to vialtranslation
Bayesian Analysis
Randomizeto placebo or optimal dose
Continue
Stop
New Patient
Terminator
50
Issues
? ? ?

• How do we do the updating ?
• How do we predict ?
• How do we choose a dose ?
• How do we stop ?
• How do we model response ?

51
The Dose-Response Model qj f( zj ,q)

• Requirements
• To model f (z ,q) we needed

• a flexible model, allowing non-monotone curves.

• analytical posterior updating (simulation
  required for terminator and allocator)
• efficient (analytic) computation of expected
  utilities

52
2nd Order Polynomial Normal Dynamic Linear Model
Evolution Variance Smoother
53
Fitting NDLM in ASTIN Study

• Original code written by Peter Müller (Duke Univ
  and MD Anderson)
• Written in C (1000s of lines)
• Re-engineered to FDA validation standards by a
  computing company Tessella
• A black box making algorithmic changes
  difficult.

54
Fitting NDLM Using WinBUGSCode by David
Spiegelhalter
55
Comparison of ImplementationsFinal ASTIN
Dose-Response
21
19
Response
17
15
13
0
10
20
30
40
50
60
70
80
90
100
110
120
Dose
Peter Müller
WinBUGS
56
General Linear State Space Model

• Standard form Observation equation
• and structural equations

Design vector (px1)
Transition Matrix (p x p )
State vector at time t (p x p)
57
2nd Order NDLM in State-Space FormLocal Linear
Trend
58
Structural ExpansionHans-Peter Piepho and Joseph
O OgutuSimple State-Space Models in a Mixed
Model Framework, Amer Stat, 61, 224-232
Substitution in the observation equation gives
General Mixed Model yX?Z??
59
SAS Code to Local Linear Trend Modelwith
Baseline (ASTIN Model)
proc mixed lognote methodreml model DSSSDose
base_SSS/outppred ddfmkr alphap0.1 random
z1_1-z1_16 z2_1-z2_16/subintercept typetoep(1)
In theory this code could be used to fit the
model used for ASTIN. Proc Mixed now performs
Bayesian Analyses using the Prior
Statement Questionable priors Jeffreys, flat
60
Conclusion

• As a practising statistician research is lead by
  need
• Consulting can provide focus for research
• Links to academia
• Academics
• REMEMBER today research funding may be linked
  to practice / industry