Lost Opportunities for Design Theory in Drug Development

1
Lost Opportunities for Design Theory in Drug
Development

• Stephen Senn

2
Basic Thesis

• Design theory has great potential in drug
  development
• But this potential is unrealised
• Those working in so-called optimal design are so
  ignorant of application realities that where
  their influence is not zero it is harmful
• On the other hand the understanding of design
  theory by biostatisticians is pitifully
  inadequate
• We must cooperate properly to cure this parlous
  state of affairs

3
Outline

• Quick tutorial on cross-over trials
• I shall then give two introductory examples of
  nonsense
• By leading design theoreticians
• By leading biostatisticians
• I shall then consider design nonsense further
• Some conclusions
• After lunch a case-study

4
Warning

• I am a biostatistician
• We are used to thinking of data matrices with
  rows as subjects and columns as measurements
• That means that we write sequences for designs
  with rows representing subjects and columns
  representing periods

5
Cross-over Trials
Definition A cross-over trial is one in which
subjects are given sequences with the object of
studying differences between individual
treatments.
6
An Example of an AB/BA cross-over in asthma
7
An Example from Rheumatism2 doses of diclofenac
and placebo
8
Carry-over
Definition Carry-over is the persistence
(whether physically or in terms of effect) of a
treatment applied in one period in a subsequent
period of treatment. If carry-over applies in a
cross-over trial we shall, at some stage, observe
the simultaneous effects of two or more
treatments on given patients. We may, however,
not be aware that this is what we are observing
and this ignorance may lead us to make errors in
interpretation.
9
Simple Carry-over

• Carry-over lasts for exactly one period
• It depends only on the engendering treatment and
  is unmodified by the perturbed treatment
• There is a huge literature proposing optimal
  designs for this model
• There is no empirical evidence that any of this
  has been useful

10
Three Period Bioequivalence Designs

• Three formulation designs in six sequences
  common.
• Subjects randomised in equal numbers to six
  possible sequences.
• For example, 18 subjects, three on each of the
  sequences ABC, ACB, BAC, BCA, CAB, CBA.
• A test formulation under fasting conditions,
• B test formulation under fed conditions
• C reference formulation under fed conditions.

11
Weights for the Three Period Design not
Adjusting for Carry-over
12
Properties of these weights

• Sum 0 in any column,
• eliminates the period effect.
• Sum 0 in any row
• eliminates patient effect
• Sum 0 over cells labelled A
• A has no part in definition of contrast
• Sum to 1 over the cells labelled B and to -1 over
  the cells labelled C
• Estimate contrast B-C

13
Weights for the Three Period Design Adjusting
for Carry-over
14
Weights for the Three Period Design Adjusting
for Carry-over
15
Properties of These Weights

• As before
• Estimates B-C contrast
• Eliminates, period and patient effect
• Eliminates A
• Sum to zero over cells labelled a,b, and c
• Eliminate simple carry-over

16
Have We Got Something for Nothing?

• Sum of squares weights of first scheme is 1/3 (or
  4/12)
• Sum of squares of weights of second scheme is
  5/12
• Given independent homoscedastic within- patient
  errors, there is thus a 25 increase in variance
• Penalty for adjusting is loss of efficiency

17
First ExampleSome Design Theory Nonsense
John, J. A., Russell, K. G., and Whitaker, D.
(2004), "Crossover An Algorithm for the
Construction of Efficient Cross-over Designs,"
Statistics in Medicine, 23, 2645 - 2658. A
cross-over experiment involves the application of
sequences of treatments to several subjects over
a number of time periods. It is thought that the
observation made on each subject at the end of a
time period may depend on the direct effect of
the treatment applied in the current period, and
the carry-over effects of the treatments applied
in one or more previous periods. Various models
have been proposed to explain the nature of the
carry-over effects. An experimental design that
is optimal under one model may not be optimal if
a different model is the appropriate one. In this
paper an algorithm is described to construct
efficient cross-over designs for a range of
models that involve the direct effects of the
treatments and various functions of their
carry-over effects. The effectiveness and
flexibility of the algorithm are demonstrated by
assessing its performance against numerous
designs and models given in the literature.
18
Whats wrong here?
Sometimes in a clinical trial it may be
necessary to modify or extend an ongoing trial.
For example, suppose that after a few periods
have been completed one of the treatments is
dropped from the trial. It will then be necessary
to re-allocate the other treatments to the
remaining periods of the trial. (John et al,
2004 p. 2653) Jones and Donev 17 also
consider augmenting a design to account for the
removal of a treatment. The initial trial to
compare four treatments A, B, C, and D in five
periods using four groups of subjects used a
Williams square 3 with the fourth period
repeated. After the first two periods had been
completed it was decided to drop treatment D
from the remainder of the trial.
19
(No Transcript)
20
The reality ..in a single-dose cross-over trial
in asthma in 12 patients reported by Palmqvist et
al. 5 patients were treated in the first period
of a cross-over on dates ranging from 5 May to 12
November 1987 4. They were treated in a second
period on dates ranging from 18 May to 26
November 1987. Eleven of the patients had
completed period two of treatment before the 12th
patient was recruited. (Senn, 2005, p3675.)
A basic fact of clinical trials
You treat patients when they fall ill
21
Multi-Story
22
Conclusion

• Multi-dose trials real scope for design theory.
• These will employ active wash-out
• Design problem is trade-off between exploiting
  correlation and eliminating carry-over.
• Short vs long active wash-out periods

23
Second ExampleSome Biostatistics Nonsense
Chow, S. C., and Liu, J. P. (2000), Design and
Analysis of Bioavailability and Bioequivalence
Studies (2nd ed.), New York Marcel Dekker. Have
several discussions of efficiency of designs in
their book which are completely beside the point.
They compare designs in terms of residual degrees
of freedom! They conclude that Balaams design,
which uses sequences TR/RT/TT/RR Is similar in
efficiency to the more conventional TR/RT
design. They write The degrees of freedom for
the intrasubject residuals for the 2 2 and 4
2 design are 22 and 21 respectively. Therefore
there is little difference in testing
power. This is nonsense
24
Second ExampleSome Biostatistics Nonsense
Design Design Design Design Design
Source 22 42 23 24 44
Between 23 23 23 23
Seq 1 3 1 1
Res 22 20 22 22
Within 24 24 48 72
Period 1 1 2 3
Form 1 1 1 1
Carry 1 1 1
Res 22 21 44 67
Total 47 47 71 95
.
25
What is wrong 1. Its not correct design theory

• As any design expert knows residual degrees of
  freedom are (nearly) irrelevant to efficiency
• It is the impact of adjustment on the degree of
  orthogonality of the design matrix that is
  important

26
What is wrong 2.Its not realistic biostatistics

• In fact as any biostatistician who has had to
  think about it will know from a practical point
  of view far from being optimal Balaams design is
  simply inadmissible
• The reasons is that only half of the resources
  are devoted to actually measuring the treatment
• The rest are devoted to providing an adjustment
  for a form of carry-over that is itself
  implausible

27
Allocation of patients for two designs
Sequence AB/BA Balaam
AB n/2 n/4
BA n/2 n/4
AA 0 n/4
BB 0 n/4
28
Investigation of the real efficiency of Balaams
design
29
Five Reasons why the Simple Carry-over Model is
not Useful

• If it applies then the investigator can design a
  trial which eliminates it . (double the periods)
• Implausible given pk/pd theory. (obvious)
• Leads to inefficient estimators. (see
  investigation to follow)
• Can lead to poor designs. (ditto)
• The models which incorporate it are
  self-contradicting. (example factorial X-overs)

30
Dose response the pharmacokineticists version
31
Dose ResponseThe Statisticians Version
Response
This is what the simple carry-over model implies
Dose
32
The Models Which use Simple Carry-over are
Inconsistent
Consider a factorial cross-over in four periods
comparing A, B and the combination of A and B to
placebo. We can represent the four treatments by
, A, B and AB. Suppose we consider a
patient who has received the sequence AB A
B. A standard parameterisation for treatment and
carry-over would be as in the following table.
33

Paramaterisation of a factorial
cross-over
Period
34
The Rhinoceros
The rhinoceros has a kind heart, if you doubt it
heres the proof That thing on his nose is for
taking stones out of a horses hoof He seldom
ever meets a horse, it is this that makes him
sad When he does, then it hasnt a stone in its
hoof But he would, if he did and it had
Flanders and
Swann
35
The Phoenix Bioequivalence Trials

• Analysed by DAngelo Potvin
• 20 drug classes
• 1989-1999
• 12 or more subjects
• 96 three period designs
• 324 two period designs

36
Three Treatment Designs P-Values for Carry-Over
37
Two Treatment Designs
38
Test of Uniformity of P-Values
39
Conclusions

• Distribution of P-values uniform
• no evidence of carry-over
• Carry-over a priori implausible
• presence testable by assay
• No point is testing for it
• leads to bias
• Or adjusting for it
• increased variance

40
Do Bayesians do Better?

• In principle the Bayesian approach ought to allow
  us to be more flexible about nuisance parameters
  such as carry-over
• However, the Bayesian track record is not
  impressive here
• Realistic models have not been employed

41
Cross-over trial in Eneuresis Two treatment
periods of 14 days each Treatment effect
significant if carry-over not fitted 2.037 (
0.768, 3.306) Treatment effect not significant
if carry-over fitted 0.451 (-2.272, 3.174)
1. Hills, M, Armitage, P. The two-period
cross-over clinical trial, British Journal of
Clinical Pharmacology 1979 8 7-20.
42
Identical uninformative prior placed on
carry-over as for treatment NB Parameterisation
here means that values of ? need to be doubled to
compare to conventional contrasts
43
Identical Priors for Treatment and Carryover?

• Patients treated repeatedly during trial
• Fourteen day treatment period
• Average time to last treatment plausibly 4 hours
• Average time to previous treatment seven days
• Saying that it is just as likely that carry-over
  could be greater than treatment is not coherent
• In any case the two cannot be independent
• Is negative carry-over as likely as positive
  carry-over?

44
So What are Acceptable Models for Carry-over?

• Ignoring carry-over altogether (not allowing for
  it because one believes one has taken adequate
  steps to eliminate it)
• This is always a reasonable strategy
• Using an integrated pharmacokinetic
  pharmacodynamic model (Sheiner et al, 1991)
• This may work for dose-finding trials
• Very difficult to implement where more than one
  molecule is involved

45
The Sheiner model
PD dose response PK model for
dose-concentration as a consequence of previous
dosing history
Steady state concentration for patient i in
period l
46
(No Transcript)
47
(No Transcript)
48
(No Transcript)
49
The difference between mathematical and applied
statistics is that the former is full of lemmas
whereas the latter is full of dilemmas
50
Advice for Design-Theoreticians

• Resist the temptation to give advice if you are
  unfamiliar with the application area
• Seek collaborators
• Ground your models in pharmacology
• Remember that the goal is good medicine not
  elegant mathematics
• Dont defend the indefensible

51
Advice for biostatisticians

• Remember that design theoreticians have many
  powerful results
• Its just conceivable that some of them may even
  be useful

52
(No Transcript)
53
References

• Senn, S.J., Is the 'simple carry-over' model
  useful?
• published erratum appears in Statistics in
  Medicine 1992 Sep 1511(12)1619.
• Statistics in Medicine, 1992. 11(6) p. 715-26.
• Senn, S.J., The AB/BA cross-over how to perform
  the two-stage analysis
• if you can't be persuaded that you shouldn't.,
  in Liber Amicorum Roel van Strik,
• B. Hansen and M. de Ridder, Editors. 1996,
  Erasmus University
• Rotterdam. p. 93-100.
• Senn, S.J., Cross-over Trials in Clinical
  Research. Second ed. 2002, Chichester Wiley.
• Senn, S.J., Statistical Issues in Drug
  Development. Statistics in Practice, ed. V.
  Barnett. 2007, Chichester John Wiley.