A Bayesian Approach to Parallelism Testing in Bioassay

1
A Bayesian Approach toParallelism Testing in
Bioassay

• Steven Novick, GlaxoSmithKline
• Harry Yang, MedImmune LLC

2
Manuscript co-author

• John Peterson, Director of statistics,
  GlaxoSmithKline

3
(No Transcript)
4
Warm up exercise
5
(No Transcript)
6
When are two lines parallel?

• Parallel Being everywhere equidistant and not
  intersecting
• Slope
• Horizontal shift places one line on top of the
  other.

7
When are two curves parallel?

• Parallel Being everywhere equidistant and not
  intersecting?
• Can you tell by checking model parameters?
• Horizontal shift places one curve on top of the
  other.

8
Where is parallelism important?

• Gottschalk and Dunn (2005)
• Determine if biological response(s) to two
  substances are similar
• Determine if two different biological
  environments will give similar doseresponse
  curves to the same substance.
• Compound screening
• Assay development / optimization
• Bioassay standard curve

9
Screening for compound similar to gold-standard

• E.g., seeking new HIV compound with AZT-like
  efficacy, but different viral-mutation profile.
• Desire for dose-response curves to be parallel.

10
Change to assay procedure

• E.g., change from fresh to frozen cells.
• Want to provide same assay signal window.
• Desire for control curves to be parallel.

11
Assess validity of bioassayused for relative
potency

• Dilution must be parallel to original.
• Callahan and Sajjadi (2003)

Original Dilution
12
Replacing biological materials used in standard
curve

• E.g., ELISA (enzyme-linked immunosorbent) assay
  to measure protein expression.
• Recombinant proteins used to make standard curve.
• Testing clinical sample
• New lot of recombinant proteins for standards.
• Check curves are parallel
• Calibrate new curve to match the old curve.

13

• Potency often determined relative to a reference
  standard such as ratio of EC50
• Only meaningful if test sample behaves as a
  dilution or concentration of reference standard
• Testing parallelism is required by revised USP
  Chapter lt111gt and European Pharmacopeia

14
Linear Two lots of Protein A
Estimated Concentrations Lot 2 is 1.4-fold higher
than Lot 1
Log10 Signal
?0.14
15
If the lines are parallel

• Shift Lot 2 line to the left by a calibration
  constant ?.
• ? is log relative potency of Lot 2.

Draft USP lt1034gt 2010
16
Testing for Parallelism in bioassay
17
Typical experimental design
Serial dilutions of each lot Several
replicates Fit on single plate (no plate effect)
Log10 Signal
18
Tests for parallel curvesLinear model

• Hauck et al, 2005 Gottschalk and Dunn, 2005
• H0 b1 b2
• H1 b1 ? b2

ANOVA T-test F goodness of fit test
?2 goodness of fit test
May lack power with small sample size Might be
too powerful for large sample size
19
A better idea

• Callahan and Sajjadi 2003 Hauck et al. 2005
• Slopes are equivalent
• H0 b1 - b2 ?
• H1 b1 - b2 lt ?

20
Nonlinear Two lots of protein B
Estimated Concentrations Lot 2 is 1.6-fold higher
than Lot 1
?0.21
21
Tests for parallel curves4-parameter logistic
(FPL) model

• Jonkman and Sidik 2009
• F-test goodness of fit statistic
• H0 A1 A2 and B1 B2 and D1 D2
• H1 At least one parameter not equal

May lack power with small sample size Might be
too powerful for large sample size
22

• Calahan and Sajjadi 2003 Hauck et al. 2005
  Jonkman and Sidik (2009)
• Equivalence test for each parameter
  intersection-union test
• H0 ?1 / ?2 ? ?1 or ?1 / ?2 ? ?2
• H1 ?1 lt?1 / ?2 lt ?2
• ?i Ai, Bi, Di , i1,2

1. Equiv. of params does not provide assurance of
   parallelism (except for linear)
2. May lack power with small sample size
3. Forces a hyper-rectangular acceptance region

23
Our proposal

• Parallel Equivalence

24
Definition of Parallel

• Two curves and are
  parallel if there exists a real number ? such
  that for all x.

25
Definition of Parallel Equivalence

• Two curves and are
  parallel equivalent if there exists a real number
  ? such that
• for all x ? xL, xU.

26

• It follows that two curves are parallel
  equivalent if there exists a real number ? such
  that
• It also follows that

27
Are these two lines parallel enough when xL lt x lt
xU ?
lt ??
28
Linear-model solution

• Linear model

Just check the endpoints
29
Parallel equivalence slope equivalence
wlog
Same as testing b1 - b2 lt ?
30
Parallel Equivalence

• FPL model
• No closed-form solution.
• Simple two-dimensional minimax procedure.

31
Are these two curves parallel enough when xL lt x
lt xU ?
lt ??
32
Testing for parallel equivalence

• H0
• H1
• Proposed metric (Bayesian posterior probability)

33
Computing the Bayesian posterior probability

• For each curve, assume

Data distribution
i 1, 2 reference or sample j 1, 2, , N
observations
Prior distribution
Posterior distribution proportional to
34

• Draw a random sample of the ?i of size K from the
  posterior distribution (e.g., using WinBugs).
• The posterior probability
• is estimated by the proportion (out of K) that
  the posterior distribution of

35

• 0.14 (100.141.4-fold shift)
• 0.07 ? 90 probability to call parallel
  equivalent

36

• 0.33 (100.332.15-fold shift)
• 0.52 ? 90 probability to call parallel
  equivalent

37
Simulation FPL ModelBased on protein B data
38
Simulation FPL model

• Similar to Protein B protein-chip data.
• Concentrations (9-point curve 0)
• 0, 102(100), 102.5625, 103.125, ,
  106.5(3,200,000)
• Three replicates
• xL 3.5log10(3162) xU 5log10(100,000)
• d 0.2.
• ? 0.02, 0.04, 0.11, and 0.21 (CV 5, 10, 25,
  50)
• For each Monte Carlo run, I computed

39
Scenario Original Original Original Original New New New New Max Diff
Scenario A1 B1 C1 D1 A2 B2 C2 D2 Max Diff
1 2 4.5 4 -1 2 4.5 4 -1 0
2 2 4.5 4 -1 2 4.5 4.7 -1 0
3 2 4.5 4 -1 2.445 4.945 4.7 -1 0.15
4 2 4.5 4 -1 2 4.5 4.7 -1.312 0.15
5 2 4.5 4 -1 2.61 5.11 4.7 -1 ?0.20
6 2 4.5 4 -1 2.25 4.75 4.7 -1.5 0.30
5,000 Monte Carlo Replicates
40
Example data (CV10)
Diff 0 0
0.15 0.15
0.20 0.30
41
Diff 0 0
0.15 0.15
0.20 0.30
42
Scenario Frequentist statistical power Frequentist statistical power Frequentist statistical power Frequentist statistical power
Scenario Max Diff CV5 CV10 CV25 CV50
1 0 1.00 1.00 1.00 0.50
2 0 1.00 1.00 1.00 0.50
3 0.15 (shape 1) 1.00 0.99 0.28 lt 0.01
4 0.15 (shape 2) 1.00 0.88 0.36 0.14
5 d 0.20 0.10 0.02 lt 0.01 lt 0.01
6 0.30 0.00 0.00 lt 0.01 lt 0.01
43
Summary

• Straight-forward and simple test method to assess
  parallelism.
• Yields the log-relative potency factor.
• Easily extended.

44
Extensions

• Instead of f(?, x), could use
• ?f(?, x) / ? x instantaneous slope
• f-1(?, y) estimated concentrations

45
Whats next?

• Head-to-head comparison with existing methods
• Choosing test level and ?, possibly based on ROC
  curve? Harry Yang paper
• Guidance for prior distribution of ?1 and ?2.

46
References

1. Callahan, J. D. and Sajjadi, N. C. (2003),
   Testing the Null Hypothesis for a Specified
   Difference - The Right Way to Test for
   Parallelism, Bioprocessing Journal Mar/Apr 1-6.
2. Gottschalk P.J. and Dunn J.R. (2005), Measuring
   Parallelism, Linearity, and Relative Potency in
   Bioassay and Immunoassay Data, Journal of
   Biopharmaceutical Statistics, 15 3, 437 - 463.
3. Hauck W.W., Capen R.C., Callahan J.D., Muth
   J.E.D., Hsu H., Lansky D., Sajjadi N.C., Seaver
   S.S., Singer R.R. and Weisman D. (2005),
   Assessing parallelism prior to determining
   relative potency, Journal of pharmaceutical
   science and technology, 59, 127-137.
4. Jonkman J and Sidik K (2009), Equivalence
   Testing for Parallelism in the Four- Parameter
   Logistic Model, Journal of Biopharmaceutical
   Statistics, 19 5, 818 - 837.

47
Thank you!

• Questions?