Unsafety: Making no mockery of honest ad-hockery

1
Unsafety Making no mockery of honest
ad-hockery

• Janet WittesStatistics Collaborative
• ASA/FDA 2005

2
Topic du jour
3
Sleight of tongue

• Game Remove safety and add virtue in two easy
  steps
• D S M C
• _ _ _
• _ _ _ _

4
Sleight of tongue

• Game Remove safety and add virtue in two easy
  steps
• D S M C
• D M C
• _ _ _ _

5
Sleight of tongue

• Game Remove safety and add virtue in two easy
  steps
• D S M C
• D M C
• I D M C

6
Efficacy

• Stop for overwhelming efficacy
• ?0.05 power 90 four looks
• Probability of stopping earlygt70
• Early stop estimate pulls toward null

7
Our current approaches

• Prespecify
• Classify precisely
• Give lots of data
• Rely on mechanism

8
Divide and (un)conquer (e.g. Neuropathy)
Event T C
Neuropathic pain 1 0
Neuropathy 1 0
Neuropathy NOS 5 2
Neuropathy peripheral 2 0



9
e.g. Neuropathy
Event T C
Anosmia
..
Autonomic neuropathy 1 0

Cranial neuropathy 2 1


10
e.g. Neuropathy
Event T C

Parathesia 3 2
Parathesia NOS 4 0
Parathesia other 0 1

Peripheral motor neuropathy 6 0
Peripheral sensory neuropathy 3 2
11
Other examples

• Heart failure
• Separate near synonyms
• Allocate to heart and lung

12
Other examples

• Heart failure
• Separate near synonyms
• Allocate to heart and lung
• Bleeding distribute over body systems

13
Consider mechanism

• If you dont get the drug, you cant react to it
• Eschew ITT safety population
• Modified Daleys Rule censor early and often
• Dont collect extraneous information
• Appeal to statistical conservatism

14
Populations

• ITT
• mITT
• Safety one dose of study med
• ATP
• Etc.

15
e.g., Vioxx- short follow-up
16
Through 36 months
17
With denominators

• Bresalier et al., NEJM, Feb 2005

18
The denominators

• Mo Rx Po
• 0 1287 1299
• 12 1129 1195
• 18 1057 1156
• 24 938 1042
• 30 896 1001
• 36 727 835

19
No data dredging

• We test hypotheses
• Too many type 1 errors if we dredge

20
What we typically present

• Current behavior
• Same tables for interim final analyses
• Long complete listings
• What we should do
• Interim data are different from final
• Presenting too much dulls the mind

21
Sentinel events

• Single event (e.g., death in a vaccine trial)
• Several events (e.g., 3 retinal thromboses)
• Sentinel event rate (e.g., WHI)
• Lachenbruch and Wittes, SIM, to appear

22
How to handle

• Identify sentinel event
• Establish statl method for future events
• Need reasonable power
• Type I error rate gt 1-sided 0.025

23
Methods

• Individual
• Number of non-events until the kth event
• Negative binomial
• SPRT
• Time to the next (or kth) event (?-distn)
• Rate event rate in future patients
• Normal
• Poisson

24
Pitfalls

• Time is subtle
• Power is low
• Censoring is tricky

25
Pull-up

• Safety hypothesis
• E.g., Cox-2 leads to 2 fold increase in MI etc.
• Design stop early if you reject
• Estimated relative risk must be pulled toward 2
• Insight from Joe Heyse

26
The option

• Respect PI Adjudicate adverse events
• Precision Reclassify, reorganize
• Mechanism Be an empiricist
• Dredging Use sentinel events