Active postmarketing surveillance for vaccine adverse events: The experience of the Vaccine Safety D

1
Active post-marketing surveillance for vaccine
adverse events The experience of the Vaccine
Safety Datalink
Katherine Yih, PhD, MPH Harvard Medical School
and Harvard Pilgrim Health Care
2
Why we need active post-marketing surveillance

• Rare adverse events may be impossible to detect
  in pre-licensure studies
• Drawbacks of passive surveillance systems (e.g.,
  the Vaccine Adverse Event Reporting System)
• Underreporting
• Reporting bias
• No denominators (so no calculation or comparison
  of rates in vaccinated vs. unvaccinated possible)
• Safety studies using traditional approaches can
  take months to years

3
Outline

• VSD background
• Active surveillance for vaccine safety in VSD
• Signals
• False signals
• True signal
• Monitoring influenza vaccine safety
• Conclusions

4
Vaccine Safety Datalink (VSD)

• Collaboration between CDC and 8 managed care
  organizations
• Data annually captured from 8.8 million members
  (2.9 of US population)

Group Health Cooperative
Northwest Kaiser Permanente
HealthPartners
Harvard Pilgrim
Marshfield Clinic
No. CA Kaiser Permanente
Kaiser Permanente Colorado
So. CA Kaiser Permanente
CDC
5
VSD data
Health Outcomes (Hospital) (Emergency
Dept) (Outpatient)
Patient Characteristics (Birth Certificate) (Censu
s / Geocode)
Vaccination Records
Linked by Study IDs Data are linked and kept at
each site, not at CDC
6
VSD features enabling active surveillance

• Population-basedsupplements VAERS, a passive
  surveillance system, where underreporting and
  lack of denominators are problems
• Fastdata updated on weekly basis
• Good data qualitydata quality checked
  frequently, feedback goes to data managers at
  sites

7
Vaccines for which VSD has conducted active
surveillance

• Adolescent and adult
• MCV4 (Menactra)
• Tdap (Adacel and Boostrix)
• HPV (Gardasil)
• Influenza (TIV and LAIV)
• Childhood
• MMRV (ProQuad)
• Rotavirus (RotaTeq)
• Influenza (TIV and LAIV)

8
Basics of active vaccine safety surveillance in
VSD

• For each vaccine, choose specific outcomes to
  monitor (5-10)
• Hypothesis-testing, not data-mining
• Each week, evaluate the number of outcomes in
  vaccinated persons
• Compare it to the expected number of outcomes
  based on a comparison group

9
Choosing outcomes

• 1. Select outcomes based on plausibility
• Pre-licensure data
• Known biologic properties of the vaccine
• VAERS reports
• Literature on this or similar vaccines
• 2. Additional criteria
• Clinically well-defined
• e.g., Guillain-Barré syndrome rather than
  neurologic problems
• Acute-onset
• Serious
• Relatively uncommon

10
Example Outcomes, HPV study

• Guillain-Barré syndrome
• Appendicitis
• Stroke
• Venous thromboembolism
• Seizures
• Syncope
• Allergic reactions

11
Historical comparison method

• Uses incidence rates from historical data
• Advantage Allows earlier recognition that a
  small number of excess cases is unusual
• Limitation Background rates may be uncertain,
  vary over time, or hold potential for confounding

12
Concurrent comparison method

• Uses matched controls, e.g., visits for other
  vaccines or just preventive care
• Advantage Avoids false signaling or missed
  signals due to error/trends/ confounding in
  historical data
• Limitations
• Need to define an appropriate control group not
  simple!
• Vaccines may be adopted rapidly, leaving few
  controls

13
Sequential analysis methods

• Each week, our analysis includes data from all
  previous weeks
• Problem Repeated testing increases the chance
  of false-positive results
• Need to adjust for this statistically
• Solution Sequential analysis

14
Maximized Sequential Probability Ratio Testing
(maxSPRT)1

• Refinement of a classical statistical method2
• Null hypothesis no excess risk
• Alternative hypothesis increase in risk
• Test statistic log likelihood ratio (LLR)
• Depends on numbers of events in exposed vs.
  unexposed observation time
• If critical value of LLR exceeded, potential
  association

• Kulldorff M et al. Working paper. Available at
  www.dacp.org/faculty_Kulldorff.html.
• 2. Wald A. Annals of Mathematical Statistics,
  16117186, 1945.

15
Sequential analysis methods

• Method to be used for comparison with
• historical data, large sample size
• - MaxSPRT, Poisson data
• historical data, small sample size
• - Conditional MaxSPRT, takes uncertainty in
  estimated historical rates into account
• concurrent matched controls
• - Flexible exact sequential analysis
• self-controls
• - MaxSPRT, binomial data

16
Example Rotashield vaccine and intussusception
(retrospective analysis)
Vaccine suspended
Vaccine licensed Aug 98 By Jul 99, 15 reports to
VAERS
Withdrawn
1999
17
Example Rotashield vaccine and intussusception
(retrospective analysis)
Vaccine suspended
Vaccine licensed Aug 98 15 VAERs reports through
Jul 99
Withdrawn
MaxSPRT analysis would have signaled in May 1999
Log likelihood ratio
Critical value 3.3
1999
18
SIGNAL ? ASSOCIATION ? CAUSALITY
19
Signals observed

• 11 signals observed in 4 studies
• Only 1 true association
• Example of false positive ?

20
HPV and appendicitis(Age group 18-26 years)

• When surveillance started, HPV had been in use
  for several months
• 1 case in Week 2 of data produced retrospective,
  transient signal
• At time of actual look, relative risk (RR) and
  LLR had decreased to null values
• Signal ascribed to chance

21
Data from 8/20/06 to 6/8/08 19 cases, 108,184
vaccines given (16.4 cases expected) RR1.2,
LLR0.1995.
22
Reasons for false signals

• Error in estimated background rates
• Very few cases in historical data
• Changes in true incidence or coding over time
• Inappropriate comparison group
• Confounding
• Miscoding of outcomes
• Chance

23
Signals sometimes do represent true increases in
risk
24
MMRV study background

• Age 12-23 months
• Outcomes monitored
• Ataxia --Thrombocytopenia
• Seizures --Arthritis
• Meningitis encephalitis --Allergic reactions
• Post-vaccination window 42 days
• Comparison group MMR visits in 2000-2006
• MMRV usage began 2006
• MMRV doses given as of March 2009 84,000

25
MMRV and seizures

• By 33rd week of data (Sept. 2006) 22 seizure
  diagnoses had occurred among 9,000 one-year-old
  MMRV recipients, vs. 10.7 expected from the
  historical incidence rate after MMR
• Relative risk 2.1
• LLR surpassed threshold ? signal

26
MMRV and seizures Log likelihood ratio and
relative risk over time
Relative risk
Log likelihood ratio
Relative risk
Log likelihood ratio
Critical value of LLR
27
Number of seizures among 12-23 mo. olds by day
after MMRV vaccination, 2006-2007
Number of Seizures
Days Post-MMRV Vaccine
(2/06-9/07, after 47,137 vaccine visits)
28
Seizure rate among 12-23 month olds by day after
vaccination, 2000-2008
Temporal Scan
29
MMRV and seizures in 1-year-olds

• 2-fold increased risk of seizure on Days 7-10
  after MMRV compared to MMRV, adjusting for age,
  VSD site, year, flu season
• For every 2,000 doses of MMRV given instead of
  MMRV, 1 additional seizure on Days 7-10
• Same result for febrile seizure after chart
  review and reanalysis
• Same result with (not yet chart-reviewed) data
  updated through 10/08 and alternative analyses

30
(No Transcript)
31
Studying influenza vaccine safety

• Cannot use usual comparison groups to study flu
  vaccine safety
• Vaccinated vs. unvaccinated
• Misclassification of vaccinated
• Confounding by indication
• Vaccinated (current) vs. vaccinated (past)
• Population recommended to receive vaccines
  changed over time
• Secular trends in potential adverse events

32
Influenza vaccine safety analytic approaches

• Self-controlled case series
• Compares risk of AEs in risk vs. control periods
  of current season
• Difference-in-difference
• Adjusts for risk vs. control periods of
  cumulative previous seasons

33
Status of adolescent and adult vaccine studies
34
Conclusions

• Sequential evaluation of accumulating data
  complements traditional, passive, adverse event
  reporting
• Makes best, early, use of routinely collected
  data
• Signals can provide early evidence of risk
• Lack of signals may provide rapid reassurance
  when assessing concerns that arise from another
  source

35
Upcoming adolescent/adult vaccines to be
monitored for safety after FDA licensure

• MCV4 (Menveo)
• HPV (Cervarix)
• Influenza A/H1N1 vaccine
• 2009-2010 influenza vaccine

36
Local VSD colleagues

• Rich Fox
• Sharon Greene
• Virginia Hinrichsen
• Martin Kulldorff
• Grace Lee
• Lingling Li

• Renny Li
• Tracy Lieu
• Rich Platt
• Ping Shi
• Irene Shui
• Ruihua Yin