The relationship between immunization registry operational status and accuracy of vaccination covera

1
The relationship between immunization registry
operational status and accuracy of vaccination
coverage estimates among children aged 19-35
months

• Diana Bartlett1, Gary Urquhart2, and Robb Linkins2

1 Association of Schools of Public Health 2
Centers for Disease Control and Prevention
2
Overview

• Background
• CDC/NIP 1999 Immunization Registry Annual Report
• National Immunization Survey (NIS)
• Study objectives
• Methods
• Results
• Recommendations

3
Background

• The CDC/NIP 1999 Immunization Registry Annual
  Report provides self-reported registry
  development data from 62 projects
• The report measures child and health provider
  enrollment, possession of 12 core registry
  functional attributes, and vaccination coverage

4
Background

• Registries are a potential source of estimates of
  population-based childhood vaccination coverage
• Currently, the NIS is the only source of reliable
  estimates of national/state/urban area infant
  vaccination coverage that can be compared across
  sites

5
Objectives

• Evaluate whether there is agreement between
  immunization coverage based on registry data and
  a gold standard
• Evaluate whether agreement varies by operational
  status
• Describe another means of assessing registry
  functionality through accuracy of vaccination
  coverage

6
Methods

• CDC/NIP 1999 Immunization Registry Annual Report
• Self-reported data current as of April 1999
• 26 projects assessed 431 coverage of infants
  19-35 months
• Projects grouped by registry saturation ( of
  eligible children enrolled in registry) and
  registry functionality (based on 12 core
  functions) into high, medium, and low operational
  status

7
Methods

• 12 core functions include
• Electronically store NVAC data
• Establish registry record within 2 months of
  birth for each child in catchment area
• Enable providers to retrieve immunization records
  at the time of encounter with child
• Ensure that providers submit information within
  one month of administering vaccine

8
Methods

• 12 core functions include
• Have confidentiality and security measures in
  place to protect medical information
• Ability to recover lost data
• Exchange immunization using HL7 standards
• Identify needed vaccines when child presents for
  a vaccination
• Identify patients overdue for vaccinations

9
Methods

• 12 core functions include
• Automatically produce vaccination coverage
  reports by providers, age group, and geographic
  area
• Produce authorized immunization records
• Consolidate immunization records from all health
  providers to decrease deduplication and optimize
  data accuracy

10
Methods

• Registry projects assessing 431 vaccination
  estimates included
• 22 states AL, AK, AZ, DE, IL, KS, MA, MI, MO,
  NE, NH, NJ, ND, OH, OK, RI, SC, SD, TN, TX, UT,
  WA
• 4 district/city registries the District of
  Columbia, New York City, Philadelphia, and
  San Antonio

11
Methods

• High operational status 80 in both registry
  saturation and functionality (Arizona and San
  Antonio)
• Medium operational status 50-79 (22
  projects)
• Low operational status lt50 in both registry
  saturation and functionality (South
  Carolina and Utah)

12
Methods

• NIS is a random-digit-dialed telephone survey
  that uses parental report and a provider record
  check to assess vaccination coverage of infants
  aged 19-35 months
• NIS vaccination rates from July 1998-June 1999
  used

13
Methods

• Registry project 431 population coverage rates
  compared with NIS vaccination rates for project
  areas by operational status
• Registry coverage infants with 431 coverage
  in the registry/ of infants in the state
• Vaccination rate differences analyzed using SAS
  6.12 and the Wilcoxon rank sum test (equality of
  medians test)

14
Results

• Registry 431 vaccination coverage estimates
  varied widely
• High operational status 58.9-89.1
• Medium operational status 1.5-72.3
• Low operational status 8.9-18.5

15
Results
16
Results

• The median absolute differences between NIS and
  registry estimates were
• 12.6 for high operational status projects
• N2, Range 11.1-14.1
• 65.7 for medium operational status
• N22, Range6.7-87.5
• 68.8 for low operational status projects
• N2, Range 66.5-71.1

17
Conclusions

• Registries with a high operational status have
  closer agreement between vaccination coverage
  estimates in the NIS and registry data
• Equality of medians test (Wilcoxon rank test had
  a p-value of 0.1, suggesting that medians might
  not be equal)
• Despite low operational status, population
  vaccination coverage is not compromised

18
Conclusions

• Registries may not yet have sufficient
  functionality and enrollment to accurately assess
  population-based vaccination coverage
• A registry with at least 80 saturation and
  functionality tend to provide coverage estimates
  closer to those of the NIS than registries will a
  lesser operational status

19
Limitations

• The sample sizes are small and not distributed
  evenly among the operational status groups
• Registries that have low saturation rates may
  provide less accurate estimates of vaccination
  coverage in the population

20
Recommendations

• Vaccination coverage estimate agreement should be
  added as a measure of registry functionality
• Reporting periods in registry saturation and
  functionality assessment should be standardized

21
Recommendations

• Continue monitoring registry data quality and
  completeness to discern the reasons for the
  NIS-registry data vaccination estimate
  differences
• Until registries have high functionality and data
  quality, policy makers should be cautious when
  interpreting registry data in decision making