Simulating chart audits using an immunization registry

1
Simulating chart audits using an immunization
registry

• M. Irigoyen, S. Findley, S. Chen,F. Chimkin, O.
  Peña
• Northern Manhattan Immunization Partnership
• Columbia University, New York, NY

2
Background

• Chart audits are the gold standard for assessing
  practice immunization coverage but they are time
  and labor intensive
• Immunization registries offer the potential for
  reliable and timely practice assessment

3
Objective

• To simulate practice immunization chart audits
  using an immunization registry
• To compare registry-based coverage rates assessed
  by two methods universal assessment and random
  sample

4
Northern Manhattan Immunization Partnership (NMIP)

• CDC funded demonstration project in Northern
  Manhattan, New York City
• Conducted semi-annual CASA audits at 23 practices
  in Northern Manhattan
• Developed a regional registry to facilitate
  sharing of records among practices and expedite
  upload to city immunization registry

5
EzVAC Immunization Registry

• Established by NMIP to serve the community of
  Northern Manhattan
• Web-based and real time
• Launched March 1999
• Currently at 30 practices
• 120,000 children in the registry
• Regular uploads and downloads to NY Citywide
  Immunization Registry

6
EzVAC Registry
EzVAC Registry
School Based Clinics
New York Presbyterian Hospital Ambulatory Care
Network
Community Providers
NYC DOH Citywide Immunization Registry
7
Sites Participating in the Audit Simulation

• 5 hospital-affiliated practices in Northern
  Manhattan, NYC
• Study population Latino, Low-income children,
  85 Medicaid
• Assessments Oct 01 and April 02, 2 years after
  registry launch
• Registry immunization capture rate 98

8
Comparison of Practice Coverage Assessments All
cases versus Sample of cases

• Universal Assessment Advantage that all eligible
  children with records in the registry are
  included, but can be time consuming
• Random sample following Chart Audit (CASA)
  procedures Includes only a random sample of
  children, minimizing run times

9
Criteria for Including Children in the Assessment
Study

• Children ages 6 - 35 months at time of audit
  (October 2001 and April 2002)
• One or more visits to the practice
• At least one immunization record in the registry

10
Immunization Outcome

• DTaPPolioMMRHibHepB (43133) age
  -appropriate immunization coverage rate
• Practice coverage of children with
  age-appropriate immunizations, as of the date of
  the assessment

11
Steps in the Universal Assessment

• Queried registry for eligible children in each of
  5 practices (n 16,021)
• Calculated proportion of children with
  age-appropriate immunizations for 3 age groups
  6-11m, 12-23m 24-35m
• Practice assessments prepared for October 2001
  and April 2002 by age group and practice site

12
Steps in Random Sampling Assessment

• Used same registry queries to prepare lists of
  eligible children by age group, in each of 5
  practices
• Sampling proportional to size of each age group,
  with sampling fraction calculated to yield 345
  children (300 15 margin) per practice

13
Steps in Random Sampling Assessment

• Used SPPS random selection procedure to generate
  list of children
• Practice assessments prepared for October 2001
  and April 2002 by age group and practice site

14
Case Sampling From 5 Practices (April 2002)
15
Comparison of Coverage Rates (6-11 month-olds)
T tests show no significant differences by type
of assessment
16
Comparison of Coverage Rates (12-23 month-olds)
T tests show no significant differences by type
of assessment
17
Comparison of Coverage Rates (24-35 month-olds)
T tests show no significant differences by type
of assessment
18
Conclusion

• When using immunization registries for practice
  coverage assessments, random sampling using CASA
  methodologies yield equivalent results to
  universal evaluation

19
Implications

• When there are time considerations, e.g. run
  time, the random sample assessment methodology
  can be used to generate practice assessments
• This methodology could be applied to generate
  HEDIS reports and city or state-wide assessments.