Classification of Emergency Department Syndromic Data for Seasonal Influenza Surveillance

1
Classification of Emergency Department Syndromic
Data for Seasonal Influenza Surveillance
Jacqueline Coberly Lang Hung Howard Burkom Wayne
Loschen Joseph Lombardo

• Atar Baer
• Jeff Duchin

International Society for Disease Surveillance
Sixth Annual Conference Indianapolis, October 2007
2
Objectives

• To evaluate how well our emergency department
  (ED) data captured the 2006-07 flu season
  compared with laboratory data (gold standard)
• To evaluate several different syndromic
  classifications of ILI and identify the best
  definition for describing seasonal flu

3
Data Sources
Rapid antigen (RA)
Influenza sentinel providers (ISP)
Public Health
Syndromic surveillance emergency department data
4
Data Sources
Rapid antigen (RA)
Influenza sentinel providers (ISP)
Public Health
Syndromic surveillance emergency department data
5
Data Sources
Rapid antigen (RA)
Influenza sentinel providers (ISP)
Public Health
Syndromic surveillance emergency department data
6
Data Sources
Rapid antigen (RA)
Influenza sentinel providers (ISP)
Public Health
Syndromic surveillance emergency department data
7
Chief Complaint Classification

• We classify chief complaints into syndromic
  categories using a SAS-based coder originally
  developed by NYC
• Code instructs SAS to look for key words or
  phrases that can either be included or excluded
  from each syndromic category
• e.g., include flu but exclude fluid, flus,
  flut, flux, stomach flu, flu shot, etc.
• Created two sets of definitions of ILI
• One set restricted to chief complaints
• Another set included diagnoses (if available)

8
Syndrome Definitions

• Three definitions of ILI within each set
• Specific mention of flu (FLU)
• Specific mention of fever (FEVER)
• A broader ILI definition (ILI)
• Flu OR
• Fever plus cough OR
• Fever plus sore throat OR
• Sepsis, bronchiolitis, bacteremia, or pneumonia

9
Signal-to-Noise Calculations (SNR)

• Study period defined based on availability of lab
  data (10/7/06-4/28/07)
• Flu season Period in which the weekly count of
  positive ISP specimens was greater than the mean
  of positive specimens for the study period
  (1/13/07-3/10/07)
• Signal level Mean during the flu period minus
  the mean during the non-flu period
• Noise level Standard error during the non-flu
  period
• Calculated the ratio of signal to noise for each
  ILI classification

Reference Marsden-Haug N, Foster VB, Gould PL,
Elbert E, Wang H, Pavlin JA. Code-based syndromic
surveillance for influenzalike illness by
International Classification of Diseases, Ninth
Revision. Emerging Infectious Diseases 13(2)
207-216.
10
Correlation Analysis

• We examined the association between syndromic and
  lab data through correlation analysis
• Correlation coefficients were also calculated by
  lagging the ISP and RA positive specimens forward
  and backward in time by 1-week increments to
  examine the timeliness of the signals

Reference Marsden-Haug N, Foster VB, Gould PL,
Elbert E, Wang H, Pavlin JA. Code-based syndromic
surveillance for influenzalike illness by
International Classification of Diseases, Ninth
Revision. Emerging Infectious Diseases 13(2)
207-216.
11
Algorithm Analysis

• Does the choice of ILI classification affect
  algorithm performance?
• Flu season Period in which the weekly count of
  positive ISP specimens was greater than the mean
  of positive specimens for the study period
  (1/13/07-3/10/07)
• Examined sensitivity and specificity of each of
  the 6 ILI classifications using several
  algorithms
• Regression/EWMA/Poisson Switch
• C2
• C3

12
Sensitivity and SpecificityCalculations

• Sensitivity
• Weeks w/gt1 Alert in Flu-season
• Weeks in Flu-Season
• Specificity
• Weeks w/no Alerts in Not-Flu Season
• Weeks in Not-Flu Season

13
Lab Data Compared with FEVER Syndromic Category
14
Lab Data Compared with FLU Syndromic Category
15
Lab Data Compared with ILI Syndromic Category
16
Signal-to-Noise Results
17
SNRs by Age Group
18
Correlation Results
19
Lagged Correlation with ISP
Reference Marsden-Haug N, Foster VB, Gould PL,
Elbert E, Wang H, Pavlin JA. Code-based syndromic
surveillance for influenzalike illness by
International Classification of Diseases, Ninth
Revision. Emerging Infectious Diseases 13(2)
207-216.
20
Lagged Correlation with RA
Reference Marsden-Haug N, Foster VB, Gould PL,
Elbert E, Wang H, Pavlin JA. Code-based syndromic
surveillance for influenzalike illness by
International Classification of Diseases, Ninth
Revision. Emerging Infectious Diseases 13(2)
207-216.
21
Algorithm EvaluationCaveats

• The results of daily detection algorithms were
  cumulated in order to compare it to weekly gold
  standard data
• Assumes the gold standard lab test data and the
  syndromic data are measuring disease status in
  the same underlying population
• Comparing sensitivity/specificity of the ILI
  definitions, not comparing the algorithms

22
Regression/EWMA/Poisson Switch Sensitivity
Specificity
Sensitivity
Specificity
23
CDC C2 Algorithm Sensitivity Specificity
Sensitivity
Specificity
24
CDC C3 Algorithm Sensitivity Specificity
Sensitivity
Specificity
25
Summary

• The flu classification had the highest SNR
• The febrile syndrome classification had the
  closest correlation with the laboratory data
• The use of diagnostic coding did not consistently
  improve these measures compared with using chief
  complaint alone
• Sensitivity and specificity varied by definition
  of flu season, flu definition, and detector,
  making comparisons by detector problematic

26
Limitations

• Many classifications of ILI can be evaluated
  other than those included in this study other
  chief complaint classifiers may yield different
  results
• The study is based on a single flu season with
  uncertain epidemic dates, and is not suitable for
  algorithm evaluation
• The gold standard is imperfect because
  laboratory orders may spuriously increase when
  healthcare providers are informed that the
  influenza season has begun

27
Next Steps

• Extend the analysis to previous flu seasons and
  see if the results are consistent across time
• Examine results in relation to other indicators
  of flu, including school absenteeism, EMS data,
  and pneumonia/influenza mortality data
• Evaluate additional classifications of flu

28
Acknowledgements

• This presentation was supported in part by Grant
  Number P01 CD000270 from the Centers for Disease
  Control and Prevention. Its contents are solely
  the responsibility of the authors and do not
  necessarily represent the official views of CDC.

29
References

• The calculation of SNRs and correlation analyses
  were based on methods published by
• Marsden-Haug N, Foster VB, Gould PL, Elbert E,
  Wang H, Pavlin JA. Code-based syndromic
  surveillance for influenzalike illness by
  International Classification of Diseases, Ninth
  Revision. Emerging Infectious Diseases 13(2)
  207-216.

30
More Information

• Atar Baer
• Public Health Seattle King County
• 206-263-8154
• atar.baer_at_kingcounty.gov
• Jacki Coberly
• Johns Hopkins University Applied Physics Lab
• [phone]
• Jacqueline.Coberly_at_jhuapl.edu

31
(No Transcript)