Recent Data Quality Initiatives at CIHI

1
Recent Data Quality Initiatives at CIHI

• Heather Richards
• Program Lead, Data Quality Department
• OHIMA Conference
• May 5, 2006

2
Topics

• Reabstraction Studies
• Case Costing Reabstraction Study
• NACRS Reabstraction Study
• Future DAD Reabstraction Studies
• Initiatives to Address Findings

3
What is a reabstraction study?

• Trained coders go back to the source documents at
  the hospital and abstract selected charts a
  second time.
• Reabstracted information is then compared to
  the originally abstracted data.

4
Why do reabstraction studies?

• To determine the degree of measurement error
  present in a database, and what causes it to be
  there.
• Measurement error
• is the difference between the reported value and
  the true value.
• affects precision and estimates
• Need to know when using data for analysis

5
Measurement error

• Components of measurement error
• Recording error - number of times a data element
  is coded incorrectly
• Objective data elements (e.g. date of birth)
• Bias - systematic errors in measurement
• Results in misleading estimates (e.g. default
  codes)
• Consistency variation in coding over repeated
  measurements
• Subjective data elements (e.g. level of pain)
• Measurement instruments (e.g. weighting scale)

6
Measurement error

• Causes of measurement error
• Measurement instrument
• software vendors, tools used by physicians
• Response error
• Misunderstanding, reporting errors
• Resulting from physician documentation, coding
  guidelines, coder education
• Method of data collection
• Each time the data is touched increases
  measurement error

7
Case Costing Reabstraction Study
8
Case Costing Reabstraction Study

• Background
• 18 Ontario case costing facilities
• CIHI, MOHLTC, CHIM
• Discharge Abstract Database, 20022003, 20032004
• 14,500 discharges reabstracted by 24 health
  information professionals (reabstractors)
• Final report posted on http//www.health.gov.on.c
  a/transformation/providers/information/data_qualit
  y/reabstraction_study.pdf

9
Case Costing Reabstraction Study

• Studied
• Original and reabstracted codes for diagnoses,
  interventions, and non-medical data elements
• Impact of coding differences on inpatient
  grouping methodology outputs (CMG, MCC, Plx, RIW,
  ELOS)
• Variability of the coding between reabstractors
• Reasons for the observed discrepancies

10
Case Costing Reabstraction Study

• 1. Findings for recording error
• High agreement rates for
• Non-medical data (97-100)
• Intervention codes (91 exact match)
• Diagnosis codes (85 exact match)
• Impact on inpatient grouping methodology outputs
• Some MCC, CMG changes

11
Case Costing Reabstraction Study

• Intervention Codes

12
Case Costing Reabstraction Study

• Diagnosis Codes

13
Case Costing Reabstraction Study

• 2. Findings for bias
• Conditions originally typed as significant were
  consistently reabstracted as secondary
• Inpatient grouping methodology outputs were
  systematically lower in value upon reabstraction
• Complexity level
• Expected length of stay
• Resource intensity weight

14
Case Costing Reabstraction Study

• Complexity Level

15
Case Costing Reabstraction Study

• 3. Findings for consistency
• Low agreement rates for
• Assignment of significance to diagnoses (65)
• Assignment of the Most Responsible Diagnosis
  (75)
• High variation between facilities in agreement
  rates for several data elements
• Even when accounting for case mix effect
• Inter-rater reliability
• Coder effect

16
Case Costing Reabstraction Study

• Most Responsible Diagnosis (coder effect and case
  mix effect)

17
Case Costing Reabstraction Study

• Assignment of Significance (coder effect and case
  mix effect)

18
Case Costing Reabstraction Study

• Findings for causes
• Issues related to the quality of the chart
  documentation was the largest contributor to the
  discrepancies.
• Inconsistencies in assigning significance to a
  condition were part due to chart documentation,
  and part due to the definition of significance.

19
NACRS Reabstraction Study
20
NACRS Reabstraction Study

• Background
• 15 Ontario emergency department facilities
• participation from each of the 14 LHINs
• CIHI, MOHLTC, CHIM
• NACRS Database, 20042005
• 9,000 discharges reabstracted (20 inter-rater)
• Out of Scope multiple contact records day
  surgery and clinic records.
• NEW Hospital Questionnaire !!

21
NACRS Reabstraction Study

• Studying
• Original and reabstracted codes for problems,
  interventions, and non-medical data elements
• Some other data elements (e.g. Helmet indicator)
• Impact of coding differences on CACS grouping
  methodology outputs (CACS cells, MAC, CACS
  weight, DPG, DPG weight)
• Variability of the coding between reabstractors
• Reasons for the observed discrepancies
• Hospital documentation and data quality processes
  via the questionnaire

22
NACRS Reabstraction Study

• Status Update
• Study planning and design complete
• Facilities selected and confirmed
• Reabstraction application in testing
• Recruitment underway
• Questionnaire ready for testing
• Schedule for data collection in progress
• Data processing and analysis in progress
• Shell of report created

23
Future DAD Reabstraction Studies
24
Future DAD Reabstraction Studies

• National focus
• Annual subset of data
• Inter-rater component
• Study design in progress

25
Initiatives to Address Findings
26
Initiatives to Address FindingsDischarge
Abstract Database

• CIHI initiatives
• Diagnosis Typing Project
• Changes to the Canadian Coding Standards
• CIHIs CMG grouping methodology
• MOHLTC initiatives
• Physician Documentation Expert Panel
• HIM Professional Practice eLearning and
  Assessment Tool
• On-line Communities of Practice for HIM
  professionals
• Coding Audit Tool for hospitals

27
Questions?