Data for Outcomes Research

1
Data for Outcomes Research

• Andy Bindman MD
• Department of Medicine, Epidemiology and
  Biostatistics

2
What is Outcomes Research

• Studies of the quality of care as judged by
  patients outcomes
• IOM domains of quality
• Effectiveness
• Safety
• Timeliness
• Equity
• Efficiency
• Patient-Centered

3
Donabedian Model of Quality

• Structure Process Outcome

4
Donabedian Model of Quality

• Structure Process Outcome

Number of nurses per hospital bed Physicians per
capita
5
Donabedian Model of Quality

• Structure Process Outcome

Beta blocker following MI Immunizations
6
Donabedian Model of Quality

• Structure Process Outcome

Survival Functional status Satisfaction
7
Which is Best to Monitor Quality?

• Structure - necessary but not sufficient
• Process - many things we do/recommend dont
  have proven health benefit
• Outcomes - our ultimate responsibility but
  related to more than just the care we
  provide

8
Predictors of Outcomes

• Outcomes intrinsic patient risk factors
• treatment effectiveness
• quality of care
• random chance

9
Goals of Risk-Adjustment

• Account for intrinsic patient risk factors before
  making inferences about effectiveness,
  efficiency, or quality of care
• Minimize confounding bias due to nonrandom
  assignment of patients to different providers or
  systems of care

10
How is Risk Adjustment Done

• On large datasets
• Uses measured differences in compared groups
• Model impact of measured differences between
  groups on variables shown, known, or thought to
  be predictive of outcome so as to isolate effect
  of predictor variable of interest

11
When Risk-Adjustment May Be Inappropriate

• Processes of care which virtually every patient
  should receive (e.g., immunizations, discharge
  instructions)
• Adverse outcomes which virtually no patient
  should experience (e.g., incorrect amputation)
• Nearly certain outcomes (e.g., death in a patient
  with prolonged CPR in the field)
• Too few adverse outcomes per provider

12
When Risk-Adjustment May Be Unnecessary

• If inclusion and exclusion criteria can
  adequately adjust for differences
• If assignment of patients is random or
  quasi-random

13
When Risk-Adjustment May Be Impossible

• If selection bias is an overwhelming problem
• If outcomes are missing or unknown for a large
  proportion of the sample
• If risk factor data (predictors) are extremely
  unreliable, invalid, or incomplete

14
Data Sources for Risk-Adjustment

• Administrative data are collected primarily for a
  different purpose (billing), but are commonly
  used for risk-adjustment
• Disease registries

15
Sources of Administrative Data

• Federal Government
• Medicare
• VA
• State Government
• Medicaid (Medi-Cal)
• Hospital Discharge Data
• Private Insurance

16
Dataset Resources

• http//www.epibiostat.ucsf.edu/courses/RoadmapK12/
  PublicDataSetResources/
• http//base.google.com/base/search?a_n0clinicalt
  rialsa_y09hlenglUS

17
Advantages of Administrative Data

• Computerized, inexpensive to obtain and use
• Uniform definitions
• Ongoing data monitoring and evaluation
• Diagnostic coding (ICD-9-CM) guidelines
• Opportunities for linkage (vital stat, cancer)

18
Administrative Hospital Discharge Data

• Admission Date Race
• Discharge Date Sex
• Type of Admission Date of Birth
• Source of Admission Zip Code
• Principal Diagnosis Patient SSN
• Other Diagnoses Total Charges
• Principal Procedure and Date Expected
  Source of Payment
• Other Procedures and Dates
• Disposition of Patient
• External Cause of Injury
• Pre-hospital Care and Resuscitation (DNR)

19
Disadvantages of Administrative Data

• No control over data collection process
• Missing key information about physiologic and
  functional status
• Quality of diagnostic coding can vary across
  sites
• Non capture of out of plan/out of hospital/out of
  state events

20
Linking Administrative Data

• Strategy for enhancing number of predictor or
  outcomes variables
• Linkage dependent on reliable shared identifiers
  such as social security numbers in both datasets
• Probabilistic matching of less specific variables
  (age, sex, race, date of birth, etc)

21
Some Routinely Available Data Linkages

• California hospital discharge data and vital
  statistics
• Example 30 day mortality following AMI
• SEER -Medicare
• Example utilization patterns for those with
  breast cancer
• National Health Interview Survey-Medical
  Expenditure Panel Survey
• Example health care costs for those with
  self-reported chronic conditions

22
California Hospital Discharge Data and Medicaid
Eligibility Files

• Creates a continuous monthly record of an
  individuals pattern of Medicaid enrollment
• Discharge data captures all hospitalizations
  regardless of whether in or out of Medicaid
• Have found a 3 fold increase in hospitalizations
  for ambulatory care sensitive conditions for
  those with interrupted Medicaid coverage

23
Health Plans/Delivery Systems

• Health insurance claims
• Inpatient, outpatient, pharmacy, diagnostics, etc
• Electronic Medical Records
• VA
• Kaiser
• SF Dept of Public Health (THREDS)

24
THREDS

• 120,000 patients per year seen in DPH
  clinics/SFGH
• Data begin in 1996 and updated daily
• Includes demographics, insurance status,visit hx,
  diagnostic codes, tests ordered and results,
  pharmacy, link to death registry
• http//gcrcsfgh.ucsf.edu/?pagethreds

25
Disease Registries

• Attempt to capture all or large sample of the
  cases of a specified condition
• Often include more clinical information than
  administrative datasets
• Many of these can support assessments of survival
  beyond acute period
• May require permission/approved protocol to
  access all or some of the data

26
Example Registries

• UNOSnational registry of patients with end stage
  renal disease
• SEER Cancer Registry
• Coronary Artery Bypass Graft Surgery California
  Office of Statewide Health Planning and
  Development

27
Doing Your Own Risk-Adjustment vs. Using an
Existing Product

• Is an existing product available or affordable?
• Would an existing product meet my needs?
• - Developed on similar patient population
• - Applied previously to the same condition or
  procedure
• - Data requirements match availability
• - Conceptual framework is plausible and
  appropriate
• - Known validity

28
Conditions Favoring Use of an Existing Product

• Need to study multiple diverse conditions or
  procedures
• Limited analytic resources
• Need to benchmark performance using an external
  norm
• Need to compare performance with other providers
  using the same product
• Focus on resource utilization, possibly mortality

29
A Quick Survey of Existing ProductsHospital/Gener
al Inpatient

• APR-DRGs (3M)
• Disease Staging (SysteMetrics/MEDSTAT)
• Patient Management Categories (PRI)
• RAMI/RACI/RARI (HCIA)
• Atlas/MedisGroups (MediQual)
• Cleveland Health Quality Choice
• Public domain (MMPS, CHOP, CSRS, etc.)

30
A Quick Survey of Existing ProductsIntensive Care

• APACHE
• MPM
• SAPS
• PRISM

31
A Quick Survey of Existing ProductsOutpatient
Care

• Resource-Based Relative Value Scale (RBRVS)
• Ambulatory Patient Groups (APGs)
• Physician Care Groups (PCGs)
• Ambulatory Care Groups (ACGs)

32
How Do Commercial Risk-Adjustment Tools Perform

• Better than age/sex to predict health care
  use/death
• Better retrospectively (30-50 of variation)
  than prospectively (10-20 of variation)
• Lack of agreement among measures
• More than 20 of in-patients assigned very
  different severity scores depending on which tool
  was used (Iezzoni, Ann Intern Med, 1995)

33
Co-Morbidity or Severity?

• Are patients at risk for an outcome because they
  have multiple conditions (co-morbidities), a more
  severe version of a disease (disease stage) or
  both?
• Before adjusting for co-morbidity and or severity
  consider whether either is a complication of
  treatment (or non treatment) rather than an
  independent health characteristic of the patient

34
Summary

• Risk adjustment is a multivariate modeling
  technique designed to control for patient
  characteristics so that judgments can be made
  about the quality of care
• Risk adjustment requires large datasets such as
  administrative datasets or disease registries
• Commercial risk adjustment products exist for
  patients in different health care settings
• There are many reasons why one might choose to
  develop a risk adjustment model - we will talk
  about how to do this next week!