Measuring and Improving Quality in Managed Care: Some Statistical and Computing Issues

1
Measuring and Improving Quality in Managed Care
Some Statistical and Computing Issues

• Randall K. Spoeri, Ph.D.
• Vice President
• Medical and Quality Informatics
• HIP Health Plans
• New York, NY

2
Coverage

• Measurement and Quality in Managed Care
• Some Statistical and Computing Issues
• Measure Definitions
• Data Availability and Quality
• Risk Adjustment
• Analytical/Interventional Use of Results
• Predictive Modeling
• Thoughts About the Future

3
About HIP Health Plans

• A managed care company with membership of over 1
  million in metropolitan NYC and FL
• Has a long history of over 50 years of service to
  NYC residents
• 2000 revenues budgeted at 2 Billion
• Has a strong commitment to quality measurement
  and improvement

4
Measurement and Quality in Managed Care

• Managed care has experienced rapid expansion in
  recent years
• With that expansion, concerns have been expressed
  about the quality of care delivered
• In response, quality management efforts have
  relied heavily on performance measurement and
  accreditation review

5
Measurement and Quality in Managed Care

• The National Committee for Quality Assurance
  (NCQA) has been the major provider of
  accreditation reviews and performance measurement
  standards
• The Health Plan Employer Data and Information Set
  (HEDIS) is the NCQA-sponsored collection of
  standardized performance measures widely used by
  Managed Care Organization (MCOs)

6
Measurement and Quality in Managed Care

• With the growth of managed care and the
  increasing focus on quality and performance
  measurement, MCOs have begun to expand their
  staff to include medical and quality informatics
  personnel

7
Measurement and Quality in Managed Care

• Skills of Medical and Quality Informatics (MaQI)
  staff include statistics and statistical
  computing, epidemiology and public health,
  psychology, and economics
• Demand is high for this type of individual, not
  only in MCOs, but also in allied areas such as
  hospital organizations and pharmaceutical
  companies

8
Measurement and Quality in Managed Care

• Statisticians and computing professionals can
  make major contributions to the health care
  industry, especially managed care
• The opportunities are many, as are the challenges

9
Some Statistical and Computing Issues

• Measure Definitions
• Data Availability and Quality
• Risk Adjustment
• Analytical/Interventional Use of Results
• Predictive Modeling

10
Measure Definitions

• In order that comparisons can be made among
  health plans, it is essential that measure
  definitions be uniform, unambiguous, and precise
• In the managed care world, HEDIS is the standard
  set of performance measures used by MCOs

11
Measure Definitions

• The recently announced HEDIS 2001 will have 52
  measures across 7 domains of care Effectiveness
  of Care, Access/Availability of Care,
  Satisfaction with the Experience of Care, Health
  Plan Stability, Use of Services, Informed Health
  Care Choices, and Health Plan Descriptive
  Information

12
Measure Definitions

• Example measures Childhood Immunizations,
  Breast Cancer Screening, Beta Blocker Treatment
  After a Heart Attack, Practitioner Turnover,
  Cesarean Section Rate
• Developing specifications (i.e., detailed
  definitions) for these measures is an arduous
  task, often involving statistical issues

13
Measure Definitions

• Some statistical issues include
• Measure reliability/validity
• Data source (administrative or medical record)
• Complete enumeration or sample (or hybrid)
• Sampling methodology
• Sample size
• Technical Panels were convened by NCQA to advise
  on these and other issues

14
Data Availability and Quality

• Encounter data are administrative data supplied
  by a provider (i.e., a physician) to a health
  plan on outpatient visits by the plans members
• The incentives to supply these data range from
  none to weak, to strong monetarily or
  contractually or both

15
Data Availability and Quality

• Consequently, encounter data repositories are
  often incomplete and frequently of poor quality
• Many physicians dont see or understand the
  importance of this encounter data, and view it as
  yet another disruption imposed by MCOs

16
Data Availability and Quality

• The movement from paper-based systems for
  submission of encounter data, to electronic ones
  has promise
• The ever-expanding use of encounter data for such
  things as physician performance measurement
  (so-called provider profiles) has begun to
  garner physician attention

17
Data Availability and Quality

• Federal and state mandates may also help to
  improve the completeness of encounter data
• But even when data repositories are complete,
  there remain major questions about the quality of
  the data contained therein
• Encounter, claim, pharmacy, lab, and other health
  care data are still of uneven quality

18
Data Availability and Quality

• One approach, taken by NCQA, in connection with
  the quality of HEDIS data is to require that the
  complete system for this datas production be
  audited (i.e., externally reviewed and verified)
  structure, process and outcome

19
Data Availability and Quality

• Other techniques used in data quality assurance
  include the examination of source code used for
  administrative data pulls, medical record reviews
  to measure the accuracy of administrative data
  counterparts, and medical record re-review to
  assess the accuracy of primary chart abstraction

20
Data Availability and Quality

• With the increasing reliance of MCOs on data and
  measurement, through complex data structures like
  data warehouses, the completeness and accuracy of
  the information becomes paramount
• Statisticians and IS/IT professionals need to
  work with clinicians and administrators to
  continuously improve these data systems

21
Risk Adjustment

• When comparing entities such as hospitals and
  physicians, it is important that an adjustment be
  made for differences in the mix of patients being
  treated
• For example, it would be inappropriate to compare
  one hospital providing mostly obstetrical care in
  the suburbs to an inner city trauma center
  hospital using mortality

22
Risk Adjustment

• Similarly, comparing a physician seeing
  predominantly young patients to another physician
  seeing a large proportion of Medicare patients is
  not appropriate when looking at pharmacy usage
• Many of the measures used to assess hospital and
  physician performance need risk adjustment, a
  leveling of the field based upon differing
  patient characteristics

23
Risk Adjustment

• Characteristics used generally include age,
  gender, diagnosis and procedure, as well as
  comorbidities, for example
• One commonly used approach to risk adjust
  performance statistics is to use the concept of
  observed and expected

24
Risk Adjustment

• The observed result could be a hospitals average
  length of stay for a certain surgical procedure
• The expected result would be that predicted using
  epidemiological or statistical methods for the
  patient population undergoing the surgical
  procedure at that hospital

25
Risk Adjustment

• Statistical issues include methodology selection,
  model calibration, and explication
• Popular methods include multiple and logistic
  regression and survival analytic (Bailey-Makeham)
• Calibration issues often surround data
  timeframe, geography, availability and quality,
  breadth

26
Risk Adjustment

• Explication Users of the data often become
  suspicious of methods they dont understand
  (e.g., complex statistical methods), even when
  the methods are sound
• Computing issues arise in building and handling
  the enormous databases common in managed care
  (e.g., encounters)

27
Risk Adjustment

• Computing challenges also arise out of building
  and using complex risk adjustment models, or even
  in using models developed by others (i.e.,
  commercial vendors)
• Nevertheless, these issues and challenges provide
  opportunities for statisticians, computing
  professionals, and clinically prepared
  individuals, working together

28
Analytical/Interventional Use of Results

• The best measurements and measurement systems are
  of no value unless they are followed by action
  and behavior change
• Some analyses prompt action and intervention by
  their very naturean example follows using real
  HIP data

29
(No Transcript)
30
Analytical/Interventional Use of Results

• Most analyses, however, require an intervention
  strategy
• For example, learning that an MCO has a low rate
  of childhood immunizations could prompt a variety
  of actions

31
Analytical/Interventional Use of Results

• Example interventions
• Reminder to Primary Care Physician
• Reminder to Pediatrician
• Reminder to Parents
• Reminder stickers for medical chart
• Promotional brochures
• Educational forums

32
Predictive Modeling

• Most MCOs have undertaken disease management
  initiatives for high prevalence/high cost
  conditions, such as asthma, diabetes, congestive
  heart failure and hypertension
• As part of these efforts, predictive modeling is
  sometimes undertaken to identify high risk
  individuals with a given disease

33
Predictive Modeling

• The goals are to
• Avoid costly ER visits or hospitalizations
• Treat individuals at higher risk earlier in the
  disease process, and monitor them more closely
• Reduce the risk of adverse outcomes
• Improve satisfaction and quality of life

34
Predictive Modeling

• Some of the methodologies used include
• Multiple Regression (Linear and Nonlinear)
• Logistic Regression
• Neural Networks
• Time Series
• Many of the disease management vendors that have
  come on the scene tout predictive modeling as a
  value-added benefit

35
Predictive Modeling

• Many MCOs do some of this work themselves or in
  partnership with disease management vendors
• There is a need, however, to assure that the
  statistical methodologies are being properly
  utilized, and that the data used to both build
  and apply the models be accurate
• Results to date have been encouraging

36
Thoughts About the Future

• MCOs will continue to expand their performance
  measurement and improvement activities
• Accreditors (e.g., NCQA), Federal and State
  Agencies (e.g., HCFA and DOH/DOI), and
  purchasers/employers are driving much of this
  expansion

37
Thoughts About the Future

• The use of statistical methods in MCO quality
  management will increase
• Administrative data systems must improve to
  greatly reduce the need for medical chart
  abstraction
• Opportunities for statisticians with a bent
  toward computing will abound in MCOs

38
Thoughts About the Future

• The future aint what it used to be - Yogi Berra