Public Reporting of Long Term Care Quality: The US Experience - PowerPoint PPT Presentation


PPT – Public Reporting of Long Term Care Quality: The US Experience PowerPoint presentation | free to download - id: 6a5bcb-NzI5M


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation

Public Reporting of Long Term Care Quality: The US Experience


... Most Medicare home ... Based Quality Improvement Risk-adjusted Home Health Outcome Report for Improvement of Activities of Daily Living CMS OASIS ... – PowerPoint PPT presentation

Number of Views:28
Avg rating:3.0/5.0
Slides: 73
Provided by: Terry
Learn more at:


Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Public Reporting of Long Term Care Quality: The US Experience

Public Reporting of Long Term Care Quality The
US Experience
  • Vincent Mor, Ph.D.
  • Brown University

  • Long history of scandals regarding long term care
    quality, particularly nursing homes
  • While preference for and supply of community
    based alternatives have grown in US, all
    acknowledge residentially based long term care
    must be part of any system
  • Home Health less scrutinized but many worry about
    care adequacy since hard to inspect

  • Institute of Medicine Report in 1987 served as
    basis for nursing home reform also adopted by
    home care
  • Uniform Resident Assessment Instrument created in
    1991 and became the basis for the creation of
    performance measures designed to stimulate
    quality competition through public reporting
  • Home Health Outcome and Assessment Information
    Set (OASIS) emerged independently

Background (cont.)
  • Using RAI Nursing Home Quality Measures tested,
    revised and published as Nursing Home Compare
    since 2002
  • More recent efforts to create composite measure
    incorporating Inspection results, Staffing Levels
    and Quality Measures have been widely promulgated
  • Home Health Quality Measures developed and tested
    and published as Home Health Compare since 2004

  • Summarize US Experience with Development of Long
    Term Care Quality Measures
  • Review Conceptual and Technical Issues Facing the
    Construction of Long Term Care Quality Measures
  • Review Literature on Effects of Public Reporting
    of Quality Measures in Long Term Care

The Nursing Home Resident Assessment Instrument
  • 1986 Institute of Medicine Report on Nursing Home
    Quality Recommended a Uniform RAI to Guide Care
    Planning --MDS
  • OBRA 87 Contained Nursing Home Reform Act
    Including RAI Requirement
  • A 300 Item, Multi-Dimensional RAI Tested for 2
  • Mandated Implementation in 1991

Clinical Planning Basis of the MDS
  • Assessment Profile in Given Domain Triggers
    Potential Risk Status
  • Resident Assessment Protocol Reviewed to
    Determine Presence of Problem or High Risk of
  • Care Planning and Treatment Directed to the
  • Data Quality Contingent upon conduct of Clinical
    Care Planning Process

MDS Background
  • MDS Version 2.0 Introduced in 1996
  • Admission, Short Term and Quarterly Reassessments
    done on all Residents
  • Inter-State Variation with some requiring
    additional data
  • Since 1998 all MDS records are computerized and
    submitted to Centers for Medicare Medicaid

(No Transcript)
(No Transcript)
(No Transcript)
CMS Quality Measures
  • The quality measures, developed under CMS
    contract to Abt Associates and a research team
    led by Drs. John Morris and Vince Mor, have been
    validated and are based on the best research
    currently available. These quality measures meet
    four criteria. They are important to consumers,
    are accurate (reliable, valid and risk adjusted),
    can be used to show ways in which facilities are
    different from one another, and can be influenced
    by the provision of high quality care by nursing
    home staff. CMS Web Site

CMS Quality Measures - Long Term
  • Percent of Long-Stay Residents Given Influenza
    Vaccination During the Flu Season
  • Percent of Long-Stay Residents Given Pneumococcal
  • Percent of Residents Whose Need for Help With
    Daily Activities Has Increased
  • Percent of Residents Who Have Moderate to Severe
  • Percent of High-Risk Residents Who Have Pressure
  • Percent of Low-Risk Residents Who Have Pressure
  • Percent of Residents Who Were Physically
  • Percent of Residents Who are More Depressed or
    Anxious (Looks back 30 days)
  • Percent of Low-Risk Residents Who Lose Control of
    Their Bowels or Bladder
  • Percent of Residents Who Have/Had a Catheter
    Inserted and Left in Their Bladder
  • Percent of Residents Who Spent Most of Their Time
    in Bed or in a Chair
  • Percent of Residents Whose Ability to Move in and
    Around Their Room Got Worse
  • Percent of Residents with a Urinary Tract
    Infection (Looks back 30 days)
  • Percent of Residents Who Lose Too Much Weight
    (Looks back 30 days)

Physical Functioning- October/December 2009
State ADL Worse Bed Bound Move Worse Decline in ROM
National 14.9  4.7  14.7  6.6 
AK 17.4  5.3  14.6  10.9 
AL 11.6  6.5  11.7  5.4 
AR 14.2  4.2  12.6  5.5 
AZ 13.9  4.9  14.6  5.5 
CA 10.4  7.1  11.6  6.2 
CO 15.3  3.0  14.9  6.4 
CT 15.4  2.5  16.3  4.8 
DC 13.4  1.7  13.2  5.4 
DE 13.8  4.1  15.5  8.1 
FL 12.7  4.6  12.5  5.2 
Psychotropic Drug Use- October/December 2009
State Anti-Psychotics Overall Anti-Psychotics LOW Risk Anti-Anxiety Agents
National 18.6  15.6  23.1 
AK 11.2  4.7  21.5 
AL 15.9  14.0  27.2 
AR 17.9  15.6  21.1 
AZ 19.2  15.8  21.5 
CA 16.8  14.0  20.4 
CO 18.6  15.1  18.1 
CT 23.7  21.2  22.7 
DC 13.6  12.4  13.4 
DE 20.2  17.8  23.3 
FL 12.2  10.1  27.5 
CMS Quality Measures Short stay
  • Percent of Short-Stay Residents Given Influenza
    Vaccination During the Flu Season
  • Percent of Short-Stay Residents Who Were Assessed
    and Given Pneumococcal Vaccination
  • Percent of Short-Stay Residents With Delirium
  • Percent of Short-Stay Residents in Moderate to
    Severe Pain
  • Percent of Short-Stay Residents With Pressure

Home Health Quality Measurement
  • OASIS began as a cooperative effort between home
    health agencies and researchers to develop simple
    outcome measures to track patients rate of
    improvement while in care
  • University of Colorado researchers worked with
    large Visiting Nurse Services to develop and test
  • CMS then funded multiple large demonstrations to
    implement the tool and use for quality
    measurement and case-mix reimbursement

Outcome Based Quality Improvement
  • Distinct measures of change in patient
    functioning, resolution of symptoms and ability
    to manage independently collected at the start
    and end of care (OR every 60 days)
  • Most Medicare home health is short term
  • Measures tested and revised with extensive case
    mix adjustment to allow for comparison across
    agencies and states

Risk-adjusted Home Health Outcome Report for
Improvement of Activities of Daily Living
  • Percent of Patients in Home Health Care whose
    ability to Groom, Bathe, Dress Upper and Dress
    Lower Body themselves improves between start of
    care and discharge

CMS OASIS Report 2009 Rates of Improvement in
State Grooming Upper Dressing Lower Dressing Bathing
Alabama 71.0  73.0  75.0  68.0 
Alaska 67.0  66.0  56.0  59.0 
Arizona 67.0  69.0  67.0  64.0 
Arkansas 68.0  69.0  70.0  65.0 
California 71.0  72.0  70.0  67.0 
Colorado 70.0  71.0  70.0  64.0 
Connecticut 69.0  69.0  68.0  62.0 
Delaware 68.0  69.0  70.0  62.0 
District of Columbia 75.0  79.0  79.0  72.0 
Florida 69.0  69.0  68.0  66.0 
Georgia 71.0  73.0  74.0  67.0 
Risk-adjusted Home Health Outcome Report for
Utilization Outcomes
  • Percent of patients who have received emergency
    care prior to or at the time of discharge from
    home health care.
  • Percent of patients who are discharged from home
    health care and remain in the community
  • Percent of patients who are admitted to an acute
    care hospital for at least 24 hours while a home
    health care patient.

Risk-adjusted Home Health Outcome Report
State Any Emergent Care Discharged to Community Acute Care Hospital
Alabama 23.0  64.0  33.0 
Alaska 20.0  71.0  25.0 
Arizona 25.0  67.0  29.0 
Arkansas 24.0  64.0  32.0 
California 18.0  72.0  25.0 
Colorado 23.0  69.0  26.0 
Connecticut 27.0  65.0  32.0 
Delaware 20.0  70.0  26.0 
District of Columbia 21.0  71.0  26.0 
Florida 18.0  70.0  26.0 
Georgia 22.0  68.0  29.0 

Conceptual Issues Inherent in Applying Quality
  • Requires shared interpretation of Quality
  • Assumes all Providers have same goals
  • Assumes Measured Quality Domains are Important
  • Indicators are NOT Quality per se, BUT often used
    as evidence in and of themselves
  • Assumes Facilities Accountable for most of the
    variation in the Indicator (e.g. outcomes)
  • Assumes Facilities Know how to Change Practice

Technical Issues That Can Compromise Validity of
  • Reliability Validity of the data
  • Multi-dimensionality of Quality Indicators
  • Stability of Estimates Sensitive to Sample Size
  • Ranks can Overestimate Differences
  • Patient Level Risk Adjustment Complex
  • Differences in Assessment Practices Influence QI
    Scores Comparisons

Reliability Studies NH
  • 219 of 462 (47.4) facilities approached chose to
    participate in full study (52.4 for HB and 45.6
    for non-HB)
  • Non-participants were more likely to be
    for-profit, less well staffed and with more
    regulatory deficiencies
  • 5758 patients (ave. 27.5/facility) included in
    reliability analyses
  • 119 patients assessed twice by research nurses
  • Patients resemble traditional US nursing home

Reliability of Gold Standard Nurses
Item Agree Kappa
DNR 91 .83
Memory 88 .63
Decisions 97 .89
Understood 96 .82
Understand 96 .80
Fears 97 .76
Wander 99 .85
Walk 95 .86
Pain Fx. 93 .78
  • Of 100 items, only 3 didnt reach Kappagt.4
  • 50 items had Kappa gt.75
  • Pct. Agreement high even for ordinal items with

Reliability of Facility RNs to Gold Standard
  • Of the 100 data items 28 had Kappa lt.4 and 15 had
    Kappa gt.75
  • Worst Kappa items were rare binary items like
    end stage, didnt use toilet, recurrent lung
    aspirations, etc.
  • ADLs and other Functioning items had Kappa values
    above .75

Reliability of Constructed Quality Indicators NH
  • Quality Indicators are composites of several RAI
    items a definition of the denominator and of the
    conditions required to meet the QI definition
  • The inter-rater reliability of a QI is a function
    of the reliability of all the component items
    defining the algorithm

Prevalence and Inter-Rater Agreement and
Reliability of Selected Facility Quality
Indicators  N209 homes
Facility QI Reliability Variation Bladder/Bowel
Facility QI Reliability Variation Inadequate
Pain Management
Reliability Studies Home Health
  • Fewer inter-rater reliability studies of OASIS
  • More expensive to send two nurses at separate
    times on the same day to do the same assessment
  • Largest Reliability Study done as part of
    research to develop case-mix reimbursement system
  • ADL and other function items yield high levels of
    reliability symptoms achieve ok reliability

Selected Inter-Rater Reliability Results from
OASIS test
Signs Symptoms Sample Size Percent Agreement Kappa
1. Diarrhea 304 93.4 0.44
2. Difficulty urinating or gt3x/night 304 91.5 0.45
3. Fever 304 96.7 0.63
4. Vomiting 304 97.4 0.49
5. Chest Pain 304 95.4 0.51
6. Constipation in 4 of last 7 days 304 92.1 0.53
7. Dizziness or lightheadedness 304 89.1 0.46
8. Edema 304 81.3 0.50
9. Delusions 304 99.0 0.66
10. Hallucinations 304 98.4 0.44
OASIS Reliability Results Function
Variable Sample Size Percent Agreement Kappa
Grooming Current ability to tend to personal hygiene needs 304 74.7 0.83
Dressing Current ability to dress upper body with or without dressing aids 304 71.1 0.83
Dressing Current ability to dress lower body with or without dressing aids 304 77.0 0.85
Bathing Current ability to wash entire body 304 64.8 0.80
Toileting Current ability to get to and from the toilet or bedside commode 304 82.6 0.86
Transferring Current ability to move from bed to chair, on/off toilet or commode, tub, 304 74.3 0.88
Ambulation/Locomotion Current ability to safely walk, use a wheelchair 304 77.6 0.87
Validity of the Data Measures
  • Validity of the data shown by the extent to which
    items and measures behave as expected relative to
    gold standard variables or hard outcomes
  • Compared MDS diagnoses to Hospital discharge
  • Looked at MDS predictors of survival
  • Related to MDS measures to research scales

MDS vs. CMS Hospital diagnoses
  • Neurological
  • Cerebrovascular disorders (ICD-9 432, 434, 436,
  • PPV 0.73
  • Parkinsons disease (ICD-9 332)
  • PPV 0.86
  • Alzheimers disease (ICD-9 331)
  • PPV 0.68
  • Brain degeneration (ICD-9 331.0, 331.2, 331.7,
  • PPV 0.84

One Year Survival by Gender Cognition Level
Women (CPS 2-4)
Men (CPS 0-1)
(No Transcript)
Construct Validity Cognitive Performance Scale
  • Cognitive Performance Scale (CPS) Derived from 5
    MDS Items
  • Strong (gt.85) Correlation with MMSE
  • High Kappa with Global Deterioration Scale (.76)
  • Percent Patients with Dementia Increases as CPS
  • MDS Communication Correlated (.85) with MMSE
  • ADL, CPS Symptoms Select Diagnoses Related to

Sample Size and QI Stability
  • Providers and Consumers want QI to reflect not
    just what WAS but what WILL BE SO
  • QI stability is desired
  • QI must be based upon minimum observations
  • Correlation between quarters among QIs varies
  • Correlation among prevalence based QIs is high
    because same individuals assessed each quarter
  • Correlation between quarters among incidence and
    change based QIs lower and VERY sensitive to
    sample size

Residents Expected Rates of Change on Quality
  • Over 90 day period 77.1 of residents still in
    facility do not change on ADL, 14.7 decline and
    8.2 improve.
  • Over 12 months 58 of residents in home dont
    change and 30.2 decline.
  • Similar pattern for Communication, Cognition and
    individual ADL items
  • Means that rates of decline are low and many
    residents are needed to estimate a homes rate of
    ADL decline with confidence.

Estimated Sample Size for Change
Long Term Predictability of Quality
Quality Fluctuation Seasonality
Transforming QI Scores into Ranks
  • Many QI score distributions are skewed many
    facilities with little or no problem and few
    facilities with many residents experiencing the
  • Median facility might be very similar to the
    best (the one with fewest problems)
  • Transforming to ranks means saying there is a
    difference between the 10th and 40th percentile
    when there is little difference

Pressure Ulcer Prevalence Facility Distribution
Meaning of Ranks
Variability in Ranking Distributions
Anti-psychotics Median Ranks
Persistent Pain Median Ranks
Complexity of Determining Appropriate Risk
  • Risk Factors May not be Measured Independent of
    the Provider (tx) Effect
  • Potential for Over Adjustment as Great as Under
  • How to Adjust for Socio-Economic Differences
    Known to Affect Health Behavior or Clinical
    Characteristics (e.g. PU not seen on African
    American NH pts until at Stage 2 OR Pain Harder
    to see in Cognitively Impaired Oldest pts)

Risk Adjustment Complexity
Why Adjust QIs
  • Facilities should be compared on level playing
    field, acknowledging differences in
  • Types of residents admitted
  • Ability to ameliorate clinical characteristics
    thought to predispose to poor outcomes
    irrespective of care quality
  • Variability in measurement acumen of assessors

Average Admission Prevalence of Pressure Ulcers
Across All States, 1999
(No Transcript)
(No Transcript)
Multi-dimensionality of QIs
  • Consumers want to know Best nursing home
    Regulators want to know where to focus their
    survey energies Purchasers want to buy best.
  • If Quality is multi-dimensional no such thing as
    the best most valuable dimension is a
    preference and will be individualized
  • Combining QIs that arent highly correlated may
    mask differences between facilities on important
    individual QIs

Does Poor Performance on One Measure Mean NF is
  • Average Correlation Among QIs is Low
  • Anti-Psychotics and Restraints Correlated .04
  • What is a Good Home if QIs not Related?
  • Can Performance Measures Help Pick Good Homes?
  • Are Some Measures More Meaningful?
  • Should Users of Performance Measures Select the
    Measures they Value Most?

Summary Results of Factoring
Functional Decline Mood/ Behavior Pressure Ulcers Treatment Condition No Factor
Worsening Bladder Poor Mood State Prevalence Worsening Pressure Ulcers Prevalent Catheter
Worsening Bowel Worsening Mood Prevalent Pressure Ulcer (Hi Risk) Prevalent Restraint
ADL Decline Poor Mood w/o Anti-depressants Prevalent Pressure Ulcer (Lo Risk) Prevalent Anti-Hypnotic Use
Mobility Decline Behavior Problems Prevalence Prevalent Anti-Psychotic Use
Cognitive Decline Worsening Behavior Weight Loss
Communication Decline Worsening Relationships Falls
Worsening Pain
Regression Modeling Results
  • Results of relating each QI to all others
    revealed very low R2 for all Treatment
  • While R2 higher for QIs within other factors,
    many conceptually unrelated QIs found to weakly
    predict other QI
  • Many QIs load (related to) on multiple factors
  • QI type (e.g. prevalence, longitudinal, change)
    as influential as QI content in factor
  • Factor structure sensitive to which QIs included
  • Many QIs totally uncorrelated with others

Provisional Test of Combining Unlike Quality
  • Use 1999 MDS 2.0 from OH, NY CA
  • Create Risk and Admission adjusted QIs for
    Pressure Ulcers, Anti-Psychotic Use and Pain
  • Correlate Measures PU Painlt.05 PU
    Anti-Psych -.15 Pain Anti-Psych .16
  • Only 13 of facilities in bottom half on all 3
    QIs 5 if use bottom third on all measures

Public Reporting of Quality
  • NURSING HOME COMPARE allows consumers and
    advocates to identify facilities in their
    geographic area and to select using a Five Star
    global rating OR based upon global domains OR
    specific measures.
  • HOME HEALTH COMPARE allows consumers and
    advocates to identify agencies in geographic area
    and presents detail of many different Quality

(No Transcript)
(No Transcript)
(No Transcript)
(No Transcript)
(No Transcript)
(No Transcript)
Effect of Public Reporting
  • Research to date all done on nursing homes
  • Broken down by market served long stay
    residential vs. short stay, post-acute
  • First wave of studies surveyed administrators to
    find out how they were responding
  • More recent studies use MDS data to examine
    changes in outcomes and admission patterns

Facility Response to Reporting
  • Castle (2005) initially found Administrators were
    skeptical and unconvinced that reporting mattered
  • Zinn colleagues (2005) surveyed leaders and
    found they were aware of their scores and those
    of closest competitors concluded spurred quality
  • Castle (2007) concurred in a separate survey that
    more competitive markets affected response

Reporting Improve Quality?
  • Werner colleagues (2009) found significant
    improvement in BOTH measured and unmeasured
    quality measures following public reporting BUT
    general improvement trend
  • Mukamel et al (2007) looked carefully at initial
    response relative to prior quality patterns and
    also found improvement on most but not all
  • Werner, et al, 2010 also found improvement in
    post-acute quality scores

Reporting Alter Admissions?
  • Werner Colleagues have examined whether
    facilities with worse quality scores in
    competitive markets manifest reductions in
  • Very complicated must infer from the data why
    someone entering a facility should affect those
    entering to stay more, but hard to know who
  • However, evidence suggest small but significant
    changes in referral patterns favoring better

  • Public Reporting of long term care providers
    quality performance is possible
  • All measures are flawed, but no more than acute
    and ambulatory care
  • Pre-requisite is to have uniform data collected
    with relevant clinical detail AND should be able
    to be audited with penalties to minimize bad data

Summary (cont.)
  • Constructing quality measures can be complex
  • Sample size, seasonality, risk adjustment are all
    important to assure the fairness of the system
  • Like case mix reimbursement, dont want
    incentives for providers to limit access to
  • Still at the infancy of understanding how
    consumers advocates use the data

Issues for the Future
  • Preferable to have common items, measures and
    metrics across different types of long term care
    options, technically AND for consumers
  • Challenge of Creating Composite Scores consumers
    want that are technically less sensitive than
    domain specific measures
  • However, Movement to Pay for Performance
    requires we develop a solution