AHRQ National Advisory Council on Healthcare Research and Quality Subcommittee on Children

1
AHRQ National Advisory Council on Healthcare
Research and Quality Subcommittee on Childrens
Healthcare Quality Measures for Medicaid and CHIP
Programs

• Rita Mangione-Smith and Jeffrey Schiff
• September 17 18, 2009

2
CHIP
3
Potential Impact of Core Measures Identification
Work 48 of Americas Children Under 19
Sources AHRQ based on Coverage estimates
Based on 2008 national participant and spending
data derived from CMS and U.S. Census Current
Population Survey data sources, reported in
National Center on Children in Poverty,
http//www.nccp.org/profiles/index_32.html.
CHIP estimates are for number of children in
separate SCHIP programs. Medicaid estimates
include children in Medicaid SCHIP programs.
Coverage estimates reflect Medicaid and CHIP
enrollees whether or not they received health
care services. Total number of children under 19
interpolated from U.S. Census Bureau figures for
number of children 17 and under (73.9 million)
and 19 and under (83 million).
4
Our charge

• Core set of measures that when
• TAKEN TOGETHER USED TO ESTIMATE THE OVERALL
  NATIONAL QUALITY OF HEALTH CARE FOR CHILDREN

5
Conceptual Framework Guiding Determination of
the Scope for Core Measurement Set

• Grounded ? Intermediate ? Aspirational
• Measures Measures Measures
• Lean towards recommending more grounded measures
• Grounded10-25, currently feasible, many already
  in place
• Intermediate category ?, good specifications,
  some States already using them
• Aspirational needed measures to fill in the gaps

6
Scope for Core Measurement Set

• 1. Must be realistic about staffing/funding needs
  for collecting/analyzing/reporting available data
• 2. Comprehensive effort to find good measures for
  all service categories, duration of enrollment,
  and other aspects of care required by the
  legislation however, if no good measures
  currently exist for a given aspect of care, a
  measure will not be recommended for the core set
• 3. Include measures not currently used by
  Medicaid/CHIP
• e.g. State and national measurement efforts
• 4. Choose measures that are actionable
• There should be clear steps a State, plan, or
  provider can take to improve on performance the
  measure should inform what these steps need to be

7
Consensus on Criteria Definitions

• Validity
• Measures must be supported by scientific evidence
  or, where evidence is insufficient, by expert
  consensus
• Measures must support a link between
• Structure and outcomes of care
• Structure and processes of care
• Processes and outcomes of care
• The measure must represent an aspect of care that
  is under the control of health care providers and
  systems
• The measure should truly assess what it purports
  to measure
• Measures supported by evidence from unpublished
  data should be considered for inclusion

8
Consensus on Criteria Definitions

• Feasibility
• The data necessary to score the measure must be
  available to State Medicaid and CHIP programs
• Administrative data, medical records data, survey
  data
• Detailed specifications must be available for the
  measure that allow for reliable and unbiased
  scoring of the measure across States and
  institutions

9
Consensus on Criteria Definitions

• Importance
• The measure should be actionable
• Cost of the condition to the Nation should be
  high
• Health care systems are clearly accountable for
  the quality problem assessed by the measure
• The extent of the quality problem should be
  substantial
• There should be documented variation in
  performance on the measure
• The measure should be representative of a class
  of quality problems sentinel measure of
  quality of care provided for preventive care,
  mental health care, or dental care, etc.

10
Consensus on Criteria Definitions

• Importance
• The measure assesses an aspect of health care
  where there are known disparities
• The core set should represent a balanced
  portfolio of measures and be consistent with the
  intent of the legislation
• Improving on performance for the core set of
  measures should have the potential to transform
  care for our Nations children

11
Consensus on Criteria Definitions

• Transparency
• For all measures recommended for inclusion in
  the core set
• The level of scientific evidence supporting the
  measure will be reported
• Example USPSTF grades A, B, C, or I Level I,
  II, III
• The level of expected burden for obtaining the
  data needed to score the measure will be reported
• Low, moderate, high

12
Delphi Scoring Process Completed by Subcommittee
Members

• Measures scored on a 9-point scale
• 7-9 Measure definitely valid, feasible, and
  important
• 4-6 Measure has uncertain validity, feasibility,
  or importance
• 1-3 Measure is not valid, feasible, or important
• Measures with a Validity score of gt 7 pass
• Measures with Feasibility and Importance scores gt
  4 pass
• Delphi Round 1 completed at July 2009 meeting
• Assessed validity, feasibility, and importance of
  measures in use by State Medicaid and CHIP
  programs

13
Delphi Scoring Process Completed by Subcommittee
Members

• Delphi Round 2 completed, Monday September 14th
  in preparation for second meeting
• Four groups of measures were assessed
• Measures that had passing scores for validity,
  feasibility, and importance in Delphi Round 1
• Measures that were judged to be controversial
  during scoring for validity and feasibility in
  Delphi Round 1
• Measures identified through environmental scans
  but that were not included on the original list
  of measures scored during Delphi Round 1
• Measures nominated by SNAC members, Federal
  partner agencies, and the public, between July
  24th and August 24th

14
65/121 Measures Passed Delphi Round 2

• Prevention and Health Promotion 27/50
• Management of Acute Conditions 16/25
• Management of Chronic Conditions 18/31
• Family Experiences with Care 3/6
• Most Integrated Health Care Systems 0/1
• Availability of Services 0/5
• Health Status 0/1
• Duration of Enrollment 1/2

15
Meeting 9/17-9/18 Goal is to identify a
parsimonious, core, grounded, and balanced set of
measures

• 65 measures passed Delphi Round 2 and will be
  discussed for final inclusion at the meeting
• Balancing grid developed to track the following
  constructs
• Ages covered by the measure
• Disparities in performance on the measure
• Sites/Types of care addressed by the measure
• Aspects of the Care Continuum addressed by the
  measure
• e.g. Outpatient, Inpatient, Mental Health, Dental
• Aspects of the System Continuum addressed by the
  measure
• e.g. Structure, Process, Outcomes, Efficiency
• Types of Entities Using the measure
• e.g. Medicaid programs, health plans,
  clinics/providers, researchers
• The Data Source for the measure
• e.g. Administrative, Medical Records, Survey

16
Process for this Meeting

• Nomination and VFI process
• (VFI Validity, feasibility, and importance)
• Review interval process
• Vote to select only one of nearly same measures

17
Process for this meeting

• Pruning
• Discuss and confirm domains on the balancing
  grid
• Reaffirm the concepts of a core, grounded,
  parsimonious set of measures
• Discuss measures by legislative category (e.g
  prevention, health promotion PHP)
• Rank order within each category or subcategory
• Retain top 33 in each category
• Present this set on the balancing grid after the
  meeting today

18
Process for this meeting

• Creation of a unified whole
• Review balancing grid from Thursday
• Identify and acknowledge holes
• Vote/rank remaining measures as a whole set
• Review on the balancing grid for varying numbers
  of measures (10-15-20-25)
• Discuss the number for a parsimonious set
• Vote for a set by size
• Record group comments about the nominated core
  set