Performance Improvement Projects Technical Assistance Health Maintenance Organizations - PowerPoint PPT Presentation

1 / 64
About This Presentation
Title:

Performance Improvement Projects Technical Assistance Health Maintenance Organizations

Description:

www.myfloridaeqro.com – PowerPoint PPT presentation

Number of Views:114
Avg rating:3.0/5.0

less

Transcript and Presenter's Notes

Title: Performance Improvement Projects Technical Assistance Health Maintenance Organizations


1
Performance Improvement Projects Technical
Assistance Health Maintenance Organizations
Provider Service NetworksWednesday, March 28,
2007300 p.m. 500 p.m.
  • Cheryl L. Neel, RN, MPH, CPHQ
  • Manager, Performance Improvement Projects
  • David Mabb, MS
  • Sr. Director, Statistical Evaluation

2
Presentation Outline
  • PIP Overall Comments
  • Aggregate MCO PIP Findings
  • Aggregate HMO Specific Findings
  • Technical Assistance with Group Activities
  • Study Design
  • Study Implementation
  • Quality Outcomes Achieved
  • Questions and Answers

3
Key PIP Strategies
  1. Conduct outcome-oriented projects
  2. Achieve demonstrable improvement
  3. Sustain improvement
  4. Correct systemic problems

4
Validity and Reliability of PIP Results
  • Activity 3 of the CMS Validating Protocol
    Evaluating overall validity and reliability of
    PIP results
  • Met Confidence/High confidence in reported PIP
    results
  • Partially Met Low confidence in reported PIP
    results
  • Not Met Reported PIP results not credible

5
Summary of PIP Validation Scores
6
Proportion of PIPs Meeting the Requirements for
Each Activity
7
Aggregate Valid Percent Met
I
II
III
IV
V
VI
VII
VIII
IX
X
8
HMO Specific Findings
  • 53 PIPs submitted
  • Scores ranged from 5 to 98
  • Average score was 65
  • Assessed evaluation elements were scored as Met
    64 of the time

9
Summary of HMO Validation Scores
10
Study Design
  • Four Components
  • Activity I. Selecting an Appropriate Study Topic
  • Activity II. Presenting Clearly Defined,
    Answerable Study Question(s)
  • Activity III. Documenting Clearly Defined Study
    Indicator(s)
  • Activity IV. Stating a Correctly Identified
    Study Population

11
Activity I. Selecting an Appropriate Study Topic
HMO Overall Score
12
Activity I. Selecting an Appropriate Study Topic
  • Results
  • 92 percent of the six evaluation elements were
    Met
  • 8 percent were Partially Met or Not Met
  • None of the evaluation elements were Not
    Applicable or Not Assessed

13
Activity I Review the Selected Study Topic
  • HSAG Evaluation Elements
  • Reflects high-volume or high-risk conditions (or
    was selected by the State).
  • Is selected following collection and analysis of
    data (or was selected by the State).
  • Addresses a broad spectrum of care and services
    (or was selected by the State).
  • Includes all eligible populations that meet the
    study criteria.
  • Does not exclude members with special health care
    needs.
  • Has the potential to affect member health,
    functional status, or satisfaction.
  • Bolded evaluation elements show areas for
    improvement

14
Activity II. Presenting Clearly Defined,
Answerable Study Question(s) - HMO Overall Score
15
Activity II. Presenting Clearly Defined,
Answerable Study Question(s)
  • Results
  • 43 percent of the two evaluation elements were
    Met
  • 57 percent were Partially Met or Not Met
  • None of the evaluation elements were Not
    Applicable or Not Assessed

16
Activity II Review the Study Question(s)
  • HSAG Evaluation Elements
  • States the problem to be studied in simple terms.
  • Is answerable.
  • Bolded evaluation elements show areas for
    improvement

17
HMO Overall Score for Each Evaluation Element
Activity III
18
Activity III. Documenting Clearly Defined Study
Indicator(s)
  • Results
  • 56 percent of the seven evaluation elements were
    Met
  • 27 percent were Partially Met or Not Met
  • 16 percent were Not Applicable or Not Assessed

19
Activity III Review Selected Study Indicator(s)
  • HSAG Evaluation Elements
  • Is well defined, objective, and measurable.
  • Are based on practice guidelines, with sources
    identified.
  • Allows for the study question to be answered.
  • Measures changes (outcomes) in health or
    functional status, member satisfaction, or valid
    process alternatives.
  • Have available data that can be collected on each
    indicator.
  • Are nationally recognized measure such as HEDIS,
    when appropriate.
  • Includes the basis on which each indicator was
    adopted, if internally developed.
  • Bolded evaluation elements show areas for
    improvement

20
Activity IV. Stating a Correctly Identified Study
Population - HMO Overall Score
21
Activity IV. Stating a Correctly Identified
Study Population
  • Results
  • 53 percent of the three evaluation elements were
    Met
  • 43 percent were Partially Met or Not Met
  • 3 percent of the evaluation elements were Not
    Applicable or Not Assessed

22
Activity IV Review the Identified Study
Population
  • HSAG Evaluation Elements
  • Is accurately and completely defined.
  • Includes requirements for the length of a
    members enrollment in the managed care plan.
  • Captures all members to whom the study question
    applies.
  • Bolded evaluation elements show areas for
    improvement

23
Group Activity
24
Study Implementation
  • Three Components
  • Activity V. Valid Sampling Techniques
  • Activity VI. Accurate/Complete Data Collection
  • Activity VII. Appropriate Improvement Strategies

25
Activity V. Presenting a Valid Sampling Technique
- HMO Overall Score
26
Activity V. Presenting a Valid Sampling Technique
  • Results
  • 12 out of the 53 PIP studies used sampling.
  • 11 percent of the six evaluation elements were
    Met.
  • 12 percent were Partially Met or Not Met.
  • 77 percent of the evaluation elements were Not
    Applicable or Not Assessed.

27
Activity V Review Sampling Methods
  • This section is only validated if sampling is
    used.
  • HSAG Evaluation Elements
  • Consider and specify the true or estimated
    frequency of occurrence. (N7)
  • Identify the sample size. (N7)
  • Specify the confidence level to be used. (N5)
  • Specify the acceptable margin of error. (N5)
  • Ensure a representative sample of the eligible
    population. (N5)
  • Ensure that the sampling techniques are in
    accordance with generally accepted principles of
    research design and statistical analysis. (N5)

28
Populations or Samples?
All?
Some?
  • Generally,
  • Administrative data uses populations
  • Hybrid (chart abstraction) method uses samples
    identified through administrative data

29
Activity VI. Specifying Accurate/Complete Data
Collection - HMO Overall Score
30
Activity VI. Specifying Accurate/Complete Data
Collection
  • Results
  • 34 percent of the eleven evaluation elements were
    Met
  • 24 percent were Partially Met or Not Met
  • 42 percent of the evaluation elements were Not
    Applicable or Not Assessed

31
Activity VI Review Data Collection Procedures
  • HSAG Evaluation Elements
  • Clearly defined data elements to be collected.
  • Clearly identified sources of data.
  • A clearly defined and systematic process for
    collecting data that includes how baseline and
    remeasurement data will be collected.
  • A timeline for the collection of baseline and
    remeasurement data.
  • Qualified staff and personnel to collect manual
    data.
  • A manual data collection tool that ensures
    consistent and accurate collection of data
    according to indicator specifications.
  • Bolded evaluation elements show areas for
    improvement

32
Activity VI Review Data Collection Procedures
(cont.)
  • HSAG Evaluation Elements
  • A manual data collection tool that supports
    interrater reliability. Clear and concise written
    instructions for completing the manual data
    collection tool. An overview of the study in the
    written instructions.
  • Administrative data collection algorithms that
    show steps in the production of indicators.
  • An estimated degree of automated data
    completeness (important if using the
    administrative method).
  • Bolded evaluation elements show areas for
    improvement

33
Where do we look for our sources of data?
34
Baseline Data Sources
  • Medical Records
  • Administrative claims/encounter data
  • Hybrid
  • HEDIS
  • Survey Data
  • MCO program data
  • Other

35
Activity VII. Documenting the Appropriate
Improvement Strategies - HMO Overall Score
36
Activity VII. Documenting the Appropriate
Improvement Strategies
  • Results
  • 28 percent of the four evaluation elements were
    Met
  • 21 percent were Partially Met or Not Met
  • 51 percent of the evaluation elements were Not
    Applicable or Not Assessed

37
Activity Seven Assess Improvement Strategies
  • HSAG Evaluation Elements
  • Related to causes/barriers identified through
    data analysis and Quality Improvement (QI)
    processes.
  • System changes that are likely to induce
    permanent change.
  • Revised if original interventions are not
    successful.
  • Standardized and monitored if interventions are
    successful.
  • Bolded evaluation elements show areas for
    improvement

38
Determining Interventions
  • Once you know how you are doing at baseline,
    what interventions will produce meaningful
    improvement in the target population?

39
First Do A Barrier Analysis
  • What did an analysis of baseline results show ?
  • How can we relate it to system improvement?
  • Opportunities for improvement
  • Determine intervention(s)
  • Identify barriers to reaching improvement

40
How was intervention(s) chosen?
  • By reviewing the literature
  • Evidence-based
  • Pros cons
  • Benefits costs
  • Develop list of potential interventions -- what
    is most effective?

41
Types of Interventions
  • Education
  • Provider performance feedback
  • Reminders tracking systems
  • Organizational changes
  • Community level interventions
  • Mass media

42
Choosing Interventions
  • Balance
  • potential for success with ease of use
  • acceptability to providers collaborators
  • cost considerations (direct and indirect)
  • Feasibility
  • adequate resources
  • adequate staff and training to ensure a
    sustainable effort

43
Physician Interventions Multifaceted Most
Effective
  • Most effective
  • real-time reminders
  • outreach/detailing
  • opinion leaders
  • provider profiles
  • Less effective
  • educational materials (alone)
  • formal CME program without enabling or
    reinforcing strategies

44
Patient Interventions
  • Educational programs
  • Disease-specific education booklets
  • Lists of questions to ask your physician
  • Organizing materials flowsheets, charts,
    reminder cards
  • Screening instruments to detect complications
  • Direct mailing, media ads, websites

45
Evaluating Interventions
  • Does it target a specific quality indicator?
  • Is it aimed at appropriate stakeholders?
  • Is it directed at a specific process/outcome of
    care or service?
  • Did the intervention begin after baseline
    measurement period?

46
Interventions Checklist
  • Analyze barriers (root causes)
  • Choose understand target audience
  • Select interventions based on cost-benefit
  • Track intermediate results
  • Evaluate effectiveness
  • Modify interventions as needed
  • Re-Measure

47
Group Activity
48
Quality Outcomes Achieved
  • Three Components
  • Activity VIII. Presentation of Sufficient Data
    Analysis and Interpretation
  • Activity IX. Evidence of Real Improvement
    Achieved
  • Activity X. Data Supporting Sustained
    Improvement Achieved

49
Activity VIII. Presentation of Sufficient Data
Analysis and Interpretation - HMO Overall Score
50
Activity VIII. Presentation of Sufficient Data
Analysis and Interpretation
  • Results
  • 35 percent of the nine evaluation elements were
    Met
  • 15 percent of the evaluation elements Partially
    Met or Not Met
  • 50 percent of the evaluation elements were Not
    Applicable or Not Assessed

51
Activity VIII Review Data Analysis and
Interpretation of Study Results
  • HSAG Evaluation Elements
  • Is conducted according to the data analysis plan
    in the study design.
  • Allows for generalization of the results to the
    study population if a sample was selected.
  • Identifies factors that threaten internal or
    external validity of findings.
  • Includes an interpretation of findings.
  • Is presented in a way that provides accurate,
    clear, and easily understood information.

52
Activity VIII Review Data Analysis and
Interpretation of Study Results (cont.)
  • HSAG Evaluation Elements
  • Identifies initial measurement and remeasurement
    of study indicators.
  • Identifies statistical differences between
    initial measurement and remeasurement.
  • Identifies factors that affect the ability to
    compare initial measurement with remeasurement.
  • Includes the extent to which the study was
    successful.
  • Bolded evaluation elements show areas for
    improvement

53
Changes in Study Design?
  • Study design should be same as baseline
  • Data source
  • Data collection methods
  • Data analysis
  • Target population or sample size
  • Sampling methodology
  • If change
  • rationale must be specified appropriate

54
Activity IX. Evidence of Real Improvement - HMO
Overall Score
55
Activity IX. Evidence of Real Improvement
  • Results
  • 22 percent of the four evaluation elements were
    Met
  • 29 percent were Partially Met or Not Met
  • 50 percent of the evaluation elements were Not
    Applicable or Not Assessed

56
Activity IX Assess the Likelihood that Reported
Improvement is Real Improvement
  • HSAG Evaluation Elements
  • The remeasurement methodology is the same as the
    baseline methodology.
  • There is documented improvement in processes or
    outcomes of care.
  • The improvement appears to be the result of
    intervention(s).
  • There is statistical evidence that observed
    improvement is true improvement.
  • Bolded evaluation elements show areas for
    improvement

57
Statistical Significance Testing
Time Periods Measurement Periods Numerator Denominator Rate or Results Industry Benchmark Statistical Testing and Significance
CY 2003 Baseline 201 411 48.9 60 N/A
CY 2004 Re-measurement 1 225 411 54.7 60 Chi-square 2.8 P-value 0.09387 NOT SIGNIFICANT AT THE 95 CONFIDENCE LEVEL
58
Activity X. Data Supporting Sustained Improvement
Achieved - HMO Overall Score
59
Activity X. Data Supporting Sustained
Improvement Achieved
  • Results
  • 6 percent of the one evaluation element was Met
  • 36 percent was Partially Met or Not Met
  • 58 percent of the evaluation element was Not
    Applicable or Not Assessed

60
Activity X Assess Whether Improvement is
Sustained
  • HSAG Evaluation Elements
  • Repeated measurements over comparable time
    periods demonstrate sustained improvement, or
    that a decline in improvement is not
    statistically significant.
  • Bolded evaluation elements show areas for
    improvement

61
Quality Outcomes Achieved
Baseline 1st Yr
Demonstrable Improvement
Sustained Improvement
62
Sustained Improvement
  • Modifications in interventions
  • Changes in study design
  • Improvement sustained for 1 year

63
HSAG Contact Information
Cheryl Neel, RN, MPH,CPHQ Manager, Performance
Improvement Projects cneel_at_hsag.com 602.745.6201
Denise Driscoll Administrative Assistant ddriscoll
_at_hsag.com 602.745.6260
64
Questions and Answers
Write a Comment
User Comments (0)
About PowerShow.com