Measuring Quality improving health system performance using indicators

1
Measuring Quality - improving health system
performance using indicators

• Presented by
• Dr Vin McLoughlinAssistant SecretaryHealth
  Priorities BranchDepartment of Health and
  Ageing12 June 2003

2

• Countries represented were UK, USA, NZ, Canada
  and Australia
• Chaired by Arnold Epstein (Harvard)
• Supported by Gerry Anderson and Peter Hussey
  (John Hopkins)

3
The Context for the Commonwealth Fund Project

• The search for indicators of quality for the
  health care system
• history
• future demands
• Why is this so difficult?
• complex product
• multiple perspectives
• multiple facets of provision of services

4
FOCUS

• Describe international work on the development of
  quality indicators through the Commonwealth Fund
• Commenced in September 1998
• OECD in January 2003

5

• International - the interest is in the system
• Is evidence global?
• What are the boundaries of public health?
• Blend of practical and conceptual development

6
Can we measure the quality of care provided by
different countries health care systems? -in
order to identify where improvements can be made?
7
The key objective

• To find 15-20 indicators of quality of care that
  can provide the basis for international
  comparison across the five countries
• The indicators need to represent the various
  aspects of quality of care that are appropriate
  to performance measurement
• The practical product - a report to each
  countrys Ministers

8
Summary of the approach

• 1 Do we have a shared view of quality of care?
• 2 What aspects of care need to be covered?
• 3 What types of measures are needed to identify
  the quality of care provided?
• 4 What measures of quality are currently being
  collected nationally, or sub-nationally?
• 5 What criteria should be used for selecting
  indicators?
• 6 Which of the indicators currently being
  collected inform a comparison of quality of care
  across the five countries?
• 7 Is there independent evidence that these
  indicators represent quality of care?

9

• 8 For each of these, are we collecting the
  same information? Are there differences
  in the definitions, eligibility criteria and
  specifications used?
• 9 If they are not the same, which differences
  matter?
• 10 What does the data look like?
• 11 What does it tell us?

10

• 12 Where are the gaps?
• 13 What do we need to do to improve our capacity
  to provide comparative assessments of quality?
• 14 Is this work applicable to other countries?

11
1 What do we mean by quality of care?

• Do we mean the same things in different countries?

12
The Group reviewed WHO, OECD, Canadian, UK and
Australian frameworks

• There were some differences across the countries
  in terms of the number of sub-domains
• The group selected the domains with the greatest
  convergence across all five countries - despite
  some differences in expression

13
National Health Performance Framework
14
National Health Performance Framework (cont)
15
2 What aspects of care should be covered? -
Shared domains for quality indicators

• Safety
• Effectiveness/appropriateness
• Patient centered /responsiveness/acceptability
• Timely/access

16
?

• Continuity, capability, competence,
  sustainability
• Efficiency
• Equity
• ? Second wave domains

17
Continuity, capability, competence, sustainability

• Measures of input
• Reliant on the overall resources available

18
Efficiency

• Measures of throughput
• Represent increased capacity to treat and this
  may be important

19
Equity

• Defined differently across countries and
  therefore measured differently
• Requires national data capable of analysis for
  population subgroups and therefore much more
  demanding of the data

20
What types of measures are needed to identify the
quality of care provided?

• Population health outcomes eg mortality, DALYs
  (Disability adjusted life years) etc are
  attributable to many social factors
• Measures indicative of quality of care - outcomes
  attributable to health care services

21
Safety - unique measurement difficulties

• Many treatments about delaying outcomes
• Judgement about contribution of harm and
  particular interventions
• Increased reporting means more vigilance
• Harm is statistically dissociated from particular
  events from complex sequence to health care
  treatments - and rare

22
Effectiveness

• Collected all the measures of quality in each
  country being used nationally or sub-nationally

23
What criteria should be used for selecting
indicators?
24
Criteria

• Be attributable to health service interventions
• Be worth measuring
• Be supported by evidence
• Be understood by people who need to act
• Be feasible to collect and report
• The report should be capable of presentation to
  the public

25
Which of the indicators currently being collected
inform a comparison of quality of care across the
five countries

• Is it clear that either a high or a low level is
  good quality?
• Variations not helpful eg Caesarean rates,
  tonsillectomies, hospitalisation rates per se

26
Is there independent evidence that these
indicators represent quality of care in each of
the five countries?
27

• For each of these, are we collecting the
• same information?
• Are there differences in the definitions,
• eligibility criteria and specifications used?

28

• If they are not the same, which differences
  matter?
• What does the data look like?
• What does it tell us?

29
Commonwealth Fund Indicator Development
30
Commonwealth Fund Indicator Development

• FINAL SET
• Limited by availability of comparable
  dataAccessibility and continuity added from CMWF
  surveysCancer incidence and survival for key
  cancers, breast, cervix, leukaemias, lung,
  colorectal, Non-Hodgkins lymphoma
• Mortality rates at 30 days for MI and stroke
• Asthma mortality rates, Suicide rates, Transport
  rates liver, kidney and transplant survival
  rates
• Other condition specific prevalence and mortality
  rates to provide some background contextual
  information within countries
• Some process indicators for population health
  focusing on prevention or early detection in the
  population breast and cervical screening,
  vaccination rates, incidence of vaccine
  preventable diseases and smoking rates
• Clinical Indicators ?
• Responsiveness indicators ?
• Total 45

31
Summary of types of Effectiveness indicators

• Unambiguous evidence that intervention works
  frequency of use
• Time from presentation to complications or death
  (but this depends on stage at presentation)

32
Commonwealth Fund Indicators

• Safety
• (Agency for Healthcare Research and Quality)
• Sentinel events, adverse event rates
• Accessibility (OECD waiting times project)
• Waiting times for hip, knee, cataract and
  cardiac surgery
• Transplants

33
Commonwealth Fund Indicators

• Responsiveness
• Commonwealth Fund 5 country questionnaire
• WHO Responsiveness survey

34
Responsiveness
35
Responsiveness

• Responsiveness is how well the health care
  system performs from the perspective of the
  population and clients it serves.This term
  includes concepts such as access (in terms of
  both time, cost and culture) and the patients
  experience of care (the way people are treated
  rather than the technical care provided), as well
  as support to ensure that patients, and people at
  risk, are able to actively participate in their
  own care.

36
What measures will tell us meaningful/useful
things for comparison of performance?

• Prompt attention (prompt access)
• Dignity/confidentiality (how treated)
• Communication/(including capacity to complain)
  (how treated)
• Choices in treatment and management (including
  self-management and social support) (how served)

37
OECD HQI PROJECT

• Extending this work to 20 countries
• Other countries have more data can re-examine
  some areas lost earlier
• Some countries have clinical values eg HbA1c lt 6
  etc

38

• Highlighted paucity of data
• Highlighted areas for improvement
• Pointed to a need for further targeted investment
  in data development