Knowing what works in Health Care: A roadmap for the nation

1
Knowing what works in Health CareA roadmap for
the nation

• IOM Committee on Reviewing Evidence to Identify
  Highly Effective Clinical Services
• January 2008

2
Committee ChargeTo recommend

• An approach to identifying highly effective
  services
• A process to evaluate evidence on clinical
  effectiveness
• An organizational framework for using evidence
  reports to make recommendations

3
-Evidence-Based Practice Centers -Proprietary
firms
4
Strengths of the system

• Some excellent role models for transparent
  guideline development
• A network of skilled SR teams
• Some GL users have a lot of muscle
• Coverage decisions, practice measures

5
The problems

• SRs and GLs often lack scientific rigor
• Body of evidence is often weak.
• Difficult for user to connect recommendations to
  the evidence
• No standard language for rating strength of
  evidence or recommendation.
• No standard process for developing
  recommendations.
• No expectation for clear explanations
• Bias and conflict hidden from view
• Duplicated effort and conflicting recommendations

6
Policy in the US

• A work in Progress a National Program for
  clinical effectiveness research
• ? Fund primary research, ? Develop GLs
• everyone is calling for it
• legislation being written
• Institute of Medicine committee ? high-level
  blueprint

7
Comparative clinical effectiveness research

• Primary investigation
• RCTs, cohort studies, measure test performance.
• Head-to-head comparisons of drugs, tests
• Summative research
• systematic reviews and meta-analyses of primary
  investigations

8
IOM committee charge

• Primary investigation
• RCTs, cohort studies, measure test performance.
• Head-to-head comparisons of drugs, tests
• Summative research
• systematic reviews
• meta-analyses

IOM report
9
The Committees recommendations
10
Recommendation high level

• Congress should direct Secretary DHHS to
  designate a single entity to produce credible,
  unbiased information on CE.
• Would set priorities for and fund SRs.
• Develop a common language and stds for SRs and
  GLs.
• A forum to address conflicting GLs.

11
Setting priorities
The Program
12
Recommendation high level

• The Program should develop standards to minimize
  bias due to conflict of interest
• For priority setting
• For evidence assessment
• For recommendations

13
Setting priorities
The Program
14
Recommendation setting priorities

• The Program should appoint a standing committee
  (the Priority Setting Advisory Committee).
• The process of priority setting should be open,
  transparent, efficient, and timely.

15
Recommendation setting priorities

• Priorities should reflect the potential to
• Improve health outcomes
• Reduce the burden of disease and health
  disparities
• Eliminate undesirable variation
• Reduce the economic burden of disease
• Reduce the economic burden of treatment
• PSAC members should have a broad range of
  expertise and interests
• Minimize committee bias due to COI

16
Setting priorities
The Program
17
Problems with systematic reviewsthe IOM
committees view

• Current practice falls short of ideal
• Methods poorly documented
• Plan poorly executed
• Quality of studies not assessed
• Inappropriate statistical techniques

18
Assessing Evidence recommendations

• The Program should develop and require
• Evidence-based standards for SRs.
• A common language for stating the strength of
  evidence
• The Program should invest in
• Developing better methods
• The professional workforce to do SRs.

19
Developing Practice Guidelines Recommendations
20
Done by the Program
21
Hybrid vs. Agency Model

• Hybrid Model

• Agency Model

22
Hybrid vs. Agency Model

• Hybrid Model

• Agency Model

23
Done by the Program
Done By Existing Entities
24
Rationale for the hybrid model

• Reduce political opposition to creating The
  Program
• Render the Program less vulnerable to political
  pressure.
• Engender trust through the clinical credibility
  of professional organizations.
• Avoid duplication of effort by The Program and
  professional organizations.

25
Wont the hybrid model simply perpetuate the
chaotic system we have now?
26
Wont the hybrid model simply perpetuate the
chaotic system we have now?

• Not if all guideline producers adhere to The
  Programs standards for process and language for
  rating the strength of the evidence and
  recommendations.

27
Tie their credibility to adherence to standards
of good practice
28
Recommendations developing guidelines

• Guideline groups should use The Programs
  standards, document their adherence, and publish
  the documentation.
• Users of guidelines should preferentially use
  recommendations developed according to Program
  standards.

29
Challenges for The Program

• Developing consensus on standards for SRs and
  GLs.
• Language
• Process
• Developing brand credibility
• Meeting the Programs standards becomes a mark of
  GL quality that GL users require as a matter of
  corporate integrity.

30
The IOM Committees recommendations about
priority-setting
31
Getting nominations
Source IOM report What works., 2008
32
Topic nominations received by AHRQ(2005 and 2006)
Source IOM report What works., 2008
33
AHRQ Categories of Topics Nominated
Source IOM report What works., 2008
34
Priority-setting Criteria
Source IOM report What works., 2008
35
The IOM committees conclusions

• We lack evidence to inform a choice between
  priority-setting methods.
• AHRQ has lots of experience with an open process.
• Consensus exists on the key variables (ex.
  Variation)
• The Program needs a fast track review process
  with its own priority setting process
• Health plans need to decide about new
  technologies quickly

36
The IOM committees conclusions(cont.)

• The Program needs a timely process for revisiting
  earlier reviews
• Another task competing for SR resources
• The evidence base changes rapidly
• Shojania et al, 2007 found that 57 of SRs had a
  signal that would change the basic conclusion in
  a median of 5 years.

37
The IOM committee recommendations

• Appoint a Priority Setting Advisory Committee
• Process should be open, transparent, efficient,
  and timely.
• Key criteria include
• Impact on health outcomes across lifespan
• Reduce disease burden and disparities
• Reduce variation
• Take costs into account
• Seek a balanced committee to reduce impact of
  bias due to conflict of interest.

38
Principles to guide priority setting process

• Consistency
• Develop rules and stick to them
• Efficiency
• Objectivity
• Stick to the criteria and the evidence avoid
  bias due to conflict of interest
• Responsiveness
• Meet the needs of decision makers
• Transparency
• Define methods and post them meetings are open
  to anyone

39
The IOM committees two paramount criteria

• Priorities should reflect what patients and
  doctors want to know.
• They should also reflect the potential of the
  intervention to improve outcomes that are
  important to patients.

40
Two Examples of Priority-setting

• 1992 IOM committees recommended process
• AHRQs process for selecting topics for the
  Clinical Effectiveness Reviews Program

41
IOM committee to recommend a priority setting
process (1992)

• The report described an explicit process for
  developing a priority list.
• Charles Phelps, Ph.D contributed the basic idea.
• The agency for which the process was intended
  folded, and the method never got much traction.

42
W the weight for a criterion S the score for
the criterion
Source IOM Committee on Priority Setting, 1992
43

• Prevalence of condition
• Burden of illness
• Cost
• Variation in rates of use
• Potential of TA to change health outcomes
• Potential of TA to change costs
• Potential of TA to change ELS issues

44
(No Transcript)
45
(No Transcript)
46
Priority Score S Wn x lnSn
47
Priority setting for AHRQs Clinical
Effectiveness Review Program

• AHRQ invites the public to nominate topics on a
  public website.
• Quarterly, AHRQ collates the topics.
• Scientific Resource Center writes a topic summary.

48
AHRQs five priority-setting criteria

• Prevalence of a condition
• Burden of a condition
• Cost of care of a condition
• Disproportionate representation of the condition
  in the Medicare population
• Potential for impact

Source Jean Slutsky, personal communication