Topic Selection for the AHRQ Effective Health Care Program

1
Topic Selection for the AHRQ Effective Health
Care Program
Evelyn P. Whitlock, MD, MPH Associate
Director Oregon Evidence-based Practice
Center Oregon Health Science University Center
for Health Research, Kaiser Permanente Northwest
Baltimore, MD July 10, 2008
2
Acknowledgments

• Mark Helfand Janelle Guirguis-Blake
• Sarah Lopez Stephanie Chang
• Alisha Wilson Bill Lawrence
• Michelle Eder Yen-Pin Chiang
• Nicole Floyd Supriya Janakiraman
• Anna Sosnowski Elise Berliner
• Pam Curtis Lia Hotchkiss
• Mylia Christensen Scott Smith
• Valerie King Jean Slutsky

3
Effective Health Care Program 2005 - Present

• Initial work in this program originates from
  Section 1013 of the Medicare Prescription Drug,
  Improvement, and Modernization Act (MMA) of 2003
• Section 1013 authorizes AHRQ to conduct and
  support research with a focus on outcomes,
  comparative clinical effectiveness, and
  appropriateness of pharmaceuticals, devices, and
  health care services
• One aspect of this work includes reviewing and
  synthesizing knowledge
• The 15 AHRQ Evidence-based Practice Centers
  systematically review published and unpublished
  scientific evidence

4
Scientific Resource Center for the Effective
Health Care Program
The SRC supports the EHC Program as a whole, with
specific responsibilities including

• Assist AHRQ with all stages of research topic
  development
• Provide scientific and technical support for
  systematic review and outcomes research
• Collaborate with partners and EHC Program
  stakeholders

5
Scientific Resource Center for the Effective
Health Care Program
The SRC supports the EHC Program as a whole, with
specific responsibilities including

• Assist AHRQ with all stages of research topic
  development
• Provide technical support for topic generation,
  topic selection, and research development
• Provide scientific and technical support for
  systematic review and outcomes research
• Collaborate with partners and EHC Program
  stakeholders

6
Historical Overview

• History of low yield of nominations that could be
  developed into quality systematic reviews
• Open, somewhat passive nomination process
• Recently revised (2008) process to address
• Volume and quality of nominations
• Transparency of process
• Pilot testing since March 2008 (W.I.P.)
• Discuss process and early results
• Challenges

7
Five Guiding Program Principles for EHC Topic
Selection (Identification Prioritization)

• Clearly identify the overall goals/strategic
  purpose of the activity
• Involve stakeholders in the selection process
• Clearly define implement criteria for selection
• Achieve transparency
• Undertake process evaluation and improvement

IOM 2008 Priority Setting Principles
Consistency (3) Efficiency (1,3) Objectivity
(2,3,4) Responsiveness (2,5) Transparency (3,4)
8
EHC Program Principle 1Overall Goals Strategic
Purpose

• Priority health conditions guide the focus of
  research
• Robust research topics represent important
  decisional dilemma for consumers, clinicians,
  patients, payers, policy-makers, and other
  stakeholders
• Research agenda is stakeholder-driven

9
DHHS Priority Conditions

• Cardiovascular disease, including stroke and
  hypertension
• Cancer
• Pulmonary disease/asthma/pneumonia
• Arthritis and nontraumatic joint disorders
• Dementia including Alzheimer's disease
• Diabetes mellitus
• Peptic ulcer disease and dyspepsia
• Depression and other mood disorders
• Developmental delays, attention-deficit
  hyperactivity disorder and autism
• Functional limitations and disability
• Infectious diseases including HIV/AIDS
• Obesity
• Pregnancy including pre-term birth
• Substance abuse

10
Principle 2 Engage Stakeholders in Topic
Selection for the EHC Program
Convened EHC Program Stakeholder Panel and
Program Priorities Work Group
Oregon EPC
11
Topic Selection ProcessFrom Ideas
Important Research Topics
Important Research Topics
Ideas/ Nominations
12
EHC Program Principle 3Define Selection Criteria
(18 criteria)
3 criteria 7 criteria 1 criterion 1
criterion 6 criteria
Same criteria considered for every nomination in
a structured, hierarchical manner
13
Selection Criteria
1a. Represents a healthcare drug, intervention,
device, or technology available (or soon to be
available) in the US 1b. Relevant to 1013
enrollees (Medicare, Medicaid, S-CHIP, other
federal healthcare programs) 1c. Represents one
of the priority conditions designated by the
Department of Health and Human Services (DHHS)
14
Experience to Date2008 Nominations Received by
Priority Condition
15
Selection Criteria
Nomination IV Catheter-Associated Bloodstream
Infection Prevention Criterion 2b Is of high
public interest affects health care
decision-making, outcomes, or costs for a large
proportion of the US population or for a priority
population in particular Example Yes, topic is
of high public interest and affects health care
decision-making for 200,000 US patients annually.
The high public interest is evident through the
recent recognition of this topic by the Joint
commission, the Institute of Medicine (IOM),
Centers for Medicare Medicaid Services (CMS),
and the Institute for Healthcare Improvement
(IHI). CMS will soon stop paying hospitals for
eight hospital-acquired conditions that have
evidence-based prevention guidelines, including
catheter-associated infections. The decision by
CMS to not reimburse hospitals for the additional
cost of treating patients who acquire a vascular
catheter-associated infection during their
hospitalization has made infection prevention
even more of a priority.
16
Selection Criteria
Nomination Over-the-Counter (OTC) Cold Remedies
for Children Criterion 3 Would not be redundant
(i.e., the proposed topic is not already covered
by available or soon-to-be available high-quality
systematic review by AHRQ or others) Example
Topic would be redundant given the Cochrane
review published in 2008. This review looked at
the limited amount of data available for this
topic and concluded that the evidence does not
support the effectiveness of over-the-counter
preparations for children with acute cough.
17
Selection Criteria
Nomination 17-OH Progesterone Criterion 4
Effectively utilizes existing research and
knowledge by considering 1) adequacy (type and
volume) of research for conducting a systematic
review, and 2) newly available evidence
(particularly for updates or new technologies) Ex
ample The most recent review was published in
2006 and had a last search date of 2004. Given
that the search dates for this review are
somewhat dated, we conducted a brief feasibility
scan to determine whether a new systematic review
would be useful at this time. In our brief scan
we found there to be several trials, case series,
and observational studies published between 2005
and 2008. There are several ongoing and
recruiting trials for this topic listed on
clinicaltrials.gov and the NICHD website.
18
Selection Criteria
Nomination Preconception Programs Criterion 5e
Addresses inequities, vulnerable populations
(including issues for patient subgroups) Example
Yes, this nomination addresses inequities
including access to preconception and
interconception care for low-income women and
addresses vulnerable populations including
pregnant women and infants.
19
EHC Program Principle 3 Implement Selection
Criteria

• Internal Topic Triage group composed to represent
  scientific, stakeholder, and programmatic
  perspectives
• Internal Topic Triage group meets biweekly to
  review and vote on topic nomination briefs
• Current focus on identification process (triaging
  topics out)
• Pilot testing processes of gathering
  appropriate information
• Working to generate more robust topics
• Prioritization among selected topics will be
  implemented when there is an adequate pipeline
• Will engage stakeholder group(s) in organized way
  to prioritize topics

20
2008 nominations Results to date
21
Principle 4 TransparencyPrinciple 5 Process
Evaluation Improvement

• Selection criteria directly map to outcomes
  (decisions)
• Will soon be linked to nomination form on public
  website so nominators can see how their topic
  will be evaluated
• Feedback to nominators at least at point of
  receipt and final disposition of nomination
• Other processes under development with
  consultation from Stakeholder Panel

22
Ongoing Challenges in Identifying Prioritizing
Topics for New Systematic Reviews

• Addressing redundancy among nominations
• Many nominations are not novel
• 12/39 represent existing systematic reviews or
  in-process topics
• Some nominations are not unique
• 5 topics were nominated by two different
  stakeholders
• Ensuring relevance to stakeholders
• Distinguishing true duplication do existing
  products meet nominators needs?
• Some apparently redundant nominations may be
  failure of translation of existing products, need
  for different products, or needed updates
• Clarifying and developing ideas into research
  topic nominations
• Engaging multiple stakeholder groups with same
  product
• Accomplishing diverse tasks in a timely and
  transparent manner

23
Challenging diversity of current research topic
nominations
24
Topic Brief Time Effort Summary (does not
include additional nominator contacts)

• Heavily depends on the complexity of the topic
  nomination and the state in which it came in.

• Topic briefs range from 3 to 44 pages and require
  10 to 40 minutes to discuss
• Redundant topics have a range of 1-20 ( x 4)
  existing systematic reviews supporting the final
  recommendation
• Topics moving forward for refinement have a range
  of 0-5 ( x 2) existing systematic reviews
  supporting the final recommendation

25
Summary

• We have implemented a revised topic selection
  process that has improved our previous process in
  important ways
• Of the four process criteria set forth by IOM
  Knowing What Works report we have made progress
  on each
• Open (s/h input broadly incorporated)
• Transparent (explicit, consistent
  methodsjudgments linked to data and available)
• Efficient (avoids duplication and waste)
• Timely

26
Comments Questions
Oregon EPC