Introduction to the Methods Guide for Medical Test Reviews - PowerPoint PPT Presentation

Loading...

PPT – Introduction to the Methods Guide for Medical Test Reviews PowerPoint presentation | free to download - id: 646044-ODRjY



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Introduction to the Methods Guide for Medical Test Reviews

Description:

Introduction to the Methods Guide for Medical Test Reviews Prepared for: The Agency for Healthcare Research and Quality (AHRQ) Training Modules for Medical Test ... – PowerPoint PPT presentation

Number of Views:197
Avg rating:3.0/5.0
Slides: 38
Provided by: AlexP153
Learn more at: http://www.baylorcme.org
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Introduction to the Methods Guide for Medical Test Reviews


1
Introduction to the Methods Guide for Medical
Test Reviews
  • Prepared for
  • The Agency for Healthcare Research and Quality
    (AHRQ)
  • Training Modules for Medical Test Reviews Methods
    Guide
  • www.ahrq.gov

2
Learning Objectives
  • Describe the purpose of the Methods Guide for
    Medical Test Reviews.
  • Understand the unique features of medical test
    reviews.
  • Review the structure of the Methods Guide for
    Medical Test Reviews.

Matchar DB. Introduction to the Methods Guide for
Medical Test Reviews. In Methods guide for
medical test reviews. Available at
www.effectivehealthcare.ahrq.gov/medtestsguide.cfm
.
3
Medical Test Reviews (1 of 4)
  • Systematic reviews of medical tests help
    determine which tests can be relied on to improve
    health outcomes and the circumstances required
    for the tests to achieve this.
  • Systematic reviews developed under the auspices
    of the Evidence-based Practice Center (EPC)
    Program
  • Need to be technically comprehensive and
    practical
  • Need to be completed with limited time and
    resources

Matchar DB. Introduction to the Methods Guide for
Medical Test Reviews. In Methods guide for
medical test reviews. Available at
www.effectivehealthcare.ahrq.gov/medtestsguide.cfm
.
4
Medical Test Reviews (2 of 4)
  • The Methods Guide for Medical Test Reviews
  • Provides practical guidance for those who prepare
    and use systematic reviews.
  • Promotes consistency and rigor in how specific
    issues are addressed.
  • Complements the Methods Guide for Effectiveness
    and Comparative Effectiveness Reviews developed
    by the Agency for Healthcare Research and
    Quality.

Matchar DB. Introduction to the Methods Guide for
Medical Test Reviews. In Methods guide for
medical test reviews. Available at
www.effectivehealthcare.ahrq.gov/medtestsguide.cfm
. Agency for Health Care Research and Quality.
Methods Guide for Effectiveness and Comparative
Effectiveness Reviews. Available at
www.effectivehealthcare.ahrq.gov/methodsguide.cfm.
5
Medical Test Reviews (3 of 4)
  • Medical tests are defined here as any test used
    in a health care context, for example, screening,
    diagnosis, or prognosis.
  • Using medical tests depends on the context.
  • Pre-existing conditions
  • Results of other tests
  • Skill and knowledge of providers
  • Availability of therapeutic resources
  • And others

Matchar DB. Introduction to the Methods Guide for
Medical Test Reviews. In Methods guide for
medical test reviews. Available at
www.effectivehealthcare.ahrq.gov/medtestsguide.cfm
.
6
Medical Test Reviews (4 of 4)
  • Researchers have tended to focus on the ability
    of medical tests to
  • Conform to technical specifications
  • Classify patients into diagnostic or prognostic
    categories
  • Influence thought or actions by clinicians and
    patients
  • Rarely are medical tests evaluated in randomized
    controlled trials with representative patient
    populations and comprehensive measures of
    patient-relevant outcomes.

Matchar DB. Introduction to the Methods Guide for
Medical Test Reviews. In Methods guide for
medical test reviews. Available at
www.effectivehealthcare.ahrq.gov/medtestsguide.cfm
.
7
Development of the Methods Guide for Medical Test
Reviews (1 of 3)
  • Creation of the Methods Guide for Medical Test
    Reviews was guided by two tenets
  • Evaluation of a medical test must be linked to
    the context of its use
  • Medical tests are about improving patient outcomes

Matchar DB. Introduction to the Methods Guide for
Medical Test Reviews. In Methods guide for
medical test reviews. Available at
www.effectivehealthcare.ahrq.gov/medtestsguide.cfm
. Agency for Health Care Research and Quality.
Methods Guide for Effectiveness and Comparative
Effectiveness Reviews. Available at
www.effectivehealthcare.ahrq.gov/methodsguide.cfm.
8
Development of the Methods Guide for Medical Test
Reviews (2 of 3)
  • The first tenet is to consider the context.
  • Medical tests are not neutral reporters of
    reality. The context often affects sensitivity,
    specificity, and the clinical implications of a
    particular test result.
  • Therefore, systematic reviews must specify the
    context in which the test is being used.
  • The second tenet is to focus on patient outcomes.
  • The ultimate purpose of medical tests is to
    improve patient outcomes.
  • Improvement occurs by guiding the judgments of
    clinicians concerning disease state or treatment
    response.

Matchar DB. Introduction to the Methods Guide for
Medical Test Reviews. In Methods guide for
medical test reviews. Available at
www.effectivehealthcare.ahrq.gov/medtestsguide.cfm
.
9
Development of the Methods Guide for Medical Test
Reviews (3 of 3)
  • Most of the published literature
  • Fails to address the clinical impact of tests.
  • Focuses instead on test development and
    performance characteristics.
  • The first step in a medical test review must be
    to establish a link between the use of a test and
    patient outcomes.

Causal Chain Diagram Test ? Result ?
Categorization ? Decision ? Patient Outcome For
example high risk, disease present, disease
progression
Matchar DB. Introduction to the Methods Guide for
Medical Test Reviews. In Methods guide for
medical test reviews. Available at
www.effectivehealthcare.ahrq.gov/medtestsguide.cfm
.
10
Medical Tests Overview (1 of 2)
  • Typical uses of medical tests are to
  • Screen for the likelihood of a disorder.
  • Diagnose the presence of disease.
  • Assess response to treatment.
  • Assess probability of desirable/undesirable
    consequences.

Matchar DB. Introduction to the Methods Guide for
Medical Test Reviews. In Methods guide for
medical test reviews. Available at
www.effectivehealthcare.ahrq.gov/medtestsguide.cfm
.
11
Medical Tests Overview (2 of 2)
  • Typical types of medical tests
  • Laboratory tests
  • Radiological imaging
  • Patient history
  • Physical examinations
  • Scored questionnaires
  • For screening
  • For determining likely prognosis
  • For assessing the potential response to therapy

Matchar DB. Introduction to the Methods Guide for
Medical Test Reviews. In Methods guide for
medical test reviews. Available at
www.effectivehealthcare.ahrq.gov/medtestsguide.cfm
.
12
Challenges Specific to Medical Tests (1 of 2)
  • Assessing the impact of medical tests can be
    challenging.
  • Medical tests have several steps between
    conducting the test and determining the outcome
    of clinical importance.
  • Conversely, most treatments lead directly to
    intended results or adverse effects.
  • Consequences result from the less direct
    relationship between medical tests and patient
    outcomes.
  • Medical tests are often evaluated in isolation
    for their ability to detect a substance or a
    particular anatomic condition.
  • Medical tests are not evaluated in terms of their
    impact on overall health outcomes.

Matchar DB. Introduction to the Methods Guide for
Medical Test Reviews. In Methods guide for
medical test reviews. Available at
www.effectivehealthcare.ahrq.gov/medtestsguide.cfm
. Siebert U. Eur J Health Econ 20034(3)143-50 Ta
tsioni A, Zarin DA, Aronson N, et al. Ann Intern
Med 2005 Jun 21142(12 Pt 2)1048-55. PMID
15968029.
13
Challenges Specific to Medical Tests (2 of 2)
  • Randomized controlled trials, generally
    considered the strongest form of evidence, are
    rarely performed for medical tests.
  • Trials are difficult to carry out because of the
    numerous steps and the variables between the test
    and patient outcomes.

Matchar DB. Introduction to the Methods Guide for
Medical Test Reviews. In Methods guide for
medical test reviews. Available at
www.effectivehealthcare.ahrq.gov/medtestsguide.cfm
.
14
Using Medical Tests (1 of 2)
  • Medical tests can be likened to an electronic
    signal detector for the presence or absence of a
    clinical condition.
  • Signal detection theory (e.g., sensitivity,
    specificity) can be used to calculate disease
    probabilities for positive or negative test
    results.

Matchar DB. Introduction to the Methods Guide for
Medical Test Reviews. In Methods guide for
medical test reviews. Available at
www.effectivehealthcare.ahrq.gov/medtestsguide.cfm
. Green DM, Swets JA. Signal detection theory and
psychophysics. New York, NY Wiley 1966.
Reprinted 1988. Ledley RS, Lusted LB. Science
1959 Jul 3130(3366)9-21. PMID
13668531. Yerushalmy J. Public Health Rep 1947
Oct 362(40)1432-49. PMID 20340527.
15
Using Medical Tests
  • Medical test evaluation studies fall along a
    continuum related to the study objectives.

Study Objective Terms Used Examples
Test conforming to technical specifications Technical efficacy Technical quality of a radiological image
Test conforming to technical specifications Analytic validity Accuracy of a chemical assay for the target analyte Concordance of a commercial genetic test with the true genotype
Test classifying a patient into a disease/phenotype or prognosis category Diagnostic accuracy efficacy Clinical validity Test accuracy Test performance Performance characteristics Operating characteristics Sensitivity and specificity Positive and negative likelihood ratios Positive and negative predictive value Test yield Receiver operating characteristic curve
Test directing clinical management and improving patient outcomes Diagnostic thinking efficacy Therapeutic efficacy Patient outcome efficacy Clinical utility Impact on mortality or morbidity Impact on clinician judgment about diagnosis/ prognosis Impact on choice of management
Test benefiting society as a whole Societal efficacy Incremental cost-effectiveness
Matchar DB. Introduction to the Methods Guide for
Medical Test Reviews. In Methods guide for
medical test reviews. Available at
www.effectivehealthcare.ahrq.gov/medtestsguide.cfm
. ? Fryback DG, Thornbury JR. Med Decis Making
1991 Apr-Jun11(2)88-94. PMID 1907710.
16
Analytic Frameworks (1 of 3)
  • Reviewers need an explicit strategy for
    summarizing complex literature in a logical way
    to respond to key questions.
  • Frameworks can be used to
  • Maintain an orderly process.
  • Clarify questions.
  • Organize evidence.

Matchar DB. Introduction to the Methods Guide for
Medical Test Reviews. In Methods guide for
medical test reviews. Available at
www.effectivehealthcare.ahrq.gov/medtestsguide.cfm
. Bravata EP, McDonald KM, Shojania KG, et al.
Ann Intern Med 2005 Jun 21142(12 Pt 2) 1056-65.
PMID 15968030. Mulrow C, Langhorne P, Grimshaw
J. Ann Intern Med 1997 Dec 1127(11)989-95.
PMID 9412305. Whitlock EP, Orleans CT, Pender N,
et al. Am J Prev Med 2002 May22(4)267-84. PMID
11988383.
17
Analytic Frameworks (2 of 3)
  • Analytic frameworks
  • Depict graphically the linkages that relate
    interventions and outcomes.
  • Identify questions to help structure the
    literature review.
  • Provide an evidence map after the review for
    identifying gaps and weakness in the evidence.

Matchar DB. Introduction to the Methods Guide for
Medical Test Reviews. In Methods guide for
medical test reviews. Available at
www.effectivehealthcare.ahrq.gov/medtestsguide.cfm
. Agency for Healthcare Research and Quality.
U.S. Preventive Services Task Force Procedure
Manual. Available at www.uspreventiveservicestaskf
orce.org/uspstf08/methods/procmanual.htm.
18
Analytic Frameworks (3 of 3)
  • An analytic framework has two key components
  • A typology for describing the context in which
    the test is used
  • Fits with the PICOTS (population, intervention,
    comparator, outcomes, timing, and setting)
    typology (the current standard approach).
  • A visual representation of the relationship
    between the application of the test/treatment and
    the outcomes of importance to decisionmaking.

Matchar DB. Introduction to the Methods Guide for
Medical Test Reviews. In Methods guide for
medical test reviews. Available at
www.effectivehealthcare.ahrq.gov/medtestsguide.cfm
.
19
PICOTS Typology (1 of 3)
  • PICOTS stands for population, intervention,
    comparator, outcome, timing, and setting.
  • This typology was developed to describe the
    context in which medical interventions might be
    used.
  • It is important for defining the key questions of
    a review.
  • It is important for assessing whether or not a
    given study is applicable.
  • Variations in the typology exist, but the
    standard unchanging elements are PICO.

Matchar DB. Introduction to the Methods Guide for
Medical Test Reviews. In Methods guide for
medical test reviews. Available at
www.effectivehealthcare.ahrq.gov/medtestsguide.cfm
. Chalmers I, Hedges LV, Cooper H. Eval Health
Prof 2002 Mar25(1)12-37. PMID 11868442.
20
PICOTS Typology (2 of 3)
PICOTS As Applied to Medical Tests Comments
Population Patient population includes results of other/previous tests Condition(s), disease severity and stage, comorbidities, and patient demographics
Intervention Index test includes clinical role of the index strategy in relation to comparator and the test-and-treat strategy in relation to clinical outcomes Description of index test includes administrator training, technology specifications, specific application issues Three main clinical roles in relation to the comparator replacement, add-on, and triage Description of index test performance and interpretation how the results of index test lead to management decisions/actions
Comparator Comparator test-and-treat strategy Description of comparator test performance and interpretation how the results of the comparator test lead to management decisions/actions
Matchar DB. Introduction to the Methods Guide for
Medical Test Reviews. In Methods guide for
medical test reviews. Available at
www.effectivehealthcare.ahrq.gov/medtestsguide.cfm
.
21
PICOTS Typology (3 of 3)
PICOTS As Applied to Medical Tests Comments
Outcomes Relevant clinical outcomes includes any intermediate outcomes of interest Patient health outcomes includes morbidity (which also includes the adverse effects of the test and treatment), mortality, quality of life Intermediate outcomes includes technical specifications, accuracy, decisional, therapeutic impact
Timing Timing of outcome assessment Duration of followup single or multiple followup assessments
Setting Setting of test assessment Ambulatory settings (including primary, specialty care) and inpatient settings
Matchar DB. Introduction to the Methods Guide for
Medical Test Reviews. In Methods guide for
medical test reviews. Available at
www.effectivehealthcare.ahrq.gov/medtestsguide.cfm
.
22
Organization of the Methods Guide for Medical
Test Reviews
  • Module 1 Introduction
  • Module 2 Topic Development and Structure
  • Module 3 Choosing Outcomes
  • Module 4 Search Strategies
  • Module 5 Risk of Bias
  • Module 6 Applicability
  • Module 7 Grading Strength of Evidence
  • Module 8 Meta-analysis With a Gold Standard
  • Module 9 Meta-analysis Without a Gold
    Standard
  • Module 10 Decision Modeling
  • Module 11 Genetic Tests
  • Module 12 Prognostic Tests

Matchar DB. Introduction to the Methods Guide for
Medical Test Reviews. In Methods guide for
medical test reviews. Available at
www.effectivehealthcare.ahrq.gov/medtestsguide.cfm
.
23
Overview of the Flow of Modules
Matchar DB. Introduction to the Methods Guide for
Medical Test Reviews. In Methods guide for
medical test reviews. Available at
www.effectivehealthcare.ahrq.gov/medtestsguide.cfm
.
24
Summary
  • Evaluation of medical tests presents challenges
  • There are several steps between a medical test
    and outcomes of interest (indirect impact).
  • Context has great importance.
  • Few randomized controlled trials are aimed at
    comparing clinical outcomes of different tests
    and test strategies.
  • Available guidance provides some suggestions
  • Use the PICOTS (population, intervention,
    comparator, outcomes, timing, and setting)
    topology to clarify the context.
  • Use an organizing framework for classifying the
    types of studies and their relationship to key
    questions.

Matchar DB. Introduction to the Methods Guide for
Medical Test Reviews. In Methods guide for
medical test reviews. Available at
www.effectivehealthcare.ahrq.gov/medtestsguide.cfm
.
25
Practice Question 1 (1 of 2)
  • How are analytic frameworks used in the context
    of this module?
  • To graphically depict the specific questions that
    need to be answered by the literature review.
  • To list the challenges associated with a review
    of a medical test.
  • To demonstrate recurrent themes from the
    literature.
  • To display all areas of potential bias.

26
Practice Question 1 (2 of 2)
  • Explanation for Question 1
  • The correct answer is a. The analytic framework
    should be used to clearly present the questions
    that need to be answered in the literature
    review. The questions are shown by linkages
    between interventions and outcomes.

27
Practice Question 2 (1 of 2)
  • Test accuracy is included in which part of the
    PICOTS typology?
  • P for patient population
  • I for intervention
  • C for comparator
  • O for outcomes

28
Practice Question 2 (2 of 2)
  • Explanation for Question 2
  • The correct answer is d. The O in the PICOTS
    typology stands for outcomes, which include
    patient health outcomes and intermediate outcomes
    such as test accuracy.

29
Practice Question 3 (1 of 2)
  • Most of the published literature on medical tests
    addresses the clinical impact of tests.
  • True
  • False

30
Practice Question 3 (2 of 2)
  • Explanation for Question 3
  • The statement is false. Most of the published
    literature on medical tests focuses on test
    development and test characteristics and not on
    the clinical impact of tests. This poses a
    challenge when reviewing medical tests.

31
Practice Question 4 (1 of 2)
  • A causal chain diagram is used to
  • Show possible sources of bias
  • Show the links between the use of a test and the
    patient outcomes
  • Help with decisionmaking based on a positive or
    negative test result
  • Display the method used for the literature search

32
Practice Question 4 (2 of 2)
  • Explanation for Question 4
  • The correct answer is b. A causal chain diagram
    is used to show visually the links between the
    use of a test and patient outcomes.

33
Authors
  • This presentation was prepared by Brooke
    Heidenfelder, Rachael Posey, Lorraine Sease, Remy
    Coeytaux, Gillian Sanders, and Alex Vaz, members
    of the Duke University Evidence-based Practice
    Center.
  • The module is based on Chapter 1, Introduction to
    the Methods Guide for Medical Test Reviews. In
    Methods Guide for Medical Test Reviews. AHRQ
    Publication No. 12-EC017. Rockville, MD Agency
    for Healthcare Research and Quality June 2012.
    www.effectivehealthcare.ahrq.gov/medtestsguide.cfm
    .

34
References (1 of 4)
  • Agency for Healthcare Research and Quality. U.S.
    Preventive Services Task Force Procedure Manual.
    Rockville, MD Agency for Healthcare Research and
    Quality July 2008. 22-4. AHRQ Publication No.
    08-05118-EF. www.uspreventive servicestaskforce.or
    g/uspstf08/methods/procmanual.htm.
  • Agency for Healthcare Research and Quality.
    Methods Guide for Effectiveness and Comparative
    Effectiveness Reviews. Rockville, MD Agency for
    Healthcare Research and Quality April 2012. AHRQ
    Publication No. 10(12)-EHC063-EF. Chapters
    available at www.effectivehealthcare.ahrq.gov/meth
    ods guide.cfm.
  • Bravata DM, McDonald KM, Shojania KG, et al.
    Challenges in systematic reviews synthesis of
    topics related to the delivery, organization, and
    financing of health care. Ann Intern Med. 2005
    Jun 21142(12 Pt 2)1056-65. PMID 15968030.

35
References (2 of 4)
  • Chalmers I, Hedges LV, Cooper H. A brief history
    of research synthesis. Eval Health Prof 2002
    Mar25(1)12-37. PMID 11868442.
  • Fryback DG, Thornbury JR. The efficacy of
    diagnostic imaging. Med Decis Making 1991
    Apr-Jun11(2)88-94. PMID 1907710.
  • Green DM, Swets JA. Signal detection theory and
    psychophysics. New York, NY Wiley 1966.
    Reprinted in 1988 with corrections and an updated
    topical bibliography (Los Altos, CA Peninsula
    Publishing).
  • Ledley RS, Lusted LB. Reasoning foundations of
    medical diagnosis symbolic logic, probability,
    and value theory aid our understanding of how
    physicians reason. Science 1959 Jul
    3130(3366)9-21. PMID 13668531.

36
References (3 of 4)
  • Matchar DB. Introduction to the Methods Guide for
    Medical Test Reviews. In Chang SM and Matchar
    DB, eds. Methods guide for medical test reviews.
    Rockville, MD Agency for Healthcare Research and
    Quality June 2012. p. 1.1-1.11. AHRQ Publication
    No. 12-EHC017. www.effectivehealthcare.ahrq.gov/
    medtestsguide.cfm.
  • Mulrow C, Langhorne P, Grimshaw J. Integrating
    heterogeneous pieces of evidence in systematic
    reviews. Ann Intern Med. 1997 Dec
    1127(11)989-95. PMID 9412305.
  • Siebert U. When should decision analytic modeling
    be used in the economic evaluation of health
    care? Eur J Health Econ. 20034(3)143-50.
  • Tatsioni A, Zarin DA, Aronson N, et al.
    Challenges in systematic reviews of diagnostic
    technologies. Ann Intern Med. 2005 Jun 21142(12
    Pt 2)1048-55. PMID 15968029.

37
References (4 of 4)
  • Whitlock EP, Orleans CT, Pender N, et al.
    Evaluating primary care behavioral counseling
    interventions an evidence-based approach. Am J
    Prev Med. 2002 May22(4)267-84. PMID 11988383.
  • Yerushalmy J. Statistical problems in assessing
    methods of medical diagnosis, with special
    reference to X-ray techniques. Public Health Rep.
    1947 Oct 362(40)1432-49. PMID 20340527.
About PowerShow.com