Assessing Risk of Bias as a Domain of Quality in Medical Test Studies

About This Presentation

Title:

Assessing Risk of Bias as a Domain of Quality in Medical Test Studies

Description:

Assessing Risk of Bias as a Domain of Quality in Medical Test Studies Prepared for: The Agency for Healthcare Research and Quality (AHRQ) Training Modules for Medical ... – PowerPoint PPT presentation

Number of Views:279

Avg rating:3.0/5.0

Slides: 40

Provided by: AlexP181

Learn more at: http://www.baylorcme.org

Category:

more less

Transcript and Presenter's Notes

Title: Assessing Risk of Bias as a Domain of Quality in Medical Test Studies

1
Assessing Risk of Bias as a Domain of Quality in
Medical Test Studies

Prepared for
The Agency for Healthcare Research and Quality
(AHRQ)
Training Modules for Medical Test Reviews Methods
Guide
www.ahrq.gov

2
Overview of a Medical Test Review
Santaguida PL, Riley CM, Matchar DB. Assessing
risk of bias as a domain of quality in medical
test studies. In Methods guide for medical test
reviews. Available at www.effectivehealthcare.ahrq
.gov/medtestsguide.cfm.
3
Learning Objectives

Identify sources of bias that may affect the
internal validity of results.
Identify validated criteria/tools and select
risk-of-bias items that can be used to assess
specific biases in medical tests.
Explain why standardizing the application of
criteria is important.
Recognize that methods for assessing study
limitations and risk of bias should be
established in advance and documented clearly.

Quality assessment is the evaluation of study
features that may influence the relative
importance placed on a study.
The evaluation process includes an examination of
the following factors
Systematic error
Random error
Adequacy of reporting
Aspects of data analysis
Applicability
Specifying ethics approval
Detailing sample size estimates
No consensus has yet been achieved on the optimal
criteria for quality assessment in systematic
reviews.

Two overarching questions for quality as value
for judgment making
Are the results for the population and test in
the study accurate and precise?
Relates to both systematic error (lack of
accuracy/bias) and random error (lack of
precision)
Is the study applicable to the patients targeted
by the review?
Relevance to both the population of interest in
the study itself (relates to potential for
selection bias) and the population represented by
the Key Questions (i.e., applicability)

This module highlights key issues in evaluating
risk of bias related to the overarching question,
Are the results for the population and test in
the study accurate and precise?
In particular, it focuses on systematic errors
resulting from
Study design
Conduct of study
Reporting of study findings
These systematic errors can lead to
overestimation or underestimation of test
performance.
Module 6 deals with the second overarching
question, Is the study applicable to the patients
targeted by the review?

Whiting et al. (2004) conducted a review of bias
in diagnostic test studies.
The results did not permit conclusions about the
direction or relative magnitude of effects for
biases but showed that bias does occur.
Some sources of bias are particularly common in
diagnostic accuracy studies. These are
Spectrum bias
Partial verification bias
Clinical review bias
Observer or instrument variation

Whiting P, Rutjes AW, Reitsma JB, et al. Ann
Intern Med 2004 Feb 3140(3)189-202. PMID
14757617. Santaguida PL, Riley CM, Matchar DB.
Assessing risk of bias as a domain of quality in
medical test studies. In Methods guide for
medical test reviews. Available at
www.effectivehealthcare.ahrq.gov/medtestsguide.cfm
.
8
Commonly Reported Sources of Systematic Error in
Studies of Medical Test Performance (1 of 2)
Source of Systematic Bias Description
Population Population
Spectrum effect Tests may perform differently in various samples. Therefore, demographic features or disease severity may lead to variations in estimates of test performance.
Context bias Prevalence of the target condition varies according to setting and may affect estimates of test performance. Interpreters may consider test results to be positive more frequently in settings with higher disease prevalence, which may also affect estimates of test performance.
Selection bias The selection process determines the composition of the study sample. If the selection process does not aim to include a patient spectrum similar to the population in which the test will be used, the results of the study may not accurately portray the results for the identified target population.
Test protocol materials and methods Test protocol materials and methods
Variation in test execution A sufficient description of the execution of index and reference standards is important because variation in measures of diagnostic accuracy result from differences in test execution.
Variation in test technology When the characteristics of a medical test change over time as a result of technological improvement or the experience of the operator of the test, estimates of test performance may be affected.
Treatment paradox Occurs when treatment is started on the basis of the knowledge of the results of the index test, and the reference standard is applied after treatment has started.
Disease progression bias Occurs when the index test is performed an unusually long time before the reference standard, so the disease is at a more advanced stage when the reference standard is performed.
Reference standard and verification procedure Reference standard and verification procedure
Inappropriate reference standard Errors of imperfect reference standard bias the measurement of diagnostic accuracy of the index test.
Differential verification bias Part of the index test results is verified by a different reference standard.
Partial verification bias Only a selected sample of patients who underwent the index test is verified by the reference standard.
Whiting P, Rutjes AW, Reitsma JB, et al. Ann
Intern Med 2004 Feb 3140(3)189-202. PMID
14757617.
9
Commonly Reported Sources of Systematic Error in
Studies of Medical Test Performance (2 of 2)
Source of Systematic Bias Description
Interpretation Interpretation
Review bias Interpretation of the index test or reference standard is influenced by knowledge of the results of the other test. Diagnostic review bias occurs when the results of the index test are known when the reference standard is interpreted. Test review bias occurs when results of the reference standard are known while the index test is interpreted.
Clinical review bias Availability of clinical data such as age, sex, and symptoms during interpretation of test results may affect estimates of test performance.
Incorporation bias The result of the index test is used to establish the final diagnosis.
Observer variability The reproducibility of test results is one determinant of the diagnostic accuracy of an index test. Because of variation in laboratory procedures or observers, a test may not consistently yield the same result when repeated. In two or more observations of the same diagnostic study, intraobserver variability occurs when the same person obtains different results, and interobserver variability occurs when two or more people disagree.
Analysis Analysis
Handling of indeterminate results A medical test can produce an uninterpretable result with varying frequency depending on the test. These problems are often not reported in test efficacy studies the interpretable results are simply removed from the analysis. This may lead to biased assessment of the test characteristics.
Arbitrary choice of threshold value The selection of the threshold value for the index test that maximizes the sensitivity and specificity of the test may lead to overoptimistic measures of test performance. The performance of this cutoff in an independent set of patients may not be the same as in the original study.
Whiting P, Rutjes AW, Reitsma JB, et al. Ann
Intern Med 2004 Feb 3140(3)189-202. PMID
14757617.
10
Evidence for Biases Affecting Medical Test
Studies (2 of 2)

Elements of study design/conduct that may
increase risk of bias vary by study type.
Criteria for rating quality of trials of tests
with clinical outcomes should not be very
different from intervention studies.
The main difference is that medical test
performance studies are typically cohort studies
(not randomized controlled trials).
Potential biases specific to this study type must
be considered
Complete ascertainment of true disease status
The adequacy of reference standard
Spectrum effect

Santaguida PL, Riley CM, Matchar DB. Assessing
risk of bias as a domain of quality in medical
test studies. In Methods guide for medical test
reviews. Available at www.effectivehealthcare.ahrq
.gov/medtestsguide.cfm. Agency for Healthcare
Research and Quality Methods Guide for
Effectiveness and Comparative Effectiveness
Reviews. Available at www.effectivehealthcare.ahrq
.gov/methodsguide.cfm.
11
Challenges Specific to Assessing Individual Study
Limitations as a Domain of Quality

How to Identify appropriate criteria
A number of instruments are available to assess
many different aspects of study quality.
Deciding which to use can be difficult.
How to apply criteria
Criteria developed for laboratory studies may not
be appropriate for studies of medical history.
However, the review should remain true to the
essence of chosen study criteria while being
clear enough for others to reproduce.
How to deal with inadequate reporting
This does not itself lead to systematic bias, but
limits assessment of risk of bias.
If a study with inadequate reporting appears to
make an important contribution, questions may
need to be addressed to the authors of the study.

Santaguida PL, Riley CM, Matchar DB. Assessing
risk of bias as a domain of quality in medical
test studies. In Methods guide for medical test
reviews. Available at www.effectivehealthcare.ahrq
.gov/medtestsguide.cfm.
12
Principles for Addressing the Challenges Specific
to Assessing Risk of Bias in Studies of Medical
Tests

Use validated criteria to address relevant
sources of bias.
Standardize the application of criteria.
Decide when inadequate reporting constitutes a
fatal flaw.

The multiple bias-assessment instruments that are
available were evaluated in the context of
diagnostic accuracy in two systematic reviews.
The first systematic review, conducted by West et
al. (2002), evaluated 18 tools.
The authors noted that all the tools were
intended for use in conjunction with other
design-specific tools.
Three scales met all six criteria the authors
considered important
Cochrane Methods Working Group checklist
Tools used by Lijmer et al. (1999)
National Health and Medical Research Council
(Australia) checklist

Santaguida PL, Riley CM, Matchar DB. Assessing
risk of bias as a domain of quality in medical
test studies. In Methods guide for medical test
reviews. Available at www.effectivehealthcare.ahrq
.gov/medtestsguide.cfm. West S, King V, Carey TS,
et al. Evid Rep Technol Assess (Summ) 2002
Mar(47)1-11. PMID 11979732.
14
Principle 1 Use Validated Criteria To Address
Relevant Sources of Bias (2 of 4)

The multiple bias-assessment instruments that are
available were evaluated in the context of
diagnostic accuracy in two systematic reviews.
The second systematic review, conducted by
Whiting et al. (2005), evaluated 91 tools. The
majority of these tools
Did not explicitly state a rationale for the
inclusion/exclusion of items.
Had not been subjected to a test-retest
reliability evaluation.
Did not provide a definition of quality
components considered in the tool.
These variations reflect inconsistency in
understanding quality assessment in the field of
evidence-based medicine.
The authors did not recommend a particular tool
and instead developed their own Quality
Assessment of Diagnostic Accuracy Studies
(QUADAS).

Santaguida PL, Riley CM, Matchar DB. Assessing
risk of bias as a domain of quality in medical
test studies. In Methods guide for medical test
reviews. Available at www.effectivehealthcare.ahrq
.gov/medtestsguide.cfm. Whiting P, Rutjes AW,
Dinnes J, et al. J Clin Epidemiol 2005
Jan58(1)1-12. PMID 15649665.
15
Principle 1 Use Validated Criteria To Address
Relevant Sources of Bias (3 of 4)

Quality Assessment of Diagnostic Accuracy Studies
(QUADAS) checklist
Its authors attempted to incorporate sources of
bias and error with empirical basis and validity.
It contains elements beyond those of systematic
bias, such as questions related to reporting.
An updated version (QUADAS-2) identifies four key
domains rated in terms of risk of bias
Patient selection
Index test(s)
Reference standard
Flow and timing

Santaguida PL, Riley CM, Matchar DB. Assessing
risk of bias as a domain of quality in medical
test studies. In Methods guide for medical test
reviews. Available at www.effectivehealthcare.ahrq
.gov/medtestsguide.cfm. Whiting P, Rutjes AW,
Reitsma JB, et al. BMC Med Res Methodol 2003 Nov
10325. PMID 14606960. Whiting PF, Rutjes AWS,
Westwood ME, et al. Ann Intern Med 2011 Oct
18155(8)529-36. PMID 22007046.
16
QUADAS-2 Questions for Assessing Risk of Bias in
Diagnostic Accuracy Studies
Domain 1 Patient Selection
Was a consecutive or random sample of patients enrolled? (Yes/No/Unclear)
Was a case-control design avoided? (Yes/No/Unclear)
Did the study avoid inappropriate exclusions? (Yes/No/Unclear)
Could the selection of patients have introduced bias? (Risk Low/High/Unclear)
Domain 2 Index Test(s) (complete for each index test used)
Were the index test results interpreted without knowledge of the reference standard? (Yes/No/Unclear)
If a threshold was used, was it prespecified? (Yes/No/Unclear)
Could the conduct or interpretation of the index test have introduced bias? (Risk Low/High/Unclear)
Domain 3 Reference Standard
Is the reference standard likely to correctly classify the target condition? (Yes/No/Unclear)
Were the reference standard results interpreted without knowledge of the results of the index test? (Yes/No/Unclear)
Could the reference standard, its conduct, or its interpretation have introduced bias? (Risk Low/High/Unclear)
Domain 4 Flow and Timing
Was there an appropriate interval between index test(s) and reference standard? (Yes/No/Unclear)
Did all patients receive a reference standard? (Yes/No/Unclear)
Did all patients receive the same reference standard? (Yes/No/Unclear)
Were all patients included in the analysis? (Yes/No/Unclear)
Could the patient flow have introduced bias? (Risk Low/High/Unclear)
Questions related to the assessment of
applicability will be addressed in Module 6
Assessing Applicability of Medical Test
Studies in Systematic Reviews. QUADAS-2 revised
Quality Assessment of Diagnostic Accuracy Studies
checklist
Whiting PF, Rutjes AWS, Westwood ME, et al. Ann
Intern Med 2011 Oct 18155(8)529-36. PMID
22007046.
17
Principle 1 Use Validated Criteria To Address
Relevant Sources of Bias (4 of 4)

Use of criteria that have been validated by an
instrument, like QUADAS-2, is recommended to
assess the risk of systematic error.
Other items that assess applicability or random
error are considered at a different stage of the
review (see Modules 6 and 8).
Systematic reviewers may need to add criteria
from other standardized checklists. For example
Standards for Reporting of Diagnostic Accuracy
(STARD)
Strengthening the Reporting of Genetic
Association Studies (STREGA), which is an
extension of Strengthening the Reporting of
Observational Studies in Epidemiology (STROBE)

Standardizing the application of criteria
maintains objectivity in an otherwise subjective
process.
It is recommended for review teams to establish
clear definitions for each criterion due to lack
of empirical evidence to inform decisions.
It can be useful to pilot test the definitions
with two or more reviewers.
Unreliable items can be revised.
Reliability of the ultimate criteria can be
measured.
Summarize the limitations across multiple items
from a single study into one of three simple
categories.
Use the terms good, fair, or poor.
Definitions should be decided in advance and
clearly reported.
It is useful to have two independent reviewers
categorize the studies and then resolve
disagreements by discussion.

Santaguida PL, Riley CM, Matchar DB. Assessing
risk of bias as a domain of quality in medical
test studies. In Methods guide for medical test
reviews. Available at www.effectivehealthcare.ahrq
.gov/medtestsguide.cfm. Agency for Healthcare
Research and Quality Methods Guide for
Effectiveness and Comparative Effectiveness
Reviews. Available at www.effectivehealthcare.ahrq
.gov/methodsguide.cfm.
19
Categorizing Individual Studies Into General
Quality Classes
Category Application to Randomized Controlled Trials Application to Medical Test Performance Studies
Good No major features that risk biased results The study avoids problems such as failure to apply true randomization, selection of a population unrepresentative of the target patients, low dropout rates, or analysis by intention to treat. Key study features are described clearly, including the population, setting, interventions, comparison groups, outcome measurements, and reasons for dropouts. Randomized controlled trials (RCTs) are considered a high-quality study design, but studies that include consecutive patients representative of the intended sample for whom diagnostic uncertainty exists may also meet this standard. A good study avoids the multiple biases to which medical test studies are subject (e.g.., use of an inadequate reference standard, verification bias), and key study features are clearly described, including the comparison groups, outcomes measurements, and characteristics of patients who failed to be have their actual state (diagnosis or prognosis) verified.
Fair Susceptible to some bias, but flaws are not sufficient to invalidate the results The study does not meet all the criteria required for a rating of good quality, but no flaw is likely to cause major bias. The study may be missing information, making it difficult to assess limitations and potential problems. Application of this category to medical test performance studies is similar to its application to RCTs.
Poor Significant flaws imply biases of various types that may invalidate the results The study has large amounts of missing information, discrepancies in reporting, or serious errors in design, analysis, and/or reporting. The study has significant biases determined a priori to be major or fatal (i.e., likely to make the results either uninterpretable or invalid).
Adapted from Agency for Healthcare Research and
Quality Methods Guide for Effectiveness and
Comparative Effectiveness Reviews. Available at
www.effectivehealthcare.ahrq.gov/methodsguide.cfm.
20
Principle 3 Decide When Inadequate Reporting
Constitutes a Fatal Flaw

Inadequate reporting does not introduce
systematic bias itself, but it limits the ability
to assess the risk of bias.
Some reviewers may assume the worst, while others
give the benefit of the doubt.
When a study makes a potentially important
contribution to the review, contacting the study
authors may resolve issues of reporting.
When it is not possible to obtain details,
standard practice is to document inadequate
reporting of particular criteria.
The reviewers must determine in advance whether
failure to report certain criteria constitutes a
fatal flaw (i.e., makes results
uninterpretable/invalid).
Example A review meant to apply to older
individuals finds a study in which age was not
reported in this scenario, the study either is
excluded or is included and marked as poor in
quality with regard to risk of bias.

Wilson et al. (2009) undertook a systematic
review to evaluate the accuracy of patient
self-reports of family history and relevant
factors likely to affect accuracy.
Index test Patient self-reports of family
history
Reference standard Verification of relatives
status from medical records or a disease/death
registry
Quality Assessment of Diagnostic Accuracy Studies
(QUADAS) criteria were used to evaluate the
quality of eligible studies. The reviewers
Excluded 4 of 14 criteria and justified those
exclusions in an appendix.
Provided contextual examples each item used.
Defined partial verification bias criteria in the
context of the index and reference tests
described above.
Clearly described decision rules for rating each
criterion as yes, no, or unclear.

Santaguida PL, Riley CM, Matchar DB. Assessing
risk of bias as a domain of quality in medical
test studies. In Methods guide for medical test
reviews. Available at www.effectivehealthcare.ahrq
.gov/medtestsguide.cfm. Qureshi N, Wilson B,
Santaguida P, et al. AHRQ Evidence
Report/Technology Assessment No. 186. Available
at www.ncbi.nlm.nih.gov/books/NBK32554/pdf/TOC.pdf
.
22
Illustrative ExampleHow Partial Verification
Bias Was Interpreted
Modified QUADAS Item (Topic/Bias) Interpretation
5. Did the whole sample or a random selection of the sample receive verification using a reference standard of diagnosis? (Partial verification bias) This item concerns partial verification bias, which is a form of selection bias that occurs when not all of the study participants receive the reference standard (in our context, confirmation of the TRUE disease status of the relative). Sometimes the reason only part of the sample receives the reference standard is that knowledge of the index test results influence the decision to perform the reference standard. Note that in the context of family history, the reference standard can only be applied to family members or relatives. The self-report by the probands or informants is the index test. We consider the whole sample to be ALL relatives for which the proband or informant provided information (including a dont know status). YES All relatives that the proband identifies/ reports upon represent the whole sample of relatives. As such, some form of verification is attempted for all identified relatives. NO Not all relatives receive verification via the reference standard. As such, we consider partial verification bias to be present in the following situations Knowledge of the index test will determine which relatives are reported to have the disease status. Often UNAFFECTED relatives do not have their disease status verified by any method (assuming the proband/informant report is the true disease status) in this case, the disease status is verified in the AFFECTED relatives only. In this situation, the outcomes of sensitivity and specificity cannot be computed. Relatives for which the proband/ informant indicates dont know status are excluded and do not have their disease status verified (no reference standard testing). Relatives who are DECEASED are excluded from having any verification undertaken (no reference standard testing). Relatives who are UNABLE TO PARTICIPATE in interviews or further clinical testing are excluded from having any verification method (no reference standard testing). UNCLEAR Insufficient information to determine whether partial verification was present.
Qureshi N, Wilson B, Santaguida P, et al. AHRQ
Evidence Report/Technology Assessment No. 186.
Available at www.ncbi.nlm.nih.gov/books/NBK32554/p
df/TOC.pdf.
23
Illustrative Example Accuracy of
PatientSelf-reports of Family History (2 of 2)

Reviewers can choose to present results of
Quality Assessment of Diagnostic Accuracy Studies
(QUADAS) criteria ratings in tables as the
percentage of studies that scored yes, no, or
unclear on any given individual item.
QUADAS developers do not recommend using
composite scores.

Assessing methodological quality is necessary.
Judging the overall quality of a study involves
examining
Study size
Direction and degree of findings
Study relevance
Risk of bias
Systematic error
Random error
Other study limitations (e.g., inadequate
reporting)
This module focused on the evaluation of
systematic bias as a distinctly important
component of quality.

Reviewers should select validated criteria that
examine the risk of systematic error when
assessing study limitations.
Reviewers should categorize individual studies as
good, fair, or poor with respect to risk of
bias.
Independent categorization by two reviewers is
recommended.
Methods for determining an overall categorization
for the study limitations should be established
beforehand and clearly documented.

Internal validity refers to
Applicability of the study to the patients
relevant to the review.
Accuracy of the results for the population and
test in the study.
Relevance of the study to the population
represented in the Key Questions.
Degree of random error.

27
Practice Question 1 (2 of 2)

Explanation for Question 1
The correct answer is b. Good internal validity
means that the results for the population and
test in the study are accurate.

28
Practice Question 2 (1 of 2)

The term applicability generally refers to the
feasibility of performing a given test.
True
False

29
Practice Question 2 (2 of 2)

Explanation for Question 2
The statement is false. Applicability generally
refers to the relevance of an intervention
(including a medical test) to a population of
interest. Applicability is an important component
of external validity.

30
Practice Question 3 (1 of 2)

What is the QUADAS-2?
A tool used to assess the risk of systematic
error in diagnostic accuracy studies
A tool used to assess applicability in diagnostic
accuracy studies
A tool used to assess random error in diagnostic
accuracy studies
Answers a and b

31
Practice Question 3 (2 of 2)

Explanation for Question 3
The correct answer is d. QUADAS-2 is an updated
version of the Quality Assessment of Diagnostic
Accuracy Studies checklist to include these four
key domains patient selection, index test(s),
reference standard, and flow and timing. It is a
checklist of potential sources of bias and error
used to assess the risk of systematic error. It
can also be used to assess applicability.

32
Practice Question 4 (1 of 2)

Inadequate reporting introduces systematic bias.
True
False

33
Practice Question 4 (2 of 2)

Explanation for Question 4
The statement is false. Inadequate reporting does
not introduce systematic bias, but it does limit
the reviewers ability to assess the risk of bias.

34
Authors

This presentation was prepared by Brooke
Heidenfelder, Rachael Posey, Lorraine Sease, Remy
Coeytaux, Gillian Sanders, and Alex Vaz, members
of the Duke University Evidence-based Practice
Center.
The module is based on Chapter 5, Assessing Risk
of Bias as a Domain of Quality in Medical Test
Studies. In Methods Guide for Medical Test
Reviews. AHRQ Publication No. 12-EC017.
Rockville, MD Agency for Healthcare Research and
Quality June 2012. www.effectivehealthcare.ahrq.g
ov/medtestsguide.cfm

35
References (1 of 5)

Agency for Healthcare Research and Quality.
Methods Guide for Effectiveness and Comparative
Effectiveness Reviews. Rockville, MD Agency for
Healthcare Research and Quality April 2012. AHRQ
Publication No. 10(12)-EHC063-EF. Chapters
available at www.effectivehealthcare.ahrq.gov/meth
odsguide.cfm.
Bossuyt PM, Reitsma JB, Bruns DE, et al. Towards
complete and accurate reporting of studies of
diagnostic accuracy the STARD Initiative. Ann
Intern Med 2003 Jan 7138(1)40-4. PMID
12513043.
Centre for Reviews and Dissemination. Systematic
reviews CRD's guidance for undertaking reviews
in health care. York, England University of
York 2008. www.york.ac.uk/inst/crd/pdf/Systematic
_Reviews.pdf. Accessed September 19, 2011.
Cochrane Methods Working Group on Systematic
Review of Screening and Diagnostic Tests.
Recommended methods. London The Cochrane
Collaboration 1996.

36
References (2 of 5)

Higgins JPT, Altman DG, Sterne JAC the Cochrane
Statistical Methods Group and the Cochrane Bias
Methods Group. Chapter 8 Assessing risk of bias
in included studies. In Higgins JPT and Green S,
eds. Cochrane handbook for systematic reviews of
interventions. Version 5.1.0. London The
Cochrane Collaboration 2011. www.cochrane-handboo
k.org. Accessed September 19, 2011.
Leeflang MM, Deeks JJ, Gatsonis C, et al
Cochrane Diagnostic Test Accuracy Working Group.
Systematic reviews of diagnostic test accuracy.
Ann Intern Med. 2008 Dec 16149(12)889-97. PMID
19075208.
Lijmer JG, Mol BW, Heisterkamp S, et al.
Empirical evidence of design-related bias in
studies of diagnostic tests. JAMA. 1999 Sep
15282(11)1061-6. PMID 10493205.
Little J, Higgins JP, Ioannidis JP, et al.
STrengthening the REporting of Genetic
Association studies (STREGA)an extension of the
STROBE statement. Eur J Clin Invest. 2009
Apr39(4)247-66. PMID 19297801.

37
References (3 of 5)

National Health and Medical Research Council. How
to review the evidence systematic identification
and review of the scientific literature.
Canberra, Australia National Health and Medical
Research Council 2000 www.nhmrc.gov.au/_files_nhm
rc/publications/attachments/cp65.pdf.
Qureshi N, Wilson B, Santaguida P, et al. Family
History and Improving Health. Evidence
Report/Technology Assessment No. 186 (Prepared by
the McMaster University Evidence-based Practice
Center under Contract No. HHSA 290-2007-10060-I).
Rockville, MD Agency for Healthcare Research and
Quality August 2009. AHRQ Publication No.
09-E016. www.ncbi.nlm.nih.govbooks/NBK32554/pdf/TO
C.pdf.
Santaguida PL, Riley CM, Matchar DB. Assessing
risk of bias as a domain of quality in medical
test studies. In Chang SM and Matchar DB, eds.
Methods guide for medical test reviews.
Rockville, MD Agency for Healthcare Research and
Quality June 2012. p. 5.1-5.10. AHRQ Publication
No. 12-EHC017. www.effectivehealthcare.ahrq.gov/me
dtestsguide.cfm.

38
References (4 of 5)

von Elm E, Altman DG, Egger M, et al. The
Strengthening the Reporting of Observational
Studies in Epidemiology (STROBE) statement
guidelines for reporting observational studies.
Lancet. 2007 Oct 20370(9596)1453-7.
PMID 18064739.
West S, King V, Carey TS, et al. Systems to rate
the strength of scientific evidence. Evid Rep
Technol Assess. (Summ) 2002 Mar(47)1-11. PMID
11979732.
Whiting P, Rutjes AW, Dinnes J, et al. A
systematic review finds that diagnostic reviews
fail to incorporate quality despite available
tools. J Clin Epidemiol. 2005 Jan58(1)1-12.
PMID 15649665.
Whiting P, Rutjes AW, Reitsma JB, et al. The
development of QUADAS a tool for the quality
assessment of studies of diagnostic accuracy
included in systematic reviews. BMC Med Res
Methodol. 2003 Nov 10325. PMID 14606960.

39
References (5 of 5)

Whiting P, Rutjes AW, Reitsma JB, et al. Sources
of variation and bias in studies of diagnostic
accuracy a systematic review. Ann Intern Med.
2004 Feb 3140(3)189-202. PMID 14757617.
Wilson BJ, Qureshi N, Santaguida P, et al.
Systematic review family history in risk
assessment for common diseases. Ann Intern Med.
2009 Dec 15151(12)878-85. PMID 19884616.

Write a Comment

User Comments (0)

About PowerShow.com

Assessing Risk of Bias as a Domain of Quality in Medical Test Studies - PowerPoint PPT Presentation

Assessing Risk of Bias as a Domain of Quality in Medical Test Studies

Assessing Risk of Bias as a Domain of Quality in Medical Test Studies Prepared for: The Agency for Healthcare Research and Quality (AHRQ) Training Modules for Medical ... – PowerPoint PPT presentation