EPI-820 Evidence-Based Medicine - PowerPoint PPT Presentation

1 / 38
About This Presentation
Title:

EPI-820 Evidence-Based Medicine

Description:

... (population unlikely to have diseases that cause FP results) ... Pancreatic cancer. 115 (men) 120 (women) Symptoms of typical angina. Yes 75% Coronary Art. ... – PowerPoint PPT presentation

Number of Views:24
Avg rating:3.0/5.0
Slides: 39
Provided by: Epidem3
Learn more at: https://www.msu.edu
Category:
Tags: epi | based | evidence | medicine

less

Transcript and Presenter's Notes

Title: EPI-820 Evidence-Based Medicine


1
EPI-820 Evidence-Based Medicine
  • LECTURE 4 DIAGNOSIS II
  • Mat Reeves BVSc, PhD

2
Objectives
  • 1. Understand the derivation and use of Bayes
    Theorem.
  • 2. Understand the Fundamental Medical Fact 1".
  • 3. Define the likelihood ratio (LR) and calculate
    LR and LR- from 2 x 2 table.
  • 4. Understand the Odds-likelihood form of Bayes
    Theorem.
  • 5. Understand what clinical conditions maximize
    the value of test results.
  • 6. Understand the biases and limitations of
    published test performance measures.

3
I. Bayes Theorem - its Derivation and Use
  • Bayes theorem a unifying methodology, based
    on conditional probabilities, for interpreting
    clinical test results.
  • PVP Se . Prev
  • Se . Prev (1 - Sp) . (1 - Prev)
  • PVN Sp . (1 - Prev)
  • Sp . (1 - Prev) (1 - Se) . Prev

4
Derivation of Bayes Equations1) 2 x 2 table
Approach

5
2) First Principles Approach Using Joint and
Conditional Probabilities
  • Example - PVP
  • Step 1 Specify prior prob. of disease P(D),
    non-disease P(D-).
  • Step 2 Calculate joint probability of disease
    and pos. test result
  • P(T, D) P(TD) . P(D)
  • N.B. equation derived from definition of joint
    probability.
  • cell a, or Se multiplied by prevalence.
  • Step 3 Calculate joint probability of
    non-disease and a pos. test result
  • P(T, D-) P(TD-) . P(D-)
  • cell b, OR FP rate multiplied by prob. of
    non-disease.

6
Example PVP
  • Step 4 Calculate probability of a positive test
    result i.e., P (T)
  • P (T) P(T, D) P(T, D-)
  • the sum of cells a and b.
  • Substituting formulas from steps 2 and 3 we get
  • P (T) P(TD) . P(D) P(TD-) . P(D-)
  • Step 5 Calculate probability of disease given a
    positive test results i.e., PVP
  • P(DT) P(T, D)
  • P(T)
  • cell a divided by cells a and b.
  • derived from formal definition of joint
    probability (Step 2)

7
Example PVP
  • So
  • P(DT) P(TD) . P(D)
  • P(TD) . P(D) P(TD-) . P(D-)
  • PVP Se . Prev
  • Se . Prev (1 - Sp) . (1 - Prev)

8
Example of the Use of Bayes TheoremIP I-125
FS for DVT (Se 90, Sp 95), Prevalence of
disease 15
  • PVP 0.95 . 0.15
  • 0.95 . 0.15 (1 - 0.95) . (1 - 0.15)
  • PVP 0.1425 0.77 or 77
  • 0.1425 0.0425
  • PVN 0.95 . (1 - 0.15)
  • 0.95 . (1 - 0.15) (1 - 0.90) . 0.15
  • PVN 0.8075 0.982 or 98.2
  • 0.8075 0.015
  • identical to Fig. 9 in lecture 3 (given rounding
    error of PVP)
  • equations are cumbersome? difficult to remember?

9
Prevalence
  • The importance of prevalence on the
    interpretation of predictive values and its
    influence on the whole framework of clinical
    practice cannot be overstated.
  • Prevalence represents the clinician's best guess
    or opinion (expressed as a probability) prior to
    ordering an actual test.

10
Table 4.1 Relationship Between History of Chest
Pain and Prevalence of Coronary Artery Disease
(CAD) (Weiner et al., NEJM, 1979).
Prevalence of CAD Prevalence of CAD
Type of History All men Men with abN ECG
Typical angina 0.89 0.96 (gain 0.07)
Atypical angina 0.60 0.87 (gain 0.27)
Non-anginal chest pain 0.22 0.39 (gain 0.17)
11
Conclusions
  • a) probability of CAD depends on patients history
  • b) probability of CAD after an abnormal test
    depends on patients history
  • c) value of the test information also depends on
    patients history

12
Fundamental medical fact 1
  • The interpretation of test results depends on the
    probability of disease before the test was run (
    prior probability or prior belief).

13
The Prior Probability of Disease
  • represents what the clinician believes (prior
    belief or clinical suspicion)
  • set by considering the practice environment,
    patients history, physical examination findings,
    experience and judgment etc
  • constantly revised in light of new information (
    Bayes theorem).

14
Fig 4.1 Use of Bayes Theorem to Revise Disease
Estimates in Light of New Test Information
Use of Bayes Formula
Before-test
After test
After - test
1.0
0
0.5
Probability of Disease
15
Figure 4.3 Advantages of a Pre-test Probability
of 40 60(Test Has Se 75, Sp 85, LR 5
and LR- 0.3 )

16
II. Generalizability of Published Test
Performance Measures (Se, Sp, LRs)
  • Test performance measures are frequently assumed
    to be intrinsic characteristics of diagnostic
    tests independent of underlying prevalence
  • It is assumed that Se and Sp (or LRs) are fixed
    and that valid post-test probabilities can be
    computed by simply varying the prior probability.
  • Problems?

17
Potential Problems
  • 1. Disease severity affects diagnostic test
    performance - the more severe the disease the
    higher the sensitivity (easier to detect).
  • test performance depends on the spectrum of
    disease severity in the source (test) population.
  • 2. Test characteristics are in fact dynamic -
    they can therefore
  • change due to alterations in underlying
    prevalence.
  • be different among subgroups defined by age,
    gender etc.

18
Potential Problems
  • 3. Published Se and Sp estimates can be highly
    biased (Ransohoff and Feinstein, 1986).
  • Spectrum bias the difference in both the
    spectrum and severity of disease between study
    population and clinically relevant population.

19
A. Selection Bias During Phase I Evaluations
  • initial evaluation of a new diagnostic test
    typically undertaken at referral centers.
  • determine if test will be positive among patients
    with severe disease, i.e., the sickest of the
    sick. --- Se is overestimated (advanced disease
    is easy to detect).
  • determine if the test is usually negative in
    normal (healthy, young) volunteers, i.e., the
    wellest of the well --- Sp is overestimated
    (population unlikely to have diseases that cause
    FP results).
  • sometimes test is applied only to the sickest of
    the sick, with the result that Sp is
    underestimated (FP results are over-inflated,
    because very sick patients have conditions that
    tend to make the index test positive).

20
B. Selection Bias During Phase II Evaluations
(Test-Referral Bias)
  • Test-referral bias occurs when the index test
    itself is used as a criterion to select which
    patients receive the definitive (gold standard)
    diagnostic procedure.
  • Test negative subjects dont go on to get the
    gold standard which results in
  • over-estimation of Se (number of FN
    underestimated)
  • under-estimation of Sp (number of TN
    underestimated).

21
Test-referral bias Example (from Cox)
Index Pop. No disease
Index Pop. Disease
Study Pop.
T
T
T
T
T-
T-
T-
T-
FPR 0.55
TPR 0.8
FPR 0.3
TPR 0.6
22
Net Result of Spectrum bias
  • Se is over-estimated
  • Sp?? - depends on relative balance of the
    possible biases.

23
Summary Published Se/Sp/LR Values and Bayes
Theorem
  • Published estimates of Se/Sp or LRs should be
    considered average values for a particular (sub)
    population
  • Theres a great deal to be learnt from mastering
    Bayes Theorem - but its not without potential
    pitfalls and errors - be careful!!!

24
III. The Likelihood Ratio (LR) - Definition
  • Alternative way of describing diagnostic test
    performance.
  • For dichotomous test results it summarizes
    exactly the same information as Se/Sp.
  • The LR for a particular value of a diagnostic
    test is defined as the probability of observing
    the test result (X) in the presence of disease
    divided by the probability of observing the test
    result in the absence of disease. 
  • LR (X) P(XD) P(XD-)
  • it is the odds that a given test result (X)
    occurs in a diseased individual compared to a
    non-diseased individual.

25
Figure 4.2 The Likelihood Ratio for a
Dichotomous Test (Positive or Negative) Example
IP and I-125 FS testing.
26
Interpretation of LRs
  • LR indicates that a positive result (IP and/or
    I-125 FS) is 18 times more likely to occur in the
    presence of DVT than in the absence of it.
  • LR- indicates that a negative result is 0.105
    times less likely to occur in the presence of DVT
    than in the absence of it (or for every negative
    result in a patient with DVT, expect 9 negative
    results in patients without DVT).

27
Note, for dichotomous tests
  • LR Sensitivity or True-positive Rate
  • 1 - Specificity False-positive Rate
  • LR- 1 - Sensitivity or False-negative Rate
  • Specificity True-negative Rate
  • Also
  • ROC curve Se versus 1 Sp
  • LR the slope of the ROC curve.

28
Table 4.2 Likelihood Ratios for Common Tests
Disease/Condition Test and Result LR
Alcohol dependency CAGE questions Yes to 3 or more Yes to any 2 Yes to any 1 No to all 4 250 7 1.3 0.2
gt75 Coronary Art. Stenosis Symptoms of typical angina Yes 115 (men) 120 (women)
Pancreatic cancer CT Scan Definitely abnormal Probably abnormal Possibly abnormal Definitely normal 26 4.8 0.35 0.11
Breast cancer Fine needle aspirate Definitely malignant Suspicious Benign Unsatisfactory 4 4.8 0.11 0.41
TB (Culture) Sputum smear Positive Negative 31 0.79
29
IV. Advantages of Using LRs
  • A. Can Apply Bayes Theorem Easily Using
  • Odds-likelihood Ratio Form of Bayes Theorem
  • Pre-test odds X LR Post-test odds
  • where
  • Pre-test odds Prevalence
  • 1 Prevalence
  • Post-test probability Post-test odds
  • 1 Post-test odds

30
What this equation is telling us
  • The environment (indicated by the pre-test odds)
    is as important as the information provided by
    the test (indicated by the LR). Must know the
    prevalence or prior probability.
  • The LR is easily obtained from reference books,
    the clinicians job is to provide an estimate of
    the pre-test odds!!

31
Example IP and I-125 FS Tests and DVT
Prevalence 15
  • LR Se/(1-Sp) 0.90/(1 - 0.95) 18.0
  • 0.15 X 18 3.176 (post-test odds)
  • 1 - 0.15
  • Post-test probability 3.176 76 1
    3.176

32
Example IP and I-125 FS Tests and DVT
Prevalence 15
  • LR - (1 - Se)/Sp 0.10/0.95 0.105
  • 0.15 X 0.105 0.0186 (post-test odds)
  • 1 - 0.15
  • Post-test probability 0.0186 0.0182
    1 0.0186
  • Note this is P(DT-) or the complement of PVN or
    (1 P(D-T-).
  • PVN is therefore 1 0.0186 98.2

33
B. Can Calculate LRs for Several Levels of Test
Results
  • Disease more likely in the presence of an very
    abnormal test result than for a marginal one.
  • LRs can be calculated for any range of test
    results, thereby preserving clinical information.

34
Table 4.3 Likelihood Ratios of MI for Six Levels
of CK
MI - Yes CK Result MI- No LR LR
50 gt 400 1 (50/230)/(1/130) 28.3
34 320-400 1 (34/230)/(1/130) 19.2
71 160-319 4 (71/230)/(4/130) 10.0
60 80-159 10 (60/230)/(10/130) 2.6
13 40-79 26 (13/230)/(26/130) 0.28
2 0-39 88 (2/230)/(88/130) 0.01
230 130
35
LRs and Multiple Levels
  • When CK results were dichotomized, the LR range
    was 0.07 to 7.6 (108 fold difference)
  • When data presented for seven levels, the LR
    range is now 0.01 to 28.3 (a 2830 fold
    difference)
  • LRs preserve the natural degree of severity in
    the original data - DONT LUMP!!!!.

36
C. Other Advantages of LRs
  • i) Robustness
  • Test performance measures (Se/Sp or LRs) are
    assumed to be independent of the underlying
    prevalence of disease.
  • But, Se and Sp may change in response to changes
    in prevalence.
  • LRs are theoretically less susceptible to these
    changes - because calculated from smaller
    slices of data

37
ii) Multivariable Modeling
  • LR is an exponential function of a linear
    combination of test data
  • LR exp (ao B1X1 .....BkXk)
  • Logistic regression models calculate the
    probability of a certain event X, given
    explanatory variables B1X1 ..... BkXk.
  • P(DX) e B0 B1X1 ..... BkXk
  • 1 e B0 B1X1 ..... BkXk
  • With some minor adjustments, the parameters of
    the logistic regression model can be used to
    estimate the LR

38
Advantages?
  • Can combine the discriminatory power of several
    variables (or tests) into a single LR (each
    variable is now independent). e.g., IP and I-125
    FS!!
  • Can adjust estimates for important covariates
    (e.g. age or gender).
  • Examples Equine colic (Reeves), BreastAid (Osuch)
Write a Comment
User Comments (0)
About PowerShow.com