Course overview, the diagnostic process, and measures of interobserver agreement - PowerPoint PPT Presentation

Loading...

PPT – Course overview, the diagnostic process, and measures of interobserver agreement PowerPoint presentation | free to download - id: baebd-ZDc1Z



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Course overview, the diagnostic process, and measures of interobserver agreement

Description:

Course overview, the diagnostic process, and measures of interobserver agreement ... (Challenges for EBD) and course review: pass out take-home exam; no HW on Ch 12 ... – PowerPoint PPT presentation

Number of Views:93
Avg rating:3.0/5.0
Slides: 29
Provided by: ThomasB62
Learn more at: http://rds.epi-ucsf.org
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Course overview, the diagnostic process, and measures of interobserver agreement


1
Course overview, the diagnostic process, and
measures of interobserver agreement
  • Thomas B. Newman, MD, MPH
  • September 18, 2008

2
Overview
  • Administrative stuff
  • Overview of the course
  • The diagnostic process
  • Interobserver agreement
  • Continuous variables
  • Categorical variables
  • Concordance
  • Kappa
  • Regular
  • Weighted

3
Administrative stuff
  • Introductions
  • Basic structure of course
  • New material each week in lecture
  • Read material before lecture if possible
  • HW on that material due the following week in
    section
  • Exceptions
  • No class October 9
  • Penultimate class 12/4 Chapter 12 (Challenges
    for EBD) and course review pass out take-home
    exam no HW on Ch 12
  • Last lecture 12/11 review of take-home exam
  • Lectures mixture of PPT and Whiteboard
  • How many want paper copies of PPT slides?

4
SECTIONS
  • Section assignments Click ROSTER on Epi 204
    website
  • Section rooms Click SCHEDULE on website
  • Faculty will rotate students, rooms and TA's
    will be constant for the quarter

5
Homework
  • Required key way of learning material
  • Which problems are assigned announced in SECTION
    and (later) posted on web
  • Not graded if late, but can still be turned in
    answers on web
  • Use fresh sheets of paper with your name on each,
    not syllabus pages, not e-mail. (You can download
    and word-process if you want, but print a copy
    unless section leader prefers electronic.)
  • Will be graded by section leaders and returned
    the following week

6
Getting help
  • Classmates, then section leaders, then faculty
  • Ambiguous/confusing problems send e-mail to
    section leader or me
  • Unless you indicate otherwise, we will assume we
    can cc the whole class when we respond if we
    think question is of general interest

7
Textbook
  • TBN and MAK have almost finished a book,
    Evidence-based Diagnosis (Cambridge University
    Press, 2009)
  • Other texts listed in on web
  • Copies of other books in bookstore and on reserve
    in the library and available for browsing here

8
Grading, honor code, etc.
  • Worst HW score dropped all other HW count
    equally
  • 2/3 Homework avg 1/3 final examination OR 1/3
    Homework avg 2/3 final examination, whichever
    is better
  • Try all problems on your own first OK to help
    each other with HW but
  • Acknowledge help
  • Write answer in own words
  • Do not collaborate on final exam
  • Honor code taken seriously

9
Course overview
  • Diagnosis
  • Theory
  • Inter-rater reliability
  • Dichotomous tests
  • Multilevel tests
  • Studies of tests
  • Combining tests
  • Screening and prognostic tests
  • Treatments randomized trials
  • Alternatives to randomized trials
  • P-values and confidence intervals Bayes' theorem
  • Clinicians and probability

10
Diagnostic process
  • Why do we want to assign a name to this persons
    illness?
  • Different reasons lead to different
    classification schemes

11
Examples
  • Acute nephrotic syndrome
  • Acute leukemia
  • Attention deficit disorder
  • Dysuria worth a course of antibiotics
  • SLUBISelf-limited undiagnosed benign illness

12
Simplified Generic Decision Problem
  • Patient either has the disease or not
  • If D, net benefit of treatment
  • If D-, better not to treat
  • (Treat could include doing more tests)

13
Simplifying assumptions (often wrong)
  • Test results are dichotomous
  • Most tests have more than two possible answers
  • Disease states are dichotomous
  • Many diseases occur on a spectrum
  • There are many kinds of nondisease

14
Evaluating diagnostic tests
  • Reliability
  • Accuracy
  • Usefulness
  • Today we do reliability

15
Types of variables
  • Categorical
  • Dichotomous 2 values
  • Nominal no intrinsic ordering
  • Ordinal intrinsic ordering
  • Continuous (infinite number of values) vs
    Discrete (limited number)

16
Measuring interobserver agreement for categorical
variables
What is agreement?
17
Concordance rate
  • What percent of the time do the 2 observers agree
    (exactly)
  • Advantage easy to understand
  • Disadvantage may be misleading if observers
    agree on prevalence of abnormality

18
Concordance rate problem
19
Unbalanced Disagreement
Lesion RATER A RATER B
1 S S
2 S S
3 S M
4 S M
5 S M
6 M M
7 M L
8 L L
9 L L
10 L L
  • What is going on here?
  • Look for lack of balance above and below diagonal
  • Results when observers have different thresholds

20
Definition of Kappa
  • The amount of agreement beyond what would be
    expected by chance
  • Formula
  • Practice
  • Obs 90, Exp 80, K
  • Obs 70, Exp 60, K
  • Obs 60, Exp 70, K

Observed agreement Expected agreement 1
Expected agreement
Given the observed marginals
21
Calculation of Expected Agreement from Marginals
22
GCS Eye opening- Observed
23
GCS Eye Opening Expected
17 x 78/116 1326/116 11.4
24
Why does multiplying row total by column total
and dividing by N give you the expected agreement?
25
Weighted Kappa
  • Weighted kappa
  • Linear
  • Quadratic
  • Custom

26
Real-life illustration Rating of neurological
examination
  • Types of weights, Stata illustration.
  • . tab ex1 ex2
  • . kap ex1 ex2, w(w)
  • . kap ex1 ex2, w(w2)
  • (See Appendix 2.1)

27
What does observed Kappa depend upon?
  • How well people agree
  • SPECTRUM within classifications
  • E.g., re the abnormal ones VERY abnormal?
  • Difficult cases can be excluded or over-sampled
  • PREVALENCE of classifications by the various
    observers (and whether they agree on prevalence)
  • Chance (random error people can get
    lucky/unlucky)
  • Weighting scheme used

28
Wireless Internet Access
  • Key is n2xa8!wr
About PowerShow.com