GRAPHICAL AND CLUSTER-ANALYTIC TECHNIQUES FOR PRELIMINARY INSPECTION OF DIAGNOSTIC TEST EVALUATION STUDIES PRIOR TO A META-ANALYSIS

1
GRAPHICAL AND CLUSTER-ANALYTIC TECHNIQUESFOR
PRELIMINARY INSPECTION OFDIAGNOSTIC TEST
EVALUATION STUDIESPRIOR TO A META-ANALYSIS

• 
  Jørgen Hilden
• 
  Univ. of
  Copenhagen
• 
  Dept. of
  Biostatistics
• 
  Copenhagen,
  Denmark
• ISCB Szeged 2005

2
GRAPHICAL AND CLUSTER-ANALYTIC TECHNIQUESFOR
PRELIMINARY INSPECTION OFDIAGNOSTIC TEST
EVALUATION STUDIESPRIOR TO A META-ANALYSIS
IN THIS VERSION OF THE SLIDES I HAVE ADDED
EXPLANATORY NOTES ALONG THE WAY IN THE
PowerPoint NOTES FIELDS

• 
  Jørgen Hilden
• 
  Univ. of
  Copenhagen
• 
  Dept. of
  Biostatistics
• 
  Copenhagen,
  Denmark
• ISCB Szeged 2005

3
Until recently, the Cochrane Collaborationwas
100 interventional research.Now, a Cochrane
Handbook for systematic reviews and
meta-analyses of diagnostic studies is under
way. For the time being, endeavours are
restricted to the classical Black-and-White
approach, i.e., 2-by-2 tables summarized in
terms of sensitivity, specificity, etc.
4
Fairly sophisticated procedures are here
available for meta-analysisChapter 8 of the
Handbook / C. Gatsonis et al. But something
is missing ! TECHNIQUES FOR PRELIMINARY
INSPECTION of the raw data, i.e., source
study 2-by-2 tables descriptors
of technical clinical aspects of each
study
5
You will agree with me thatgraphical and
descriptive (data-analytic) procedures for
preliminary mustering of the data form an
indispensable part of statistical craftmanship.
Checking for oddities and
outliers
6

• Diagnostic studies are
• highly variable
• in scope and
• sophistication
• ? oddities are frequent
• and
• important to detect
• to put it politely

7

• One may want to see
• an array of 2-by-2 tables,
• arranged by summary statistics, or
• arranged according to descriptors
• of the technical clinical setting
• One may want to visualize a
• clustering of primary studies
• that throws light on heterogeneity
• and its causes

8
Displaying 2-by-2 tables

• Despite their key rôle in epidemiology,
• even epidemiologists do not have
• any standards how to
• display and visually compare such tables.
• ?Two main challenges that I see
• The observations are inherently
• 3- or 4-dimensional
• ROC diagrams show only 2 ds
• Near-zero frequencies are hard to distinguish
• but differences may be crucial

9

• Read off
• Fraction
• diseased
• Sensitivity
• Specificity

10
False positives upper rightfalse
negatives lower left(as before)
Paddocks for the black sheep

11
False pos/neg minorities still in
area-truerepresentation but linear pen
sizeis sqroot().False positives
upper rightfalse negatives lower left(as
before) The black sheep

.dropped something
.

12

13

• Note 1.96 SE marks (all approx. equal)

14

• Variant draw confidence ellipses ltclutter!gt
• Variant low-quality source studies in GREY

15

16

• Comet graph
• ROC and
• posterior
• counterpart (PVs)

17

18

19

20
CLUSTER-ANALYTIC TECHNIQUES

• Exploring heterogeneity
• there are endless variants

21

22
?

• faneg
• fapos true neg
• true pos

23

24

25

• Five studies used a different protein marker

26

• To reduce clutter, should only comparatively
  homogeneous groups be shown?
• The
• black-sheep plots are perhaps not so useful for
  hierarchical graphs

27

• Variant using 2X2 table distance horiz.
  vertically

28

• Those inter-study discrepancies you may wonder
• which ones are at all statistically
  significant?
• The majority!
• The next graph pushes
• the study tree to the bottom and
• displays the 16 lowest
• inter-study chi-squares.
• Out of 78 pairs, only 9 (16) are not
  significant
• at the 0.05 (0.001) level.

29

30

Graphical presentation ofmeta-analytical results

A well-known device is to show summary estimates
along with the source studies estimates exampl
es already shown But beware of the fixed-effect
fallacy / heresy. How do we summary-display the
inter-centre variation? Think!
31
Outlook

• Audiences are conservative when it comes to
  inspecting data in novel ways, and
• the graphs that one person finds informative
  others find unintelligible. Also,
• clinical problems,
• with their human and economic stakes,
• are so diversified. So,

a spectrum of graphical tools ought to
be made available to diagnostic test evaluators.
32
IS THE SUN RISING?

• I HOPE SO
• j.hilden_at_biostat.ku.dk

33
IS THE SUN RISING?
Thank you for your attention. Comments are most
welcome.

• I HOPE SO
• j.hilden_at_biostat.ku.dk