Imaging Informatics

1
Imaging Informatics

• Marine Biological Laboratory / National Library
  of Medicine
• BioMedical Informatics Course
• Woods Hole, MA
• June 5, 2008
• Zohara Cohen, PhD
• Program Director, NIBIB / NIH / DHHS

2
Biomedical Informatics is...

• The art and science of organizing knowledge of
  human health and disease, and making it useful
  for problem solving

Imaging Informatics is
3
Health Information in the Case
4
A Working Definition

• Using computers to get at the useful information
  available in medical images.

Using computers to help us deal with the data
overload that current medical imaging technology
presents Using computers to help us extract even
more information than we thought we had acquired
with those imaging methods
5
Outline

• What is imaging informatics?
• Too much information!
• Too little information!
• Summary and conclusion

6
Outline

• What is imaging informatics?
• Too much information!
• Too little information!
• Summary and conclusion

• Image segmentation
• Teleradiology / Telepathology
• Satisfaction of Search

7
Image Segmentation
Image segmentation is the process of dividing
images up into meaningful subsets that correspond
to surfaces or objects
Manual Segmentation Pro often most accurate
method available Con labor-intensive and
time-consuming requires subject expertise high
intra and inter-observer variability
8
Challenges Image Artifacts
9
Tissue Classification
10
Cortical Subcortical Parcellation
11
Incorporating Prior Information into Automatic
SegmentationPolina Goland, Killian Pohl, MIT and
BWH, U54 EB005149
Tool
MRI

Label Map
Atlas
12
Hierarchical Analysis
13
Design of Algorithm
14
Modify the Tree
15
Segmentation of 31 Structures
Pohl et al., ISBI 04
16
Application Neuroscience Studies
17
Segmentation in 3D Slicer
National Alliance for Medical Image Computing
(NA-MIC)
18
Autosegmentation for Head and Neck Radiotherapy
PlanningBenoit Dawant, Vanderbilt University,
R01 EB006193
19
Outline

• What is imaging informatics?
• Too much information!
• Too little information!
• Summary and conclusion

• Image segmentation
• Teleradiology / Telepathology
• Satisfaction of Search

20
Image Compression for TelepathologyElizabeth
Krupinski, University of Arizona, R01 EB008055

• Example
• Tissue sample size 2.25 cm2 (1-4 cm2 is
  typical)
• Scanner samples resolution 0.47 microns/pixel
• Color resolution 3 bytes/pixel (1 byte per
  color)
• 200 MB to 1 GB of data for one image
• Why compress?
• transmission rates
• retrieval rate
• storage space they occupy
• concern that DICOM cannot handle images larger
  than 2 GB
• It is difficult, if not impossible, to define a
  single minimum level of compression for use
  across all clinical questions.

21
JND Point for Virtual Pathology Slides
22
JND Point for Virtual Pathology Slides
23
Human Observers of Breat Biopsies
Krupinski EA, et al., Hum Path, 2006.
The red circles are those from the pathologists,
the green squares from the residents and the blue
triangles from the medical students. The black
circles represent areas that were considered
common selections.
24
Model Observers
Krupinski EA, et al., Hum Path, 2006.
Regions of interest selected by a preliminary
model algorithm for the virtual slide shown in
previous slide.
25
Outline

• What is imaging informatics?
• Too much information!
• Too little information!
• Summary and conclusion

• Image segmentation
• Teleradiology / Telepathology
• Satisfaction of Search

26
(No Transcript)
27
(No Transcript)
28
Observer Errors in Diagnostic Imaging Kevin
Berbaum, University of Iowa, R01EB000863,
R01EB006638

• Two primary ways to reduce diagnostic error in
  radiology
• improve imaging technology so that abnormalities
  are more visible
• improve the interpretive process itself
• Many diagnositc errors have been attributed to
  satisfaction of search, which occurs when a
  lesion is not reported because discovery of
  another abnormality has satisfied the goal of
  the search.
• With the increasing number of imaging studies and
  increasing amounts of information available,
  satisfaction of search is expected to be a
  substantial contributor to observer error.

29
(No Transcript)
30
(No Transcript)
31
WorkstationJ
32
New Tools for Perception Studies in Advanced
Imaging
33
Outline

• What is imaging informatics?
• Too much information!
• Too little information!
• Summary and conclusion

• Image Registration
• Knowledge repositories
• Content-based image retrieval

34
Outline

• What is imaging informatics?
• Too much information!
• Too little information!
• Summary and conclusion

• Image Registration
• Knowledge repositories
• Content-based image retrieval

35
Image Registration

• Image registration is the process of transforming
  different image sets into a common coordinate
  system so that data from the two sets can be
  compared or integrated.

from Matlab Central File Exchange
36
Image Registration

• How are image registration methods classified?
• Single modality vs. multiple modality

• Single subject with multiple time points vs.
  multiple subjects
• Single subject vs. subject and atlas/population
  template
• 2D vs. 3D
• Rigid vs. nonrigid (aka elastic)
• Area-based vs. feature-based
• Linear vs. nonlinear transformation model

37
Improved Targeting in Minimally Invasive Radio
Frequency AblationNeculai Archip, Brigham and
Womens Hospital, R03 EB006515

• Radio Frequency Ablation performed under CT
  guidance
• targeting tumor with diameter smaller than 3 cm
• limited conspicuity of tumor margins on
  unenhanced CT

Pre-procedural contrast- enhanced MRI
Intra-procedural un-enhanced CT
38
Neculai Archip, Brigham and Womens Hospital, R03
EB006515

• Existent rigid registration algorithms
• Pros robust and fast Cons prone to errors due
  to liver deformation
• Existent non-rigid registration algorithms
• Pros accurate alignment between images Cons
  slow for clinical use

50-year-old male with right hepatectomy due to
metastatic colon cancer. On this contrast
enhanced MRI, there is a metastatic mass at the
surgical margin. By fusing pre-procedural MRI and
intra-procedural CT during RF ablations, position
of RF applicator can be accurately depicted with
respect to the tumor margin.
39
Non-rigid Image Registration Evaluation Project
(NIREP)Gary E. Christensen, The University of
Iowa, R33 EB004126

• Gold standards rarely exist for image
  registration.
• NIREP to develop, establish, maintain, and
  endorse a standardized set of relevant benchmarks
  and metrics for performance evaluation.
• Tools and evaluation databses are distributed via
  www.nirep.org.

40
Outline

• What is imaging informatics?
• Too much information!
• Too little information!
• Summary and conclusion

• Image Registration
• Knowledge repositories
• Content-based image retrieval

41
Computer-Assisted Functional NeurosurgeryBenoit
Dawant, Vanderbilt University, R01 EB006136
Patient Selection
Outcome Assessment
Deep Brain Stimulation Repository
Planning
Programming
Placement
42
Snapshot of the planning software developed to
design platforms used for the placement of Deep
Brain Stimulators
43
Knowledge RepositoryMIND Research Network
Database
44
Outline

• What is imaging informatics?
• Too much information!
• Too little information!
• Summary and conclusion

• What is imaging informatics?
• Too much information!
• Too little information!
• Summary and conclusion

• Image Registration
• Knowledge repositories
• Content-based image retrieval

45
Content-Based Image Retrieval (CBIR)

• Methods to enable the (automatic) capture of
  information (e.g. color, shape, texture, size)
  about an image that allows you to relate it to
  other images.
• ? index and query images
• ? retrieve them from a large database
• ? assists diagnosis
• A method to deal with both the absence of
  information, as well as data overload.

46
Content Based Indexing and Retrieval of
Cervigrams Greenspan, Gordon, et al. Tel Aviv
UniversityAntani, Long at NLM
cervicogram
Worldwide, 275,000 women per year die of cervical
cancer. This constitutes 4 of cancer deaths for
women.
47
CBIR for Information Management of the Uterine
Cervix Database
100,000 cervigrams with diagnostic classification
from longitudinal multi-year study carried out in
Costa Rica and the United-States
Data (NCI)

• Tissue Segmentation
• Parameter extraction per tissue (size,
  color-texture specifications)

Method (NLMTel Aviv University)
Content-Based Image Indexing and Retrieval (CBIR)
Example Find all images where the regions of
type 1 entail at least 0.1 of the total image
area.
web-based database of digitized cervix images
Goal (NLM)
48
CBIR Typical Approach

• Choose features
• e.g. color, texture, shape
• Choose representation of features
• e.g. color histograms, clusters texture
  intensity distribution in a region
• Define equivalence of features (similarity)
• Associate words with features (semantics)

49
CBIR Indexing Phase
Region Localization Segmentation
50
CBIR Indexing Phase
Region Localization Segmentation

• Feature organization improves retrieval
  efficiency
• Approaches include
• SLIM Trees
• kD-Trees
• Aglomerative clustering
• R-Trees

Feature Computation Representation

• Color Features
• color moments
• histograms
• Texture Features
• wavelet-based
• co-occurrence matrices
• Size Features
• absolute
• relative
• Location
• Feature Combination

Feature Organization
Multimedia DB Development
51
CBIR Retrieval Phase

• Query Paradigms
• Query by Image Example
• Query by Feature Color, Texture, Shape,
  Size, Location
• Weighted Features
• Hybrid (Text Image) Query
• Retrieval Strategies
• Target and Range Search
• Interactive Refinement
• Result combination

Query Comprehension
Segmentation
Feature Extraction Representation
52
CBIR Retrieval Phase
Query Comprehension
Segmentation
Feature Extraction Representation
Search Multimedia DB
User Feedback
Results Visualization
53
CervigramFinder Query Specification
User can specify type of desired region e.g.
blood, mucus, acetowhite lesion, etc.
54
CervigramFinder Query Specification
User can also specify weights (importance) of
various image features
55
CervigramFinder Results Screen
56
BRISC
Daniela Raicu, DePaul University, supported by
NSF Grant 0453456 http//brisc.sourceforge.net/
57
Lung Imaging Database Consortium
58
XML Nodule Data
lt?xml version"1.0"?gt ltnodulesgt ltnodulegt
ltnoduleIDgt4026lt/noduleIDgt ltnodule_nogt1lt/nodule
_nogt ltminXgt87lt/minXgt ltmaxXgt95lt/maxXgt
ltminYgt287lt/minYgt ltmaxYgt292lt/maxYgt
ltactualWidthgt5.080077lt/actualWidthgt
ltactualHeightgt3.386718lt/actualHeightgt
ltseriesInstanceIDgt1.3.6.1.4.1.9328.50.3.272lt/serie
sInstanceIDgt ltimageSOP_UIDgt1.3.6.1.4.1.9328.50
.3.289lt/imageSOP_UIDgt ltannotationsgt
ltcalcificationgt6lt/calcificationgt
ltinternalStructuregt1lt/internalStructuregt
... lt/annotationsgt ltharalickgt
ltcontrastgt192.219776785714lt/contrastgt
ltcorrelationgt-0.274325990655673lt/correlationgt
... lt/haralickgt ltgaborgt
ltgaborhist orientation"0" scale"0"gt0.0555
0.1111 0.2037 0.4074 0.0925 0.0555 0.0555
0.01851lt/gaborhistgt ltgabormean
orientation"0" scale"0"gt249.628474435425lt/gaborm
eangt ...
59
Schematic of BRISC retrieval system
60
Snapshot of BRISC Interface
Retrieved Images
61
Retreival Precision
62
Filling the Semantic GapDaniela Raicu, DePaul
University, supported by NSF Grant 0453456
63
LIDC Semantic Concepts
64
Extracted Image Features
65
Correlations between Image Features and Concepts
66
Texture Regression Model
67
Outline

• What is imaging informatics?
• Too much information!
• Too little information!
• Summary and conclusion

68
Applications of Imaging Informatics

• basic science research
• diagnosis
• surgical planning
• treatment guidance
• treatment assessment

69
To Realize Full Benefit

• development of cutting-edge technologies
• assessment and validation
• FDA approval
• integration into clinical workflow
• interoperability
• human factors
• training and adoption
• psychosocial considerations
• policies and incentives

70
Zohara Cohen, PhD

• Program DirectorDivision of Discovery Science
  TechnologyNational Institute of Biomedical
  Imaging and BioengineeringNational Institutes of
  Health
• zcohen_at_mail.nih.gov
• 301-402-1127

71
AAAS Science and Technology Policy Fellowships