Medical Image Segmentation

1
Medical Image Segmentation

• Ralph Müller, Ph.D.
• Institute for Biomechanics, ETH Zürich,
  Switzerland

2
Adapted from

• Pham DL, Xu C, Prince JL.Current methods in
  medical image segmentation.Annu Rev Biomed Eng.
  20002315-37

3
Image Segmentation

• Group similar components (such as, pixels in an
  image, image frames in a video) to obtain a
  compact representation.
• Applications Finding tumors, veins, etc. in
  medical images, finding targets in
  satellite/aerial images, finding people in
  surveillance images, summarizing video, etc.
• Methods Thresholding, Clustering, etc.

4
Image Segmentation

• Segmentation algorithms for monochrome images
  generally are based on one of two basic
  properties of gray-scale values
• Discontinuity
• The approach is to partition an image based on
  abrupt changes in gray-scale levels.
• The principal areas of interest within this
  category are detection of isolated points, lines,
  and edges in an image.
• Similarity
• The principal approaches in this category are
  based on thresholding, region growing, and region
  splitting/merging.

5
Definitions

• Image segmentation is the partitioning of an
  image into nonoverlapping, constituent regions
  that are homogeneous with respect to some
  characteristics

6
Definitions
7
Definitions

• When the constraint of connected regions is
  removed, then determining the sets Sk is called
  pixel classification.
• Desirable when disconnected regions belonging to
  the same tissue class require identification.
• Determining the total number of classes K in
  pixel classification can be a difficult problem.
• Mostly based on apriori knowledge of anatomy.

8
Definitions

• Labeling is the process of assigning a meaningful
  designation to each region or class.
• Often performed separately from segmentation.
• In medical imaging, labels are often visually
  obvious and determined on inspection by a
  physician or technician.
• Computer-automated labeling is desirable when
  labels are not obvious or in automat. processes.

9
Dimensionality

• Dimensionality refers to whether a segmentation
  method operates in a 2D or 3d image domain.
• Methods that rely solely on image intensities are
  independent of the image domain.
• Generally, 2D methods are applied to 2D images,
  and 3D methods are applied to 3D img.
• In some cases, 2D methods are applied
  sequentially to the slices of a 3D image.

10
Soft Segm. and Partial-Volume Effects

• Partial-volume effects are artifacts that occur
  where multiple tissues types contribute to a
  single pixel, resulting in blurring of
  intensities.

11
Soft Segm. and Partial-Volume Effects

• The most common approach to addressing
  partial-volume effects is to produce
  segmentations that allow regions or classes to
  overlap, called soft segmentations.
• Soft segmentations retain more information than
  typical hard segmentation from the original image
  by allowing uncertainty in the location of the
  object boundaries.

12
Characteristic and Membership Function
13
Membership Functions

• Membership functions can be derived from
• Fuzzy clustering
• Classifier algorithms
• Statistical algorithms using probability
  functions
• Estimates of partial-volume fractions
• Soft segmentations based on membership functions
  can be easily converted to hard segmentations by
  assigning a pixel to the class with the highest
  membership value.

14
Intensity Inhomogeneities

• Major difficulty of MR imaging is the intensity
  inhomogeneity artifact.
• Degrades performance of methods that assume the
  constant tissue intensity value over image.
• Some methods suggest a prefiltering operation.
• Methods that simultaneously segment the image and
  estimate inhomogeneity off the advantage of being
  able to use intermediate information.

15
Intensity Inhomogeneities

• Two prevailing approaches
• Assumes that mean intensity for each tissue class
  is spatially varied and independent of one
  another.
• Models the inhomogeneities as a multiplicative
  gain field or additive bias field.
• The second approach can also be used for removing
  inhomogeneities by simply multiplication of the
  acquired image by the reciprocal of the estimated
  gain field.

16
(No Transcript)
17
Interaction

• The tradeoff between manual interaction and the
  performance is important consideration in segm.
• Manual interaction can improve accuracy.
• The type of interaction can vary from completely
  manual delineation of an anatomical structure to
  the selection of a seed point for region growing.
• Even automated segmentation requires
  specification of some initial parameters.

18
Validation

• Validation experiments are necessary to quantify
  the performance of a segmentation method.
• This is typically performed with one of two types
  of truth models
• Compare automated segmentation with manually
  obtained segmentations.
• The use of physical or computational phantoms.

19
Phantom Limitations

• Manually obtained segmentations do not guarantee
  perfect truth model because of inherent operator
  flaws
• Physical phantoms provide accurate depiction of
  image acquisition process but typically do not
  present a realistic representation of anatomy.
• Computational phantoms represent anatomy
  realistically, but usually simulate the image
  acquisition process by using simplified models.

20
Figure of Merit

• Once a truth model is available, a figure of
  merit must be defined for quantifying accuracy
  and precision.
• Typical definitions include
• Region information, such as number of pixels
  misclassified.
• Boundary information, such as distance to the
  true boundary.

21
Segmentation Methods

• Thresholding approaches
• Region Growing approaches
• Classifiers
• Clustering approaches
• Markov random fields (MRF) models
• Artificial neural networks
• Deformable models
• Atlas-guided approaches

22
Thresholding

• Suppose that an image, f(x,y), is composed of
  light objects on a dark background, and the
  following figure is the histogram of the image.
• Then, the objects can be extracted by comparing
  pixel values with a threshold T.

23
Thresholding

• It is also possible to extract objects that have
  a specific intensity range using multiple
  thresholds -gt multipthresholding.

24
Thresholding

• Non-uniform image acquisition may change the
  histogram in a way that it becomes impossible to
  segment the image using a single global
  threshold.
• Choosing local threshold values may help.

25
Thresholding
26
Adaptive Thresholding
27
Adaptive Thresholding
Almost constant illumination ?Separation of
objects
28
Region-Oriented Segmentation

• Region Growing
• Region growing is a procedure that groups pixels
  or subregions into larger regions.
• The simplest of these approaches is pixel
  aggregation, which starts with a set of seed
  points and from these grows regions by appending
  to each seed points those neighboring pixels that
  have similar properties (such as gray level,
  texture, color, shape).
• Region growing based techniques are better than
  the edge-based techniques in noisy images where
  edges are difficult to detect.

29
Region-Oriented Segmentation

• Segment the image in different regions Ri
• The regions cover the whole image
• Two regions do not have the same elements
• A region fulfils some property P
• The union of two regions does not satisfy the P

30
Region Growing
Element in L
Seed points

• Define seed point
• Add n-neighbors to list L
• Get and remove top of L
• Test n-neighbors pif p not treatedif
  P(p,R)True then p?L and add p to region else p
  marked boundary
• Go to 2 until L is empty
• Two Regions R and R

Border element
Region element
31
Region Growing

• P is a predicate that defines whether an element
  belongs to a region, i.e.
• Compares the new element with the mean value of
  region.
• Compares the new element with the neighbor value.
• L is ordered, for example, according to P
• We can define several seed points
• If a point touches more than one Ri, a measure
  defines to which region it belongs to.

32
Region Growing

• Current region dominates the growth process.
• Ambiguities around edges of adjacent regions may
  not be resolved correctly.
• Different choices of seeds may give different
  segmentation results.
• Results are completely dependent on the choice of
  the predicate P.

33
Classifiers

• Classifier methods are pattern recognition
  techniques that seek to partition a feature space
  derived from the image by using data with know
  labels
• A feature space is any function of the image
• most commonly this is the image intensity
• Classifiers are known as supervised method
  because they require training data

34
Types of classifiers

• Nearest-neighbor classifier
• k-nearest-neighbor classifier
• same class as majority of k-closest training set
• Parzen window
• Maximum-likelihood or Bayes classifiers

35
Classifiers

• Structure to be segmented possesses distinct
  quantifiable features
• Being noniterative, classifiers are relatively
  computationally efficient
• Can be applied to multichannel images
• Disadvantage is
• that they do not perform any spatial modeling
• that they require manual interaction to obtain
  training data

36
Examples
Extract fish from background
Extract ROI with tumor
Volume, circularity, moments ...
length, width, lightness, weight ...
Salmon
Malign
Sea Bass
Benign
37
Classifier

• Model
• Different models for each class (sea bass longer
  than salmon, benign tumor is smooth circular)
• Take noise out of the sensed data to identify the
  model
• Training samples look at the feature

Salmon/Benign
Sea bass/Malign
t ?
38
Classifier
Salmon/Benign
Sea bass/Malign
t1
t2

• Costs per decision
• Equal - minimum misclassification error t1
• Different costs t2
• Better some salmon with sea bass than otherwise
• Wrong classification of malign tumor is worse
  than benign tumor

39
Feature Extraction

• Multidimensional features improve performance

How many features are necessary?
Salmon/Benign
Sea bass/ Malign
40
Decision Boundary
Salmon/Benign

• Tune decision boundary model
• Complexity
• Overfitting
• Less performance in training data better
  performance in novel patterns

Decision Boundary
Sea bass/ Malign
41
Feature Extraction vs. Classification

• Small number of features ? simpler decision
  regions, easier to train and quicker response
• Ideal Feature Extraction ? Trivial Classifier
• Perfect Classifier ? Simple Features
• General purpose recognition system is a very
  difficult challenge
• Practice defines what is the best approach

42
Clustering

• Perform the same function as classifier methods
  without the use of training data
• Unsupervised methods
• Three commonly used clustering algorithms
• K-means or ISODATA algorithm
• Fuzzy c-means algorithm
• Expectation-maximization (EM) algorithm

43
K-means

• Assumes a fixed number of classes
• Iteratively computes a mean intensity for each
  class
• Segments the image by classifying each pixel in
  the class with the closest mean
• No incorporation of spatial modeling and are
  therefore sensitive to noise

44
Figure 4. Segmentation of a magnetic resonance
brain image. (a) Original image. (b)
Segmentation using the K-means algorithm. (c)
Segmentation using the K-means algorithm with a
Markov random field.
45
Markov random field (MRF) models

• Not a segmentation method but statistical model
  that can be used within segmentation methods
• MRF models the spatial interactions between
  neighboring or nearby pixels
• In medical images most pixels belong to the same
  class as their neighboring pixels
• MRF is often incorporated into clustering
  segmentations such as K-means

46
Deformable Models

• Segmentation methods until now (no knowledge of
  shape
• Thresholding
• Edge based
• Region based
• Deformable models
• Knowledge of the shape of the object

47
Deformable Models

• Initial shape (curve or surface)
• Move the shape according
• Internal forces (curve/surface properties)E.g.
  Curvature to keep the object smooth
• External forces (image properties)E.g. To track
  the object to the boundary
• 2D Snakes Kass, Witkin and Terzopoulos 1987

48
Example

• Animation with a 2D countour adapting to the edge

49
Deformable Models

• Various names for the same
• 2D snakes, active contours, and deformable
  contours ...
• 3D ballons, active surfaces, and deformable
  surfaces ...
• Two main groups
• Parametric deformable models (based on parametric
  form of models)
• Geometric deformable models (curve evolution or
  level sets)

50
Figure 6 An example of using a deformable surface
in the reconstruction of the cerebral cortex.
51
Figure 7 A view of the intersection between the
deformable surface and orthogonal slices of the
MR image.
52
Atlas-Guided Approaches

• Atlas is generated by compiling information on
  the anatomy that requires segmentation
• Atlas is then used as a reference frame for
  segmenting new images
• Conceptually similar to classifier methods but
  implemented in spatial domain rather than feature
  space

53
Atlas-Guided Approaches

• Standard atlas-guided approach treats
  segmentation as registration problem
• Finds one-to-one transformation that maps a
  presegmented atlas image to the target image
• Atlas warping uses sequential application of
  linear and non-linear transformations
• Advantage is that
• labels are transferred with the segmentation
• they provide standard system for morphometry

54
(No Transcript)
55
(No Transcript)
56
Watershed Segmentation Algorithm

• Visualize an image in 3D spatial coordinates and
  gray levels.
• In such a topographic interpretation, there are 3
  types of points
• Points belonging to a regional minimum
• Points at which a drop of water would fall to a
  single minimum. (?The catchment basin or
  watershed of that minimum.)
• Points at which a drop of water would be equally
  likely to fall to more than one minimum. (?The
  divide lines or watershed lines.)

57
Watershed Segmentation Algorithm

• The objective is to find watershed lines.
• The idea is simple
• Suppose that a hole is punched in each regional
  minimum and that the entire topography is flooded
  from below by letting water rise through the
  holes at a uniform rate.
• When rising water in distinct catchment basins is
  about the merge, a dam is built to prevent
  merging. These dam boundaries correspond to the
  watershed lines.

58
Watershed Segmentation Algorithm
59
Watershed Segmentation Algorithm

• Start with all pixels with the lowest possible
  value.
• These form the basis for initial watersheds
• For each intensity level k
• For each group of pixels of intensity k
• If adjacent to exactly one existing region, add
  these pixels to that region
• Else if adjacent to more than one existing
  regions, mark as boundary
• Else start a new region

60
Watershed Segmentation Algorithm
Watershed algorithm might be used on the gradient
image instead of the original image.
61
Watershed Segmentation Algorithm
Due to noise and other local irregularities of
the gradient, oversegmentation might occur.
62
Watershed Segmentation Algorithm
A solution is to limit the number of regional
minima. Use markers to specify the only allowed
regional minima.
63
Watershed Segmentation Algorithm
A solution is to limit the number of regional
minima. Use markers to specify the only allowed
regional minima. (For example, gray-level values
might be used as a marker.)
64
Figure 10 Segmentation in digital mammography.
(a) Digitized mammogram and radiologists
boundary for biopsy-proven malignant tumor. (b)
Result of watershed algorithm. (c) Suspicious
regions determined by automated method. (Images
provided courtesy of CE Priebe.)
65
Conclusions

• Future research will strive towards improving
• accuracy and precision
• computational speed
• automation and reduction of manual interaction
• The big question is whether these system will be
  more often used in the clinical setting
• extensive validation is required
• demonstrate significant performance advantage
• Will not replace physicians but be valuable tools
  i.e. image-guided surgery