Active Shape Models: Their Training and Applications Cootes, Taylor, et al.

1
Active Shape ModelsTheir Training and
ApplicationsCootes, Taylor, et al.

• Robert Tamburo
• July 6, 2000
• Prelim Presentation

2
Other Deformable Models

• Hand Crafted Models
• Articulated Models
• Active Contour Models Snakes
• Fourier Series Shape Models
• Statistical Models of Shape
• Finite Element Models

3
Motivation Prior Models

• Lack of practicality
• Lack of specificity
• Lack of generality
• Nonspecific class deformation
• Local shape constraints

4
Goals of Active Shape Model (ASM)

• Automated
• Searches images for represented structures
• Classify shapes
• Specific to ranges of variation
• Robust (noisy, cluttered, and occluded image)
• Deform to characteristics of the class
  represented
• Learn specific patterns of variability from a
  training set

5
Goals of ASM (contd.)

• Utilize iterative refinement algorithm
• Apply global shape constraints
• Uncorrelated shape parameters
• Better test for dependence?

6
Point Distribution Model (PDM)

• Captures variability of training set by
  calculating mean shape and main modes of
  variation
• Each mode changes the shape by moving landmarks
  along straight lines through mean positions
• New shapes created by modifying mean shape with
  weighted sums of modes

7
PDM Construction
8
Labeling the Training Set

• Represent example shapes by points
• Point correspondence between shapes

9
Aligning the Training Set

• xi is a vector of n points describing the the ith
  shape in the set
• xi(xi0, yi0, xi1, yi1,, xik,
  yik,,xin-1, yin-1)T
• Minimize
• Ej (xi M(sj, ?j)xk tj)TW(xi
  M(sj, ?j)xk tj)
• Weight matrix used

10
Alignment Algorithm

• Align each shape to first shape by rotation,
  scaling, and translation
• Repeat
• Calculate the mean shape
• Normalize the orientation, scale, and origin of
  the current mean to suitable defaults
• Realign every shape with the current mean
• Until the process converges

11
Mean Normalization

• Ensures 4N constraints on 4N variables
• Equations have unique solutions
• Guarantees convergence
• Independent of initial shape aligned to
• Iterative method vs. direct solution

12
Aligned Shape Statistics

• PDM models cloud variation in 2n space
• Assumptions
• Points lie within Allowable Shape Domain
• Cloud is hyper-ellipsoid (2n-D)

13
Statistics (contd.)

• Center of hyper-ellipsoid is mean shape
• Axes are found using PCA
• Each axis yields a mode of variation
• Defined as , the eigenvectors of covariance
  matrix
• , such that
• ,where is the kth eigenvalue of S

14
Approximation of 2n-D Ellipsoid

• Most variation described by t-modes
• Choose t such that a small number of modes
  accounts for most of the total variance

• If total variance

and the
approximated variance , then
15
Generating New Example Shapes

• Shapes of training set approximated by
• , where is the matrix
  of the first t eigenvectors and
  is a vector of weights
• Vary bk within suitable limits for similar shapes

16
Application of PDMs

• Applied to
• Resistors
• Heart
• Hand
• Worm model

17
Resistor Example

• 32 points
• 3 parameters capture variability

18
Resistor Example (cont.d)

• Lacks structure
• Independence of parameters b1 and b2
• Will generate legal shapes

19
Resistor Example (cont.d)
20
Resistor Example (cont.d)
21
Resistor Example (cont.d)
22
Heart Example

• 66 examples
• 96 points
• Left ventricle
• Right ventricle
• Left atrium
• Traced by cardiologists

23
Heart Example (cont.d)
24
Heart Example (cont.d)

• Varies Width

• Varies Septum

• Vary LV

• Vary Atrium

25
Hand Example

• 18 shapes
• 72 points
• 12 landmarks at fingertips and joints

26
Hand Example (cont.d)

• 96 of variability due to first 6 modes
• First 3 modes
• Vary finger movements

27
Worm Example

• 84 shapes
• Fixed width
• Varying curvature and length

28
Worm Example (cont.d)

• Represented by 12 point
• Breakdown of PDM

29
Worm Example (cont.d)

• Curved cloud
• Mean shape
• Varying width
• Improper length

30
Worm Example (cont.d)

• Linearly independent
• Nonlinear dependence

31
Worm Example

• Effects of varying first 3 parameters
• 1st mode is linear approximation to curvature
• 2nd mode is correction to poor linear
  approximation
• 3rd approximates 2nd order bending

32
PDM Improvements

• Automated labeling
• 3D PDMs
• Nonlinear PDM
• Polynomial Regression PDMs
• Multi-layered PDMs
• Hybrid PDMs
• Chord Length Distribution Model
• Approximation problem

33
PDMs to Search an Image - ASMs

• Estimate initial position of model
• Displace points of model to better fit data
• Adjust model parameters
• Apply global constraints to keep model legal

34
Adjusting Model Points

• Along normal to model boundary proportional to
  edge strength

• Vector of adjustments

35
Calculating Changes in Parameters

• Initial position
• Move X as close to new position (X dX)
• Calculate dx to move X to dX
• Update parameters to better fit image
• Not usually consistent with model constraints
• Residual adjustments made by deformation

36
Model Parameter Space

• Transforms dx to parameter space giving allowable
  changes in parameters, db
• Recall
• Find db such that
• - yields
• Update model parameters within limits

37
Applications

• Medical
• Industrial
• Surveillance
• Biometrics

38
ASM Application to Resistor

• 64 points (32 type III)
• Adjustments made finding strongest edge
• Profile 20 pixels long
• 5 degrees of freedom
• 30, 60, 90, 120 iterations

39
ASM Application to Heart

• Echocardiogram
• 96 points
• 12 degrees of freedom
• Adjustments made finding strongest edge
• Profile 40 pixels long
• Infers missing data (top of ventricle)

40
ASM Application to Hand

• 72 points
• Clutter and occlusions
• 8 degrees of freedom
• Adjustments made finding strongest edge
• Profile 35 pixels long
• 100, 200, 350 iterations

41
Conclusions

• Sensitivity to orientation of object in image to
  model
• Sensitivity to large changes in scale?
• Sensitive to outliers (reject or accept)
• Sensitivity to occlusion
• Quantitative measures of fit
• Overtraining
• Occlusion, cluttering, and noise
• Dependent on boundary strength
• Real time
• Extension to 3rd dimension
• Gray level PDM

42
MR of Brain2

• Improves ASM
• Tests several model hypotheses
• Outlier detection adjustment/removal
• 114 landmark points
• 8 training images
• Model structures of brain together
• Model brain structures

43
MR Brain (cont.d)
44
MR Brain (cont.d)
Separate
Together
45
MR Brain (cont.d)
46
(No Transcript)
47
Thank You For
Your Time!
THE END
48
References

• 1 Cootes, Taylor, et al., Active Shape Models
  Their Training and Application. Computer Vision
  and Image Understanding, V16, N1, January, pp.
  38-59, 1995
• 2 - Duta and Sonka, Segmentation and
  Interpretation of MR Brain Images An Improved
  Active Shape Model. IEEE Transactions on Medical
  Imaging, V17, N6, December 1998

49
Hand Crafted Models

• Built from subcomponents (circles, lines, arcs)
• Some degree of freedom
• May change scale, orientation, size, and position
• Lacks generality
• Detailed knowledge of expected shapes
• Application specific

back
50
Articulated Models

• Built from rigid components connected by sliding
  or rotating joints
• Uses generalized Hough transform
• Limited to a restricted class of variable shapes

back
51
Active Contours Snakes

• Energy minimizing spline curves
• Attracted toward lines and edges
• Constraints on stiffness and elastic parameters
  ensure smoothness and limit degree to which they
  can bend
• Fit using image evidence and applying force to
  the model and minimize energy function
• Uses only local information
• Vulnerable to initial position and noise

back
52
Spline Curves

• Splines are piecewise polynomial functions of
  order d
• Sum of basis functions with applied weights
• Spans joined by knots

back
53
Fourier Series Shape Models

• Models formed from Fourier series
• Fit by minimizing energy function (parameters)
• Contains no prior information
• Not suitable for describing general shapes
• Finite number of terms approximates a square
  corner
• Relationship between variations in shape and
  parameters is not straightforward

back
54
Statistical Models of Shape

• Register landmark points in N-space to
  estimate
• Mean shape
• Covariance between coordinates
• Depends on point sequence

back
55
Finite Element Models

• Model variable objects as physical entities with
  internal stiffness and elasticity
• Build shapes from different modes of vibration
• Easy to construct compact parameterized shapes

next