Some Rambling (M

1
Some Rambling (Müjdat)and Some More Serious
Discussion (Ayres, Junmo, Walter)on Shape
Priors
2
Desire to use Shape Priors in Segmentation

• The posterior

• The most common (implicit) prior used is the
  curve length penalty

• Want to be able to use better prior models

3
Challenges and remarks

• Need probabilistic descriptions in the space of
  shapes
• A non-linear, infinite-dimensional manifold
• Distance (similarity) measures in the shape space
• Of course, a statistical description for shapes
  has uses other than segmentation as well
• Sampling from a shape density
• Recognition of objects
• Completion of incomplete shapes

4
The PCA World

• Cootes Taylor
• Shape representation using marker points
• Leventon, Tsai
• PCA and level sets
• More
• Want a more principled approach

5
Some Literature

• Active shape models - their training and
  application,
• T. Cootes, C. J. Taylor, D.H. Cooper, J. Graham,
  1995.
• Embedding Gestalt laws in Markov random fields,
• Song-Chun Zhu, 1999.
• On the incorporation of shape into geometric
  active contours,
• Y. Chen, H.D. Tagare et al., 2001.
• Image segmentation based on prior probabilistic
  shape models,
• A. Litvin and W. C. Karl, 2002.
• Shape priors for level set representations,
• M. Rousson and N. Paragios, ECCV, 2002.
• Nonlinear shape statistics in Mumford-Shah based
  segmentation,
• D. Cremers, T. Kohlberger, and C. Schnorr, 2002.
• Geometric analysis of constrained curves for
  image understanding,
• A. Srivastava, W. Mio, E. Klassen, X. Liu, 2003.
• Analysis of planar shapes using geodesic paths
  on shape spaces,
• E. Klassen, A. Srivastava, and W. Mio (in
  review)
• Gaussian distributions on Lie groups and their
  application to stat. shape analysis,
• P. T. Fletcher, S. Joshi, C. Lu, and S. Pizer,
  2003.

6
Overview of Anuj Srivastavas Work

• Specify a space of continuous curves with
  constraints (e.g. simple closed) and exploit the
  differential geometry of this space
• Use geodesic paths for deformations between
  curves and to compute distances do not have
  analytical exps. for geodesics
• To move in the shape manifold, first move in a
  linear space and then project back (using
  tangents/normals)
• Given two curves, solve an optimization problem
  to find the geodesic path between them (find a
  local min)
• Build statistical descriptions based on Karcher
  means
• covariance of (Fourier coefficients of) tangent
  vectors
• Use such descriptions as priors (very
  preliminary)
• Some current limitations
• Cannot handle topological changes
• Extension to surfaces in 3-D not straightforward

7
Outline

• Some metrics proposed for shape similarity
  (Junmo)
• Anuj Srivastavas work
• Geometric representations of curves
• and shape spaces (Walter)
• Tangents, normals, geodesics (Ayres)
• Statistical models and application to
  segmentation (Junmo)
• Brief highlights from Pizers work (Walter)
• Discussion on all of this

8
Some Metrics on the Space of Shapes

• Notion of similarity between shapes
• Basic task of vision system is to recognize
  similar objects which belong to the same
  category.
• Describing shapes landmarks, level set
• Shape can be described as
• There are several metrics
• for two shapes

9
Hausdorff Metric

• Given
• Very sensitive to any outlier points in and

10
Template Metric

• Totally insensitive to outliers

11
Transport Metric

• Fill with stuff and find the shortest
  paths along which to move this stuff so that it
  now fills

12
Optimal Diffeomorphism

• If and are topologically different, the
  distance would be infinite.
• E.g. is minus a pinhole
• E.g. A small break cuts a shape into two

13
Geometric Representations of Curves and Shape
Spaces

• Restrict attention to closed curves in R2.
• Classify curves which differ only by orientation
  preserving rigid motions (rotation and
  translation) and uniform scaling as the same
  shape.
• Consider two different representations of planar
  curves for simple closed curves
• Using direction functions.
• Using curvature functions.

14
Preliminaries

• First, the issue of scaling is resolved by fixing
  the length of all curves to 2?.
• Curves are parameterized by arc length
• with period 2?
• Define the unit tangent function
• where S1 is the unit circle,
• where ?(s) is the direction function.
• ?(s2?) - ?(s) 2?n (n rotation index 1)

15
Curves to Consider

• Consider entire set of curves with rotation index
  1 because this set is complete.
• Contains its own limit points.
• Note that the set of simple closed curves is an
  open subset of this larger set.

16
Shape Using Direction Functions

• Direction function for S1 is ?0(s) s. All
  other closed curves have direction function
• ? ?0 f, where f is L2 periodic on 0,2?.
• To adjust for rigid rotations, restrict attention
  to ? s.t.
• To ensure curve closure, require

17
More Direction Functions

• Define a map
  by
• The pre-shape space C1 is ?1-1(?,0,0).
• Multiple elements of C1 may denote the same
  shape. An adjustment of the reference point
  (s0) handles this.

18
Geometry of Shape Manifolds

• Constraints define a manifold embedded in
• q0 L2
• Move along manifold by moving in tangent space
  and projecting back to manifold
• Tangent space is infinite dimensional, but normal
  space is characterized by three constraints
  defined in f1

19
Tangents and Normals

• The derivative of f1 in the direction of f at ?
  is
• Implies df1 is surjective
• If f is orthogonal to 1, sin q, cos q, then
  df10 in the direction of f and hence f is in the
  tangent space

20
Projections

• Want to find the closest element in C1 to an
  arbitrary q ? q0 L2
• Basic idea move orthogonal to level sets so
  projections under f form a straight line in R3
• For a point b ? R3, we define the level set as
• Let b1(p,0,0). Then its level set is the
  preshape space C1

21
Approximate Projections

• If points are close to C1, then one can use a
  faster method
• Let dq be the normal vector at q for which
  f(qdq)b1. Can do first order approximation to
  compute this
• Approximate Jacobian as

22
Iterative algorithm

• Define the residual (error) vector as
• Then
• where
• Iteratively update q dq? q until the error goes
  to zero
• Call this projection operator P

23
Example Projections
Fig. 1 Projections of arbitrary curves into C1
24
Geodesics

• Definition For a manifold embedded in Euclidean
  space, a geodesic is a constant speed curve whose
  acceleration vector is always perpendicular to
  the manifold
• Define the metric between two shapes as the
  distance along the manifold between the shapes
  with respect to the L2 inner product
• Nice features
• Defined for all closed curves
• Interpolants are closed curves
• Finds geodesics in a local sense, not necessarily
  global

25
Paths from initial conditions

• Assume we have a q in C1 and an f in the tangent
  space
• Approximate geodesic along manifold by moving to
  qfDt and projecting that back onto the manifold
  (Dt is step size)
• So q(tDt) P(q(t)f(t)Dt)

26
Transporting the tangent vector

• Now f(t) is not in the tangent space of q(tDt)
• Two conditions for a geodesic
• The acceleration vector must be perpendicular to
  the manifold simply project f into the next
  tangent space
• The curve must move at constant speed
  renormalize so f(t1)f(t)
• hk is the orthonormal basis of the normal space

27
Geodesics on shape spaces

• S1 is a quotient space of C1 under actions of S1
  by isometries, so finding geodesics in S1
  equivalent to finding geodesics in C1 which are
  orthogonal to S1 orbits
• S1 acting by isometries implies that if a
  geodesic in preshape space is orthogonal to one
  S1 orbit, its orthogonal to all S1 orbits which
  it meets
• So now normal space has one additional component
  spanned by
• The algorithm is the same as detailed earlier
  except with an expanded normal space

28
Geodesics between shapes

• We know how to generate geodesic paths given
• q and f
• Now we want to construct a geodesic path from
• q1 to q2
• So we need to find all f that lead from q1 to an
  S1 orbit of q2 in unit time, and then choose the
  one that leads to the shortest path
• Let Y define the geodesic flow, with ?(q1,0,f)q1
  as the initial condition
• We then want Y(q1,1,f)q2

29
Finding the geodesic

• Define an error functional which measures how
  close we are to the target at t1
• Choose the geodesic as the flow Y which has the
  smallest initial velocity f
• i.e., min f s.t. Hf0
• Hard because infinite dimensional search

30
Fourier decomposition

• f ? L2, so it has a Fourier decomposition
• Approximate f with its first m1 cosine
  components and its first m sine components
• Let a be the vector containing all of the Fourier
  coefficients
• Now optimization problem is min a s.t. Ha0

31
Geodesic paths
Fig. 2 Geodesic paths between two shapes
32
Statistical Modeling of Shapes

• Given example shapes
• Mean shape
• Shape variation
• Shape prior
• Sampling from the prior
• Using shape prior for segmentation of occluded
  images

33
Mean Shape

• Given the geodesic distance function
  ,
• Karcher mean of shapes is defined to be a
  shape
• for which the variance function
• is a local minimum
• The Karcher mean exists, but may not be unique

34
Variances on Shape Spaces

• Model the variation from the mean shape
• as ,
• an element of the tangent space at the mean shape
• Represent by its Fourier expansion
• Model as multivariate normal with
  mean 0 and covariance matrix

35
Shape Sampling

• Sample Fourier coefficients of tangent vectors
  from the multivariate normal distribution
• Move along the geodesic path starting from the
  mean shape in the direction of by distance
  

36
Shape Sampling Examples
Random samples from the Gaussian model
Mean shape
Observed shapes
37
Shape Prior

• the space of curves (larger than shape
  space)
• can be represented as pairs
• parameters for translation, rotation,
  and scaling
• the shape
• Gaussian density with a mean shape with the
  shape dispersion

38
Bayesian Discovery of Objects
39
Some closing thoughts on Anuj Srivastavas work

• Most energy has been spent on manipulating the
  shape manifold
• Work on using these models as priors preliminary
• Non-diagonal covariance explore modes of
  variation
• Non-Gaussian?
• Mean in manifold?
• Other thoughts
• Non-Fourier representations for tangents KL
  expansion?
• Could similar ideas be used with representations
  based on signed distance functions?

40
Overview of Steve Pizers Work

• Medial representations (m-reps) are used to model
  the geometry of anatomical objects.
• Medial parameters are not in a Euclidean space
  so, PCA cannot be used. However, m-reps model
  parameters are elements of a Lie group.
• Gaussian distributions on this Lie group are
  considered, with the max likelihood estimates of
  mean and covariance derived.
• Similar to PCA for Euclidean spaces, principal
  geodesic analysis (PGA) on Lie groups are defined
  for the study of anatomical variability.
• Framework is applied to hippocampi in a
  schizophrenia study.
• 86 m-rep figures are first aligned
  (translation/rotation/scaling)
• Intrinsic mean is then computed
• PGA (modes of variability) are then computed
• Results yield smoother deformations when compared
  with PCA

41
Medial Representation

• Introduced by Blum (1978), a 3-D object is
  represented by a set of connected continuous
  medial manifolds formed by the centers of all
  spheres are are interior to the object and
  tangent to the object boundary at two or more
  points. The figure below illustrates this

42
Medial Atom Lie Groups

• A medial atom is represented by
• The location in space (R3)
• The radius of the sphere (R)
• The local frame (SO(3))
• The object angle (SO(2))
• R3 is a Lie group under vector addition, R is a
  multiplicative Lie group, and SO(2) SO(3) are
  Lie groups under composition of rotations.
• The direct product of Lie groups is a Lie group.

43
Lie Groups and Lie Algebras

• A Lie group is a group G that is a finite-dim
  manifold such that the two group operations of G,
  multiplication and inverse, are C2 mappings.
• If e is the identity of G, the tangent space at e
  forms a Lie algebra. The exponential map
  provides a method for mapping vectors in the
  tangent space into the Lie group.

44
Alignment and PGA

• Translation each model is situated so that the
  average of its medial atoms is at the origin
• Rotation and scaling are done in a manner which
  minimizes the total sum-of-squared distances
  between m-rep figures.
• After alignment, principal directions in the
  geodesic are computed, and the analog to PCA is
  performed.

45
Results

• The analysis, performed on 86 aligned hippocampus
  m-reps, shows smooth deformations (compared with
  PCA). The mean shape is top left, the medial
  atoms are overlaid lower left, and the first 3
  PGA modes are shown right.

46
Shape Using Curvature Functions

• Alternatively, curves can be represented by
  curvature fcns. Since the rotation index is 1,
• Using , the closure
  condition

47
More Curvature Functions

• Define a map
  by
• then the pre-shape space C2 is ?2-1(2?,0,0).
• As with direction functions, different
• placements of s0 result in different C2 shapes.
• Thus, re-parameterization is needed.