Metric approaches to

1
Metric approaches to nonrigid shape analysis
nonrigid
Michael M. Bronstein
Department of Computer Science Technion Israel
Institute of Technology cs.technion.ac.il/mbron
18 February 2008

2
Collaborators
Alexander Bronstein
Ron Kimmel
Alfred Bruckstein
Dan Raviv
3
Nonrigid world from macro to nano
4
Feuille-caillou-ciseaux
Rock
Scissors
Paper
5
Feuille-caillou-ciseaux
Hands
Rock
Scissors
Paper
6
Invariance
Rigid motion
Elastic deformation
Inelastic deformation
7
Metric model
Shape metric space
Invariance isometry w.r.t.
EXTRINSIC GEOMETRY

• Euclidean metric
• Isometry rigid motion

8
Metric model
Shape metric space
Invariance isometry w.r.t.
INTRINSIC GEOMETRY
EXTRINSIC GEOMETRY

• Geodesic metric
• Isometry inelastic deformation

• Euclidean metric
• Isometry rigid motion

9
Extrinsic vs. intrinsic similarity
INTRINSIC SIMILARITY
EXTRINSIC SIMILARITY

• Similarity isometry
• Invariant to inelastic deformation

• Similarity congruence
• Invariant to rigid motion

10
Extrinsic similarity
Subsets of the same metric space
Minimize the Hausdorff distance over all the
possible congruences
Rotation
Translation
Chen Medioni, 1991 Besl McKay, PAMI 1992
11
Iterative closest point (ICP)

• Correspondence

• Best rigid alignment

Chen Medioni, 1991 Besl McKay, PAMI 1992
12
And now, intrinsic similarity
INTRINSIC SIMILARITY
EXTRINSIC SIMILARITY
Part of the same metric space
Two different metric spaces
SOLUTION Find a representation of
and in a common metric space
13
Canonical forms
Multidimensional scaling (MDS)
Isometric embedding
A. Elad, R. Kimmel, CVPR 2001
14
Canonical forms
?
INTRINSIC SIMILARITY
Compute canonical forms
EXTRINSIC SIMILARITY OF CANONICAL FORMS
INTRINSIC SIMILARITY
A. Elad, R. Kimmel, CVPR 2001
15
Mapmakers problem
?
16
Mapmakers problem
Bad news exact canonical forms usually do not
exist (embedding error)
Karl Friedrich Gauss (1777-1985)
17
Gromov-Hausdorff distance
Isometric embedding
Isometric embedding
Compute Hausdorff distance
Gromov-Hausdorff distance
Change the embedding space
M. Gromov, 1981
COMPUTATIONALLY UNTRACTABLE!
18
Gromov-Hausdorff distance (cont.)
where
F. Memoli, G. Sapiro, FCM 2005 A. Bronstein, M.
Bronstein, R. Kimmel, PNAS 2006, SIAM JSC 2006
19
Minimum distortion embedding
Multidimensional scaling (MDS)
Generalized multidimensional scaling (GMDS)
A. Bronstein, M. Bronstein, R. Kimmel, PNAS
2006, SIAM JSC 2006
20
MDS vs GMDS
Generalized MDS
MDS

• Stress

• Generalized stress

• Analytic expression for
• Nonconvex problem
• Variables Euclidean coordinates
• of the points

• must be interpolated
• Nonconvex problem
• Variables points on in
• barycentric coordinates

21
Gromov-Hausdorff vs. canonical forms
CANONICAL FORMS (MDS, 500 points)
GROMOV-HAUSDORFF DISTANCE (GMDS, 50 points)
A. Bronstein, M. Bronstein, R. Kimmel, SIAM JSC
2006
22
Extrinsic similarity limitations
EXTRINSICALLY SIMILAR
EXTRINSICALLY DISSIMILAR
Suitable for nearly rigid shapes
Unsuitable for nonrigid shapes
23
Intrinsic similarity limitations
Suitable for near-isometric shape deformations
Unsuitable for deformations modifying shape
topology
24
Topological noise
3D scan
Incorrect reconstruction
Correct reconstruction
Images A. Sharf et al., SIGGRAPH 2007
25
Extrinsically similar Intrinsically dissimilar
Extrinsically dissimilar Intrinsically similar
Extrinsically dissimilar Intrinsically dissimilar
Desired result
THIS IS THE SAME SHAPE!
A. Bronstein, M. Bronstein, R. Kimmel, ICCV 2007
26
Joint extrinsic / intrinsic similarity
?
DEFORM X TO MATCH Y EXTRINSICALLY
CONSTRAIN THE DEFORMATION TO BE AS ISOMETRIC AS
POSSIBLE
A. Bronstein, M. Bronstein, R. Kimmel, ICCV 2007
27
How to fit a glove?
Misfit Extrinsic dissimilarity
Stretching Intrinsic dissimilarity
A. Bronstein, M. Bronstein, R. Kimmel, ICCV 2007
28
If it doesnt fit, you must acquit!
Image Associated Press
29
?
Extrinsic dissimilarity
Intrinsic dissimilarity
A. Bronstein, M. Bronstein, R. Kimmel, ICCV 2007
30
Computation of the joint similarity

• Optimization variable the deformed shape vertex
  coordinates
• Assuming has the connectivity of
• Split into computation of and
• Gradients w.r.t. are required for
  optimization
• Intrinsic distance distortion only
  (correspondence is known)

A. Bronstein, M. Bronstein, R. Kimmel, ICCV 2007
31
Numerical example dataset
topology change
Data tosca.cs.technion.ac.il
32
Numerical example intrinsic similarity
no topological changes
33
Numerical example intrinsic similarity
Insensitive to strong deformations
Sensitive to topological changes
topology change
topology-preserving
34
Numerical example extrinsic similarity
Sensitive to strong deformations
Insensitive to topological changes
topology change
topology-preserving
35
Numerical example joint similarity
Insensitive to topological changes...
and to strong deformations
topology change
topology-preserving
36
An identity crisis
?
?
I am a centaur.
Am I human?
Am I equine?
Partial similarity is a non-metric relation
Yes, Im partially human.
Yes, Im partially equine.
37
Partial similarity
X and Y are partially similar
X and Y have
significant
similar parts
Illustration Herluf Bidstrup
38
Recognition by parts

• Divide the shapes into significant parts
• Compare each part separately using a full
  similarity criterion
• Merge the part similarities

Problem how to find significant parts?
39
Significance
Measure of insignificance of the parts
INSIGNIFICANCE
A. Bronstein, M. Bronstein, A. Bruckstein, R.
Kimmel, SSVM 2007, IJCV 2008
40
Multicriterion optimization
Measure of insignificance of the parts
Intrinsic full dissimilarity between parts
Insignificance of parts
Simultaneously minimize dissimilarity and
insignificance over all possible parts
DISSIMILARITY
INSIGNIFICANCE
A. Bronstein, M. Bronstein, A. Bruckstein, R.
Kimmel, SSVM 2007, IJCV 2008
41
Partial similarity
Intrinsic full dissimilarity between parts
Insignificance of parts
Simultaneously minimize dissimilarity and
insignificance over all possible parts
DISSIMILARITY
Pareto frontier set-valued similarity
INSIGNIFICANCE
UTOPIA
A. Bronstein, M. Bronstein, A. Bruckstein, R.
Kimmel, SSVM 2007, IJCV 2008
42
Scalar- vs. set-valued similarity
DISSIMILARITY
INSIGNIFICANCE
UTOPIA
A. Bronstein, M. Bronstein, A. Bruckstein, R.
Kimmel, SSVM 2007, IJCV 2008
43
Scalar-valued partial similarity
DISSIMILARITY
Scalar partial similarity distance from Pareto
frontier to utopia point
INSIGNIFICANCE
UTOPIA
A. Bronstein, M. Bronstein, A. Bruckstein, R.
Kimmel, SSVM 2007, IJCV 2008
44
Fuzzy setting
CRISP PART
FUZZY PART
A. Bronstein, M. Bronstein, A. Bruckstein, R.
Kimmel, SSVM 2007, IJCV 2008
45
Partial similarity computation

• Find correspondence minimizing
  distortion between current parts

• Select parts minimizing the
  distortion with current correspondence
  subject to

A. Bronstein, M. Bronstein, A. Bruckstein, R.
Kimmel, IJCV 2008
46
Partial similarity computation (cont.)

• Intrinsic correspondence

• Part selection

A. Bronstein, M. Bronstein, A. Bruckstein, R.
Kimmel, IJCV 2008
47
Partial similarity
FULL SIMILARITY
SCALAR PARTIAL SIMILARITY
Partial similarity
A. Bronstein, M. Bronstein, A. Bruckstein, R.
Kimmel, IJCV 2007
48
Not only size matters
What is better?...
Many small parts
or one large part?
A. Bronstein, M. Bronstein, Submitted to CVPR
2008
49
Boundary regularization
Measure of part irregularity
Large irregularity
Small irregularity
Mumford-Shah functional (fuzzy formulation)
A. Bronstein, M. Bronstein, Submitted to CVPR
2008
50
Regularized partial similarity
DISSIMILARITY
INSIGNIFICANCE
IRREGULARITY
A. Bronstein, M. Bronstein, Submitted to CVPR
2008
51
Correspondence
52
Correspondence
?
53
Minimum distortion correspondence
A. Bronstein, M. Bronstein, R. Kimmel, IEEE
TVCG, 2006
54
Minimum distortion correspondence
A. Bronstein, M. Bronstein, R. Kimmel, IEEE
TVCG, 2006
55
Partial correspondence
A. Bronstein, M. Bronstein, R. Kimmel, IEEE
TVCG, 2006
56
Partial correspondence
A. Bronstein, M. Bronstein, R. Kimmel, IEEE
TVCG, 2006
57
Texture transfer
Reference
Transferred texture
A. Bronstein, M. Bronstein, R. Kimmel, IEEE
TVCG, 2006
58
Virtual body painting
A. Bronstein, M. Bronstein, R. Kimmel, IEEE
TVCG, 2006
59
Shape space
Extrapolation
Interpolation
A. Bronstein, M. Bronstein, R. Kimmel, IEEE
TVCG, 2006
60
Metamorphing
61
Face caricaturization
Interpolation
Extrapolation
Extrapolation
0
1
1.5
0.5
-0.5
62
Shameless advertisement

• Published by Springer
• To appear in early 2008
• 350 pages
• Over 50 illustrations
• Color figures

Additional information
tosca.cs.technion.ac.il
63
Workshop on Nonrigid Shape Analysis and
Deformable Image Alignment (NORDIA)
June 2008, Anchorage, Alaska
in conjunction with CVPR08