Template based shape descriptor

1
Template based shape descriptor

• Raif Rustamov
• Department of Mathematics and Computer Science
• Drew University, Madison, NJ, USA

2
Components of descriptors in general

• Selection of surface feature
• Mapping
• Signal Processing
• Need this discussion to set up the context for
  our approach

3
Selection of surface feature

• A function on the surface that captures a
  property relevant to shape description
• constant function (restriction of the surface's
  characteristic function to the surface itself)
• distance to the center of mass
• curvature
• components of the normal vector
• We refer to the selected function as the feature
  function.

4
Mapping

• The feature function is used to construct a new
  function defined on some predetermined domain
• The new domain called the mapping domain
• the new function the mapped feature function
• Common mapping domains
• Spheres
• Planes
• the 3D space (surface's bounding volume)
• surface itself
• Mapping procedures
• projection
• Identity, if mapping domain surface itself

5
Signal Processing

• Extract concise noise-robust numerical descriptor
  from the mapped feature function.
• Depends on the mapping domain
• Sphere Spherical Harmonic Transform
• Plane or box volume 2D or 3D Fourier transform
• Ball volume 3D Zernike Transform.
• Mapped feature function is expanded in a series
  in terms of the relevant basis
• Expansion coefficients are used as the shape
  descriptor

6
Example I Saupe, Vranic 2001

• Shoot rays from the origin (center of mass),
  determine the distance to the farthest
  intersection point with the bounding mesh
• Parameterize the rays by the unit sphere to
  obtain a function on the sphere
• Use spherical harmonic transform on this function
  to extract the numerical shape descriptor.

7
Example I Saupe, Vranic 2001

• Surface feature
• Distance to the origin
• Mapping
• Mapping domain the unit sphere
• Mapping procedure project onto the sphere,
  resolve collisions by selecting the larger
  function value
• Signal processing
• Spherical harmonic transform

8
Example II Depth Buffer

• Heczko, Keim, Saupe, Vranic 2002
• Place a normalized mesh into a unit cube
• Generate six gray-scale images on each face of
  the cube by parallel projection
• The grayness value is the distance from the cube
  face to the model
• Apply 2D Fourier transform to each of the six
  gray-scale images

9
Example II Depth Buffer

• For each cube face
• Surface feature
• distance from mesh point to the face
• Mapping
• Mapping domain cube face
• Mapping procedure project onto the face, resolve
  collisions by selecting the smaller function
  value
• Signal processing
• 2D Fourier transform

10
Generality

• More examples easily generated
• Compare to classification in Bustos et al. survey
• Mapping object abstraction
• Signal processing numerical transformation

11
Observations

• Mapping
• Mapping domain
• ? original surface
• a primitive geometry sphere, plane etc
• Mapping procedure
• Projection
• Signal processing
• well established Fourier, Zernike, Spherical
  Harmonics
• limits possible mapping domains

12
Contributions

• Mapping
• Mapping domain
• any fixed surface template
• Mapping procedure
• interpolation mean-value coordinates, Shepard
• Signal processing
• via manifold harmonics eigenfunctions of
  Laplace-Beltrami operator

13
Why templates?

• Mademlis, Daras, Tzovaras, Strintzis 2008
• Since ellipses approximate elongated shapes
  better than spheres
• Mapping domain ellipsoid
• Signal processing ellipsoidal harmonics
• Showed experimentally better retrieval results
  than sphere spherical harmonics
• We take this idea further
• Mapping domain any fixed surface

14
Why template ?surface itself?

• Expand the feature function in terms of the
  manifold harmonics of the original surface?
• Problem notoriously difficult to match the
  harmonics coming from different surfaces
• Sign flipping
• Eigenfunction switching
• Linear combinations
• Fixed template extracted expansion coefficients
  are in direct correspondence

15
Why interpolation?

• Projection
• Mapped feature function can be discontinuous at
  overlaps
• Gibbs effect may render low-frequency expansion
  coefficients used as the shape descriptor
  inadequate for representing the function

Feature function is distance to the origin Jump
discontinuity
16
Why interpolation?

• Projection
• Redundancy
• the value sets of the mapped feature functions on
  various templates will be almost the same
• limits the gains of concatenating descriptors
  obtained from different templates

17
Why interpolation?

• Interpolation
• No Gibbs effect
• mapped feature function is smooth
• Less redundancy
• the value sets of the mapped feature functions on
  various templates depend on relative positions
• mean-value coordinates can inject more shape
  information into the mapped feature function
• a mesh can be reconstructed given the mean-value
  coordinates

18
Construction of the descriptor

• Selection of surface feature
• Mapping
• Signal Processing
• Now discuss details

19
Selection of surface feature

• All models are normalized using shift, continuous
  PCA, isotropic scaling
• Many possibilities, but not
• the characteristic function
• nor linear function of coordinates
• To focus discussion
• f distance from a mesh point to the origin
• Similar to Saupe, Vranic 2001

20
Mapping

• Model surface S, Template surface T
• Given
• Construct
• Shepard interpolation
• Mean-value interpolation

21
Mapping Shepard

• Model surface S, Template surface T
• Given
• Construct
• 0th order precision constant functions
  reproduced

22
Mapping Barycentric

• Model surface S, Template surface T
• Given
• Construct
• are barycentric coordinates of point p with
  respect to vertex
• 1st order precision linear functions reproduced

23
Mapping Barycentric

• A few different kinds of barycentric coordinates
• Mean-value, positive mean-value
• Harmonic
• Maximum Entropy
• Green coordinates, Complex in 2D
• We use mean-value coordinates
• Closed formula
• Fastest to evaluate

24
Signal processing

• We have a function
• Need a compact representation
• Expand the function into series
• Use low-frequency coefficients
• Need a function basis on template surface T
• Manifold harmonics Laplace-Beltrami
  eigenfunctions

25
Signal Processing

• Manifold harmonics generalize Fourier basis to
  Riemannian manifolds
• Spherical harmonics manifold harmonics on the
  sphere
• Have similar properties
• Orthogonal
• Concept of frequency
• low-frequency coefficients are noise-robust
• convey essential information about function

26
Signal Processing

• Egenvalues, eigenfunctions solve
• Evaluation procedures well known
• Solve symmetric eigenvalue problem for a matrix
• We use cotangent Laplacian with voronoi
  point-areas
• Pre-compute for the given templates and store
• Our templates have about 500 vertices, the
  process takes less than 3 seconds
• The storage for each template 10,500 floats
  20500500 eigs vertices vertices

to store eigenvectors
27
Resulting shape descriptor

• Feature function
• Mapped feature function
• The templates feature function
• Quotient function
• Expand into series

28
Resulting shape descriptor

• Feature vector, N20
• For template surface T,
• Normalization scale to get a 1
• Use L2 distance

29
Experiments Benchmark

• Models watertight benchmark
• 400 closed surface models
• 20 equal object classes

30
Experiments Implementation

• Implemented in MATLAB
• Use C for mean-value coordinate computation
• Timing
• About 1 minute per model when mean-value
  coordinates used
• Could make faster if simplified the models

31
Experiments Templates

• Templates randomly chosen models
• Simplified using Qslim
• Makes mean-value computation faster

32
Compare Mapping Methods

• Template sphere
• Projection vs. Shepard (a1,2,3) vs. Mean-value
• At long distance behavior of mean-value
  interpolant is similar to that of Shepard with
  a2

33
Compare templates

• Mapping via mean-value interpolation
• M

34
Compare templates

• Beneficial to combine relative independence
• All templates are normalized as objects in the
  benchmark span similar spatial regions
• The descriptors could have been made even more
  independent if the templates were differently
  posed.

35
Future work

• Investigate dependency between the nature of the
  template and the produced retrieval results
• No ideal" template for all kinds of shapes
• Flexibility of our approach choose optimal
  templates based on the shape database at hand
• How to choose?
• Can we design a rotationally invariant
  descriptor?