Nonlinear Dimensionality Reduction Approach ISOMAP

1
Nonlinear Dimensionality Reduction Approach
(ISOMAP)

• 2006. 2. 28
• Young Ki Baik
• Computer Vision Lab.
• Seoul National University

2
References

• A global geometric framework for nonlinear
  dimensionality reduction
• J. B. Tenenbaum, V. De Silva, J. C. Langford
  (Science 2000)
• LLE and Isomap Analysis of Spectra and Colour
  Images
• Dejan Kulpinski (Thesis 1999)
• Out-of-Sample Extensions for LLE, Isomap, MDS,
  Eigenmaps, and Spectral Clustering
• Yoshua Bengio et.al. (TR 2003)

3
Contents

• Introduction
• PCA and MDS
• ISOMAP
• Conclusion

4
Dimensionality Reduction

• The goal
• The meaningful low-dimensional structures hidden
  in their high-dimensional observations.
• Classical techniques
• PCA (Principle Component Analysis)
• preserves the variance
• MDS (MultiDimensional Scaling)
• - preserves inter-point distance
• ISOMAP
• LLE (Locally Linear Embedding)

5
Linear Dimensionality Reduction

• PCA
• Finds a low-dimensional embedding of the data
  points that best preserves their variance as
  measured in the high-dimensional input space.
• MDS
• Finds an embedding that preserves the inter-point
  distances, equivalent to PCA when the distances
  are Euclidean.

6
Linear Dimensionality Reduction

• MDS
• distances
• Relation

7
Nonlinear Dimensionality Reduction

• Many data sets contain essential nonlinear
  structures that invisible to PCA and MDS
• Resort to some nonlinear dimensionality reduction
  approaches.

8
ISOMAP

• Example of Non-linear structure (Swiss roll)
• Only the geodesic distances reflect the true
  low-dimensional geometry of the manifold.
• ISOMAP (Isometric feature Mapping)
• Preserves the intrinsic geometry of the data.
• Uses the geodesic manifold distances between all
  pairs.

9
ISOMAP (Algorithm Description)

• Step 1
• Determining neighboring points within a fixed
  radius based on the input space distance
  .
• These neighborhood relations are represented as a
  weighted graph G over the data points.
• Step 2
• Estimating the geodesic distances
  between all pairs of points on the manifold by
  computing their shortest path distances in the
  graph G.
• Step 3
• Constructing an embedding of the data in
  d-dimensional Euclidean space Y that best
  preserves the manifolds geometry.

10
ISOMAP (Algorithm Description)

• Step 1
• Determining neighboring points within a fixed
  radius based on the input space distance
  .
• e-radius
  K-nearest neighbors
• These neighborhood relations are represented as a
  weighted graph G over the data points.

K4
e
i
j
k
11
ISOMAP (Algorithm Description)

• Step 2
• Estimating the geodesic distances
  between all pairs of points on the manifold by
  computing their shortest path distances in the
  graph G.
• Can be done using Floyds algorithm or Dijkstras
  algorithm

j
i
k
12
ISOMAP (Algorithm Description)

• Step 3
• Constructing an embedding of the data in
  d-dimensional Euclidean space Y that best
  preserves the manifolds geometry.
• Minimize the cost function

13
Experimental results

• FACE Hand
  writing
• face pose and illumination bottom
  loop and top arch

MDS open triangles Isomap filled circles
14
Discussion

• Isomap handles non-linear manifold.
• Isomap keeps the advantages of PCA and MDS.
• Non-iterative procedure
• Polynomial procedure
• Guaranteed convergence
• Isomap represents the global structure of a data
  set within a single coordinate system.

15
(No Transcript)