The Earth Mover

1
The Earth Movers Distance as a Metric for Image
Retrieval and as a Metric for Distributions with
Applications to Image DatabasesRanjini
Swaminathan University of Arizona
2
The Problem

• To employ EMD as a method to find dissimilarities
  between images.
• Image Parameters
• Images are described by features such as
• color (3-D representation)
• Texture (a distribution of energy in the
  frequency domain)

3
Need for methods
Image Retrieval-based on dissimilarity between
two images. Steps - Compute histograms/signatures
- Implement EMD /other method - Navigation
4
Histograms
A histogram hi is a mapping from a set of
d-dimensional integer vectors i to the set of
nonnegative reals. An example - Consider a
grey-level histogram where d1 - split possible
grey values into N intervals - image pixels fall
into respective bins
5
Histograms contd.
Disadvantage -fail to strike a balance between
expressiveness and efficiency. Adaptive Binning
of Histograms -Need prior knowledge of
distribution of features for all
images. -significant information restricted to
few bins.
6
Signatures
Signature sj(mj,wj) - a set of clusters mj
- d-dimensional mode Wj - number of pixels
belonging to the cluster A histogram is a
special case of a signature.
7
Signatures contd.
Clusters
8
Methods

• Bin-By-Bin Dissimilarity measures
• - Minkowski-Form Distance
• dLr(H,K) (??hi-ki?r)1/r
• - Histogram Intersection
• d(H,K) 1 - ?i min(hi,ki)/?i ki
•  

9
Methods contd.

• - K-L Divergence and Jeffrey Divergence
• dKL(H,K) ?i hi log hi/ki
• dJ(H,K) ?i (hi log hi/mi ki log ki/mi)
• - X2 Statistics
• d?2(H,K) ?i(hi-mi)2/mi

10
Methods contd.

• Cross-Bin Dissimilarity measures
• Quadratic-form distance
• Match distance
• - Kolmogorov-Smirnov distance

11
Signatures vs Histograms
Histograms - fixed size structures - efficiency
vs expressiveness Signatures - representative of
features - less info yet better retrieval
12
EMD
-measures the amount of work done in
moving mass over holes in the same
space. -not a matching problem but a
transportation problem -increased efficiency
13
Flow equations
WORK(P,Q,F) ?i?jdijfij with the constraints
fij ? 0 ?j fij ? wpi ?i fij ? wqj
?i?jfij min(?i wpi , ?j wqj ) where 1 ?
i ? m , 1 ? j ? n
14
EMD-Features
EMD(P,Q) ?i?jdijfij/?i?jfij Features -applies
to signatures which subsume histograms - avoids
quantization problems -allows partial matches
variable length descriptions -metric
15
MDS
Multi Dimensional Scaling as a Perceptual
Evaluation Tool ?ij is the original distance
between objects i and j dij is the distance
between the points in a low dimensional
space STRESS (?i,j(dij - ?ij )2/ ?i,j
?ij 2)1/2 Zero Stress perfect fit
16
Color
Courtesy www.cs.ucsd.edu/
17
Color Distributions
Given Set of images Do cluster obtain
signatures compute EMD with a stress
value Result images aligned along lightness and
chroma MDS displays more subtle similarities
between closely related images
18
Color distributions contd.
Test results show that EMD with signatures
outperforms all other methods. Navigation -
Embeddings are adaptive - Zooming into query -
Use of dont cares
19
Efficiency
EMD efficiency depends on Transportation
algorithm Easy to compute lower bounds However,
EMD is Computationally intensive Works better
for smaller sample sizes
20
Notes
Metric (Merriam-Webster) a mathematical
function that associates with each pair of
elements of a set a real nonnegative number with
the general properties of distance such that the
number is zero only if the two elements are
identical, the number is the same regardless of
the order in which the two elements are taken,
and the number associated with one pair of
elements plus that associated with one member of
the pair and a third element is equal to or
greater than the number associated with the other
member of the pair and the third element. Proof
Triangle Inequality
21
Future Work
Other vision problems classification,
recognition and segmentation Outside computer
vision????
-in speech recognition -in biology