Segmentation and Boundary Detection Using Multiscale Measurements

1
Segmentation and Boundary Detection Using
Multiscale Measurements
Ronen BasriAchi BrandtMeirav GalunEitan Sharon
2
Image Segmentation
3
Local Uncertainty
4
Global Certainty
5
Local Uncertainty
6
Global Certainty
7
Coarse Measurements for Texture
8
A Chicken and Egg Problem
Problem Coarse measurements mix neighboring
statistics
Solution support of measurements is determined
as the segmentation process proceeds
9
Segmentation by Weighted Aggregation

• Normalized-cuts measure in graphs
• Complete hierarchy in linear time
• Use multiscale measures of
• intensity, texture, shape, and boundary
  integrity

10
Segmentation by Weighted Aggregation

• Normalized-cuts measure in graphs
• Complete hierarchy in linear time
• Use multiscale measures of
• intensity, texture, shape, and boundary
  integrity

11
Segmentation by Weighted Aggregation

• Normalized-cuts measure in graphs
• Complete hierarchy in linear time
• Use multiscale measures of
• intensity, texture, shape and boundary
  integrity

12
The Pixel Graph
Couplings Reflect intensity similarity

Low contrast strong coupling
High contrast weak coupling
13
Hierarchical Graph
14
Hierarchyin SWA
15
Normalized-Cut Measure
16
Normalized-Cut Measure
Minimize
17
Normalized-Cut Measure
Low-energy cut
Minimize
18
Segment Detection
19
Coarse-Scale Measurements

• Average intensities of aggregates
• Multiscale intensity-variances of aggregates
• Multiscale shape-moments of aggregates
• Boundary alignment between aggregates

20
Adaptive vs. Rigid Measurements
Original
Averaging
Geometric
Our algorithm - SWA
21
Adaptive vs. Rigid Measurements
Original
Interpolation
Geometric
Our algorithm - SWA
22
Use Averages to Modify the Graph
23
Use Averages to Modify the Graph
24
Texture Examples
25
Isotropic and Oriented Filters
A brief tutorial
Textons by K-Means Malik et al IJCV2001
26
Oriented Texture in SWA
with Meirav Galun
Shape Moments

• center of mass
• width
• length
• orientation

Oriented Texture of aggregate orientation,
width and length in all scales
27
Boundary Integrity in SWA
28
Hierarchyin SWA
29
SWA
Detects the salient segments
Hierarchical structure
Linear in of points (a few dozen operations per
point)
30
Experiments

• Our SWA algorithm (CVPR00 CVPR01)
• run-time 5-10 seconds.
• Normalized cuts (Shi and Malik, PAMI00 Malik
  et al., IJCV01)
• run-time about 10-15 minutes.
• Software courtesy of Doron Tal, UC Berkeley.

31
Isotropic Texture - Horse I
Our Algorithm (SWA)
Normalized Cuts
32
Isotropic Texture - Horse II
Our Algorithm (SWA)
Normalized Cuts
33
Isotropic Texture - Tiger
Our Algorithm (SWA)
Normalized Cuts
34
Isotropic Texture - Butterfly
Our Algorithm (SWA)
Normalized Cuts
35
Isotropic Texture - Leopard
Our Algorithm (SWA)
36
Isotropic Texture - Dalmatian Dog
Our Algorithm (SWA)
37
Isotropic Texture - Squirrel
Our Algorithm (SWA)
Normalized Cuts
38
Full Texture - Squirrel
Our Algorithm (SWA)
Normalized Cuts
with Meirav Galun
39
Full Texture - Composition
Our Algorithm (SWA)
with Meirav Galun
40
Full Texture Lion Cub
Our Algorithm (SWA)
with Meirav Galun
41
Full Texture Polar Bear
Our Algorithm (SWA)
with Meirav Galun
42
Full Texture Penguin
Our Algorithm (SWA)
with Meirav Galun
43
Full Texture Leopard
Our Algorithm (SWA)
with Meirav Galun
44
Full Texture Leopard
Our Algorithm (SWA)
with Meirav Galun
45
Full Texture Owl
Our Algorithm (SWA)
with Meirav Galun
46
Full Texture Bird
Our Algorithm (SWA)
with Meirav Galun
47
Separation of Parts
Poissonian u ?u 1 u 0 outside the segment