Learning to Detect Natural Image Boundaries Using Local Brightness, Color, and Texture Cues

1
Learning to Detect Natural Image Boundaries Using
Local Brightness, Color, and Texture Cues

• David Martin, Charless Fowlkes, Jitendra Malik
• dmartin,fowlkes,malik_at_eecs.berkeley.edu
• UC Berkeley Vision Group
• http//www.cs.berkeley.edu/projects/vision

2
(No Transcript)
3
Multiple Cues for Grouping

• Many cues for perceptual grouping
• Low-Level brightness, color, texture, depth,
  motion
• Mid-Level continuity, closure, convexity,
  symmetry,
• High-Level familiar objects and configurations
• This talk Learn local cue combination rule from
  data

4
(No Transcript)
5
Goal and Outline

• Goal Model the posterior probability of a
  boundary Pb(x,y,?) at each pixel and orientation
  using local cues.
• Method Supervised learning using dataset of
  12,000 segmentations of 1,000 images by 30
  subjects.
• Outline of Talk
• 3 cues brightness, color, texture
• Cue calibration
• Cue combination
• Compare with other approaches
• Canny 1986, Konishi/Yuille/Coughlan/Zhu 1999
• Pb images

6
Brightness and Color Features

• 1976 CIE Lab colorspace
• Brightness Gradient BG(x,y,r,?)
• ?2 difference in L distribution
• Color Gradient CG(x,y,r,?)
• ?2 difference in a and b distributions

r
(x,y)
?
7
Texture Feature
TextonMap

• Texture Gradient TG(x,y,r,?)
• ?2 difference of texton histograms
• Textons are vector-quantized filter outputs

8
Cue Calibration

• All free parameters optimized on training data
• Brightness Gradient
• Scale, bin/kernel sizes for KDE
• Color Gradient
• Scale, bin/kernel sizes for KDE, joint vs.
  marginals
• Texture Gradient
• Filter bank scale, multiscale?
• Histogram comparison L1, L2, L?, ?2, EMD
• Number of textons
• Image-specific vs. universal textons
• Localization parameters for each cue (see paper)

9
Classifiers for Cue Combination

• Classification Trees
• Top-down splits to maximize entropy, error
  bounded
• Density Estimation
• Adaptive bins using k-means
• Logistic Regression, 3 variants
• Linear and quadratic terms
• Confidence-rated generalization of AdaBoost
  (SchapireSinger)
• Hierarchical Mixtures of Experts (JordanJacobs)
• Up to 8 experts, initialized top-down, fit with
  EM
• Support Vector Machines (libsvm, ChangLin)
• Range over bias/variance, parametric/non-parametri
  c, simple/complex

10
Classifier Comparison
11
Cue Combinations
12
Alternate Approaches

• Canny Detector
• Canny 1986
• MATLAB implementation
• With and without hysteresis
• Second Moment Matrix
• Nitzberg/Mumford/Shiota 1993
• cf. Förstner and Harris corner detectors
• Used by Konishi et al. 1999 in learning framework
• Logistic model trained on eigenspectrum

13
Two Decades of Boundary Detection
14
Pb Images I
Canny
2MM
Us
Human
Image
15
Pb Images II
Canny
2MM
Us
Human
Image
16
Pb Images III
Canny
2MM
Us
Human
Image
17
Summary and Conclusion

• A simple linear model is sufficient for cue
  combination
• All cues weighted approximately equally in
  logistic
• Proper texture edge model is not optional for
  complex natural images
• Texture suppression is not sufficient!
• Significant improvement over state-of-the-art in
  boundary detection
• Pb useful for higher-level processing
• Empirical approach critical for both cue
  calibration and cue combination
• Segmentation data and Pb images on the
  webhttp//www.cs.berkeley.edu/projects/vision