Regression based Bandwidth Selection for Segmentation using Parzen Windows

1
Regression based Bandwidth Selection for
Segmentation using Parzen Windows
Maneesh Singh and Narendra Ahuja ECE and Beckman
Institute, Univ. of Illinois at Urbana-Champaign
2
Topics

• Introduction
• Model advantages
• Bandwidth Selection
• Scale-guided segmentation
• Results
• Conclusions

3
Topics

• Introduction
• Model advantages
• Bandwidth Selection
• Scale-guided segmentation
• Results
• Conclusions

4
Introduction
Original
Segmented
5
Introduction

• We consider segmentation of non-stationary image
  signals.
• Non-stationarity modeled by
  which varies with location
• Relationship of and
  segmentation
• multimodal.
• Each mode corresponds to a segment.

6
Introduction

• is estimated using Parzen windows.
• Bandwidth selection critical for above estimate.
• This paper is about bandwidth selection.
• Segmentation done using Mean-Shift.

7
Introduction

• Notations
• - conditional image
  PDF.
• - estimated cond.
  image PDF (n samples, H bandwidth).
• - MSE.
• - Integrated MSE
  (IMSE)
• - Integrated
  Asymptotic MSE (IAMSE)

8
Topics

• Introduction
• Model advantages
• Bandwidth Selection
• Scale-guided segmentation
• Results
• Conclusions

9
Model

• Image model
• Non-stationary error PDF modeled using Parzen
  Windows
• modeled using a plug-in regression
  estimator.

10
Model

• Segmentation assumes implicit/ explicit image
  model.
• Our Model (flexible)
• Image signal noise
• Signal (regression) function - piecewise cont.
• Noise
• Non-stationary with unknown statistics.
• PDF smooth.
• Estimated from image data.

11
Approach

• Segmentation via
• Estimation of PDF modes
• Pixel association with modes.
• We use mean-shift for above.
• Issue kernel bandwidth selection

12
Topics

• Introduction
• Model advantages
• Bandwidth Selection
• Scale-guided segmentation
• Results
• Conclusions

13
Bandwidth Selection

• Several Bandwidth selection schemes in
  literature.
• We note
• Plug-in bandwidth estimators are simplest and
  most popular.
• Assume an underlying data distribution (Gaussian,
  locally Gaussian).
• But images highly non-Gaussian.

14
Bandwidth Selection
Typical 2-dim image
15
Bandwidth Selection

• Histograms
• Raw values
• Wavelet residues
• Median residues

16
Bandwidth Selection

• Observations
• Plug-in bandwidth estimation feasible for image
  residuals (use GGD) though not for image data.
• Computing residuals requires a regression
  estimate
• Above observation true for a wide variety of
  regression functions.
• Good regression-models already available
• Scale-based regression ? scale-based segmentation

17
Bandwidth Selection

• Bandwidths selected to minimize the Integrated
  Mean Square Error (IMSE) between the true and
  estimated image PDF.
• We derive expressions for AIMSE based on the
  error PDF and image regression estimates.

18
Bandwidth Selection

• PDF estimator

• Consistency conditions should be met

19
Bandwidth Selection

• Conditional PDF Estimate

20
Bandwidth Selection

• Bandwidth estimated to minimize the MSE

• Asymptotic analysis required (IAMSE)
• Note we drop the conditional notation for
  brevity.

21
Bandwidth Selection

• Using

22
Bandwidth Selection

• H assumed diagonal

where

• requires knowledge of and

23
Bandwidth Selection

• Difficult optimization problem.
• We derive an upper bound
• Optimal bandwidths for this upper bound.
• Bound asymptotically tight.
• Detailed derivations in the paper.

24
Topics

• Introduction
• Model advantages
• Bandwidth Selection
• Scale-guided segmentation
• Results
• Conclusions

25
Scale Guided Segmentation

• Algorithm
• Choose scale
• Get plug-in regression estimate
• Find residuals (errors)
• Use GGD as a plug-in for error PDF
• Estimate global bandwidth parameters
• Optionally, find local BW parameters using
  Abramsons Law
• Use Mean-shift to find modes

26
Topics

• Introduction
• Model advantages
• Bandwidth Selection
• Scale-guided segmentation
• Results
• Conclusions

27
Segmentation Results

• Results using N(0,42I) smoothing function

28
Segmentation Results

• Results using N(0,42I) smoothing function

29
Segmentation Results

• Results using N(0,42I) smoothing function

30
Segmentation Results

• Results using N(0,42I) smoothing function

31
Topics

• Introduction
• Model advantages
• Bandwidth Selection
• Scale-guided segmentation
• Results
• Conclusions

32
Conclusions

• Proposed a regression-based conditional PDF model
  for images (Similar to Bashtannyk Hyndman,
  Comp. Stats. Data Anal., 36(3), 279-298).
• Regression using wavelets, median similar
  results.
• Noise PDF estimation using Parzen windows, GGD.
• Scale estimates easier for noise PDF.
• Better image model (than Comaniciu Meer, ITIP,
  02).
• Derived a Bandwidth estimation scheme.
• Leads to (spatial) scale-based segmentation
  framework.

33
The Object Selection Problem

• Interactively extract objects embedded in images
  and video for compositing and manipulation
• Application areas special effects, graphic
  design, TV broadcast
• Key to enabling intelligent editing operators

34
Related Work

• Vector graphic editors
• GRANDMA CMU
• PerSketch Xerox PARC
• Edge-based selection
• Intelligent Scissors BYU, Adobe
• Active Contours Various
• Alpha channel-based
• Commercial Tools - Photoshop, Ultimatte, KnockOut
• Alpha Estimation - Ruzon-Tomasi, Chuang et al.

35
Segmentation-based Selection with Freehand
Sketches

• Photoshop

• Segmentation-based

36
Rose Example Selection Results
Photoshop
Segmentation-based
It can be seen that the segmentation-based tool
produced higher-quality object boundaries
37
Limitation of Segmentation and Edge-based
Representation

• Diffused Edges are present in most images due to
• Optical defocus (intentional or otherwise)
• Motion blur
• Finite Sensor Resolution
• Antialiasing (synthetic images)
• Compression
• Texture boundary
• Need an alpha channel to represent diffused edges
  and recover foreground and background

38
Boundary Region Decomposition ICCV01

• Use simple segmentation algorithm that decomposes
  image into approximately convex segment
• Use the Delaunay triangulation of the segment
  centroids as decomposition

Lines can now be matched to centroids using the
triangles.
39
Local Boundary Analysis

• Within each triangle
• Find linear approximation of boundary using
  initial selection
• Finer-scale segmentation
• Classify new segment centroids using discriminant
• Extract pixel samples in a window around centroid
• Compute alpha channel

40
Test Images
41
Selection with Points
Click around and select
42
Selection with Loops
43
Selection with Loops
44
A Better Triangulation CVPR01

• Want triangles to satisfy the following
  properties
• Vertices lie on medial axis of segments
• Each triangle is completely contained in at most
  three regions
• For an edge between two vertices, one in region A
  and one in region B, the edge must lie entirely
  in the union of A and B.

1. Cover junctions
2. Cover openings
45
Selection with New Triangulation
Details
46
Contact

• msingh_at_uiuc.edu , n-ahuja_at_uiuc.edu