Computer and Robot Vision I

1
Computer and Robot Vision I

• Chapter 7
• Conditioning and Labeling

Presented by ??? ??? d01922027_at_csie.ntu.edu.tw
???? ??? ??
2
7.1 Introduction

• Conditioning
• noise removal, background normalization,
• Labeling
• thresholding, edge detection, corner finding,

3
7.2 Noise Cleaning

• noise cleaning
• uses neighborhood spatial coherence
• uses neighborhood pixel value homogeneity

4
7.2 Noise Cleaning
box filter computes equally weighted average box
filter separable box filter recursive
implementation with two, two-, one/
per pixel
5
7.2 Noise Cleaning
6
7.2 Noise Cleaning

• Gaussian filter linear smoother

weight matrix
for all where
size of W two or three from center
linear noise-cleaning filters defocusing images,
edges blurred
7
A Statistical Framework for Noise Removal

• Idealization if no noise, each neighborhood
• the same constant

8
Outlier or Peak Noise

• outlier peak noise pixel value replaced by
• random noise value
• neighborhood size larger than noise, smaller
• than preserved
  detail
• center-deleted neighborhood pixel values in
• neighborhood except
  center

9
Outlier or Peak Noise
center-deleted neighborhood
center pixel value
mean of center-deleted neighborhood
10
Outlier or Peak Noise
output value of neighborhood outlier removal
not an outlier value if reasonably close to
use mean value when outlier
threshold for outlier value
too small edges blurred
too large noise cleaning will not be good
11
Outlier or Peak Noise
center-deleted neighborhood variance
use neighborhood mean if pixel value
significantly far from mean
12
Outlier or Peak Noise

• smooth replacement
• instead of complete replacement or not at
  all

convex combination of input and mean
use neighborhood mean
weighting parameter
use input pixel value
13
K-Nearest Neighbor

• K-nearest neighbor
• average equally weighted average of k-nearest
• neighbors

14
Gradient Inverse Weighted

• gradient inverse weighted
• reduces sum-of-squares error within regions

15
Take a Break
16
Order Statistic Neighborhood Operators

• order statistic
• linear combination of neighborhood
  sorted values
• neighborhood pixel values
• sorted neighborhood values from
  smallest to
• largest

17
Order Statistic Neighborhood Operators

• Median Operator

median most common order statistic
operator median root fixed-point result of a
median filter median roots comprise only
constant-valued neighborhoods,
sloped edges
18
Median Root Image
Original Image
19
Order Statistic Neighborhood Operators

• median effective for impulsive noise (salt and
  pepper)
• median distorts or loses fine detail such as
• thin lines

20
Order Statistic Neighborhood Operators
inter-quartile distance
21
Order Statistic Neighborhood Operators

• Trimmed-Mean Operator

trimmed-mean first k and last k order statistics
not used trimmed-mean equal weighted average of
central N-2k order
statistics
22
Order Statistic Neighborhood Operators

• Midrange

midrange noise distribution with light and
smooth tails
23
Hysteresis Smoothing

• hysteresis smoothing
• removes minor fluctuations, preserves major
  transients
• hysteresis smoothing
• finite state machine with two states UP, DOWN
• applied row-by-row and then column-by-column

24
Hysteresis Smoothing

• if state DOWN and next one larger, if next local
  maximum does not exceed threshold then stays
  current value i.e. small peak cuts flat
• otherwise state changes from DOWN to UP and
  preserves major transients

25
Hysteresis Smoothing

• if state UP and next one smaller, if next local
  minimum does not exceed threshold then stays
  current value i.e. small valley filled at
• otherwise state changes from UP to DOWN and
  preserves major transients

26
Hysteresis Smoothing
27
Sigma Filter

• sigma filter average only with values within
• two-sigma interval

28
Selected-Neighborhood Averaging

• selected-neighborhood averaging
• assumes pixel a part of homogeneous
  region
• (not required to be squared, others can
  be diagonal, rectangle, three pixels vertical and
  horizontal neighborhood)
• noise-filtered value
• mean value from lowest variance
  neighborhood

29
Minimum Mean Square Noise Smoothing

• minimum mean square noise smoothing
• additive or
  multiplicative noise
• each pixel in true image
• regarded as a
  random variable

30
Minimum Mean Square Noise Smoothing
31
Noise-Removal Techniques-Experiments
32
Noise-Removal Techniques-Experiments
types of noise

• uniform
• Gaussian
• salt and pepper
• varying noise
• (the noise energy varies across the
  image)

33
Noise-Removal Techniques-Experiments

• salt and pepper

minimum/ maximum gray value for noise pixels
fraction of image to be corrupted with noise
uniform random variable in 0,1
gray value at given pixel in input
image
gray value at given pixel in
output image
34
Noise-Removal Techniques-Experiments

• Generate salt-and-pepper noise

35
Noise-Removal Techniques-Experiments

• S/N ratio (signal to noise ratio)
• VS image gray level variance
• VN noise variance

36
Take a Break
37
Noise-Removal Techniques-Experiments
uniform noise
Gaussian noise
varies
salt and pepper noise
38
Uniform
Original
noise
Gaussian
S P
39

• default peak noise removal

40
(No Transcript)
41

• contrast-dependent noise removal

42
Uniform
contrast-dependent outlier removal
Gaussian
S P
43

• smooth replacement

44
Uniform
contrast-dependent outlier removal with smooth
replacement
Gaussian
S P
45

• hysteresis smoothing

46
Uniform
hysteresis smoothing
Gaussian
S P
47

• inter-quartile mean filter

48
Uniform
inter-quartile mean filter
Gaussian
S P
49

• neighborhood midrange filter

50
Uniform
neighborhood midrange filter
Gaussian
S P
51

• neighborhood running-mean filter

52
Uniform
neighborhood running-mean filter
Gaussian
S P
53

• sigma filter

54
Uniform
sigma filter
Gaussian
S P
55

• neighborhood weighted median filter
• (7X7)

56
Uniform
neighborhood weighted median filter
Gaussian
S P
57
Noise-Removal Techniques-Experiments
gain in S/N ratio (in decibels)
1.5057
Contrast-dependent peak noise removal
0.9380
Contrast-dependent peak noise removal
1.5445
Contrast-dependent peak noise removal
1.5057
Default peak noise removal
0.9380
Default peak noise removal
1.5445
Default peak noise removal
1.5692
Midrange filter
58
Noise-Removal Techniques-Experiments with Lena
Uniform Gaussian_30 S P 0.1
Contrast-dependent outlier removal 26.408 25.758 20.477
Smooth replacement 29.344 26.821 26.792
Outlier removal 26.407 25.768 20.406
Hysteresis 18.278 13.804 1.9040
Interquartile mean filter 29.193 24.517 35.720
Midrange filter 22.171 19.629 -0.1658
Running-mean filter 29.522 28.278 21.194
Sigma filter 34.717 18.831 -2.9151
Weighted-median filter 33.964 30.720 36.033
59
7.3 Sharpening

• unsharp masking subtract fraction of
• neighborhood mean and scale
  result

possible to replace neighborhood mean with
neighborhood median
60
Extremum Sharpening

• extremum-sharpening output closer of
• neighborhood maximum or
  minimum

61
7.4 Edge Detection

• digital edge boundary where brightness
• values
  significantly different
• edge brightness value appears to jump

62
(No Transcript)
63
(No Transcript)
64
(No Transcript)
65
Gradient Edge Detectors

• Roberts operators two 2X2 masks to calculate
  gradient

• gradient magnitude
• where are values from first, second
  masks respectively

66
Roberts operators
67
Gradient Edge Detectors

• Prewitt edge detector two 3X3 masks in row
  column direction

P1
P2

• gradient magnitude
• gradient direction
  clockwise w.r.t. column axis
• where are values from first, second
  masks respectively

68
Gradient Edge Detectors
69
Prewitt edge detector
70
Gradient Edge Detectors

• Sobel edge detector two 3X3 masks in row
  column direction

• gradient magnitude
• gradient direction
  clockwise w.r.t. column axis
• where are values from first, second
  masks respectively

71
Gradient Edge Detectors

• Sobel edge detector 2X2 smoothing followed by
  2X2 gradient

72
Gradient Edge Detectors
73
Sobel edge detector
74
Gradient Edge Detectors

• Frei and Chen edge detector two in a set of
  nine orthogonal
• 
  masks (3X3)

• gradient magnitude
• gradient direction
  clockwise w.r.t. column axis
• where are values from first, second
  masks respectively

75
Gradient Edge Detectors

• Frei and Chen edge detector nine orthogonal
  masks (3X3)

76
Frei and Chen edge detector
77
Gradient Edge Detectors

• Kirsch set of eight compass template edge masks

• gradient magnitude
• gradient direction

78
Kirsch
79
Gradient Edge Detectors

• Robinson compass template mask set with only

• done by only four masks since negation of each
  mask is also a mask
• gradient magnitude and direction same as Kirsch
  operator

80
Robinson
81
Gradient Edge Detectors

• Nevatia and Babu set of six 5X5 compass
  template masks

82
Nevatia and Babu
83
Gradient Edge Detectors

• edge contour direction
• along edge, right side bright, left
  side dark
• edge contour direction
• more than gradient direction

84

• Robinson and Kirsch compass operator detect
  lineal edges

85
(No Transcript)
86
Gradient Edge Detectors
four important properties an edge operator might
have

• accuracy in estimating gradient magnitude
• accuracy in estimating gradient direction
• accuracy in estimating step edge contrast
• accuracy in estimating step edge direction

• gradient direction used for edge organization,
  selection, linking

87
7.4.1 Gradient Edge Detectors
88
Take a Break
89
Zero-Crossing Edge Detectors

• first derivative maximum exactly where second
  derivative zero crossing

90
Zero-Crossing Edge Detectors
Laplacian of a function
91
Zero-Crossing Edge Detectors

• two common 3X3 masks to calculate digital
  Laplacian

92
Zero-Crossing Edge Detectors

• the 3X3 neighborhood values of an image function

93
Zero-Crossing Edge Detectors
( r, c)
(-1,-1) (-1, 0) (-1, 1)
( 0,-1) ( 0, 0) ( 0, 1)
( 1,-1) ( 1, 0) ( 1, 1)
(-1,-1) (-1, 0) (-1, 1)
( 0,-1) ( 0, 0) ( 0, 1)
( 1,-1) ( 1, 0) ( 1, 1)
(-1,-1) (-1, 0) (-1, 1)
( 0,-1) ( 0, 0) ( 0, 1)
( 1,-1) ( 1, 0) ( 1, 1)
(-1,-1) (-1, 0) (-1, 1)
( 0,-1) ( 0, 0) ( 0, 1)
( 1,-1) ( 1, 0) ( 1, 1)
(-1,-1) (-1, 0) (-1, 1)
( 0,-1) ( 0, 0) ( 0, 1)
( 1,-1) ( 1, 0) ( 1, 1)
(-1,-1) (-1, 0) (-1, 1)
( 0,-1) ( 0, 0) ( 0, 1)
( 1,-1) ( 1, 0) ( 1, 1)
(-1,-1) (-1, 0) (-1, 1)
( 0,-1) ( 0, 0) ( 0, 1)
( 1,-1) ( 1, 0) ( 1, 1)
(-1,-1) (-1, 0) (-1, 1)
( 0,-1) ( 0, 0) ( 0, 1)
( 1,-1) ( 1, 0) ( 1, 1)
(-1,-1) (-1, 0) (-1, 1)
( 0,-1) ( 0, 0) ( 0, 1)
( 1,-1) ( 1, 0) ( 1, 1)
(-1,-1) (-1, 0) (-1, 1)
( 0,-1) ( 0, 0) ( 0, 1)
( 1,-1) ( 1, 0) ( 1, 1)



94
Zero-Crossing Edge Detectors
95
Zero-Crossing Edge Detectors
96
Zero-Crossing Edge Detectors

• 3X3 mask computing
• a digital Laplacian

97
Zero-Crossing Edge Detectors
98
Zero-Crossing Edge Detectors

• 3X3 mask for computing minimum-variance digital
  Laplacian

99

• Laplacian of the Gaussian kernel

-2 -9 -23 -1 103 178 103 -1 -23 -9 -2
100
(No Transcript)
101
Zero-Crossing Edge Detectors
102
(No Transcript)
103
(No Transcript)
104
(No Transcript)
105
Zero-Crossing Edge Detectors
106
Zero-Crossing Edge Detectors

• A pixel is declared to have a zero crossing if it
  is less than t and one of its eight neighbors
  is greater than t or if it is greater than t
  and one of its eight neighbors is less than t
  for some fixed threshold t

107
Zero-Crossing Edge Detectors
108
Take a Break
109
Edge Operator Performance

• edge detector performance characteristics
• misdetection/false-alarm rate

110
(No Transcript)
111
(No Transcript)
112
(No Transcript)
113
(No Transcript)
114
(No Transcript)
115
(No Transcript)
116
7.5 Line Detection

• A line segment on an image can be characterized
  as an elongated rectangular region having a
  homogeneous gray level bounded on both its longer
  sides by homogeneous regions of a different gray
  level

117
7.5 Line Detection

• one-pixel-wide lines can be detected by compass
  line detectors

118

• semilinear line detector
• by step edge on
• either side of the line

119

• detecting lines of one to three pixels in width

120
Project due Nov. 27

• Write the following programs

1. Generate additive white Gaussian noise
2. Generate salt-and-pepper noise
3. Run box filter (3X3, 5X5) on all noisy images
4. Run median filter (3X3, 5X5) on all noisy images
5. Run opening followed by closing or closing
   followed by opening

121
Project due Nov. 27

• box filter on white Gaussian noise with amplitude
  10

Gaussian noise
122
Project due Nov. 27

• box filter on salt-and-pepper noise with
  threshold 0.05

salt-and-pepper noise
123
Project due Nov. 27

• median filter on white Gaussian noise with
  amplitude 10

Gaussian noise
124
Project due Nov. 27

• median filter on salt-and-pepper noise with
  threshold 0.05

salt-and-pepper noise
125
Project due Nov. 27

• Generate additive white Gaussian noise

Gaussian random variable with zero mean and st.
dev. 1
amplitude determines signal-to-noise ratio, try
10, 30
126
Project due Nov. 27

• Generate additive white Gaussian noise
• 
  with amplitude 10

127
Project due Nov. 27

• Generate additive white Gaussian noise
• 
  with amplitude 30

128
Project due Nov. 27

• Generate salt-and-pepper noise

129
Project due Nov. 27

• Generate salt-and-pepper noise
• 
  with threshold 0.05

130
Project due Nov. 27

• Generate salt-and-pepper noise
• 
  with threshold 0.1

131
Project due Dec. 26
Write programs to generate the following gradient
magnitude images and choose proper thresholds to
get the binary edge images

• Roberts operator
• Prewitt edge detector
• Sobel edge detector
• Frei and Chen gradient operator
• Kirsch compass operator
• Robinson compass operator
• Nevatia-Babu 5X5 operator

132
Project due Dec. 26

• Roberts operator with threshold 30

133
Project due Dec. 26

• Prewitt edge detector with threshold 30

134
Project due Dec. 26

• Sobel edge detector with threshold 30

135
Project due Dec. 26

• Frei and Chen gradient operator with threshold
  30

136
Project due Dec. 26

• Kirsch compass operator with threshold 30

137
Project due Dec. 26

• Robinson compass operator with threshold 30

138
Project due Dec. 26

• Nevatia-Babu 5X5 operator threshold 30

139
Project due Jan. 2

• Write the following programs to detect edge

• Zero-crossing on the following four types of
• images to get edge images (choose proper
• thresholds), p. 349
• Laplacian
• minimum-variance Laplacian
• Laplacian of Gaussian
• Difference of Gaussian, (use tk to generate
  D.O.G.)
• dog (inhibitory , excitatory
  ,
• kernel size11)

140
Difference of Gaussian
141
Difference of Gaussian
142
Project due Jan. 2

• Zero-crossing on the following four types of
• images to get edge images (choose proper
• thresholds t1)

143

• Laplacian

144

• minimum-variance Laplacian

145

• Laplacian of Gaussian

146

• Difference of Gaussian
• (inhibitory , excitatory
  , kernel size11)