Einfhrung in die erweiterte Realitt Bildverarbeitung 2D Prof' G' Klinker, Prof' B' Brgge 23' Juni 20

1
Einführung in die erweiterte Realität
Bildverarbeitung (2D) Prof. G. Klinker, Prof.
B. Brügge23. Juni 2000
2
1. Image Formation

• Radiometry
• Image geometry

3
Image Geometry
y
Object point (x,y,z)
Image plane (inverted)
x
r
f
z
z
r
(x,y)
x
y
4
Image Geometry
y
Image plane
Object point (x,y,z)
y
x
(x,y)
x
r
r
f
z
5
Image Geometry
Image plane
Image array
y
0 column j m-1
0 . . . . n-1
pixel ai,j
(x,y)
row i
x
x j - m2- 1
y - (i - n2- 1)
6
2. Levels of Computation

• Object level (3D)
• Object level (2D features)
• Global level
• Local level
• Point level
• Bottom-up vs. top-down

7
3. Point Level

• fB i,j Opoint fAi,j
• Thresholding fBi,j
• Lookup tables fBi,j lookupfAi,j
• Image inversion fBi,j 255 - fAi,j
• Image arithmetic fBi,j fA1i,j
  fA2i,j
• Color transformations (rgb -gt IHS, LUV, ...)

0, if fAi,j lt t 1, if fAi,j gt t
8
4. Local Level

• fB i,j Olocal fAik,jl ik,jl e Ni,j
• Image correlation
• Linear space-invariant image filters (image
  convolution)
• Smoothing
• Edge detection
• Corner detection

9
4.1 Region Correlation

• unnormalized r(i,j) S t(x,y)
  s(ix,jy)
• normalized

St(x,y)-Ts(ix,jy)-Sij
r(i,j)
S t(x,y) - T2 S s(ix,jy) - Sij2
Image 2
Image 1
j
i
template t, mean T
search region s mean Sij (i,j)
10
4.2 Linear Space-Invariant (LSI) Systems

• Input impulse d(x,y)
• Impulse response g(x,y)
• g(x,y) independent of the position of the input
  impulse d(x-x0,y-y0) g (x-x0,y-y0)
• System completely describable by g(x,y)
• Closed under linear combinations af1 bf2

Linear space invariant system g(x,y)
Input f(x,y)
Output h(x,y)
11
Image Convolution

• h(x,y) f(x,y) g(x,y) f(x
  x, y y) g(x, y) dx dy
• h i,j f i - n/2 k, j - n/2 l
  gk, l

-
-
m-1
n-1
l0
k0
12
Discrete Image Convolution
m-1
n-1

• h i,j f i - n/2 k, j - n/2 l
  gk, l

l0
k0
j
input image f(i,j)
i
convolution mask g(k,l)
13
4.3 Smoothing Filters
1 1 1 1 1 1 1 1 1

• Local pixel averages
• Gaussian low-pass filter
• rotationally symmetric
• single lobe
• not corrupted by high-frequencysignals
• parameterized by s
• efficient separable filter
• cascadable (scale space, image pyramids)

1/9
1 2 1 2 4 2 1 2 1
1/16
- (k2l2) 2s2
with k -n/2 .. n/2 l -m/2 .. m/2
gk,l e
1 1 2 2 2 1 1 1 2 2 4 2 2 1 2 2 4
8 4 2 2 2 4 8 16 8 4 2 2 2 4 8 4 2
2 1 2 2 4 2 2 1 1 1 2 2 2 1 1
1 / 144
14
4.4 Edge Detection

• Image gradient (first derivative of f(x,y))
• orientation (direction of steepest ascent)
  a (x,y) tan-1 (Gy / Gx)
• magnitude Gf(x,y) (Gx2
  Gy2)
  Gx Gy

df dx df dy
-1 1
Gx
Gx Gy
Gf(x,y)
1 -1
Gy
15
Simple Edge Detectors

• Roberts Cross
• Sobel
• Prewitt

1 0 0 -1
0 -1 1 0
-1 0 1 -2 0 2 -1 0 1
1 2 1 0 0 0 -1 -2 -1
-1 0 1 -1 0 1 -1 0 1
1 1 1 0 0 0 -1 -1 -1
16
Robust Edge Detection

• Combined filters for
• Smoothing (noise reduction)
• Enhancement (edge detection)
• Detection (magnitude thresholding)
• Localization (subpixel precision)
• Detection problems (classification errors)
• Lack of acuracy (position, orientation)
• False edges (false positives)
• Missing edges (false negatives)

17
4.5 Laplacian Edge Detection

• Optima of 1. derivative of f(x,y)
• Zero-crossings of 2. derivative of f(x,y)
• Problem very sensitive to noise!

d2f dy2
d2f dx2
D2f
fi,j1 - 2fi,j fi,j-1
d2f dx2
1 4 1 4 -20 4 1 4 1
0 1 0 1 -4 1 0 1 0
d2f dy2
fi1,j - 2fi,j fi-1,j
18
Mexican Hat Operator

• Laplacian of Gaussian (LoG)
• Smoothing with a Gaussian filter
• Enhancement by 2. derivative edge detection
• Detection of zero crossings in 2. derivative in
  combination with large peak in 1. derivative
• Localization with subpixel resolution using
  linear interpolation

19

• LoG-Operator

h(x,y) D2g(x,y) f(x,y)
D2g(x,y) f(x,y)
0 0 0 0 0 0 -1 -1 -1 -1 -1
0 0 0 0 0 0 0 0 0 0 -1 -1
-1 -1 -1 -1 -1 -1 -1 0 0 0 0
0 0 -1 -1 -1 -2 -3 -3 -3 -3 -3
-2 -1 -1 -1 0 0 0 0 -1 -1 -2 -3
-3 -3 -3 -3 -3 -3 -2 -1 -1 0 0
0 -1 -1 -2 -3 -3 -3 -2 -3 -2 -3 -3
-3 -2 -1 -1 0 0 -1 -2 -3 -3 -3 0
2 4 2 0 -3 -3 -3 -2 -1 0 -1
-1 -3 -3 -3 0 4 10 12 10 4 0
-3 -3 -3 -1 -1 -1 -1 -3 -3 -2 2 10
18 21 18 10 2 -2 -3 -3 -1 -1 -1 -1
-3 -3 -3 4 12 21 24 21 12 4 -3 -3
-3 -1 -1 -1 -1 -3 -3 -2 2 10 18 21
18 10 2 -2 -3 -3 -1 -1 -1 -1 -3 -3
-3 0 4 10 12 10 4 0 -3 -3 -3 -1
-1 0 -1 -2 -3 -3 -3 0 2 4 2
0 -3 -3 -3 -2 -1 0 0 -1 -1 -2 -3
-3 -3 -2 -3 -2 -3 -3 -3 -2 -1 -1
0 0 0 -1 -1 -2 -3 -3 -3 -3 -3 -3
-3 -2 -1 -1 0 0 0 0 -1 -1 -1 -2
-3 -3 -3 -3 -3 -2 -1 -1 -1 0 0
0 0 0 0 -1 -1 -1 -1 -1 -1 -1
-1 -1 0 0 0 0 0 0 0 0 0 0
-1 -1 -1 -1 -1 0 0 0 0 0 0
0 0 -1 0 0 0 -1 -2 -1 0 -1 -2 16
-2 -1 0 -1 -2 -1 0 0 0 -1 0 0
20
Canny Edge Detector

• First derivative of a Gaussian
• Nonmaxima suppression (ridge thinning)
• Double thresholding to detect and link edges

Si,j Gi,j s Ii,j Pi,j - Si,j
Si,j1 - Si1,j
Si1,j1 Qi,j Si,j Si,j1
- Si1,j - Si1,j1
-1 1 -1 1
1 1 -1 -1
21
--- Levels of Computation ---

• Object level (3D)
• Object level (2D features)
• Global level
• Local level
• Point level
• Bottom-up vs. top-down

22
5. Global Level

• P Oglobal fAi,j
• Histograms, histogram-based region segmentation
• Operations on image masks
• Computation of region statistics
• Connected component analysis contour detection
• Morphological operators medial axis
  transformation
• Region segmentation (growing, split merge)
• Hough transformations aligned features
• Fourier transformation

23
5.1 Histogram-based Region Segmentation
h(I)

• P-tile method
• Mode method
• Iterative threshold selection
• Adaptive, variable thresholding
• Double thresholding

I
24
5.2 Operations on Image Masks

• Geometric properties (moments)
• Size (zeroth-order moment) A S S Bi,j
• Position (first-order moments) mi S S i
  Bi,j / A mj S S j Bi,j / A
• Orientation (second-order moments) a S S (I-
  mi)2 Bi,j b 2 S S (I- mi)(j- mj) Bi,j c
  S S (j- mj) 2 Bi,j tan 2q b / (a-c)

0 0 0 1 1 0 0 1 0 0 0 0 1 1 0 1 1 0 1 1 1 1 1 1 1
1 0 0 1 1 1 1 0 0 0 0 0 0 0 1 1 0 1 1 0
25
Photometric Properties of a Region

• Mean intensity (color)
• Intensity variation (color variation 3
  eigenvectors)
• Repetitive patterns (textures)

26
Connected Component Analysis- Pixel
Neighborhoods -
0 1 0 1 1 1 0 1 0

• 4-neighbors
• 8-neighbors
• 4-path, 8-path
• Connected components 8 4
• Background and holes 4 8
• Boundary

1 1 1 1 1 1 1 1 1
0 1 0 1 0 1 0 1 0
0 1 0 1 0 1 0 1 0
27
Connected Component Analysis- Algorithms -

• Two-pass approach progressingin row-major order
• Recursive region growing
• find seed
• check each neighborthat hasnt been visited yet
• if neighbor is inside the region,
• label it
• recurse

0 0 0 1 1 0 0 1 0 0 0 0 1 1 0 1 1 0 1 1 1 1 1 1 1
1 0 0 1 1 1 1 0 0 0 0 0 0 0 1 1 0 1 1 0
28
Region Properties

• Region boundary
• Crab algorithm
• Area A and perimeter P
• Compactness
• Isoperimetric inequality P2 / A gt 4 p

29
Morphological Operators

• Intersection of two binary images
• Union of two binary images
• Complement of a binary image
• Dilation of a binary image by a structuring
  element (simple case region expansion)
• Erosion (simple case region shrinking)
• Opening erosion dilation
• Closing dilation erosion

30
Medial Axis Transform - Image Distance Measures -

• Euclidean
• City-block
• Chessboard

31
Medial Axis Transformation- Algorithm -

• Find pixels pi,j with locally maximal distances
  in S from S w.r.t 4-neighbors u,v d(i,j, S)
  gt d(u,v,S)

000000000000 010000000010 001000000100 00011111100
0 001000000100 010000000010 000000000000
000000000000 011111111110 011111111110 01111111111
0 011111111110 011111111110 000000000000
32
6. Object Level (2D)

• Bottom-up Contour detection
• Line fitting
• Curve fitting
• Determination of corner points
• Top-down
• Shape fitting
• Image segmentationDistinction between objects
  and background

33
6.1 Digital Curves

• parametric curve description P(ui) (x(ui),
  y(ui))
• k-slope
• k-curvature
• curve lengthx(add 1 along sides, 1.5 along
  diagonals)

34
Chain Codes
2 3 4 1 x 5 8 7 6

• Eight directions
• Clockwise
• Rotation by 45 degrees add 1
• Difference code (1. derivative of chain code)
  rotation-invariant
• Area, corners can be computed
• But limited set of tangent directions

35
Slope Representation y -s Plot

• Tangent y versus arc length s
• Lines in image Horizontal lines in plot
• Circular arcs in image Sloped lines in plot
• Periodic description
• Compact description of contour shape
• Watch for
• Changing starting point
• Rotated contours

36
Slope Density Function

• Histogram of contour slopes
• Useful descriptor for recognition

37
6.2 Curve Fitting

• Goodness of fit
• Maximum absolute error
• Mean squared error
• Normalized maximum error
• Number of sign changes
• Ratio of curve length to end point distance

38
Curve Models

• Line segments (Polylines)
• Circular arcs
• Conic sections
• Cubic splines

39
Polylines

• Sequence of line segments
• implicit line representation f(x,y) 0
• distance from line -gt f(x,y) d
• Polyline splitting (recursive subdivision)
• Segment merging (bottom-up approach)
• Split and merge
• Hop-along algorithm

40
6.3 Curve Approximation

• Total regression
• Estimating corners
• Robust regression

41
(No Transcript)
42
n
-
k1
a bc d ef g hij k l m n o p qr s t u v w x y
z ABCDEFGHIJK LM NO PQR S TU V W XYZ