Lecture Two for teens

1
Lecture Two for teens
2
Introduction to Morphological Operators
3
About this lecture

• In this lecture we introduce INFORMALLY the most
  important operations based on morphology, just to
  give you the intuitive feeling.
• In next lectures we will introduce more formalism
  and more examples.

4
Binary Morphological Processing

• Non-linear image processing technique
• Order of sequence of operations is important
• Linear (32)3 (5)315
• 33239615
• Non-linear (32)2 (5)2 25 sum, then square
• (3)2 (2)2 9413 square, then sum
• Based on geometric structure
• Used for edge detection, noise removal and
  feature extraction
• ? Used to understand the shape/form of a binary
  image

5
Introduction

• Structuring Element
• Erosion
• Dilation
• Opening
• Closing
• Hit-and-miss
• Thinning
• Thickening

6
1D Morphological Operations
7
Example for 1D Erosion
Input image
1 0 0 0 1 1 1 0 1 1
Structuring Element
0
Output Image
8
Example for Erosion
Input image
1 0 0 0 1 1 1 0 1 1
Structuring Element
0 0
Output Image
9
Example for Erosion
Input image
1 0 0 0 1 1 1 0 1 1
Structuring Element
0 0 0
Output Image
10
Example for Erosion
Input image
1 0 0 0 1 1 1 0 1 1
Structuring Element
0 0 0 0
Output Image
11
Example for Erosion
Input image
1 0 0 0 1 1 1 0 1 1
Structuring Element
0 0 0 0 1
Output Image
Structured element is completely included in set
of ones
12
Example for Erosion
Input image
1 0 0 0 1 1 1 0 1 1
Structuring Element
0 0 0 0 1 0
Output Image
13
Example for Erosion
Input image
1 0 0 0 1 1 1 0 1 1
Structuring Element
0 0 0 0 1 0 0
Output Image
14
Example for Erosion
Input image
1 0 0 0 1 1 1 0 1 1
Structuring Element
0 0 0 0 1 0 0 0
Output Image
15
Example for 1D Dilation
Input image
1 0 0 0 1 1 1 0 1 1
Structuring Element
1
Output Image
Structuring element overlaps with input image
16
Example for Dilation
Input image
1 0 0 0 1 1 1 0 1 1
Structuring Element
1 0
Output Image
17
Example for Dilation
Input image
1 0 0 0 1 1 1 0 1 1
Structuring Element
1 0 1
Output Image
18
Example for Dilation
Input image
1 0 0 0 1 1 1 0 1 1
Structuring Element
1 0 1 1
Output Image
19
Example for Dilation
Input image
1 0 0 0 1 1 1 0 1 1
Structuring Element
1 0 1 1 1
Output Image
20
Example for Dilation
Input image
1 0 0 0 1 1 1 0 1 1
Structuring Element
1 0 1 1 1 1
Output Image
21
Example for Dilation
Input image
1 0 0 0 1 1 1 0 1 1
Structuring Element
1 0 1 1 1 1 1
Output Image
22
Example for Dilation
Input image
1 0 0 0 1 1 1 0 1 1
Structuring Element
1 0 1 1 1 1 1 1
Output Image
23
2D Morphological Operations
24
Image Set of Pixels

• Basic idea is to treat an object within an image
  as a set of pixels (or coordinates of pixels)
• In binary images, pixels that are off, set to
  0, are background and appear black.
• Foreground pixels (objects) are 1 and appear white

25
Neighborhood

• Set of pixels defined by their location relation
  to the pixel of interest
• Defined by structuring element
• Specified by connectivity
• Connectivity-
• 4-connected
• 8-connected

26
Labeling Connected Components

• Label objects in an image
• 4-Neighbors
• 8-Neighbors

p
p
27
4 and 8 Connect
Input Image 8
Connect 4 - Connect
28
Morphological Image Processing
From Digital Image Processing, Gonzalez,Woods
And Eddins
Basic operations on shapes
29
Morphological Image Processing
From Digital Image Processing, Gonzalez,Woods
And Eddins
30
Operations on binary images in MATLAB
From Digital Image Processing, Gonzalez,Woods
And Eddins
31
Translation of Object A by vector b

• Define Translation ob object A by vector b
• At t ? I2 t ab, a ? A
• Where I2 is the two dimensional image space
  that contains the image
• Definition of DILATION is the UNION of all the
  translations
• A? B ? t ? I2 t ab, a ? A for all bs
  in B

32
Structuring Element (Kernel)

• Structuring Elements can have varying sizes
• Usually, element values are 0,1 and none(!)
• Structural Elements have an origin
• For thinning, other values are possible
• Empty spots in the Structuring Elements are dont
  cares!

Box Disc
Examples of stucturing elements
33
Reflection of the structuring element
From Digital Image Processing, Gonzalez,Woods
And Eddins
34
Idea of DILATION versus TRANSLATION
A
A1
Object B is one point located at (a,0)
A1 Object A is translated by object B Since
dilation is the union of all the translations, A?
B ? At where the set union ? is for all the bs
in B, the dilation of rectangle A in the positive
x direction by a results in rectangle A1 (same
size as A, just translated to the right)
35
DILATION B has 2 Elements
A
A2 A1 (part of A1 is under A2)
-a a
-a a
Object B is 2 points, (a,0), (-a,0)
There are two translations of A as result of two
elements in B Dilation is defined as the UNION
of the objectsA1 and A2. NOT THE INTERSECTION
36

• 2D DILATION

37
DILATION
Rounded corners
Round Structuring Element (SE) can be
interpreted as rolling the SE around the contour
of the object. New object has rounded corners and
is larger in every direction by ½ width of the SE
38
DILATION
Rounded corners
Square Structuring Element (SE) can be
interpreted as moving the SE around the contour
of the object. New object has square corners and
is larger in every direction by ½ width of the SE
39
DILATION

• The shape of B determines the final shape of the
  dilated object.
• B acts as a geometric filter that changes the
  geometric structure of A

40

• 2D EROSION

41
Another important operator

• Introduction to Morphological Operators
• Used generally on binary images, e.g., background
  subtraction results!
• Used on gray value images, if viewed as a stack
  to binary images.
• Good for, e.g.,
• Noise removal in background
• Removal of holes in foreground / background
• Check www.cee.hw.ac.uk/hipr

42
A first Example Erosion

• Erosion is an important morphological operation
• Applied Structuring Element

43
Dilation versus Erosion

• Basic operations
• Are dual to each other
• Erosion shrinks foreground, enlarges Background
• Dilation enlarges foreground, shrinks background

44
Erosion

• Erosion is the set of all points in the image,
  where the structuring element fits into.
• Consider each foreground pixel in the input image
• If the structuring element fits in, write a 1
  at the origin of the structuring element!
• Simple application of pattern matching
• Input
• Binary Image (Gray value)
• Structuring Element, containing only 1s!

45
Another example of erosion

• White 0, black 1, dual property, image as a
  result of erosion gets darker

46
Introduction to Erosion on Gray Value Images

• View gray value images as a stack of binary
  images!

Intensity is lower so the image is darker
47
Erosion on Gray Value Images

• Images get darker!

48
Example Counting Coins

• Counting coins is difficult because they touch
  each other!
• Solution Binarization and Erosion separates them!

49

• DILATION
• more details

50
Example Dilation

• Dilation is an important morphological operation
• Applied Structuring Element

51
Dilation

• Dilation is the set of all points in the image,
  where the structuring element touches the
  foreground.
• Consider each pixel in the input image
• If the structuring element touches the foreground
  image, write a 1 at the origin of the
  structuring element!
• Input
• Binary Image
• Structuring Element, containing only 1s!!

52
Another Dilation Example

• Image get lighter, more uniform intensity

53
Dilation on Gray Value Images

• View gray value images as a stack of binary
  images!

54
Dilation on Gray Value Images

• More uniform intensity

55
Edge Detection

• Edge Detection
• Dilate input image
• Subtract input image from dilated image
• Edges remain!

56
From Digital Image Processing, Gonzalez,Woods
And Eddins
Illustration of dilation
57
Example of Dilation in Matlab
gtgt I zeros(13 19) gtgt I(6,68)1 gtgt I2
imdilate(I,se)
58
Imdilate function in MATLAB
IM2 IMDILATE(IM,NHOOD) dilates the image IM,
where NHOOD is a matrix of 0s and 1s that
specifies the structuring element
neighborhood. This is equivalent to the syntax
IIMDILATE(IM, STREL(NHOOD)). IMDILATE
determines the center element of the
neighborhood by FLOOR((SIZE(NHOOD) 1)/2). gtgt
se imrotate(eye(3),90) se 0 0
1 0 1 0 1 0 0 gtgt
ctrfloor(size(se)1)/2 ctr 2 2
59
MATLAB Dilation Example
gtgt I zeros(13 19) gtgt I(6, 612)1 gtgt SE
imrotate(eye(5),90) gtgt I2imdilate(I,SE) gtgt
figure, imagesc(I) gtgt figure, imagesc(SE) gtgt
figure, imagesc(I2)
60
DILATED IMAGE
INPUT IMAGE
SE
61
I I2
gtgt I(69,613)1 gtgt figure, imagesc(I) gtgt
I2imdilate(I,SE) gtgt figure, imagesc(I2)
SE
62
I I2
SE 1 1 1 1 1 1
1 1 1
63
Dilation and Erosion
64
Dilation and Erosion

• DILATION Adds pixels to the boundary of an
  object
• EROSIN Removes pixels from the boundary of an
  object
• Number of pixels added or removed depends on size
  and shape of structuring element

65
Illustration of Erosion
From Digital Image Processing, Gonzalez,Woods
And Eddins
66
MATLAB Erosion Example
2 pixel wide
SE 3x3
I3imerode(I2,SE)
67
Illustration of erosion
From Digital Image Processing, Gonzalez,Woods
And Eddins
68
Combinations

• In most morphological applications dilation and
  erosion are used in combination
• May use same or different structuring elements

69
Opening Closing

• Important operations
• Derived from the fundamental operations
• Dilatation
• Erosion
• Usually applied to binary images, but gray value
  images are also possible
• Opening and closing are dual operations

70

• OPENING

71
Opening

• Similar to Erosion
• Spot and noise removal
• Less destructive
• Erosion next dilation
• the same structuring element for both operations.
• Input
• Binary Image
• Structuring Element, containing only 1s!

72
Opening

• Take the structuring element (SE) and slide it
  around inside each foreground region.
• All pixels which can be covered by the SE with
  the SE being entirely within the foreground
  region will be preserved.
• All foreground pixels which can not be reached by
  the structuring element without lapping over the
  edge of the foreground object will be eroded
  away!
• Opening is idempotent Repeated application has
  no further effects!

73
Opening

• Structuring element 3x3 square

74
Opening Example

• Opening with a 11 pixel diameter disc

75
Opening Example

• 3x9 and 9x3 Structuring Element

39
93
76
Opening on Gray Value Images

• 5x5 square structuring element

77
Use Opening for Separating Blobs

• Use large structuring element that fits into the
  big blobs
• Structuring Element 11 pixel disc

78

• CLOSING

79
Closing

• Similar to Dilation
• Removal of holes
• Tends to enlarge regions, shrink background
• Closing is defined as a Dilatation, followed by
  an Erosion using the same structuring element for
  both operations.
• Dilation next erosion!
• Input
• Binary Image
• Structuring Element, containing only 1s!

80
Closing

• Take the structuring element (SE) and slide it
  around outside each foreground region.
• All background pixels which can be covered by the
  SE with the SE being entirely within the
  background region will be preserved.
• All background pixels which can not be reached by
  the structuring element without lapping over the
  edge of the foreground object will be turned into
  a foreground.
• Opening is idempotent Repeated application has
  no further effects!

81
Closing

• Structuring element 3x3 square

82
Closing Example

• Closing operation with a 22 pixel disc
• Closes small holes in the foreground

83
Closing Example 1

1. Threshold
2. Closing with disc of size 20

Thresholded closed
84
Closing Example 2

• Good for further processing E.g. Skeleton
  operation looks better for closed image!

skeleton of Thresholded
skeleton of Thresholded and next closed
85
Closing Gray Value Images

• 5x5 square structuring element

86
Opening Closing

• Opening is the dual of closing
• i.e. opening the foreground pixels with a
  particular structuring element
• is equivalent to closing the background pixels
  with the same element.

87
Morphological Opening and Closing

• Opening of A by B ? A B
• Erosion of A by B, followed by
• the dilation of the result by B
• Closing of A by B ? A B
• Dilation of A by B, followed by
• the erosion of the result by B
• MATLAB imopen(A, B)
• imclose(A,B)

88
MATLAB Function strel

• strel constructs structuring elements with
  various shapes and sizes
• Syntax se strel(shape, parameters)
• Example
• se strel(octagon, R)
• R is the dimension see help function

89

• Opening of A by B ? A B
• Erosion of A by B, followed by the dilation of
  the result by B

Erosion- if any element of structuring element
overlaps with background output is zero
FIRST - EROSION
gtgt se strel('square', 20)fe
imerode(f,se)figure, imagesc(fe),title('fe')
90
Dilation of Previous Result Outputs 1 at center
of SE when at least one element of SE overlaps
object
SECOND - DILATION
gtgt se strel('square', 20)fd
imdilate(fe,se)figure, imagesc(fd),title('fd')
91
FOimopen(f,se) figure, imagesc(FO),title('FO')
92
What if we increased size of SE for DILATION
operation??
se 25 se 30
se strel('square', 25)fd imdilate(fe,se)figu
re, imagesc(fd),title('fd') se strel('square',
30)fd imdilate(fe,se)figure,
imagesc(fd),title('fd')
93
Closing of A by B ? A B Dilation of A by B
Outputs 1 at center of SE when at least one
element of SE overlaps object
se strel('square', 20)fd imdilate(f,se)figur
e, imagesc(fd),title('fd')
94
Erosion of the result by B
Erosion- if any element of structuring element
overlaps with background output is zero
95
ORIGINAL
OPENING
CLOSING
96
Fingerprint problem
From Digital Image Processing, Gonzalez,Woods
And Eddins
97

• HIT and MISS

98
Hit or Miss Transformation

• Hit or Miss Called also Hit and Miss is
  useful to identify specified configuration of
  pixels.
• For instance, such combinations as
• isolated foreground pixels
• or pixels at end of lines (end points)
• A? B (A ? B1) ? (Ac ? B2)
• A eroded by B1, intersection A complement eroded
  by B2 (two different structuring elements)

99
Hit or Miss Example Find cross shape pixel
configuration
0 1 0
1 1 1
0 1 0
MATLAB Function C bwhitmiss(A, B1, B2)
100
Original Image A and B1 A eroded by
B1 Complement of Original Image and
B2 Erosion of A complement And
B2 Intersection of eroded images
From Digital Image Processing, Gonzalez,Woods
And Eddins
101
Hit or Miss

• Have all the pixels in B1, but none of the pixels
  in B2

102
Hit or Miss Example 2

• Locate upper left hand corner pixels of objects
  in an image
• Pixels that have east and south neighbors (Hits)
  and no NE, N, NW, W, SW Pixels (Misses)
• B1 B2

0 0 0
0 1 1
0 1 0
1 1 1
1 0 0
1 0 0
Dont Care about SE
103
Hit or Miss in Matlab
G bwhitmiss(f, B1, B2) Figure, imshow(g)
From Digital Image Processing, Gonzalez,Woods
And Eddins
104
bwmorph(f, operation, n)

• Implements various morphological operations based
  on combinations of dilations, erosions and look
  up table operations.
• Example Thinning
• gtgt f imread(fingerprint_cleaned.tif)
• gtgt g bwmorph(f, thin, 1)
• gtgt g2 bwmorph(f, thin, 2)
• gtgt g3 bwmorph(f, thin, Inf)

105
Chapter 9 Morphological Image Processing
Input
From Digital Image Processing, Gonzalez,Woods
And Eddins
gtgt f imread(fingerprint_cleaned.tif) gtgt g
bwmorph(f, thin, 1) gtgt g2 bwmorph(f, thin,
2) gtgt g3 bwmorph(f, thin, Inf)
106
From Digital Image Processing, Gonzalez,Woods
And Eddins
107
Hit-and-miss Transform is used for Pattern
Matching

• Hit and Miss is used to look for particular
  patterns of foreground and background pixels
• It allows a recognition of very simple objects
• All other morphological operations can be derived
  from it!!
• Input
• Binary Image
• Structuring Element, containing 0s and 1s!!

108
Example for a Hit-and-miss Structuring Element

• Contains 0s, 1s and dont cares.
• Usually a 1 at the origin!

109
Hit-and-miss Transform as pattern matching

• It is one variant of a general to Pattern
  Matching approach
• If foreground and background pixels in the
  structuring element exactly match foreground and
  background pixels in the image,
• then the pixel underneath the origin of the
  structuring element is set to the foreground
  color.

110
EXAMPLE Corner Detection with Hit-and-miss
Transform

• Structuring Elements representing four corners

111
Corner Detection with Hit-and-miss Transform

1. Apply Hit--Miss with each Structuring Element
2. Use OR operation to combine the four results

Allows to describe approximate shape of an object
112

• Basic THINNING

113
Thinning

1. Used to remove selected foreground pixels from
   binary images
2. After edge detection, lines are often thicker
   than one pixel.
3. Thinning can be used to thin those line to one
   pixel width.

114
Definition of Thinning

• Let K be a kernel and I be an image
• with 0-10!!
• If foreground and background fit the structuring
  element exactly, then the pixel at the origin of
  the SE is set to 0
• Note that the value of the SE at the origin is 1
  or dont care!

115
Example Thinning with two HM transforms
We use two Hit-and-miss Transforms
0
0
116

• Basic THICKENING

117
Thickening

• Used to grow selected regions of foreground
  pixels
• E.g. applications like approximation of convex
  hull

118
Definition Thickening

• Let K be a kernel and I be an image
• with 111
• If foreground and background match exactly the
  SE, then set the pixel at its origin to 1!
• Note that the value of the SE at the origin is 0
  or dont care!

119
Example Thickening
If foreground and background match exactly the
SE, then set the pixel at its origin to 1!
1
1
120
Skeletonization
From Digital Image Processing, Gonzalez,Woods
And Eddins
Bone Image
Skeleton obtained using function bwmorph
Resulting Skeleton obtained after pruning with
function endpoints
121
Objects in Images
In many cases we want to find some known objects
in images
Image containing ten objects
A subset of pixels from the Image
From Digital Image Processing, Gonzalez,Woods
And Eddins
122
Finding objects in pictures
Pixel p and its diagonal neighbors
Pixel p and its 8- neighbors
Pixel p and its 4-neighbors
Pixels that are 8 adjacent but not 4 adjacent
4 and 8 adjacent pixels
These pixels are 4 and 8 connected
These pixels are 8 connected but not 4 connected
From Digital Image Processing, Gonzalez,Woods
And Eddins
123
How many objects are really in a picture?
From Digital Image Processing, Gonzalez,Woods
And Eddins
124
Connected Components
Center of mass is another useful concept in
object recognition
From Digital Image Processing, Gonzalez,Woods
And Eddins
125
Morphological Reconstruction
Original image (the mask)
Intermediate image after 100 iterations
Marker image
126
Morphological Reconstruction
Chapter 9 Morphological Image Processing
From Digital Image Processing, Gonzalez,Woods
And Eddins
127
Translation and Reflection
From Digital Image Processing, Gonzalez,Woods
And Eddins
128
Reflection

• Dilation definition
• Dilation of A by B is the set consisting of all
  structuring element origin locations where the
  reflected and translated B overlaps at least some
  portion of A
• If structuring element is symmetric with respect
  to origin, reflection of B has no effect

129

• PROBLEMS TO THINK

130
Images of lanes and corridors
131
Problems

1. Consider the images on slide 17! Why are the
   images getting darker under erosion? Explain!
2. Consider the images on slide 31! Why are the
   intensities becoming more uniform? Explain!
3. Compare Dilatation and Erosion! How are they
   related? Verify your answers with Matlab!
4. Apply Erosion and Dilatation for noise removal!
5. Consider slide 38 Remove the artefacts remaining
   with the horizontal lines.

132
Problems

1. Derive dilatation and erosion from the
   Hit-and-miss transformation
2. How to use these all operations to find some good
   features in our FAB building corridors and halls
   so that the robot can recognize the object such
   as door or windows?

133
Sources Used

1. Volker Krüger
2. Rune Andersen
3. . Roger S. Gaborski