Lecture 003: Image Filters and Convolution

1
Lecture 003Image Filters and Convolution
CSE398/498 31 Aug 04
2
Administration

• Lab sessions MAY start on Thursday
• Still awaiting the delivery of cameras
• If you dont hear from me via e-mail that we are
  a go, come to the lecture instead

3
References Supporting Todays Lecture

• Digital Image Processing, R. Gonzalez R. Woods
• HIPR2 Image Processing Learning Resources
• http//homepages.inf.ed.ac.uk/rbf/HIPR2/
• Introductory Techniques for 3-D Computer Vision,
  E. Trucco A. Verri

4
Images are Discrete Functions!

• Digital images are discrete functions that
  correspond to the average scene luminance as
  perceived by the camera over a period of time
• Discrete spatially
• Discrete quantization

5
Images are Discrete Signals!

• Signal is visible light (the scene radiance)
• Collector is our CCD array
• While the signal is continuous, we sample at
  discrete intervals (e.g. 30 Hz)
• Both spatial and frequency representations
• Image formed in the spatial domain
• Can obtain frequency info through Discrete
  Fourier Transform (DFT) of the image
• We will focus on the spatial domain for now

6
Since Images are Signals

• They can be corrupted
• Subject to random and additive noise (e.g. from
  electronics)
• Often assumed to be zero-mean Gaussian
• Mathematical convenience
• Some theoretical basis

7
Effects of Gaussian Noise (s3)
8
Also from Impulsive Noise

• Aka salt pepper noise
• Less chronic but more acute than Gaussian noise
• Causes
• Transmission errors
• Faulty CCD elements
• External noise in AD conversion
• Algorithm artifacts

9
Effects of Salt Pepper Noise
10
Noise Filtering

• Q Given a noise corrupted image, how to we go
  about attenuating the noise while minimizing the
  impact on the true signal ?
• A Signal processing.
• We will look at filters for handling both random
  and impulsive noise
• First, we need to go way back in our minds to
  calculus and rediscover

11
Convolution

• Definition
• Q What is the result of the convolution of 2
  functions?
• A Another function.

12
So What Exactly does Convolution Do?

• For a given value of t
• Take the mirror of g
• Shift it by a given value of t
• Multiply by f(t)
• Integrate from
• Repeat for every value of t from
• Since our images are only defined over a finite
  region, our range of t will be limited

13
Sample Convolutions
http//mathworld.wolfram.com/Convolution.html
14
Fun with Gaussians!

• The convolution of 2 gaussians function is itself
  a gaussian
• Convolution can be used to motivate the central
  limit theorem
• Convolve 2 equally sized rectangle functions

15
More fun with Gaussians!

• Now convolve the rectangle function with the
  result of the initial convolution
• Is this starting to look familiar?

16
Discrete Convolution

• For discrete functions, integration is replaced
  by a summation
• Discrete convolution can then be defined as
• In our case, the function f will correspond to
  our image and g the filter kernel that we will
  use to suppress noise

17
Kernels are just Discrete Functions

• A filter kernel or mask is an n x m array of
  numbers
• It is no different than an image in that it is a
  discrete function defined over the n x m array
  and 0 everywhere else
• 1-D Example
• Kernel is -1 0 1

Im2
Im1
18
Noise Filtering Mean Smoothing

• Linear Filter
• Takes a weighted average of the neighborhood.
  For a 3x3 we have
• Also referred to as the box filter

19
Noise Filtering Mean Smoothing

• In the frequency domain, this corresponds to the
  sinc function
• This acts as a low-pass filter by weighting
  frequencies in the main lobes higher
• Because of the secondary lobes, some noise can
  still enter the filtered image

20
Noise Filtering Gaussian Smoothing

• Linear filter, low-pass filter
• Based upon Gaussian distributions
• Typically use a 5x5 (min and max)
• Subtends 98.8 of the area when s1 pixel
• Because of discretization, some high frequency
  noise is not attenuated

21
Generating Gaussian Kernels

• Discrete 1-D kernel coefficients can be generated
  from Pascals triangle
• 2-D coefficients can be obtained from convolving
  2 1-D kernels (horizontal vertical components)

1 1 1 1 2 1 1 3 3 1 1 4 6 4 1
22
Properties of Convolution

• Commutative
• Associative
• Distributive
• Linear

23
Separability of the 2-D Gaussian
24
Separability of the 2-D Gaussian

• This means that rather than convolve the image
  with a 2-D n x n Gaussian kernel, we can first
  convolve it by a horizontal 1-D gaussian,
  followed by a 1-D vertical kernel
• Theoretically, computation then will only
  increase by a factor of 2n vice n2
• In practice, we will use the 2-D convolution
  because
• Moving image data to/from memory costs
  instructions
• We typically will use a 5x5 (or even 3x3) kernel

25
Synthetic Example Gaussian Smoothing
26
Noise Filtering Median Filter

• Non-linear Filter useful for impulsive noise
• Generate an n x n neighborhood around each pixel
  in the original image
• Take the median value of this neighborhood as the
  value in the new image
• n is typically small (3-5 are common)

27
Noise Filtering Median Filter
Image 1
Image 2
(10,11,11,12,13,13,15,150,255)
28
Synthetic Image Example
29
Handling Edge Effects

• No perfect solution
• Only convolve valid areas
• Mirror the borders outside the image
• Pad the image border with zeros adapt kernel

30
Putting the Pieces TogetherLaboratory 1

• Construct a vision system for automatic scene
  surveillance
• Part 1 Establish the Background of the Scene
• Part 2 Separate the background from the
  foreground
• Part 3 Eliminate outliers from the data

31
POP QUIZ ?How do we do this???

• Part 1
• ?
• Part 2
• ?
• ?
• Part 3
• ?

32
Other Applications of ConvolutionEdge Detection

• Convolution with an appropriate kernel can yield
  the image gradient
• Discontinuities in the gradient correspond to
  edges. Well talk more about this later

33
Summary

• Images are discrete digital signals subjected to
  additive, random and impulse noise
• We can attenuate this noise through a variety of
  filters (mean, gaussian, median, et al)
• This is accomplished by convolving the original
  image with a discrete kernel generated from the
  filter
• Convolution associativity can be used to speed up
  processing when multiple convolutions are
  required (sometimes)
• Because we are operating in a discrete world,
  results are imperfect