CIS303 Advanced Forensic Computing

1
CIS303Advanced Forensic Computing

• Dr Giles Oatley

2
Image Segmentation

• Neighbourhood operations allow us to create
  algorithms to detect edges, reduce noise, define
  regions in various ways, locate specific features
  and detect edges. All are vital activities
  employed in many algorithms devised for object
  recognition.
• Sub topics
• Neighbourhood operations (spatial filtering)
• Convolution and correlation
• Spatial Filtering
• Edge detection (Sobel, Laplace Canny)

3
Terminology

• In this lecture, we are interested in filtering
  operations that are performed directly on the
  pixels of an image. Use of the term linear
  spatial filtering differentiates this type of
  process from frequency domain filtering.
• These operations consist of multiplying each
  pixel in the neighbourhood by a corresponding
  coefficient and summing the results to obtain the
  response at each point (x,y) in the image.
• If the neighbourhood is of size m x n, mn
  coefficients are used, where the coefficients are
  arranged as a matrix.
• This matrix is called a filter, mask, filter
  mask, kernel, template or window (the first four
  are most popular).
• The terms convolution filter/mask/kernel are also
  used.

4
A neighbourhood

• The set of pixels that surround a target pixel
  form the neighbourhood of that pixel. There are
  no absolute and fast rules about the dimensions
  of a neighbourhood but it is customary to use an
  odd number of rows and columns to define a
  symmetrical neighbourhood.
• Example a 5 x 5 neighbourhood.

Pi-1,j





Pi,j
Pi,j-1
Pi1,j
5
How it works

• Thus, in a neighbourhood operation, the value of
  the target pixel is a function of the values of
  all other pixels in the neighbourhood, including
  the target pixel.
• The operation (whatever it happens to be more
  soon) is performed on every pixel in the image,
  by using a mask.

6
Linear spatial filtering

• Note that at the edges, it is normal to pad f
  with as many 0s as necessary to ensure that there
  will always be corresponding points for a full
  excursion of w past f.
• See function imfilter for details of padding
  options.

7
Using the mask in practice

• The 3 x 3 neighbourhood is usually referred to as
  a) mask, b) structure element or c) as a
  convolution kernel.
• To see how the mask is used in practice let us
  take a numerical example in 1 dimension.

8
Convolution and correlation

• The masking operation just described is
  technically a correlation. The alternative is a
  convolution.
• If the mask is symmetrical (m1 m3) which is the
  most common case the distinction is not important
  in practice. Theoretically the difference is very
  important

9
Convolution and correlation in 2D

• For an image f and convolution kernel h (2L1
  x 2L1) the definition of convolution is
• and the definition of correlation is
• Applying these equations in practice requires
  careful algorithm design for efficiency and
  consideration needs to be given as to the
  handling of edge pixels in the image.

10
Filtering

• The following mask will have the effect of
  replacing the central pixel by the central pixel
  plus the sum of its neighbourhood pixels.
• Dividing the result by 9 gives the average
  intensity over the nine pixels. This is a useful
  technique for reducing noise in the image but it
  also results in a loss of sharpness in the image.
• The process is equivalent to suppressing higher
  frequency components and is thus a form of low
  pass filtering
• We will see later that by using the frequency
  domain far more efficient methods exist.

11
A simple MatLab filtering programme

• Here we generate a simple filter using the ones
  function. The result is normalised by dividing by
  n2.
• function onesfilter(n)
• To test simple filtering
• A imread('church.jpg')
• vary this
• n12
• SE ones(n)/n2
• BW imfilter(A, SE)
• imshow(BW)

12
Results on the church image
N 5
Original
N 12
N 9
13
Noise reduction
(a)
(a) Shows a picture of a moth taken against a
cloth background which can clearly be seen in
the enlargement (b). Simple smoothing reduces the
background but looses definition (c)
14
Gaussian filtering
The 2D Gaussian function provides a powerful
filtering function. It is particularly attractive
as we shall see because the corresponding
function in the frequency domain is also Gaussian
The function is rotationally symmetric so there
is no bias in the filtering and more weight is
given to the central pixels.
15
Application of Gaussian filtering
To apply the Gaussian filter it is possible to
use the equation on the previous slide directly
but in practice it is much more efficient to
create a mask of the appropriate size. MatLab
provides the fspecial function for creating 2D
filters. One form of fspecial is h
fspecial('gaussian',hsize,sigma) hsize is the
dimension of the square mask and sigma the
standard deviation of the Gaussian. Examples
h fspecial('gaussian',5,2)
h fspecial('gaussian',3,1)
h 0.0232 0.0338 0.0383 0.0338
0.0232 0.0338 0.0492 0.0558 0.0492
0.0338 0.0383 0.0558 0.0632 0.0558
0.0383 0.0338 0.0492 0.0558 0.0492
0.0338 0.0232 0.0338 0.0383
0.0338 0.0232
h 0.0751 0.1238 0.0751 0.1238
0.2042 0.1238 0.0751 0.1238 0.0751
16
The Gaussian filter in practice

• Repeating the example of removing the background
  from the moth image does not show a significant
  improvement with a Gaussian filter.
• function Gaussfilter
• To demonstrate a simple application of a
  Gaussian filter
• T imread('moth9.jpg')
• imshow(T)
• title('Original image')
• SE fspecial('gaussian',3,1)
• FX imfilter(T,SE)
• figure,imshow(FX)
• title('mask 3 x 3 sigma 1')

17
The original image
18
A 3 x 3 filter with ? 1

• Some improvement and not too much blurring

19
A 5 x 5 filter with ? 1

• A further improvement and still acceptable
  blurring

20
A 5 x 5 filter with ? 2

• Background completely removed but excessive
  blurring

21
Imfilter options (DIPUM)
22
Imfilter options example (DIPUM)
23
fspecial filters supported (DIPUM)
24
Filter examples (1 - DIPUM)
25
Filter example (2 - DIPUM)
26
Introduction to image segmentation

• Segmentation subdivides an image into its
  constituent regions or objects.
• The level to which the subdivision is carried
  depends on the problem being solved stop when
  the objects of interest have been isolated.
• Segmentation of nontrivial images remains one
  of the most difficult tasks in image processing
  and segmentation accuracy determines the eventual
  success or failure of computer vision tasks.
• Control may be exercised in two ways environment
  (e.g. lighting, position of objects of interest)
  sometimes possible in industrial applications,
  sensors (i.e. choice of imaging sensors/cameras).
• Segmentation algorithms often partition an image
  based on abrupt changes in intensity e.g. edges.
  We will look at edge detection in monochrome
  images here.

27
Modelling the intensity gradient
We can approximate the intensity gradient in the
x and y directions as follows
Where f(I,J) is the input pixel at I,J etc. We do
not divide by the distance between the pixels
because this is only a scaling factor and
division is expensive on computer time. We get
some noise reduction if we apply the formulae to
a 3 x 3 neighbourhood.
28
Example using an artificial image

• To demonstrate the use of this simple gradient
  edge detection scheme we can use the MatLab
  function imfilter
• function EdgeDetector
• To demonstrate some basic edge detection
  routines on a square image
• T imread('Square.jpg')
• subplot(3,1,1),imshow(T)
• title('Original image')
• XSE -1 0 1-1 0 1-1 0 1
• YSE -1 -1 -10 0 01 1 1
• FX imfilter(T,XSE,'conv')
• subplot(3,1,2),imshow(FX)
• title('X convolution')
• FY imfilter(T,YSE,'conv')
• subplot(3,1,3), imshow(FY)
• title('Y convolution')

XSE and YSE are the masks and the filter is
applied as a convolution
29
Results
Direction of calculation
In both the X and Y directions we have clearly
detected a falling edge but failed to detect a
rising edge. What is happening here ?
30
The results across the edge in more detail
The rising edge
The falling edge
The imfilter function truncates the output to
the allowed range of the image type. Thus
negative values are truncated to 0. This is why
we do not use imfilter directly for edge
detection.
31
Matlab edge function

• The edge function is used to detect edges.
• To find edges, this function looks for places in
  the image where the intensity changes rapidly,
  using one of these two criteria
• Places where the first derivative of the
  intensity is larger in magnitude than some
  threshold.
• Places where the second derivative of the
  intensity has a zero crossing. (More about this
  shortly)
• edge takes an intensity image I as its input, and
  returns a binary image BW of the same size as I,
  with 1's where the function finds edges in I and
  0's elsewhere.
• A threshold value of the intensity change
  determines whether or not an edge is detected.
  This can be given as a parameter to edge,
  otherwise it is calculated automatically.
• BW edge(I,'sobel')
• BW edge(I,'sobel',thresh)

32
The Prewitt edge detector
The pair of masks defined earlier form the basis
of the Prewitt edge detector.
However the gradients (gx gy) are computed
without truncation and then combined to give a
gradient magnitude and direction.
The correct expression for magnitude
Is not used because it is computationally too
intensive.
33
Edge detection in practice

• The MatLab edge function allows us to test a
  whole range of edge detectors with very simple
  code. As an example of a real image we will take
  Monkwearmouth Church
• function ChurchEdge
• To test the various edge detectors
• A imread('church1.jpg')
• tic
• BW edge(A,'prewitt')
• t toc
• imshow(BW)

34
Prewitt edge detector applied to a real image
35
The Sobel edge detector
The Sobel edge detector is a variant on the
Prewitt in which more prominence is given to on
axis pixels.
This detector generally performs better than the
Prewitt and is easy to implement in hardware.
As previously mentioned, in MatLab edge detectors
can be used with parameters BW
edge(I,'sobel',thresh,direction) Edges weaker
than thresh are not displayed and direction can
be horizontal, vertical or both (the default)
36
The Sobel detector in practice
37
The Canny edge detector
Noise in an image can lead to errors in edge
detection usually a proliferation of weak
edges. For single step edges in the presence of
Gaussian noise the Canny Edge detector can be
shown to perform optimally.
? Is the standard deviation of the Gaussian
filter. The gradient detector could be any
suitable. Non maximal suppression reduces all
detected edges to one pixel wide. tmin to tmax
defines the intensity range of edges detected.
The weaker edges are only included if they are in
the 4 or 8 neighbourhood of a strong edge.
38
The Canny detector in practice
39
Second derivative edge detector

• The second derivative seems to give the
  possibility of a more precise location of an edge
  by detecting the zero crossing point.

40
Using the Laplacian
The Laplacian operator provides a direction
insensitive second derivative edge detector.
Using the Taylor expansion we can derive 3 x 3
mask for this operator.
41
The Laplacian mask
This suggests for the Laplacian mask
The Laplacian is very rarely used by itself
because it is very sensitive to noise. Normally
it is used as part of the Laplacian of Gaussian
(LoG) filter. A Gaussian filter is used to reduce
noise then the Laplacian operator is used to
enhance edges and finally a zero crossing
detector is used to identify the edges.
42
LoG example
43
Tutorial

• In this weeks laboratory, investigate the
  following
• Create M-files to demonstrate the Matlab code
  given on these slides.
• Investigate the following Matlab commands (use
  the Matlab Help information) imfilter fspecial
  bwperim bwfill, edge.
• Run the Matlab demo edgedemo
• Write an M-file to measure the perimeter of
  apple, banana and orange in the images apple.jpg,
  banana.jpg and orange.jpg. Comment on how useful
  this measure would be when trying to identify
  whether an unknown fruit is an apple, orange or
  banana.
• Write another M-file to try and identify what
  fruit is in each image automatically, using any
  method you like (suggestion segment out the
  fruit, then examine its colour).