# Image deblocking using local segmentation - PowerPoint PPT Presentation

PPT – Image deblocking using local segmentation PowerPoint presentation | free to download - id: 5586c-ZWI0M

The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
Title:

## Image deblocking using local segmentation

Description:

### An image which is divided into 8x8 blocks has 64 possible sub-band images ... Combine multiple slightly different images to form one higher quality image ... – PowerPoint PPT presentation

Number of Views:160
Avg rating:3.0/5.0
Slides: 34
Category:
Tags:
Transcript and Presenter's Notes

Title: Image deblocking using local segmentation

1
Image deblocking using local segmentation
• Supervisor Dr. Peter Tischer

2
Presentation Outline
• An introduction to JPEG
• Lossy image compression
• Discrete Cosine Transform (DCT) and quantization
• Local segmentation and prior research
• Measuring image quality
• Deblocking Filter
• Super-Resolution Filter
• Conclusion

3
Lossy image compression
• JPEG is the most common lossy image compression
format
• Easy to implement
• Good quality with high compression ratios
• Block based transform approach
• Divide image into 8x8 blocks
• Perform discrete cosine transform (DCT) on each
8x8 block
• Quantize the DCT coefficients

4
Discrete Cosine Transform
DC coefficient
• Transform 8x8 block of pixels into a set of
weighted basis functions (DCT coefficients)

AC coefficients
5
DCT - Subbands
• An image consisting of just one coefficient from
each 8x8 block is called the sub-band image
• An image which is divided into 8x8 blocks has 64
possible sub-band images

An 8x8 image split into 2x2 blocks has 4 subbands
6
Quantization
• Exploit visual redundancy
• Quantization is a many-to-one mapping
• Divide each DCT coefficient by a value and round
to the nearest integer
• The decoder makes a guess from a range of values
(pick midpoint by default)

5 6 7 8 9 10 11 12 13 14
Quantizer 10
1
7
Quantization
• Coarse quantization introduces artifacts into
reconstructed image
• DCT coefficients are reconstructed inaccurately
• Most visually distracting artifact is blockiness

JPEG compressed image at quality 10 PSNR 30.41 dB
8
Deblocking Techniques
• Three approaches in the literature
• Filter the reconstructed pixel values
• Attempt to reconstruct DCT coefficients more
accurately
• A hybrid approach

9
Local Segmentation
• Most deblocking filters introduce excessive
blurring
• Destroys the structure of the image
• Edges lose their sharpness
• Local segmentation takes into account the
structure of the image

10
Local Segmentation
• Divide a mask of pixels into N segments and
filter each segment independently

Average of whole mask 43.22, average of yellow
segment 20.8
• Two questions
• How many segments do we use?
• How do we segment a mask of pixels?

11
Prior Research
• Lukasz Kizewski, BSE (hons) 2004
• DC subband approach
• Filter using a mask of DC subbands
• How do we segment a mask of pixels?
• Segment the pixel mask using thresholding
• How many segments do we use?
• Do-No-Harm heuristic

12
Prior Research
• Do-No-Harm heuristic
• Try a 1-segment model (average of the whole mask)
• If filtered value is implausible reject and try a
2-segment model
• A plausible value is one which falls inside the
quantization range midpoint /- ½ Quantum
• Try a 2-segment model
• If still implausible then dont filter

13
Prior Research
• Results

(a) Unfiltered image
(a) Filtered image
14
Room for improvement
• No objective measure used to test the
effectiveness of the filter
• Difficult to make comparisons
• Difficult to rate changes in filter
• Works only on DC subband
• AC subbands contain edge and texture information
• A very simple local segmentation method

15
Measuring image quality
• Peak-signal-to-noise-ratio
• Most commonly used metric
• Does not necessarily reflect the subjective
visual quality
• Generalized Block-Edge Impairment Metric (GBIM)
H.R. Wu, M. Yuen

16
Measuring image quality - GBIM
• Measures the quality of DCT encoded images
• Assume that what happens inside a block is the
same as what happens across blocks
• Take absolute mean difference of pixels inside a
block (vertical/horizontal)
• Take absolute mean difference of pixels across
blocks (vertical/horizontal)
• Compare them, if the two differ greatly than it
is a sign of blockiness

2x2 block example with vertical blockiness
17
A new deblocking filter
• Filter all coefficients
• Treat each 8x8 block as a 64 element vector where
each value in the vector is one of the DCT
coefficients
• Local segmentation no longer as simple (need to
• K-Means or K-Nearest Neighbours
• DNH needs to work on vectors

18
A new deblocking filter
• Basic structure of filter is same
• Create an NxN mask where each item is a 64
element DCT coefficient vector
• Segment mask using K-Means or K-Nearest
Neighbours segmentation
• Check if segmentation produces valid result using
DNH, if not, try different segment
• If no segmentation produces valid result, leave
alone

19
K-Means
• If segmentation is invalid, increase number of
segments by one until a maximum number of
segments is reached
• Try largest change first

20
K-Nearest Neighbours
• Set the number of nearest neighbours to find as
the number of items in the entire mask
• Keep decreasing by segment size by one until a
valid segment is found

21
Vector DNH
• Center DNH
• Compare the filtered vector against only the
• Strict Segment DNH
• Compare the filtered vector against all the
vectors in the segment the center vector is in
• Lesser Strict Segment DNH
• Same as strict segment DNH with some error
tolerance

22
Results
• Best found parameters for the filter
• 3x3 vector mask (covers 24x24 pixels)
• K-Nearest Neighbour segmentation
• Lesser strict segment DNH with 1 error tolerance
• Tested
• 5x5 mask, K-Means, Center DNH, Strict DNH etc.
• Many variations of the filter parameters

23
Results
(a) JPEG compressed image
(b) Filtered image
• An improvement of 0.07 dB in PSNR and a reduction
of 0.62 in GBIM

24
Results
(a) JPEG compressed image
(b) Filtered image
• An improvement of 0.07 dB in PSNR and a reduction
of 0.55 in GBIM

25
Other uses for local segmentation
• Super-resolution
• Combine multiple slightly different images to
form one higher quality image
image?
• Neighbouring pixels are similar and share
information
• Use local segmentation

26
Super-resolution filter
• Very similar to the deblocking filter
• Instead of using a 64 element vector for each DCT
coefficient, use one element vector containing
each pixel value in the mask
• How to test if it works?
• Convert 8 bit image to 4 bits and attempt to
reconstruct back an 8 bit image

27
Results
• Best found parameters for filter
• K-Means segmentation with up to three segments
• Center DNH
• More than one iteration of the filter can further
improve PSNR

28
Results
(a) An image rounded to 4 bits per pixel
(a) Filtered image (8 bits per pixel)
• An improvement of 1.38 dB in PSNR

29
Results
(a) An image rounded to 4 bits per pixel
(a) Filtered image (8 bits per pixel)
• An improvement of 1.32 dB in PSNR

30
Conclusion
• Filtering AC subbands is difficult because most
have been quantized to zero or have very large
quantization ranges
• Most improvements in image quality are from the
stricter DNH and better local segmentation
techniques
• The increase in computational complexity may not
be worth the increase in image quality

31
Conclusion
• Super-resolution filter shows a lot of promise
• Large increase in PSNR and image quality
• A different DNH heuristic may work better with
the super-resolution filter

32
Future Research
• Allow more variation in pixels for the strict DNH
• Assume local mask is linear and not constant
• Try different segmentation techniques
• Region growing
• Further investigate iterative filtering

33
The End
• Questions?