Visual Artifact Reduction By PostProcessing - PowerPoint PPT Presentation

1 / 69
About This Presentation
Title:

Visual Artifact Reduction By PostProcessing

Description:

Visual artifacts in digital image and video. Post-processing Method. Enhancement. Restoration ... Image and video processing procedure applied after normal ... – PowerPoint PPT presentation

Number of Views:347
Avg rating:3.0/5.0
Slides: 70
Provided by: YuHe8
Category:

less

Transcript and Presenter's Notes

Title: Visual Artifact Reduction By PostProcessing


1
Visual Artifact Reduction By Post-Processing
Presented at Carnegie Mellon University Dept.
ECE, November 1, 2001
  • Yu Hen Hu (hu_at_engr.wisc.edu)
  • University of Wisconsin -- Madison
  • Dept. Electrical and Computer Engineering
  • in collaboration with Dr. Seungjoon Yang

2
Outline
  • Visual artifacts in digital image and video
  • Post-processing Method
  • Enhancement
  • Restoration
  • Blocking Artifact Reduction using GenLOT-embedded
    Inverse DCT
  • Ringing Artifact Reduction Using Robust ML filters

3
Visual Artifacts in Digital Visual Materials
  • Digital Images and Videos
  • Obtained by digitizing analog visual materials or
    direct capture.
  • Visual quality remain unchanged after copying.
  • Visual Artifacts in digital images and videos
  • Artifacts due to inadequate acquisition
  • Out of focus, smearing, dust, scratch, blotches
  • Artifacts due to inadequate processing
  • Lossy compression coding artifact
  • Watermarking
  • Artifacts due to transmission error
  • Lost blocks, frames, discoloring

4
Types of Visual Artifacts
  • Spatial artifacts
  • Content independent
  • Blotches
  • Scratches
  • Additive noise
  • Content dependent
  • Blocking
  • Ringing
  • Smearing
  • Temporal artifacts
  • Frame jittering
  • Successive image frames at slightly different
    spatial positions
  • Line jittering
  • Individual raster line mis-aligned, causing
    vertical straight line jitters
  • Rapid, uneven camera motion
  • Uneven panning, tracking, etc.
  • Discontinued motion of objects
  • Due to frame skipping

5
Coding Artifacts
  • Depending on the type of the transform

(a) Blocking Artifact
(b) Ringing Artifact
Non-Overlapping Transforms
Overlapping Transforms
6
Visual post-processing
  • Image and video processing procedure applied
    after normal processing (including compression,
    decompression), prior to final presentation.
  • Purpose to enhance visual quality of images and
    videos by removing unsightly visual artifacts.
  • Artifact reduction is accomplished in two phases
  • Artifact detection
  • Artifact removal

7
Artifact Detection
  • A statistical signal detection problem.
  • Exploit unique feature of different types of
    artifacts
  • Difficult for content dependent artifacts
  • Temporal artifact detection must be performed by
    examining consecutive frames (spatial-temporal
    analysis)
  • Must exploit human visual system (HVS)
    characteristics
  • Visual masking effect

8
Artifact Reduction
  • Two steps
  • Visual quality enhancement reducing unsightly
    artifact
  • Restoring original content by reversing the
    degradation process
  • Need to estimate lost information
  • Artifact may cover original contents e.g.
    blotches
  • Artifact may be due to lost of original content
    e.g. coding artifact

9
Transformed Image Coding Artifact
ReductionProblem Statements
10
Transformed Image Coding
f
F
Fq
Transform
Quantizatoin
Lossless Coding
Bit Allocation
(a) Encoder
Fq
g
Lossless Decoding
Dequantization
Inverse Transform
(b) Decoder
11
Coding Artifacts
  • Blocking artifact
  • Cause
  • Lossy quantization of frequency coefficients
  • Symptom
  • Spurious edges at block boundary (known position)
  • Issues
  • preserving true edges that cross or overlap with
    block boundary
  • Common approaches
  • spatial domain filtering
  • DCT coefficients adjustment
  • Ringing Artifact
  • Cause
  • Lossy quantization of frequency coefficients
  • Symptom
  • Spurious edges along major edges
  • Issues
  • difficult to separate ringing artifact with true
    edge of texture
  • Common approaches
  • Restoration while preserving major edges
  • Medium filtering

12
Blocking Artifact
  • The edge map is enhanced to show spurious edges
    along block boundaries.
  • Visual masking effect Blocking artifact is more
    visible at relatively flat region of an image.

13
Blocking Artifact
  • With block-based transform coding, pixels
    acrossing the block boundary are encoded with
    different set of basis functions.
  • Reconstructed Block (g pixel value, Fq freq.
    Coef., ? basis)
  • Discontinuity at Block Boundary
  • where
  • If the basis function overlaps between adjacent
    blocks, the blocking effect may be alleviated.

14
Ringing Artifact
  • Ringing Artifact Oscillation (spurious edges) at
    the Vicinity of major (high contrast, large
    scale) edges
  • Visual masking effect prominent in areas with
    relatively smooth background

Ringing artifact Enhanced edge map
15
Cause of Ringing Artifact
  • Gibbs phenomenon -- Truncation of frequency
    domain coefficients corresponds to convolving the
    time domain sequence with a sinc function. The
    side lobes manifest themselves as oscillations in
    spatial domain around step discontinuities.
  • The cause of the ringing artifact is similar to
    that the of Gibbs phenomenon when long basis
    functions are cut short due to heavy lossy
    quantization

Truncation of DCT coefficients from 128 down to
32 leads to ringing artifact Original is a step
function. With only 32 low frequency coefficients
left, ringing occurs.
16
New Approaches
  • Blocking Artifact Reduction using
    GenLOT-embedded IDCT
  • Embedding DCT/IDCT in a more general lapped
    orthogonal filter bank transformation.
  • Modifying the GenLOT coefficients to reduce
    blocking artifact.
  • Performed only at decoding end without altering
    encoding process.
  • Maximum Likelihood estimation of ringing
    artifact free image
  • Use a flat-surface image model to distinguish
    true edges from ringing edges that have smaller
    magnitudes.
  • Use ML method to estimate true image value within
    a sliding window.
  • Implemented as a data-adaptive, nonlinear robust
    filter

17
Blocking Artifact Reduction using Embedded IDCT
S. Yang, S. Kittitornkun, Y. H. Hu, T. Q. Nguyen,
and D. L. Tull, Generalized Lapped Biorthogonal
Transform embedded inverse discrete cosine
transform, IEEE Trans. Image Processing, pp.
submitted, 2000.
18
Existing Blocking Artifact Reduction Methods
  • MAP based restoration Approach ORourke
    Stevenson, 95
  • POCS (Projection onto convex set)
  • Constraints can be imposed to ensure the
    quantized bit stream of smoothed image is close
    to that of decoded image. E.g. POCS Yang,
    Galatsanos, Katsaggelos, 95
  • Location-specific smoothing (filtering)
  • Replacing step edges along block boundary by
    smoothed pixel values. Nonlinear smoothing may be
    used (H.263, annex J)
  • Can be applied in spatial as well as frequency
    domain
  • Optimization Approach Minami Zakhor, 95

19
MAP Estimation
  • Given g, find f such that

Feasible Image Set
Prior Knowledge on Estimate
Improved Image Decompression for Reduced
Transform Coding Artifacts, T.P. O'Rourke and
R.L. Stevenson
20
MAP Estimation
  • Gibbs Image Prior Distribution
  • With Gibbs Image Prior Distribution

c clique
? potential function
21
MAP Estimation with Line Process
  • Take away the option for discontinuity at block
    boundaries.

?
Line Process
At block boundaries
22
POCS
  • Find f such that
  • Actual Implementation

Projection-Based Spatially Adaptive
Reconstruction of Block-Transform Compressed
Images, Y. Yang and N.P. Galatsanos
and A.K. Katsaggelos
23
Nonlinear Filtering
  • Deblocking option in H.263 Annex J

where d1,d2,clip() are designed for appropriate
smoothing
Annex J of H.263,
24
Blocking Artifact Reduction using Overlapped
Transformation
  • Blocking artifact is due to block-based transform
    of images.
  • At low bit-rate compression, many frequency
    coefficients are quantized to zero
  • Overlapped Orthogonal Transform (LOT) compute
    frequency coefficients of the same block size
    using longer bases that overlap adjacent blocks.
    As a result, blocking effect is less prominent
    using LOT in place of DCT
  • However, LOT is not standard-compliant!
  • Our approach
  • Realize blocking artifact reduction in overlapped
    transform coefficient domain
  • Maintain standard compliance by embedding
    DCT/IDCT pair within an overlapped transformation.

25
Overlapped and Non-overlapped Transforms
  • 1D linear transform projection of 1D signal
    onto basis function of the transform.
  • Base on how the basis functions are interlaid,
    can be classified into overlapped and
    non-overlapped transform.
  • Let
  • Non-Overlapping Transforms (e.g. blocked DCT)

(b) Overlapping Transforms, e.g. LOT
26
Generalized Lapped Biorthogonal Transform (GLBT)
Transformed coefficients from adjacent blocks
T
G0(z)
GK-1(z)
GK-1-1
Forward Transform
Inverse Transform
The Generalized Lapped Biorthogonal Transform,
T.D. Tran and T.Q. Nguyen
27
DCT is Part of GLBT
DCT
GLBTs Frist Stage
T
28
GLBT Embedded IDCT (ge-IDCT)
Standard non-compliant
Forward Transform
Inverse Transform
Tdct-1
G
G-1
Tdct
Standard compliant
DCT
ge-IDCT
29
ge-IDCT based Transform
  • ge-IDCT is standarad compliant!

Tdct
Tdct-1
G
G-1
Quantization
Same as standard DCT based Encoder
Ge-IDCT based Decoder
30
Blocking Artifact Reduction in Embedded Lapped
Transform Domain
Perform blocking artifact reduction in GLBT
coefficient domain.
Tdct
Tdct-1
Post-Processing
G-1
Quantization
G
Standard Encoder
Improved Decoder
31
Deblocking
  • Blocking Artifact Discontinuity between 2 DCT
    blocks
  • Empirically, odd-symm. frequency coefficients
    contribute more to blocking effects.

32
Frequency Weighting
  • Solution
  • reduce excessive energy in odd-sym. coef.s.
  • Method
  • After lapped transform G of DCT coefficients,
    apply weights on two lowest odd-symm.
    coefficients
  • Features
  • Selective Smoothing
  • Preservation of Major Structure Robustness

33
New Inverse Transforms
(a) ge-IDCT
(b) le-IDCT
34
PSNR MSDS
  • Peak Signal to Noise Ratio
  • Mean Square Difference of Slope

An Optimization Approach for Removing Blocking
Effects in Transform Coding, S. Minami and A.
Zakhor
35
Evaluation PSNR MSDS
PSNR improvement
MSDS reduction
(a) Airplane
(b) Lena
(c) Peppers
36
Evaluation Subjective Quality
(a) JPEG
(b) ge-IDCT
At quality factor 15
37
Comparative Study
  • Existing Methods
  • Maximum a posteriori (MAP) Estimation (MAP)
  • Projection onto Convex Set (POCS)
  • Nonlinear Filtering (NF)
  • We also implement
  • MAP Estimation with Line Process (MAP-L)

38
Comparison PSNR MSDS
(a) PSNR
(b) MSDS
39
Comparison Less Over-smoothing
MAP
ge-IDCT
MAP-L
40
Comparison Less Under-smoothing
POCS
le-IDCT
NF
41
Comparison Simplicity
  • Unitary transformation in ge-IDCT and le-IDCT can
    be implemented with Lattice Structures and
    CORDIC.

Iterative Algorithms
Non-Iterative Algorithms
MAP, MAP-L, POCS
NF, ge-IDCT, le-IDCT
42
Summary
  • New Inverse Transforms
  • (DCT, ge-IDCT) replaces (DCT, IDCT)
  • Processing in Embedded Lapped Transform Domain
  • Effective and Efficient Removal of Blocking
    Artifact

43
Ringing Artifact ReductionUsingMaximum
Likelihood Robust Filtering
S. Yang, Y. H. Hu, D. L. Tull, and T. Q. Nguyen,
Maximum likelihood parameter estimation for
image ringing artifact removal, IEEE Trans.
Circuits and Systems for Video Technology, vol.
11, 2001, (to appear).
44
MAP Estimation
  • Wavelet Bit plane Coding Based Codecs.
  • MAP Estimation
  • Redesending Functions

45
Feasible Image Set
  • For Bit Plane Coding
  • Feasible Image Set

46
Approximated MAP Estimation
  • Enforcement of the Feasible Image Set
  • Replacement

Artifact Removal in Low Bit Rate Wavelet
Coding with Robust Nonlinear Filtering,
M. Shen and C.C.J. Kuo
47
Morphological Filter
  • Detection Phase
  • Removal Phase

Image coding ringing artifact reduction using
morphological post-filtering, part I the
algorithm, S.H.Oguz, Y.H.Hu, and T.Q.Nguyen
48
Flat Surface Model
  • Replace ripped surface with flat surface.
  • Flat Surface Model Images are montage of flat
    surfaces.
  • Each surface consists of a cluster of pixel
    intensity values.

K the number of surfaces
? averaged grayscale value of each surface
z surface information (membership of individual
pixel)
49
Examples
50
ML Parameter Estimation
  • Estimate model parameter from the samples.
  • Maximum Likelihood Parameter Estimation

A pair of (G,z) is regarded as complete data G
pixel values Z membership of each pixel to a
particular surface (cluster)
51
k-means Algorithm
  • K-means clustering is an implementation of the EM
    (expectation-maximization) Algorithm
  • k-Means Algorithm

Expectation Maximization
52
Hierarchical K-Cluster Model
  • Similarity between Two Clusters
  • Insensitivity
  • Modification of Vapniks ?-Insensitive
    Function

? variance
? cluster center
53
Hierarchical K-Cluster Model
  • Cluster Similarity Measure (CSM)
  • Optimal Number of Clusters

54
Hierarchical K-Cluster Model
  • Algorithm
  • Step1 For LKmax to 2 Run k-means algorithm
    with L clusters. Measure the max similarity
    between cluster pairs. Calculate CSM(L) Merge
    two most similar clusters.
  • Step 2 Determine the number of clusters K.
  • Step 3 Run k-means algorithm with K clusters.

55
Example
56
Example
57
Three Cluster Model
  • Cluster Validation leads to a complex algorithm.
  • Simplify to a Three Cluster Model
  • Reduce the k-means algorithm to one iteration.

Gc the center pixel of G
58
Robust Filter
  • Let C(i,j) be index set of pixels in G centered
    at (i,j)
  • Define the set
  • Define the function
  • ML Estimation with

59
Applications
Lossless Decoding
Dequantization
Inverse Transform
(a) Standard Decoder
Lossless Decoding
Dequantization
Inverse Transform
Post- Processing
(b) Improved Decoder
60
VRM
  • Visible Ringing Measure
  • Average Local Variance around Major Edges

(a) Image
(b) Edge Map
(c) Filtering Mask
(d) VRM Mask
Image coding ringing artifact reduction using
morphological post-filtering, part I the
algorithm, S.H.Oguz, Y.H.Hu, and T.Q.Nguyen
61
Evaluation PSNR VRM
(a) PSNR
(b) VRM
62
Evaluation Subjective Quality
JPEG2000
K-means
RF
At 0.125 bpp
63
Comparative Study
  • Existing Methods
  • Approximated MAP Estimation (MAP-A)
  • Morphological Filtering (MF)
  • We implement
  • MAP Estimation (MAP)

64
Comparison PSNR VRM
(a) PSNR
(b) VRM
65
Comparison Miss of MF
(b) Robust Filter
(a) Morphological Filter
66
Comparison Staircase Effect of MAP-A
(a) Approximated MAP
(b) Robust Filter
67
Comparison Simplicity
(a) Memory Usage in byte
(b) Execution Time in sec
RF
MF
MAP-A
With JPEG2000 compressed 2048x2560 size images
at 0.125 bpp
On 333MHz Dual Pentium,512M Ram, WindowsNT, in
sec
68
Summary
  • ML Model Parameter Estimation
  • Hierarchical Clustering Algorithm
  • k-means algorithm
  • CSM
  • Robust Filter
  • Based on three cluster model
  • New Modeling of Pgf
  • Effective and Efficient Removal of Ringing
    Artifact

69
Conclusion
  • Post-processing can effectively reduce image
    coding artifact.
  • An embedded LOT based blocking artifact reduction
    algorithm is presented.
  • A maximum likelihood based robust filter is
    proposed to reduce ringing artifact
  • These coding artifact reduction methods are
    amenable for hardware implementation for embedded
    applications in digital cameras as well as
    digital photo printers.

The ppt file can be downloaded from http//www.ec
e.wisc.edu/hu/postproc.ppt
Write a Comment
User Comments (0)
About PowerShow.com