ERROR CONCEALMENT TECHNIQUES IN H.264/AVC, FOR VIDEO TRANSMISSION OVER WIRELESS NETWORKS - PowerPoint PPT Presentation

Loading...

PPT – ERROR CONCEALMENT TECHNIQUES IN H.264/AVC, FOR VIDEO TRANSMISSION OVER WIRELESS NETWORKS PowerPoint presentation | free to download - id: 6b58fe-NzYwN



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

ERROR CONCEALMENT TECHNIQUES IN H.264/AVC, FOR VIDEO TRANSMISSION OVER WIRELESS NETWORKS

Description:

ERROR CONCEALMENT TECHNIQUES IN H.264/AVC, FOR VIDEO TRANSMISSION OVER WIRELESS NETWORKS Vineeth Shetty Kolkeri University of Texas, Arlington – PowerPoint PPT presentation

Number of Views:41
Avg rating:3.0/5.0
Slides: 43
Provided by: utaEdufacu6
Learn more at: http://www.uta.edu
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: ERROR CONCEALMENT TECHNIQUES IN H.264/AVC, FOR VIDEO TRANSMISSION OVER WIRELESS NETWORKS


1
ERROR CONCEALMENT TECHNIQUES IN H.264/AVC, FOR
VIDEO TRANSMISSION OVER WIRELESS NETWORKS
  • Vineeth Shetty Kolkeri
  • University of Texas, Arlington

2
Outline
  • Introduction
  • What is Error Concealment?
  • Error Concealment Architecture
  • H.264/MPEG-4 AVC Overview
  • Error Concealment algorithm
  • Error Concealment performance analysis test
    results
  • Conclusions
  • Future Work
  • References

3
Introduction
Figure 1 Typical Situation of 3G/4G cellular
telephony
4
What is Error Concealment?
  • The operation adopted to reconstruct lost
    information for video transmission over wireless
    networks.
  • Need Compatible with all video streaming
    devices.
  • Applications Recovery of lost information which
    is comparable with encoded video sequence with
    minimal complexity.

5
Error Concealment Architecture
6
Overview of H.264 / AVC
  • Latest Video coding standard
  • Basic design architecture similar to MPEG-x or
    H.26x
  • Better compression efficiency
  • Upto 50 bitrate reduction from the preceding
    video codec standard
  • Subjective quality is better
  • Advanced functional element
  • Wide variety of applications such as video
    broadcasting, video streaming, video
    conferencing, D-Cinema, HDTV.
  • Layered structure - consists of two layers
    Network Abstraction Layer (NAL) and Video Coding
    Layer (VCL) supports 420 chroma sampling
    picture format including QCIF and CIF formats

7
Overview of H.264 / AVC (contd.)
  • Uses hybrid block based video compression
    techniques such as
  • Transformation - reduction of spatial correlation
  • Quantization - bit-rate control
  • Motion compensated prediction - reduction of
    temporal correlation
  • Entropy coding - reduction in statistical
    correlation
  • Includes the following features
  • Intra-picture prediction
  • 4x4 integer transform
  • Multiple reference pictures
  • Variable block sizes
  • Quarter pel precision for motion compensation
  • In-loop de-blocking filter
  • Improved entropy coding

8
H.264/AVC Profiles
  • Profiles and Levels for particular applications
  • Profile a subset of entire bit stream of
    syntax,
  • different decoder design based on the
    Profile
  • Four profiles Baseline, Main, Extended and High

Applications
Profile
Video Conferencing Videophone
Baseline
Digital Storage Media Television Broadcasting
Main
Streaming Video
Extended
High
Studio editing
9
Specific coding parts for the Profiles
10
Specific coding parts for the Profiles (contd.)
  • Common coding parts for the Profiles
  • I slice (Intra-coded slice) the coded slice by
    using prediction only from decoded samples within
    the same slice
  • P slice (Predictive-coded slice) the coded
    slice by using inter prediction from
    previously-decoded reference pictures, using more
    than one motion vector and reference index to
    predict the sample values of each block
  • CAVLC (Context-based Adaptive Variable Length
    Coding) for entropy coding

11
Specific coding parts for the Profiles (contd.)
  • Coding parts for Baseline Profile
  • Common parts I slice, P slice, CAVLC
  • FMO Flexible macroblock order macroblocks may
    not necessarily be in the raster scan order. The
    map assigns macroblocks to a slice group
  • ASO Arbitrary slice order the macroblock
    address of the first macroblock of a slice of a
    picture may be smaller than the macroblock
    address of the first macroblock of some other
    preceding slice of the same coded picture
  • RS Redundant slice This slice belongs to the
    redundant coded data obtained by same or
    different coding rate, in comparison with
    previous coded data of same slice

12
H.264 Encoder (contd.)
13
H.264 - Transform
  • 4x4 multiplier-free integer transform
  • Transform coefficients perfectly invertible
  • Hierarchical structure - 4 x 4 Integer DCT,
    Hadamard transform
  • Hadamard transform applied when (16x16) intra
    prediction mode is used with (4x4) integer DCT
  • MB size for chroma depends on 420, 422 and
    444 formats

14
H.264 Transform (contd.)
YUV different formats
  • 444 is full bandwidth YUV video, and each
    macroblock consists of 4 Y blocks, and 4 U/V
    blocks. Being full bandwidth, this format
    contains as much as data would if it were in the
    RGB color space.
  • 422 contains half as much chrominance
    information as 444 and 420 contains one
    quarter of the chrominance information.

15
H.264 Transform (contd.)
  • Codec - A video codec is software that can
    compress a video source (encoding) as well as
    play compressed video (decompress).
  • CIF - Common Intermediate Format - a set of
    standard video formats used in videoconferencing,
    defined by their resolution. The original CIF is
    also known as Full CIF (FCIF).
  • QCIF - Quarter CIF (resolution 176x144)
  • SQCIF - Sub quarter CIF (resolution 128x96)
  • 4CIF - 4 x CIF (resolution 704x576)
  • 16CIF - 16 x CIF (resolution 1408x1152

16
H.264 - Scaling and Quantization
  • Multiplication operation for exact transform
    combined with multiplication of scalar
    quantization
  • Scale factor for each element in each sub-block
    varies as a function of quantization parameter
    associated with macro-block that contains sub
    block position of element within sub-block
  • Rate-control algorithm in encoder controls value
    of quantization parameter
  • Encoder performs post-scaling and quantization

17
Deblocking filter Adaptive
  • To reduce the blocking artifacts in the block
    boundary and prevent the propagation of
    accumulated coded noise.
  • Filtering is applied to horizontal or vertical
    edges of 4 x 4 blocks in a macroblock, adaptively
    on the several levels (slice, block-edge,
    sample).

18
H.264 Motion Compensation
  • Predicted rectangular arrays of pixels 4x4, 4x8,
    8x4, 8x8, 16x8, 8x16, and 16x16.
  • Translation from other array positions in
    reference picture specified with quarter pixel
    precision.
  • In case of 420 format, the chroma MVs have a
    resolution of 1/8 of a pixel, derived from
    transmitted luma MVs of 1/4 pixel resolution.

19
Prediction of variable block size
  • A MB can be partitioned into smaller block sizes
  • 4 cases for 16 x 16 MB, 4 cases for 8 x 8 Sub-MB
  • Large partition size homogeneous areas, small
    detailed areas

20
Prediction of variable block size (contd.)
Frame divided into multiple macroblocks of 16 x
16, 8 x 8, 4 x 4 variable size to represent
coding profiles
No. of bits in I and P frames
I P I
Graph shows the size of the different I and P
frames obtained after encoding 19 frames of the
Foreman QCIF video sequence. Green line shows the
average values of the bit lost when it is passed
through the lossy algorithm after encoding in a
video sequence
21
H.264 Entropy Coding
  • CAVLC (Context-based Adaptive Variable Length
    Coding).
  • CABAC (Context-based Adaptive Binary Arithmetic
    Coding).
  • CAVLC makes use of run-length encoding.
  • CABAC utilizes arithmetic coding codes both MV
    and residual transform coefficients.
  • Typically CABAC provides 10-15 reduction in bit
    rate compared to CAVLC, for the same PSNR.
  • All syntax elements are encoded by Exp-Golomb
    codes (Universal Variable Length Codes (UVLC)).

22
Error Concealment Algorithm
  • 1.Spatial Concealment weighted averaging
  • Estimate missing pixels by smoothly extrapolating
    surrounding pixels
  • Correctly recovering missing pixels is extremely
    difficult, however correctly estimating the DC
    (average) value is very helpful
  • 2.Temporal Concealment copy algorithm
  • Copy the pixels at the same spatial location in
    the previous frame
  • Effective when there is no motion, potential
    problems when there is motion
  • 3.Motion compensated temporal Concealmentmotion
    vector interpolation
  • Estimate missing block as motion-compensated
    block from previous frame
  • Can use coded motion vector, neighboring motion
    vector, or compute new motion vector

23
Spatial Concealment weighted averaging
Block based weighted averaging
24
Spatial Concealmentweighted averaging (contd.)
Macroblock based weighted averaging
25
Spatial Concealment weighted averaging (contd.)
Recovery of the damaged macroblock in Foreman and
Akiyo video sequence (a) distorted image lying
within a smooth area b) macroblock based
weighted averaging applied on a white smooth
area c) block based weighted averaging applied
on a white smooth area.
26
Spatial Concealment weighted averaging (contd.)
Recovery of the damaged macroblock in Foreman and
Akiyo video sequence (a) distorted image lying
within a smooth area b) macroblock based
weighted averaging applied on a white smooth
area c) block based weighted averaging applied
on a white smooth area.
27
Temporal Concealment Frame Copy
Frames 5, 6 and 7 are the output of H.264
encoded frames after it is transmitted in the
error prone wireless medium
Frame 5 is the decoded frame. Here Frame 6
successfully copied lost information from Frame 5
by copy algorithm Frame 7 is degraded (Because
Frame7 is reconstructed bycollecting the
information from previous reference frames)
28
Temporal Concealment MV Interpolation
Motion vector recovery by a) Using the motion
vectors from the surrounding macroblocks after
frame decoding b) Using the motion vectors from
the surrounding macroblocks during macroblock
decoding
29
Temporal Concealment MV Interpolation (contd.)
Four Prior-decoded pictures
Current Picture as references
30
Motion Vector Interpolation
Frames 5, 6 and 7 of the Original Sequence
Frame 5 of the decoded frame, Successfully
decoded lost Frame 6. Frame 6 was
reconstructed by Motion Copy algorithm. Frame 7
is degraded.
31
Motion Vector Interpolation (contd.)
Recovery of the damaged macroblock in Foreman
video sequence (a) original sequence b) Distorted
Sequence c) Concealed Output using Motion
Estimation.
32
Error Concealment performance analysis test
results (Foreman Sequence)
33
Error Concealment performance analysis test
results (contd.) (Foreman Sequence)
34
Error Concealment performance analysis test
results (contd.) (Foreman Sequence)
35
Error Concealment performance analysis test
results (contd.) (Foreman Sequence)
36
Error Concealment performance analysis test
results (contd.)
Simulation results of different error concealment
algorithms for Foreman QCIF176x144 video sequence.
37
Error Concealment performance analysis test
results (contd.)
Simulation results of different error concealment
algorithms for Foreman QCIF176x144 video sequence.
38
Conclusions
  • Implementation of spatial concealment performed
    better in a video with constant background.
  • Implementation of temporal concealment performed
    better in a video with linear motion between
    consecutive frames.
  • Implementation of temporal concealment performed
    better in a video with dynamic motion between
    consecutive frames.
  • At higher bit rates spatial and temporal
    concealments achieved better results.
  • Complexity of implementation is negligible and
    does not degrade in the processor performance.

39
Future Research
  • Implementing Error Concealment algorithm in
    H.264/SVC video codec as it doesnt support error
    concealment in current implementation.
  • Use forward and backward MVs.
  • Implementing Error Concealment algorithm in H.264
    extended and High profiles.
  • Find a block which has uniform motion within
    small duration.
  • Implementing Error Concealment model for real
    time application like video surveillance.

40
References
  • T. Stockhammer, M. M. Hannuksela and T. Wiegand,
    H.264/AVC in Wireless Environments, IEEE Trans.
    Circuits and Systems for Video Technology, Vol.
    13, pp. 657- 673, July 2003.
  • 2. Soon-kak Kwon, A. Tamhankar and K.R. Rao,
    Overview of H.264 / MPEG-4 Part 10, J. Visual
    Communication and Image Representation, vol. 17,
    pp.186-216, April 2006.
  • 3. S. Wenger, H.264/AVC over IP IEEE Trans.
    Circuits and Systems for Video Technology, vol.
    13, pp. 645-656, July 2003.
  • 4. M. Wada, Selective Recovery of Video
    Packet Loss using Error Concealment, IEEE
    Journal on Selected Areas in Communication, vol.
    7, pp. 807-814, June 1989.
  • 5. I.C.Todoli Performance of Error
    Concealment Methods for Wireless Video, Diploma
    Thesis, Vienna University of Technology, 2007 .
  • 6. Video Trace research group at ASU, YUV
    video sequences, http//trace.eas.asu.edu/yuv/ind
    ex.html.
  • 7. A.B. Watson, "Toward a perceptual video
    quality metric", SPIE Human Vision, Visual
    Processing, and Digital Display VIII, vol. 3299,
    pp 139-147, 1998.
  • 8. F. Xiao, DCT-based video quality
    evaluation, Final Project for EE392J Stanford
    Univ. 2000. http//compression.ru/video/quality_me
    asure/vqm.pdf
  • Z. Wang, The SSIM index for image quality
    assessment, http//www.cns.nyu.edu/zwang/files/re
    search/ssim/.

41
References (contd.)
  • 10. Z. Wang, et al, Image Quality Assessment
    From Error Visibility to Structural Similarity,
    IEEE Trans. Image Processing, vol. 13,
    pp.600-612, April 2004.
  • 11. ISO/IEC JTC1, Joint Draft 8 of SVC
    Amendment, ISO/IEC JTC1/SC29/WG11 and ITU-T SG16
    Q.6, Doc. JVT-U201, Oct. 2006.
  • 12. ISO/IEC JTC1, Joint Scalable Video Model
    8.0, ISO/IEC JTC1/SC29/WG11 and ITU-T SG16 Q.6,
    Doc. JVT-U202, Oct. 2006.
  • 13. Yi-Hau Chen, et al, Bandwidth-efficient
    encoder framework for H.264/AVC scalable
    extension Ninth IEEE International Symposium on
    Multimedia, pp 401-406, Dec 2007.
  • 14. DSP Design Line article on SVC
    http//www.dspdesignline.com/products/206902239.
  • 15. eInfochips HD Codecs - H.264 SVC - for
    Digital Media Processors From Texas
    Instruments
  • 16. H.264/AVC Reference Software Download
  • http//iphome.hhi.de/suehring/tml/download/
  • 17. H.264/SVC Reference Software download
    http//ip.hhi.de/imagecom_G1/savce/downloads/SVC-R
    eference-Software.htm
  • 18. AVI to YUV converter http//www.sunrayimage.c
    om/

42
  • Thanks for your attention!
  • Q/A
About PowerShow.com