Adaptive Multi-path Prediction for Error Resilient H.264 Coding

1
Adaptive Multi-path Prediction for Error
Resilient H.264 Coding

• Xiaosong Zhou, C.-C. Jay Kuo
• University of Southern California
• Multimedia Signal Processing 2006

2
Outline

• Introduction
• Adaptive Multi-path Prediction for H.264
• Observation and Research Motivation
• Computation of Expected Decoder Distortion
• Adaptive Reference Selection (ARS) Scheme
• Experimental Results
• Conclusion and Future Work

3
Introduction

• Video communication problem

4
Introduction

• Error Resilient tools
• Error Resilient Entropy Coding (EREC)
• Unequal Error Protection by Layered Coding
• Mismatch exists, error propagation cant be
  stopped properly
• Not compatible with H.264

5
Introduction

• One way for reducing error propagation
• Intra Refreshing insert intra macroblocks in
  temporally coded (P or B) video frames
• Intra macroblocks in H.264 are coded by intra
  prediction based on neighbors
• Intra macroblock have much lower coding
  efficiency than inter macroblocks
• Not suitable for H.264

6
Introduction

• Multiple reference (long-term reference) motion
  compensation predictive coding (LTMCP)
• Enhance coding efficiency
• Proposed for error resilience
• T. Wiegand et. al, Error-resilient video
  transmission using long-term memory
  motion-compensated prediction
• Select the best reference frame by evaluating the
  expected reconstruction calculated based on the
  error feedback and an error propagation model

7
Observation and Research Motivation

• Sequential prediction
• In H.264, LTMCP allows encoder to choose the best
  prediction from a number of reference frames
• The best reference of some blocks may exist in a
  long-term reference frame
• But, sequential prediction is still common in
  H.264

8
Observation and Research Motivation

• Utilize long term reference frames for error
  resilience
• Error resilience performance is improved using
  alternative prediction patterns

Different predictive patterns
9
Observation and Research Motivation

• If a video stream is encoded into these fixed
  prediction patterns
• As most video frames are forced to use a distant
  reference frame, coding efficiency is likely to
  be sacrificed
• Performance vary for different macroblocks since
  its largely dependent on video content
• Its difficult to design a fixed prediction
  pattern at the frame level
• In H.264, reference frame selection is done at
  macroblock level
• Incorporate the idea of multi-path predictive
  coding at the macrokblock level

10
Computation of Expected Decoder Distortion

• Error map
• Created and maintained for each allowed reference
  frame in buffer
• Store the absolute value of the expected error e
  of every pixel in the frame

11
Computation of Expected Decoder Distortion

• can not be obtained directed
• For a pixel in the nth frame, update its value of
  e by
• pe channel error rate
• ep,n expected error from error propagation
• ec,n expected error from error concealment when
  pixel is lost

12
Computation of Expected Decoder Distortion

• To calculate ec,n, the mismatch caused by
  reconstruction of error concealment scheme
• Consider a simple error concealment method
• Intra block copy pixel from the boundary of
  correctly reconstructed block above the target
  block
• Inter block copy the block from the same
  position in the previous frame
• Should also consider de-blocking operation which
  attenuate the error generated by error
  concealment
• d
• en-1 error value of the pixel where MV points
  in reference frame
• an-1 attenuation factor of the propagating
  error from n-1st to nth frame
• Expected decoder distortion

13
Adaptive Reference Selection (ARS)

• in LTMCP, multiple predictions can be created to
  encode block M
• The predictions generated from multi-reference
  frames are evaluated based on both coding and
  error resilience performance

14
Adaptive Reference Selection (ARS)

• Three used vectors
• xN (X1, , XN) the set of all N MBs in a GOP
• mN (M1, , MN) all modes selected by each MB
• qN (Q1, , QN) quantization parameters used
  to encode these MBs
• The overall mode decision problem is

15
Adaptive Reference Selection (ARS)

• Convert to an unconstraint optimization problem
  using Lagrange Multiplier method

16
Adaptive Reference Selection (ARS)

• Overall expected distortion through erroneous
  channel
• expected error that has mean zero
• decoder error and uncorrelated to
• Therefore

17
Adaptive Reference Selection (ARS)

• Rewrite mode decision problem equation
• Assumption rate and
  distortion of MB i have no
  impact to other MBs
• expected overall distortion
  of the GOP due to error propagation

18
Adaptive Reference Selection (ARS)

• d
• a attenuation factor
• M expected number of frames in the future
  prediction path of the pixel
• The method to calculate M

19
Experimental Results
20
Experimental Results
21
Experimental Results

• Averaged Frame Index

22
Experimental Results

• R-D performance comparison under test condition T2

23
Experimental Results

• R-D performance comparison under test condition T7

24
Conclusion and Future Work

• An adaptive prediction selection scheme was
  proposed in this work to create multiple
  prediction paths in the compressed video stream
• The proposed scheme is able to maintain good
  coding efficiency of the compressed stream while
  serve as an effective error resilience tool in
  visual communication applications
• In the future, we plan to develop a new model to
  simplify the calculation of the expected decoder
  distortion to reduce the complexity of the
  proposed scheme