Fine%20Granularity%20Video%20Compression%20and%20Optimal%20FEC%20Assignment%20for%20FG%20Video%20Streaming%20over%20Burst%20Error%20Channel

1
Fine Granularity Video Compression and Optimal
FEC Assignment for FG Video Streaming over Burst
Error Channel

• Yih-Ching Su
• Department of Computer Science and Engineering,
  National Sun Yat-Sen University

2
Contents

• Introduction
• Gilbert Channel with Loss Rate Feedback
• Optimal FEC Assignment for FG Video
• HSDD Motion Estimation Metric
• HMRME Motion Estimation Algorithm
• ABEC Embedded Coder
• Conclusions Future Works

3
1. Introduction
4
Research Focuses

• Optimal FEC assignment scheme for FG video
  transmission over burst error channel (as
  wireless Internet) with or without loss rate
  feedback.
• Wavelet domain video compression algorithms with
  high-performance or low-complexity features.

5
Research Focuses (cont.)
ABEC
Source Coder
Motion Estimation
Transform
Quantization Entropy Coding
Raw Video
HSDD HMRME
Channel Coder
FEC Protection
Error-Resilient Video Packets
Optimal FEC Assignment
6
Definition of Fine Granularity Video Stream

• Bit stream is scalable (layered).
• Rate can be precisely controlled.

7
Merits of Fine Granularity Video Stream

• Precise rate control
• Bandwidth adaptation

EL
Client
BL
Media Server
FG Video Encoder
Heterogeneous Internet Environment
EL
FG
BL
EL
Client
No transcoding!
BL
8
Merits of Fine Granularity Video Stream (cont.)

• Content-adaptive error protection

BL
Unequal Error Protection
Equal Error Protection
9
Fine Granularity Video Compression Systems

• DCT based
• MPEG-4 FGS
• ISO/IEC 14496-22001/Amd 22002
• Base layer plus enhancement layer
• DWT based
• Multirate 3-D subband coding of video, D.
  Taubman et al., 1994.
• 3D SPIHT, B.-J. Kim et al., 2000.
• HSDD, Y.-C. Su et al., 2003.

10
2. Gilbert Channel with Loss Rate Feedback
11
Packet Loss

• Packet loss can severely affect the quality of
  delay sensitive multimedia applications.
• FEC (Forward Error Correction) technique can be
  used when delay time is strictly restricted.

BOP len n pkts
data
FEC redundancy
data len k pkts
12
Gilbert Channel Model

• The ability of the application to react is
  enhanced by the availability of simple and
  efficient loss models.
• A two state Markov model or Gilbert-model is
  often used to simulate burst loss patterns over
  wired/wireless channel.

C. C. Tan, N. C. Beaulieu, On first-order Markov
modeling for the rayleigh fading channel, IEEE
Commun., 2000.
13
Enhanced Video Transmission over Gilbert Channel

• Feedback loss rate.
• Decide FEC protection ratio relying on a new
  probability function which is conditioned on loss
  rate feedback.

14
Renewal Error Process
Gap probabilities

• Packet loss over Gilbert-model can be modeled
  with a renewal error process.
• The lengths of consecutive inter-error intervals
  (also called gaps) are independently and
  identically distributed.

Probability that m-1 packet losses occur in
the next n-1 packets following an error
Probability that m packet losses occur within a
block of n packets
E. N. Gilbert, "Capacity of a burst-noise
channel," Bell Syst. Tech. J., vol.39,
pp.1253-1265, Sept. 1960. E. O. Elliott, "A model
of the switched telephone network for data
communications," Bell Syst. Tech. J., 1965.
15
Probability Toolbox
16
Probability Toolbox (cont.)
17
Probability Toolbox (cont.)
n
E
E
n
E
S
n
S
S
n
S
E
18
Probability Toolbox (cont.)
19
Iterative Equation Set
20
Initial Conditions
21
Conditional Probability Function
22
Validation of Correctness
23
Performance Evaluation
24
3. Optimal FEC Assignment for FG Video
25
FEC Assignment Schemes

• Equal error protection
• Content-adaptive unequal error protection
• Content-adaptive plus channel-adaptive unequal
  error protection

26
Block of Packets (BOP) Structure
27
Complete Expected Quality
28
Simplified Expected Quality
29
The Optimization Problem
Constrained by
30
Dynamic Programming
31
Validation of Correctness
(i) frame resolution CIF format (352x288) (ii)
constant stream rate 256 Kbps (iii) 1 GOP 1
intra frame accompanied with 14 inter frames and
frame rate 15 fps (iv) sequence length 9 GOPs
32
Performance Discrepancy between Complete
Simplified Models
33
Performance Evaluation
34
Performance Evaluation (cont.)
35
Performance Evaluation (cont.)
36
Performance Evaluation (cont.)
37
Performance Evaluation (cont.)
38
4. HSDD Motion Estimation Metric
39
Bit-Plane Coding

• The Core of FGS or Embedded Coder
• Just bit-plane coding!

40
Zero-Tree Coding

• Natural images in general have a low pass
  spectrum.
• Large wavelet coefficients are more important
  than small wavelet coefficients.
• A zero-tree is a quad-tree of which all nodes are
  equal to or smaller than the root.

41
Hierarchical Sum of Double Difference Metric

• Zero-tree coding aware
• Jointly constrain motion vector searching for
  both temporal and spatial (quad-tree) directions
• Fewer bits are spent later for describing
  isolated zeros

42
Sum of Absolute Difference Metric
Current block's pixel (block size nxn)
Reference block's pixel within search area
(2p1)x(2p1)
SAD metric conflicts with the zerotree rule
often, because the goal of SAD metric is just to
minimize the temporal difference, and it is
irrelevant to the magnitude hierarchy of the
spatial quad-trees.
43
HSDD Metric Calculation
Current block's pixel (block size nxn)
Reference block's pixel within search area
(2p1)x(2p1)
44
Observations on HSDD Metric

• HSDD value may be negative, but a larger positive
  one is preferred.
• Given any parent pixel information, the maximal
  HSDD(MV) occurs if and only if the perfect SAD
  matching exists, that is SAD(MV)-gt0.

45
Motion Estimation Applying HSDD Metric
46
Layered Magnitude Distributions for HSDD SAD
47
Performance Evaluation
48
5. HMRME Motion Estimation Algorithm
49
Half-Pixel Multi-Resolution Motion Estimation

• Combine transform-adapted half-pixel
  interpolation with anti-aliasing under complexity
  constraints.
• Avoid multiple inverse transforms.
• Can be united with the conventional wavelet
  domain motion estimation algorithms.

50
H-Transform
h H ? a
51
Aliasing
52
Half-Pixel Interpolation
53
Horizontal Interpolation
54
Vertical Interpolation
55
Diagonal Interpolation
56
Performance Evaluation
MRME Y. Q. Zhang, S. Zafar, Motion- Compensated
Wavelet Transform Coding for Color Video
Compression, IEEE CSTV, 1992.
AMRME M. K. Mandal, E. Chan, X. Wang and
S. Panchanathan, Multiresolution Motion
Estimation Techniques for Video Compression,
Optical Engineering, 1996
57
6. ABEC Embedded Coder
58
Array-Based Embedded Coder

• Performance similar to SPIHT (Amir Said and
  William A. Pearlman, A New Fast and Efficient
  Image Codec Based on Set Partitioning in
  Hierarchical Trees, IEEE CSVT, 1996)
• One pass processing no link lists
• Hardware implementation friendly
• R.O.C. patent no. 141267, 2001

59
ABEC Encoding Flow
60
Significance Map
61
ABEC Encoder Structure
Zero-tree

• Definitions of ABEC Status Bits
• P parents significance bit
• S parents sign bit
• R parents refinement bit
• C childrens significance bit

62
7. Conclusions Future Works
63
Conclusions

• Joint optimization for wavelet domain ME
  zero-tree coding can raise the compression
  performance significantly (HSDD).
• According to the prediction for DC coefficients
  in wavelet domain, the ideas of fast
  anti-aliasing transform-adapted half-pixel
  interpolation can be combined (HMRME).

64
Conclusions (cont.)

• One pass processing no link lists fast
  hardware friendly zero-tree coding is possible
  (ABEC).
• The loss probability function for Gilbert channel
  conditioned on past loss rates can be calculated
  out by an iterative equation set.

65
Conclusions (cont.)

• Content-adaptive plus channel-adaptive (loss rate
  feedback) unequal error protection can further
  enhance FG video transmission efficiency.
• Simplified quality prediction formulas can be
  used with trivial performance degradation while
  significant speeding up.

66
Future Works

• Exploit possible optimal or sub-optimal weighting
  rules for the two difference terms in HSDD
  metric.
• Extend HMRME (by lifting scheme?) to be available
  for overlapped transforms.
• Try to find some other better estimation method
  for ho in HMRME.

67
Future Works (cont.)

• Upgrade to an context-based entropy-constrained
  version of ABEC coder.
• Investigate the affection of packet length to FG
  video transmission over bit-error channel.

68
Thank You!