DistortionAware Retransmission of Video Packets and Error Concealment using Thumbnail

1
Distortion-Aware Retransmission of Video Packets
and Error Concealment using Thumbnail

• EE398 Course Project
• Winter 07/08
• Presenter Zhi Li

2
Presentation Outline

• Idea in a nutshell
• Realizations
• Distortion estimation
• Retransmission decision
• Adaptive error concealment
• Experiment results
• Conclusions

3
Retransmission-Based Robust Video Streaming over
Lossy Network
0101
1101
0001
0100
NACK
NACK
1101
0001
What if retransmission is subjected to a rate
constraint?
4
Prior Related Work

• Soft ARQ Vetterli98
• avoids retransmitting packets that already passed
  deadline
• Syntax-aware (frame-aware) Zakhor02
• gives retransmission priority to I packets over P
  and B packets
• Analysis-by-synthesis De Martin07
• With NACKs, Sender simulates decoding (with ErC)
  multiple times and decides retransmission based
  on distortion

5
Proposed Approach Thumbnail-Aided Retransmission
and Error Concealment
0101
1101
0001
0100
NACK
NACK
1101
0001
6
System Overview
Raw video
Video Packets
Lossy video
Video Encoder
Lossy Network
Video Decoder w/ErC
Locate Error Packets
Send NACK
Thumb. Generator
Thumbnail
7
Decomposition of Module
Raw video
Video Packets
Lossy video
Video Encoder
Lossy Network
Video Decoder w/ErC
Locate Error Packets
Send NACK
Thumb. Generator
Thumbnail
Lossy video
Slice Dist. Estimation
Pkt Dist. Estimation
Retrans. Decision
Thumbnail
8
Thumbnail Generation Slice Distortion
Estimation

• Two types of projections
• Projection into the mean
• Random projection (J240)

Z
X
Y
Block
Projection
Projection
Thumbnail pixel
Welsh- Hardamard Transform (WHT)
Est. MSE
e.g. 4 bits
9
Comparing Two Projections
Projection into the mean
J240 random projection
(Foreman CIF)
10
Packet Distortion Estimation

• Distortion model decaying and additive
• Knowing slicing distortion, we can solve
  distortion contribution of lost packets through a
  set of linear equations

11
Packet Distortion Estimation (Contd)
(Foreman CIF)
12
Retransmission Decision
.
.
TR

• Retransmission priority based on
  (assume packet size known)
• Sender randomly drops B packets to maintain 100
  transmission rate (also compensating thumbnail
  rate)

13
Adaptive Error Concealment
Control Data
DCT Coefs
Decoder
Deq./Inv. Transform
0
Motion- Compensated Predictor
ErC
Intra/Inter
Motion Data
Thumbnail
14
Comparing Various Schemes

• Oracle assume receiver knows orginal video
• Thumbnail-aided proposed method using mean-based
  thumbnail
• Frame-aware receiver requests retransmission
  based on the assumption that I packets are more
  important than P packets
• No retransmission

15
PSNR vs. Packet Loss Rate (default ErC)
(Foreman CIF)
16
PSNR vs. Packet Loss Rate (Adaptive ErC
default ErC)
(Foreman CIF)
17
Visual Quality
Thumbnail-aided retran. default ErC (26.7 dB)
Thumbnail-aided retran. adaptive ErC (27.0 dB)
Experiment settings 20 packet loss, 100
bandwidth
18
Conclusions

• Thumbnail-aided distortion-aware retransmission
  can achieve gains of 0.5 1.5 dB over
  distortion-unaware heuristic methods
• Thumbnail-aided adaptive error concealment can
  achieve additional gains of 0.5 1 dB under
  severe distortion conditions
• Key ingredient leading to gain is content-level
  error detection and correction

19
Acknowledgement

• I would like to thank Prof. Girod, Yao-Chung Lin,
  Xiaoqing Zhu, David Varodayan and Pierpaolo
  Baccichet for extremely helpful discussions.

20
Basic Settings
GOP
Video Sequence


I
B
P
B
P
B
Picture
Packet
0101
Thumbnail
Slice

• Packetization
• one slice per packet
• Thumbnail generation
• I and P pictures only
• For example, 32x32 block maps to 1 pixel of 4-bit
  in thumbnail

21
Analysis of Two Projections

• Analysis model
• Channel distortion non-zero mean (to
  characterize local distortion) and s.d. ,
  i.i.d.
• Want to estimate the noise power
• Mean-based estimator
• Random-projection estimator
• where with equal prob.

22
Analysis of Two Projections (Contd)

• Mean-based estimator
• Mean
  i.e. biased, always
  underestimate
• Variance as
• Random-projection estimator
• Mean , i.e. unbiased
• Variance as
  i.e. large variance if noise is
  not zero-mean

23
PSNR vs. Packet Loss Rate (w/o Error
Concealment)