Modeling and Evaluating Feedback-Based Error Control for Video Transfer

1
Modeling and Evaluating Feedback-Based Error
Control for Video Transfer
PhD Candidate Yubing Wang - Computer
Science, WPI, EMC Corp. Committee Prof. Mark
Claypool - Computer Science, WPI Prof. Robert
Kinicki - Computer Science, WPI Prof. Dan
Dougherty - Computer Science, WPI Prof.
Ketan Mayer-Patel Computer Science, UNC at
Chapel Hill
2
Video Transfer
5
4
3
2
1
Frame Loss
Video Frames
Server
Error Propagation
Internet
Capacity Constraint
Delay Constraint
Client
3
Error Control
5
4
3
2
1
3
Retransmission
Video Frames
Server
Change Coding Parameter
Internet
Local Concealment
Client
3
NACK
4
Motivation

• Frame loss degrades video quality
• Feedback-based error control techniques use
  information from decoder to repair
• Feedback indicates damage location.
• Encoder and decoder cooperate in error control
  process.
• Better than error control techniques where no
  interaction between encoder and decoder
• Major techniques RPS, Intra Update,
  Retransmission
• Choice and Effectiveness depends on packet loss,
  RTT, video content and GOP size
• No systematic exploration and comparison of
  impact of video and network conditions on the
  performance of feedback-based error control
  techniques

5
The Dissertation

• Analyze video quality with feedback based error
  control
• Develop analytical models to predict quality of
  videos streamed with RPS NACK, RPS ACK, Intra
  Update or Retransmission
• Conduct systematic study of effects of reference
  distance on video quality
• Validate analytical models through simulations
• Analysis of loss rate, round-trip time, video
  content, Group Of Pictures (GOP)
• Determine choice between RPS NACK, RPS ACK, Intra
  Update or Retransmission
• Publications
• Impact of Reference Distance for Motion
  Compensation Prediction on Video Quality, MMCN07
• An Analytic Comparison of RPS Video Repair,
  MMCN08
• Modeling RPS and Evaluating Video Repair with
  VQM, IEEE Transactions on Multimedia, 2009, (to
  appear)

6
Outline

• Introduction
• Background
• RPS ACK
• RPS NACK
• Intra Update
• Retransmission
• Impact of Reference Distance on Video Quality
• Analytical Models and Results
• Model Validations
• Conclusions

7
Reference Picture Selection (ACK)
1
3
4
5
6
7
2
ACK(1)
ACK(2)
ACK(3)

• The decoder acknowledges all correctly received
  frames
• Only the acknowledged frames are used as a
  reference
• Error propagation is avoided entirely
• Distance from reference frame is reference
  distance
• Reference distance increases with round-trip
  delay
• Coding efficiency decreases as reference distance
  increases
• Video quality degrades as coding efficiency
  decreases

8
Reference Picture Selection (NACK)
1
3
4
5
6
7
8
2
NACK(3)

• The previous frame is used as a reference for
  encoding during the error-free transmission.
• Reference distance is always 1 regardless of RTT
• The decoder sends a NACK for the erroneous frame
  along with a reference frame number
• Error propagation
• Impact of loss increases with RTT

9
Intra Update
Intra-coded
1
3
4
5
6
7
8
9
2

• Upon receiving a NACK from the decoder, encodes
  the current frame with intra mode
• Frame is independently encoded without using any
  information from previous frames
• Coding efficiency is reduced because of intra
  coding

10
Retransmission
Encoder
1
3
4
5
6
7
8
9
2
Decoder
1
3
4
5
6
7
8
9
2

• Retransmission of lost frames needs extra
  bandwidth
• Packets arriving after their display times are
  not discarded but instead are used to reduce
  error propagation

11
Outline

• Introduction
• Background
• Impact of Reference Distance on Video Quality
• Hypothesis
• Methodology
• Results and Analysis
• Analytical Models and Results
• Model Validations
• Conclusions

12
Impact of Reference Distance on Video Quality

• RPS selects one of several previous frames as a
  reference frame during compression
• Distance from selected frame is reference
  distance
• Higher reference distance, lower quality and vice
  versa
• How reference distance affects video quality has
  not been quantified
• A systematic study of the effects of reference
  distance on video quality
• Data is needed for modeling RPS

13
Hypothesis

• Low Motion
• The similarities among frames are high
• More macro-blocks are inter-coded
• High motion
• The similarities among frames are low
• More macro-blocks are intra-coded

• The y-intersect is determined by motion and scene
  complexity.
• High-motion video sequences starts with low
  quality, degrade slower.
• Low-motion video sequence starts with high
  quality, degrade faster.

14
Methodology

• Select a set of non-compressed video clips with a
  variety of motion content.
• All in YUV 422, CIF (352x288)
• Each video sequence contains 300 video frames
  with a frame rate of 30 fps.
• Change reference distances for each selected
  video sequence
• Encode the video clips using H.264
• Measure video quality using
• Peak-Signal-to-Noise-Ratio (PSNR)
• Video Quality Metric (VQM)
• Analyze the results.

15
PSNR vs. Reference Distance
Video Clips a b R-Squared
Akiyo -2.0116 47.965 0.9953
Container -1.9023 44.838 0.9948
News -1.8556 43.295 0.9984
Silent -1.5283 41.41 0.9929
Mom Daughter -1.4581 41.442 0.9904
Foreman -1.1681 38.511 0.9265
Mobile -1.1553 26.663 0.9754
Coastguard -0.8626 35.582 0.9975

• The relationship between PSNR and reference
  distance can be characterized using a logarithmic
  function

16
VQM vs. Reference Distance
Video Clips a b R-Squared
Akiyo -0.0113 0.9847 0.9869
Container -0.0114 0.9766 0.9848
News -0.0115 0.9732 0.9931
Silent -0.0124 0.9606 0.9937
Mom Daughter -0.0085 0.9217 0.9821
Foreman -0.0068 0.9059 0.9779
Mobile -0.0022 0.8055 0.9076
Coastguard -0.0014 0.8423 0.9671

• The relationship between VQM and reference
  distance can be characterized using a linear
  function

17
Outline

• Introduction
• Background
• Impact of Ref. Distance on Video Quality
• Analytical Models and Results
• Assumptions
• RPS ACK
• RPS NACK
• Intra Update
• Retransmission
• Result Analysis
• Model Validations
• Conclusions

18
Assumptions

• Each GOB is independent from other GOBs in the
  same frame.
• An independent video sub-sequence is referred to
  as a reference chain.
• Each GOB is carried in a single network packet.
• Reliable transmission of feedback messages are
  assumed.
• Erroneously-decoded GOBs are repaired by local
  concealment.
• Make no assumption on specific local concealment
  techniques.

• Assume independent packet loss with a random loss
  distribution.
• In this talk, GOB and Frame is exchangeable.

19
Model Parameters
20
Modeling of RPS ACK
p Packet loss probability
Probability of GOB (n-d-i) being successfully decoded
Round-trip time
Time-interval between two frames
1
3
4
5
2
ACK(1)
ACK(2)

• The probability of decoding GOB (n) correctly
  using GOB (n-d-i) as a reference
• The probability of GOB (n) being successfully
  decoded is

21
RPS ACK Modeling (cont.)

• The expected video quality for n-th GOB

Average video quality for a GOB encoded using the GOB that is r GOBs backward.
Average video quality for a Intra-Coded GOB
Average PSNR value for a GOB that is repaired using local concealment
22
RPS NACK -- Model

• The probability of GOB (n) being successfully
  decoded

--- the probability of decoding GOB (n)
correctly using GOB (n- d -i) as a
reference
GOB Dependency Tree
23
Intra Update -- Model

• The probability of GOB (n) being successfully
  decoded

-- the probability of decoding GOB (n)
correctly using Intra coding
Intra-coded
1
3
4
5
2
NACK
GOB Dependency Tree
24
Retransmission

• Capacity constraint

• The n-th GOB in the reference chain being
  successfully decoded

• The expected video quality for GOB (n)

25
Outline

• Introduction
• Background
• Impact of Ref. Distance on Video Quality
• Analytical Models and Results
• Assumptions
• RPS ACK
• RPS NACK
• Intra Update
• Retransmission
• Result Analysis
• Model Validations
• Conclusions

26
Analytic Experiments

• Our analytical models consider a number of
  factors that may affect feedback-based repair
  performance
• Reference distance change
• Loss probability
• Round-trip time
• Bitrate constraint
• Video content
• GOP Size
• Select a set of video clips
• with a variety of motion content

27
Quality versus Round-Trip Time
RPS ACK
RPS NACK

• Quality degrades with round-trip time increase
• NACK resistant to degradation with round-trip
  time for low loss
• ACK degrades uniformly with round-trip time

28
Quality versus Loss Rate
RPS NACK
RPS ACK

• Quality degrades with loss rate increase
• NACK degrades faster with high round trip times
• ACK uniform degradation

29
RPS NACK vs. RPS ACK

• Above trend line, ACK better. Below trend line,
  NACK better
• Crossover points for low-motion are higher than
  for high-motion
• Error propagation more harmful to quality than
  reference distance

30
Comparison

• RPS NACK performs best in low loss
• RPS ACK performs best in high loss
• RPS ACK performs worst in low loss
• Retransmission performs worst in high loss
• Intra Update performs as well as RPS NACK as RTT
  increases

RTT80 ms
RTT240 ms
31
Outline

• Introduction
• Background
• Impact of Ref. Distance on Video Quality
• Analytical Models and Results
• Model Validations
• Methodology
• Results
• Conclusions

32
Validation -- Methodology

• Randomly drop controllable number of frames in
  input sequence based on given loss probability
• Based on given round-trip time and randomly
  selected lost frames, regenerate video sequence
• Encode video sequence generated in step 2 using
  H.264
• Measure average PSNR and VQM for encoded H.264
  video sequence
• Calculate average PSNR and VQM based upon video
  quality measured in step 4

RPS NACK, round-trip time 2 frames, frame 3 is
lost
33
Validation RPS NACK

• Error bar represents 95 confidence interval
• As loss probability or round-trip time
  increases, the variance is increased
• Simulation results are consistent with values
  predicted by analytical model for both PSNR and
  VQM

34
Outline

• Introduction
• Background
• Impact of Ref. Distance on Video Quality
• Analytical Models and Results
• Model Validations
• Conclusions

35
Major Contributions

• Systematic study of effects of reference distance
  on video quality for a range of video coding
  conditions
• Two utility functions that characterize impact of
  reference distance on video quality based upon
  study
• Modeling prediction dependency among GOBs for RPS
  NACK and Intra Update using binary tree
• Analytical models for feedback-based error
  control techniques including Full Retransmission,
  Partial Retransmission, RPS ACK, RPS NACK and
  Intra Update
• Simulations that verify accuracy of our
  analytical models
• Analytic experiments over a range of loss rates,
  round-trip times and video content using our
  models

36
Future Work

• Explore and incorporate other existing video
  quality metrics or develop a new quality metric
• Investigate how local concealment may affect the
  choice of feedback-based repair techniques
• Investigate the impact of the extra bandwidth
  consumed by feedback messages on performance
• Build a videoconference system that automatically
  adapts to the best repair techniques

37
Conclusions

• Degree of video quality degradation is affected
  by video content
• High-motion video sequences starts with lower
  quality, degrade slower.
• Low-motion video sequences starts with higher
  quality, degrade more rapidly.
• Mathematical Characterization of the
  relationship between video quality and reference
  distance
• PSNR
• VQM
• Analytical models reveal
• RPS NACK performs best in low loss
• RPS ACK performs best in high loss, worst in low
  loss
• RPS NACK outperforms RPS ACK over a wider range
  for low motion videos than for high motion videos
• Retransmission performs worst in high loss
• Intra Update performs as well as RPS NACK as RTT
  increases

38
Acknowledge

• Prof. Claypool and Prof. Kinicki
• Prof. Dougherty
• Prof. Mayer-Patel from UNC at Chapel Hill
• Faculty/Staff of Computer Science Dept., WPI
• Huahui Wu, Mingze Li, Feng Li, and everyone from
  PEDS and CC groups
• Attendees today
• My Family