Reminder

1
Reminder

• Project Proposal Due Next Mon.
• Submit by Email

2
Problems with Internet

• Limited Bandwidth
• Varying Bandwidth
• Delay
• Delay Jitter
• Packet Loss
• Heterogeneity

3
Error Recovery
4
You Are Here
Encoder
Decoder
Middlebox
Sender
Receiver
Network
5
Overview

• Characteristics of the Internet
• General techniques
• Error recovery for audio
• Effect of loss on MPEG
• Error recovery for MPEG

6
Loss Characteristics of The Internet
7
Characteristics of Internet

• 60-70 of paths do not show any loss
• Those with loss have an average of 4.5 6
  packet loss
• Paxson97 End-to-end Internet packet dynamics

8
Packet Loss Pattern
1000
30
9
Characteristic of Internet

• Bursts of loss are typically short (2-3
  consecutively loss packets)
• Long burst do occur
• Burst may occur periodically

10
Wireless Link

• Loss rate measured in my office 10 20
• up to 50 reported!

11
A Survey of Packet-Loss Recovery Techniques for
Streaming Audio

• Perkins, Hodson and Hardman
• IEEE Network Magazine 1998

12
Retransmission
1
2
3
4
3
1
2
4
3
13
Redundant Data
1
2
3
4
2
3
4
5
1
2
4
2
3
5
3
14
Error Concealment
1
2
3
4
2
4
1
15
Pros and Cons

• retransmission
• redundant data
• error concealment

16
Retransmission

• for audio

17
Retransmission is Bad

• Need scalable feedback
• NACK consumes bandwidth
• May need to retransmit frequently
• Retransmission maybe useless

18
Retransmission is Good

• Only retransmit when needed

19
Scalable Retransmission

• On packet loss
• T random(0, RTT)
• wait for T
• multicast NACK
• On receiving NACK from others
• suppress own NACK

20
Retransmit when

• group size is small
• loss rate is low
• large latency acceptable

21
Redundant Data

• for audio

22
Parity FEC
1011001
XOR
1000010
0010010
0001001
23
Parity FEC
1011001
1000010
0001001
0010010
24
Parity FEC Ordering
25
Parity FEC Ordering
26
Parity FEC Ordering
27
Parity FEC Ordering
28
Parity FEC Ordering
29
Reed-Solomon Code

• RS(n,k)

RS






30
Reed-Solomon Code








31
Media Specific FEC
1
2
3
4
2
3
4
5
1
2
4
2
3
5
3
32
Interleaving
33
Error Concealment
34
Effect of Loss on Audio

• Speech
• Human ears can interpolate
• Loss up to length of phoneme can still be
  tolerable

35
Insertion-based Repair

• Splice
• Silence Substitution
• Noise Substitution
• Repetition

36
Other Repair Methods

• Interpolation
• Regeneration

37
Colins Recommendations
38
Non-Interactive Apps

• Interleaving
• FEC
• Retransmission for unicast only

39
Interactive Applications

• Media Specific FEC

40
Error Concealment

• Repeat

41
Packet Loss Effects on MPEG Video Sent over the
Public Internet

• Jill Boyce and Robert Gaglianello
• ACM Multimedia 1998

42
Measurement-based Study

• Need to understand the problem before proposing
  solution

43
(No Transcript)
44
Data Gathering Method

• From NYC13 , Austin21, London18
• To Holmdel, NJ

45
Data Gathering Method

• Video
• Two 5-mins MPEG
• 30 fps
• 384 kbps and 1 Mbps
• IBBPBBPBBPBBPBB
• QSIF 176x112 and SIF 352x240
• One row per slice

46
Average Packet Loss
47
Packet Loss vs Time
48
How to Fill Packets

• B 1 frame 1 packet
• P 1 slice 1 packet
• I 1 slice 1 packet

49
Packet Size Distributions
50
Packet Size vs Loss Rate
51
Lesson

• 1 slice per packet even if larger than MTU
• If smaller packets means higher loss rate fill
  packet until MTU
• Else one slice per MTU
• Alt Change size of slice

52
Frame Effected by Errors
53
Correlation of Packet Loss
54
Overview of Error Recovery for Video
55
Methods

• Retransmission
• Error Concealment
• FEC
• Limiting Error Propagation
• Reference Frame Selection

56
Reference Frame Selection
I P B B P

I
X
57
Methods

• Retransmission
• Error Concealment
• FEC
• Limiting Error Propagation
• Reference Frame Selection
• Changing Temporal Pattern

58
Error Control Techniques for Interactive Low
Bitrate Video Transmission over The Internet

• Injong Rhee
• SIGCOMM 98

59
Basic Idea

• Better Late Than Never!, or
• Late packet is still useful

60
MPEG Frame Pattern
I
B
B
P
B
B
P
I
P
B
B
P
B
B
61
H.261 Error Propagation
I
P
P
P
P
P
P
X
X
X
X
X
X
loss
X
X
X
X
retransmission
62
H.261 Frame Pattern
I
P
P
P
P
P
P
X
loss
63
H.261 Frame Pattern
X
X
X
retransmission
loss
64
PTDD

• Periodic Temporal Dependency Distance
• Large PTDD
• Later Deadlines
• Longer Error Propagations
• Less Temporal Dependencies

65
QAL
I
P
P
P
Enhancement Layer
I
P
P
P
Base Layer
66
QAL
I
P
P
P
X
Enhancement Layer
I
P
P
P
Base Layer FEC
67
QAL PTDD
I
P
P
P
Enhancement Layer
I
P
P
P
Base Layer
68
Error Propagation
69
Recovery from Error Propagation
70
Packet Loss Recovery for Streaming Video

• N. Feamster, H. Balakrishnan
• PV02

71
Overview

• An analytical model for packet loss
• SR-RTP
• Post Processing

72
Mathematical Model

• Lets model packet loss effects and see what it
  can tell us

73
p prob. packet loss SI mean num. of packets
in I-frames P(I) prob. frame type is I P(F)
prob. frame is useless
74
p prob. packet loss SP mean num. of packets
in P-frames Np number of P-frames in
GOP P(P) prob. frame type is P P(F) prob. frame
is useless
75
p prob. packet loss SB mean num. of packets
in B-frames NB number of B-frames in
GOP P(B) prob. frame type is B P(F) prob. frame
is useless
76
Average Frame Rate
77
Is it accurate?
78
SR-RTP

• Extensions to RTP for selective reliability

79
SR-RTP
0
Length
ADU Sequence Number
ADU Length
ADU Offset
Priority
Layer Number
80
Feamster Balakrishnans ADU

• 1 ADU 1 Frame

81
Loss Detection
LEN 50 ADU 0 ADU Size 100 ADU Off 0
LEN 20 ADU 0 ADU Size 100 ADU Off 80
LEN 50 ADU 1 ADU Size 150 ADU Off 40
LEN 50 ADU 3 ADU Size 50 ADU Off 0
82
Retransmission Request

• Based on priority
• Example
• I-Frame highest priority
• P-Frame higher priority if closer to prev I
• B-Frame no retransmition

83
Error Concealment for Video

• Repeat pixels from previous frame
• Interpolate pixels from neighbouring region
• Interpolate motion vectors from previous frame

84
Inferred MV for P Frame
I
P
85
Inferred MV for I Frame
P
P
I
86
Evaluation

• Peak Signal to Noise Ratio (PSNR)
• 2552 over average square of differences in pixel
  values
• Calculated in dB (10 log 10)

87
Performance of SR-RTP
88
Todays Summary
89
How to recover packet loss

• Retransmission
• FEC
• Error Concealment

90
How to limit the damage of error

• Interleaving
• Key frame selection
• Change reference frame

91
Demo
92
Relate to CS5248

• Really bursty audio loss
• Error concealment with noise
• Really bursty video loss, or long video
  propagation
• Need to protect important content (content-based
  unequal error protection)