Transporting RealTime Video Over the Internet: Challenges and Approaches

1
Transporting Real-Time Video Over the Internet
Challenges and Approaches

• Dapeng Wu, Yiwei Thomas Hou, and Ya-Qin Zhang
• PROCEEDINGS OF THE IEEE, VOL. 88, NO. 12,DECEMBER
  2000

???? ??? ?? ??????? R89725048 ???
2
Introduction

• Internet multimedia applications video
  conference, distance learning, digital libraries,
  and video-on-demand.
• There is no quality of service (Qos) guarantee
  for video transmission over the current Internet.

3
Challenging QoS issues

• Bandwidth Current Internet does not provide
  bandwidth reservation
• Delay Real-time video requires bounded
  end-to-end delay. Internet does not offer delay
  guarantee.
• Loss Packet loss ratio required to be kept below
  a threshold. Internet does not provie any loss
  guarantee.

4
Unicast and Multicast video distribution
5
Tradeoff between bandwidth and flexibility
6
Another challenge Heterogeneity

• Network heterogeneity Different processing,
  bandwidth, storage, and congestion control
  policies.
• Receiver heterogeneity Receivers have different
  latency requirement, visual quality requirement,
  and processing capability.

7
Two general approaches to the above challenges

• Network-centric Routers/switches in the network
  are required to provide QoS support.
• End system based Guarantee QoS without
  imposing any requirements on the network.

8
Congestion control

• Rate control
• Rate-adaptive
• Video encoding
• Rate shaping

9
Rate control

• Windows-based Like TCP, using congestion
  window. Increase the window slowly, but decrease
  the window greatly. (may introduce intolerable
  delay)
• Rate-based Source-based, receiver-based, and
  hybrid rate control

10
Source-based rate control

• The sender is responsible for adapting the
  transmission rate of the video stream. It can
  minimize the amount of packet loss by matching
  the rate of the video stream to the available
  network bandwidth.
• Feedback is employed to convey the changing
  status of the Internet.hhg

11
Source-based approach for unicast (contd)

• lt1gt Probed-based approach additive increase and
  multiplicative decrease (AIMD) MIMD

p packet loss ratio Pth threshold for the
packet loss ratio AIR additive increase
rate r sending rate at the source a
multiplicative decrease factor
12
Source behavior under the AIMD rate control
13
Source-based approach for unicast (contd)

• lt2gt Model-based approach also called TCP
  friendly rate control
  

? throughput of a TCP connection MTU maximum
transit unit RTT round trip time p packet loss
ratio
14
Single channel Multicast

• Only the probe-based rate control can be employed

Fcon fraction of congested Fun fraction of
unloaded Tcon threshold AIR additive increase
rate r sending rate at the source
15
Receiver-based control

• Target at solving heterogeneity problem
• Layered multicast video
• Probe-based approach and model-based approach

16
Layered multicast video

• Raw video sequence is compressed into multi
  layers a base layer and one or more enhancement
  layers

17
Probe-Based approach

• When no congestion, a receiver probes for the
  available bandwidth by joining a layer, which
  leads to an increase of its receiving rate.
• When congestion id detected, the receiver drops a
  layer, resulting in reduction of its receiving
  rate.

18
Model-based approach
?i transmission rate of layer i
19
Hybrid rate control

• Receiver regulate the receiving rate of video
  streams by adding/dropping channels while sender
  also adjusts the transmission rate of each
  channel based on feedback information from the
  receiver.

20
Rate-adaptive video encoding A compression
approach

• Video conference with H.261 and H.263
• MPEG-1, MPEG-2, and MPEG-4
• Maximize the perceptual quality under a given
  encoding rate.
• Encoders quantization parameter (QP)

21
Rate-distortion (RD) theory
22
RD theory approaches

• Model-based approach Assumes various input
  distribution and quantizer characteristics.
  Close-form solutions can be obtained by using
  continuous optimization theory.
• Operational R-D based approach The admissible
  quantizers are used by the rate control
  algorithm to determine the optimal strategy to
  minimize the distortion under the constraint of a
  given bit budget.

23
Rate shaping

• A rate shaper is a filter between the encoder and
  the network, with which the encoders output rate
  can be match to the available network bandwidth.
• It is applicable to any video coding scheme and
  both live and stored video.

24
Rate shaping approaches (contd)

• Transport perspective Server selective frame
  discard.
• Two advantage Taking the network bandwidth and
  client buffer constraint into account take
  advantage of application-specific information
  such as regions of interest and group of pictures
  structure, in its decision in discarding frames.

25
Rate shaping approaches (contd)

• Compression perspective Based on the R-D theory,
  the dynamic rate shaper selectively discards the
  discrete cosine transform (DCT) coefficients of
  the high frequency so that the target rate can be
  achieved.

26
Error control

• Forward error correction (FEC)
• Retransmission
• Error resilience
• both the source and receiver side
• Error concealment
• only receiver side

27
FEC

• Add extra information to a compressed video bit
  stream.
• Channel coding
• Source coding-based FEC
• Joint source/channel coding

28
Channel coding FEC

• Disadvantages It increases the transmission rate
  and delay. It is not adaptive to varying loss
  characteristics and works best only when the
  packet loss rate is stable.
• Unequal error protection In MPEG I -frame
  gtP -frame gtB -frame
• Hierarchical FEC The FEC stream is used for
  recovery of a different video layer. More
  flexibility and bandwidth efficiency.

29
Source coding-based FEC

• The redundant information added by SFEC is more
  compressed versions of the raw data.
• SFEC recovers the video with reduced quality.
• Advantage Lower delay
• Disadvantage It increases the transmission rate
  and is inflexible to varying loss character.

30
Joint source/channel coding
31
Retransmission (contd)
32
Retransmission (contd)
33
Retransmission (contd)

• Hybrid control It could achieve better
  performance at the cost of higher complexity.
• Multicast Retransmission has to be restricted
  within closely located multicast member.

34
Optimal mode selection

• Intermode vs Intramode

35
Multiple description coding

• A raw video sequence is compressed into multiple
  streams.
• Advantages
• lt1gtrobustness to loss even if a receiver gets
  only one description, it can still reconstruct
  video with acceptable quality.
• lt2gtenhance quality

36
Error concealment A compression approach

• Human eyes can tolerate a certain degree of
  distortion in video signals, error concealment is
  a viable technique to handle packet loss.
• Spatial and temporal interpolation

37
Error resilience A compression approach

• Prevent error propagation or limit the scope if
  the damage.
• Optimal mode selection
• Multiple description coding

38
Simple error-concealment shemes

• The receiver replaces the whole frame with the
  previous reconstructed frame.
• The receiver replaces a corrupted block with the
  block at the same location from the previous
  frame.
• The receiver replaces the corrupted block with
  the block from the previous frame pointed by a
  motion vector.

39
(No Transcript)
40
(No Transcript)
41
MPEG-4

• Transport MPEG-4 video over Internet is expected
  to be available !

42
Introduction to MPEG-4 (contd)

• ?????????????
• ?????????, ?????????

43
Introduction to MPEG-4 (contd)

• ????????????, Base layer???????, Enhancement
  layer???????.
• ????Video object plane??, ???????? shape ?
  texture data.
• Spatial scalability ????????????
• Temporal scalability ?????????

44
MPEG?????
 
45
MPEG-7

• MPEG-7???????????????,????????????????????????????
  ??Multimedia Content Description
  Interface????MPEG-7????????,???????????,??????????
  ???

46
MPEG-21

• MPEG-21???????????????, ???????????????????,
  ???????????????????
• ???? ??????????????????????????????????????????
  ???????

47
On End-to-End Architecture for Transporting
MPEG-4 Over the Internet

• Dapeng Wu, Yiwei Thomas Hou, Wenwu Zhu, Hung-Ju
  Lee, Tihao Chiang, Ya-Qin Zhang, and H.Jonathan
  Chao
• IEEE TRASACTIONS ON CIRCUITS AND SYSTEMS FOR
  VIDEO TECHNOLOGY, VOL. 10, NO. 6, SEPTEMBER 2000

48
An end-to-end architecture for transporting
MPEG-4 video
49
(No Transcript)
50
MPEG-4 layers

• Compression layer compresses the visual
  information and generates elementary streams,
  which contain the coded representation of the
  VOs.
• SyncLayer packetized streams, and provide time
  and synchronization information.
• The SL-packetized streams are multiplexed into a
  FlexMux Stream at the TransMux layer.

51
RTP/RTCP protocol

• RTP support the packet sequence number,
  timestamps, and some application-specific
  profiles.
• RTCP provides QoS feedback through use of
  receiver report (RR), sender report (SR), source
  description items (SDES) , BYE, and application
  specific functions (APP) .

52
(No Transcript)
53
End-to-end feedback control protocol
54
Adaptive encoding rate control (ARC) for MPEG-4
video
55
Initialization stage
ßt remaining available bit count for encoding
the subsequent P -frames at the coding time
instant t T duration of the video sequence (in
second) r bit rate for the sequence (in bps) I
number of bits actually used for the first I
frame
Channel output rate ß0 /N0 where N0 is the
number of P -frames in the sequence Buffer size
0.5 r
56
Pre-encoding stage (contd)
Rt1 the target bit count for the P -frame at
time t1 Nt the remaining number of P -frames at
time t S the weight factor in target bit
estimation, default value is 0.05 At the actual
bits used for the P -frames at time t
57
Pre-encoding stage (contd)
Ft the current buffer fullness at time t T
buffer size
µ Ft /T
58
Pre-encoding stage (contd)
? application bit rate ?f frame rate of the
source video
59
Pre-encoding stage (contd)

• Avoid buffer fullness

m safety margin

• Avoid buffer underflow

C ß0 /N0 channel output rate
60
Dynamic target Bit-rate distribution among VOs
Target bit budget for the i th VO at time t
MAD mean absolute difference, which is computed
after the motion compensation for the luminance
component
61
Encoding stage

• Encoding the video frame and recording all actual
  bit-rate.
• Activating macroblock-layer rate control.

62
Post-encoding stage (contd)
R-D Model update
63
Post-encoding stage (contd)
64
Shape-threshold control
65
Frame-skipping control
66
Packetization algorithm
67
Simulation result lt1gt
68
(No Transcript)
69
Simulation result lt2gt
70
(No Transcript)
71
(No Transcript)
72
Future work

• MPEG-4 video over ADSL
• MPEG-4 video over Cable-modem
• MPEG-4 video over GPRS mobile phone
• MPEG-4 video over 3G wireless mobile devices

73
Thank You!