Video Communications over the Internet Jozsef Vass 1162000 Multimedia Communications and Visualizati

1
Video Communications over the Internet Jozsef
Vass 1/16/2000Multimedia Communications and
Visualization LaboratoryDept. of Computer
Engineering and Computer ScienceUniversity of
Missouri-ColumbiaColumbia, MO 65211,
USAhttp//meru.cecs.missouri.edu
2
Presentation Overview

• References
• Multimedia Requirements
• Internet Protocol
• Real-Time Transport Protocol
• Resource Reservation Protocol
• Multicast
• Multiresolution Coding
• Packetization
• Error Control
• Flow Control
• Conclusions

3
References

• S. McCanne, M. Vetterli, and V. Jacobson,
  Low-complexity video coding for receiver driven
  layered multicast, IEEE Journal on Selected
  Areas in Communications, vol. 15, no. 6, pp.
  983-1001, Aug. 1997.
• W.-T. Tan and A. Zakhor, Real-time Internet
  video using error resilient scalable compression
  and TCP-friendly transport protocol, IEEE
  Transaction on Multimedia, vol. 1, no. 2, pp.
  172-186, June 1999.
• T. Turletti, S.F. Parisis, and J.-C. Bolot,
  Experiments with a layered transmission scheme
  over the Internet, INRIA Technical Report, Nov.
  1997.
• F. Kuo, W. Effelsberg, and J.J.
  Gracia-Luna-Aceves, Multimedia Communications,
  Prentice Hall, 1998.
• A.S. Tanenbaum, Computer Networks, Prentice Hall,
  1998.

4
Multimedia Requirements

• High bandwidth
• Delay (critical for interactive applications)
• Variation of delay (jitter)
• Reliability
• Commonly referred as quality-of-service (QoS)
  parameters
• Low complexity codecs Software only
  encoder/decoder
• Multicasting Bandwidth reduction
• Support for heterogeneous system

5
Connection Type

• Connectionless (Datagram) Each packet is routed
  individually
• Currently used in the Internet
• Efficient for short connections
• Robustness
• Easy internetworking
• Connection-oriented (Virtual Circuit) Route is
  established at connection setup
• Currently used in X.25, Frame Relay, ATM
• Efficient routing at runtime
• Call acceptance to avoid congestion

6
Internet Protocol version 4

• Provide reliable transmission in the case of link
  failure
• Datagram service, each packet is routed
  independently
• Best effort service, does not support QoS (fair)
• Unicast (multicast addresses are reserved)
• No provisions for supporting continuous media
• No flow control
• No error control

7
Internet Protocol version 4

• 20 byte header

Type of Service
Header Length
Version (4)
Total Length
Identification
M F
D F
Fragment Offset
Header Checksum
Transport Protocol
Time to Live
Source Address
Destination Address
Options (optional)
8
Internet Protocol version 4

• Packets can be segmented as traversing through
  the network
• Identification Fragmentation information
• DF Do not fragment. Each router shall handle
  packets of 576 bytes
• MF More fragments
• Fragment Offset Tells the place of the fragment

9
Internet Protocol version 6

• New version of Internet Protocol
• New features
• Introduction of streams Flow gt Packet stream
  between sender and receiver
• Larger address space 128 bit
• Improved multicast addressing
• Priority
• Security (authentication, integrity, and
  encryption)
• Backward compatible to IPv4
• Connectionless
• No flow control
• No error control

10
Internet Protocol version 6

• 40 byte header

Version (6)
Priority
Flow Label
Payload Length
Next Header
Hop Limit
Source Address
Source Address
Source Address
Source Address
Destination Address
Destination Address
Destination Address
Destination Address
11
Internet Protocol version 6

• Flow label Multimedia flow can be identified
• Special handling by the router
• Soft state Keep the state at the router until a
  time-out is reached

12
Migration to IPv6

• Requires infrastructure changes
• Internet providers work with small profit margins
• IPv4 is good enough
• May take several years

13
User Datagram Protocol

• Application interface to IP
• Send datagrams without establishing connection
• Eight byte header

Source Port
Destination Port
UDP Checksum
UDP Length
14
Real-Time Transport Protocol (RTP)

• Transport protocol for multimedia streams over
  the Internet
• Main functionality Timing and synchronization
• Used over UDP
• Lightweight protocol
• No error correction
• No flow control
• No packet reordering
• No retransmission of lost packets
• Does not support either resource reservation or
  QoS
• Multicast capable
• Associated control protocol Real-Time Control
  Protocol (RTCP)
• Measure connection parameters and send
  transmission records (participants can monitor
  the quality of the media flow)
• Parameter negotiation

15
Real-Time Transport Protocol (RTP)

• 12 byte header

Sequence Number
Payload Type
M
CC
V
P
Timestamp
Synchronization Source (SSRC) Identifier
Contributing Source (CSRC) Identifier (Optional,
up to 15 fields)
V Version Number P Padding On/Off CC CSRC
Counter M Mark
16
Real-Time Transport Protocol (RTP)

• Timestamp Intrastream and interstream
  synchronization
• Synchronization Source (SSRC) Identifier Random
  number generated by the source, unique
  identification of the stream
• Contributing Source (CSRC) Identifier If a
  router (mixer) combines media streams, the SSRC
  of contributing sources are recorded

17
Real-Time Transport Protocol (RTP)

• Implemented as part of the application
• Relatively new protocol
• Used in vic and vat
• Expected to be used in next generation WWW
  browsers

18
Resource Reservation Protocol

• Earlier protocols
• ST-II (Internet Stream Protocol) Parallel to IP
• Tenet Protocol Suit (UC Berkeley)
• Resource Reservation Protocol (RSVP)
• To be used with IPv6 (may also be used with IPv4)
• Reservations are made for flows (specified in IP
  header)
• Based on the flow label, the router schedules
  transmission in accordance with the reservation
  setup
• RSVP keeps soft states
• Large amount of information
• Must be refreshed within a given period

19
Resource Reservation Protocol

• Receiver oriented Each receiver may join or
  leave session
• Each receiver decides based on its own
  characteristics and requirements gt Heterogeneous
  reservation (multicast)
• The path may change Cannot provide hard QoS
  guarantees
• Tunneling Traffic may flow through routers that
  do not support RSVP gt no guarantees can be given
• Broken links are not handled

20
Multicast Distribution

• Unicast Send a copy of each packet to each
  receiver individually
• Multicast Send each packet simultaneously to all
  the interested receivers
• Multicast Backbone (MBone) of Internet
• Multicast IP address space was reserved (group
  addresses)
• Routers must be extended
• Understand group addresses
• New routing mechanism
• Duplicate incoming packets if necessary
• Forward packets on the correct links

21
Multicast Distribution

• IGMP Internet Group Management Protocol
• Multicast routing
• Joining and leaving multicast groups
• Multicasting still considered experimental
• Multicasting will be fully integrated to IPv6
• Experimental tools vat, vic, wb, etc.

22
Multiresolution Coding

• Network heterogeneity
• Local network
• Dial up network (PSTN, ISDN, etc.)
• Wide area network
• Earlier video coding techniques (H.261, MPEG 1,
  H.263, etc.) do not support scalable coding
• Scalability in recent video coding standards
• MPEG-2 Spatial and quality scalability
• H.263 Spatial, quality, and temporal
  scalability
• MPEG-4 Spatial, quality, temporal, and object
  scalability

23
Multiresolution Coding

• Simulcast Set of independent encoders each
  producing a different output rate gt Low
  compression ratio since cross stream correlation
  is not exploited
• Multiresolution coding Correlation across
  streams is exploited resulting in high
  compression efficiency
• The more layer the decoder receives the better
  the visual quality
• Each layer may be carried by a different
  multicast group gt Multiresolution-multicast
  framework

24
Multiresolution Coding

• Hierarchical encoding
• Used in MPEG-2, H.263, etc.
• Base layer gt Coarsest resolution signal,
  received by all receivers
• Enhancement layers continuously refine the
  quality
• Only layers received that the physical link can
  support
• Layers must be received in importance order

25
Multiresolution Coding

• Hierarchical encoding Some layers are mandatory
  to reconstruct the signal
• The network must provide prioritized transmission
  gt In the case of packet loss, always the lowest
  priority layer must be dropped
• ATM supports two levels of priority
• IPv4 does not support priorities
• IPv6 support priorities

26
Multiresolution Coding

• Solutions for IPv4
• Produce substreams that are of equal importance.
  No matter which substream is received, the
  quality can be continuously improved (Multiple
  Description Coding)
• Ensure that higher priority streams are more
  reliably transmitted gt Use unequal error
  protection (UEP)

27
Multiresolution Coding

• Audio coding Subsampling
• Each flow is separately encoded
• Each flow has the same importance
• The larger number of flows received, the better
  the quality

1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
8 kHz
2.7 kHz
2.7 kHz
2.7 kHz
28
Multiresolution Video Coding
Block-Based Techniques

• High coding performance
• Most standards (H.261, H.263, MPEG-1, MPEG-2,
  MPEG-4, etc.) based on this technique
• Temporal correlation is exploited by block-based
  motion estimation and compensation
• Remaining spatial redundancy is exploited by
  transform coding
• High complexity of encoder
• Error sensitivity
• Error propagation
• Scalability is hard to support

29
Multiresolution Video Coding
Block-Based Techniques

• Conditional replenishment
• Only blocks are encoded that change from previous
  frame
• Blocks are encoded in intra mode, no prediction
  is applied
• Lower performance than prediction
• Higher error resilience
• Only good for videoconferencing (large portion of
  the scene is unchanged)

30
Multiresolution Video Coding
Three-Dimensional Techniques

• Block-based techniques does not naturally support
  scalability
• Wavelet decomposition naturally provides
  multiresolution representation
• Wavelets can be extended to 3-D for video coding
• Error resilience
• No recursive loop Reduces temporal error
  propagation
• No spatial prediction Reduces spatial error
  propagation

31
Multiresolution Video Coding
Three-Dimensional Techniques

• Nine substreams Each substream is of
  approximately equal importance

32
Packetization

• Before transmission, the stream must be
  packetized
• Due to significant overhead of IP, large packet
  size is preferred
• IPv4UDPRTP2081240 bytes
• IPv6UDPRTP4081260 bytes
• Packetization delay Time to wait for data to
  fill the packet
• Critical for low bit rate interactive
  applications (especially for speech)
• Multilayer transmission increases the
  packetization delay
• Error protection increases the packetization delay

33
Packetization

• Speech example
• Sampling rate 8 kHz, eight bits/sample 64 kbps
• It is not beneficial to use high compression
  efficiency algorithms

34
Error Control

• Type of errors
• Bit errors (rare due to high quality fiber)
• Packet loss due to congestion
• Retransmission
• Most widely used in the current infrastructure
  (TCP, etc.)
• Receiver send ACK or NACK
• Computationally inexpensive
• Introduces delay May not be tolerated in
  real-time application
• Requires backchannel
• Does not work for multicast
• Eventually all the packets need to be transmitted
• NACK impulsion problem

35
Error Control

• Forward Error Correction (FEC)
• Requires processing at the host
• Increases the bandwidth gt May worsen the problem
  during congestion
• Interlaced protection for packet loss (introduces
  packetization delay)
• Reed-Solomon codes
• Maximum error correction capability
• Low computational complexity

Information Packet
Information Packet
Information Packet
Parity Packet
Parity Packet
36
Flow Control

• Rate of insertion packets into the network
• Each coded layer is transported by a different
  flow
• How many flow to subscribe
• On congestion gt Drop a layer (easy to detect)
• On spare capacity gt Add a layer (hard to detect)
• Join experiments gt May overload the network for
  large multicast groups

Layer 4
Layer 3
Layer 2
Layer 1
37
Flow Control

• The most dominant traffic of Internet is TCP
  (WWW)
• TCP is adaptive flow Reduces rate when network
  is congested
• Fair sharing with TCP is required Match the rate
  of TCP
• k 0.7,1.3
• MSS Maximum segment size
• RTT Round trip time
• p Packet loss rate
• These parameters must be measured

38
Conclusions

• Internet video is very challenging
• Large overhead of Internet protocols
• Best effort service
• Low delay for interactive applications
• Error control
• Multicasting
• Lack of prioritization
• The situation is improving
• IPv6 enables prioritization
• Resource Reservation Protocol