Fine Granularity Scalability in MPEG4 Video by Weiping Li Presentation by Warren Cheung

1
Fine Granularity Scalability in MPEG-4 Videoby
Weiping LiPresentation by Warren Cheung
2
Fine Granularity Scalability

• Amendment to MPEG-4
• Bitplane coding of the DCT coefficients

3
Motivation

• Video optimised for bit rate range
• Limited Channel Capacity
• Partial Decoding

4
Nonscalable Encoding

• Fixed bit rate
• Bandwidth efficiency depends on
• Channel bit rate
• Encoded bit rate

5
Techniques in MPEG

• MPEG-2, MPEG-4
• incorporate layered scalability techniques
• SNR scalability
• Temporal scalability
• Spatial scalability

6
SNR scalability

• Signal-to-noise ratio (SNR)
• Encode two layers at same frame rate and spatial
  resolution
• Use different quantization accuracy

7
SNR scalability(2)

• Two layers base layer, enhancement layer
• Base layer decodes DCT coefficients as before
• Can decode and add enhancement layer DCT
  coefficients
• Drift can occur

8
Drift

• Drift occurs in the base layer if
• Enhancement layer information used in motion
  prediction loop and
• Enhancement layer is not decoded by the decoder
• Drift occurs in the enhancement layer if
• Enhancement layer information not used in motion
  prediction loop and
• Enhancement layer is not decoded

9
Temporal Scalability

• Two layers at different frame rates
• Base layer with only P-type prediction
• Enhancement layer adds
• Extra P and B frames based on base layer
• Extra P frames based on enhancement layer

10
Spatial Scalability

• Two layers same frame rate but different spatial
  resolution
• Base layer has lower resolution
• Enhancement layer upsamples the base layer and
  for prediction
• No drift occurs if enhancement layer is not
  decoded (but unavailable for prediction of base
  layer)

11
Performance

• Enhancement layer must be completely decoded,
  otherwise it is useless
• Jagged stair performance curve
• Want to be able to truncate the enhancement
  bitstream and still provide partial enhancement

12
FGS streaming
13
Bit-Plane Coding

• Two Layers base and enhancement
• DCT coefficients normally encoded using run-level
  encoding
• Use Bit-Plance Coding for enhancement layer
• Reorganise the bitstream such that even if it is
  truncated, we get a usable set of DCT
  coefficients

14
Sending Numbers

• Given a set of numbers
• Which bits are the most significant?

15
Bit-Plane Coding Example

• Suppose we wish to send
• 10,0,6,0
• 10 is the largest number and requires 4 bits so
  we have 4 bit-planes
• 1, 0, 0, 0 (MSB)
• 0, 0, 1, 0
• 1, 0, 1, 0
• 0, 0, 0, 0

16
Bit-Plane Coding of DCT Coefficient

• We apply Bit-Plane coding for each 8x8 block of
  DCT coefficients
• Bit-Planes are encoded using (RUN, EOP) symbols
• RUN Number of zeroes before next one
• EOP Is this the last one in the plane

17
Additional Details

• Each of the YUV color components may have a
  different number of bit-planes in the same frame
• Y luminance (black and white)
• U, V chrominance (color difference)
• MSB (most significant bit plane) is the first
  plane that is not all zeroes

18
Advanced Features

• Frequency Weighting Low frequency DCT
  coefficients are more important than
  high-frequency coefficients and should be
  transmitted first
• Shift the bits for DCT coefficients up/down
• Parts of the picture may be more significant
• Shift the bit-planes for these elements up

19
Additional Features(2)

• Error Resilience
• Resynchronization markers
• bit-plane start code
• Temporal Scalability
• Add additional frames that can be decodedto
  improve bitrate
• Predictions not allowed in the additional frames,
  but quantization can be scaled and bit-plane
  coding can also be used

20
References

• Scalability Extensions (brief overview)
• http//www.cs.ucl.ac.uk/staff/jon/mmbook/book/node
  133.html
• FGS and applications
• http//www.eesi.tue.nl/VCA/projects/fine_granular_
  scalability/

21
Questions and Comments

• Advantages/Disadvantages of FGS (and the other
  techniques)
• Applications and Related Issues
• Future Directions for improvement
• Wavelet based methods

22
Discussion Topics

• Overhead
• Noticeable degradation of image quality
• Quantization is only a coarse way to manipulate
  the image quality (powers of two)
• Many people share similar connections
• Vary due to path through network, physical
  location
• Wireless connection
• Other people on the network
• Unreliable bandwidth

23
Discussion(2)

• New MPEG standard
• Soliciting solutions to scalability issues
• Future work may be based on wavelates
• No current mainstream video format currently uses
  wavelets
• Used in static image format JPEG-2000