Video Transcoding in H.264

1
Video Transcoding in H.264
PISATEL

• Prof. Maurizio Bonuccelli
• Francesca Martelli
• Francesca Lonetti

2
Outline
PISATEL

• H.264 coding and transcoding
• Motion Vector Composition algorithms
• Overview of our temporal transcoding results
  (MPEG4, H.263)

3
H.264 coding
PISATEL

• H.264 mean features
• Variable and small block sizes (44)

• Quarter-pixel resolution
• NAL units
• New entropic coding algorithms (CABAC)
• Deblocking filter
• Moto compensation with multiple reference pictures

Average bit rate reduction!
High complexity!
4
H.264 temporal transcoding
PISATEL

• Skipping frames to reduce the output bit-rate
• Three main issues
• Motion Vector Computation(MVC)
• Prediction Error Computation
• Frame Skipping Policy

1
3
1
2
3
Transcoder
Output bit-rate 32 Kb
Input bit-rate 64 Kb
5
Motion Vector Composition

• The motion vectors are computed by
• Motion Vector Composition Algorithms (BI, TVC,
  FDVS, ADVS)
• Restricted Motion Estimation (RME)

6
Telescopic Vector Composition
MVn
(skipped)
MV1n-1
MV2n-1
MBn
MB1n-1
MB2n-1
MV4n-1
MBn
MV3n-1
MB3n-1
MB4n-1
F(n)
F(n-1)
New reference area
MVnMVnMVTVC
F(n-2)
7
Bilinear Interpolation
MVBI(1-a)(1-ß)MV1n-1(a)(1-ß)MV2n-1(1-a)(ß)MV3n-
1(a)(ß)MV4n-1
MVn
(skipped)
MBn
MV2n-1
MB1n-1
a
MB2n-1
MV1n-1
ß
MV4n-1
MVBI
MBn
MV3n-1
F(n)
MB4n-1
MB3n-1
New reference area
F(n-1)
MVnMVnMVBI
F(n-2)
8
H.264 features

• New in H.264 variable macroblock partition (16
  motion vectors for each macroblock)
• Transcoder keeps the same partitions of the
  remote encoder (most efficient solution)
• How to apply MVC in H.264?
• BI and TVC adaptation
• New MVC algorithm

H.264
?
9
MVC in H.264
10
A new MVC algorithm

• Basic Idea looking for a reference area, the
  most similar to the current macroblock
• Two steps
• For each motion vector of a macroblock (or block)
  of the reference area in the skipped frame,
  compute the difference between the macroblock and
  an area pointed by the motion vector with the
  same size of the current macroblock
• Choose the motion vector of the reference area
  that minimizes this difference

11
New MVC algorithm example
Vfargmin i ? S MSE(A, Ai) argmin i ? S (1/NxM
A-Ai2)
12
MVC performance (time)
PISATEL
13
MVC performance (quality)
Akiyo
PSNR(dB)
Frames
14
MVC performance (quality)
Akiyo
PSNR(dB)
Frames
15
MVC performance (quality)
Coastguard
PSNR(dB)
Frames
16
MVC performance (quality)
Coastguard
PSNR(dB)
Frames
17
Our activities in video transcoding

• We started in studying video features in MPEG4
• With this codec, we developed two temporal
  transcoders
• One based on an architecture known as FSC (Frame
  Skipping Control)
• One based on an architecture known as DFS
  (Dynamic Frame Skipping)
• By simulation, we realized that the second
  architecture results better in terms of video
  quality, while the first one results better in
  terms of processing times

18
MPEG4 transcoder architectures

• After a skipped frame
• In the FSC architecture, motion vectors and
  prediction errors are computed without executing
  motion estimation ? worse quality, re-encoding
  errors amassment
• In the DFS architecture, motion vectors and
  prediction errors are computed with a restricted
  motion estimation ? better quality, but worse
  time performance

19
Motion Vector Composition

• In literature we found 4 MVC algorithms
• Bilinear interpolation
• Telescopic Vector Composition
• Forward Dominant Vector Selection
• Activity Dominant Vector Selection
• By using the DFS architecture (that performs
  motion estimation) we realized by simulation that
  these algorithms are equivalent

20
Architectures and MVC algorithms

• M. A. Bonuccelli, F. Lonetti, F. Martelli. Video
  Transcoding Architectures for Multimedia Real
  Time Services, ERCIM News No. 62, pp.
  39-40, July 2005.

21
Skipping policies

• We developed 4 skipping policies, all of them
  based on the transcoder buffer occupancy
• Buffer based
• Motion activity
• Consecutive skipping
• Random
• M. A. Bonuccelli, F. Lonetti, F. Martelli.
  Temporal Transcoding for Mobile Video
  Communication. In Proceedings of 2nd Annual
  International Conference on Mobile and Ubiquitous
  Systems Networking and Services (Mobiquitous
  2005), pp.502-506, July 17-21, 2005, San Diego,
  CA.

22
H.263 Video Transcoder

• Then, we studied the H.263 video codec, and we
  developed a temporal transcoder, based on DFS
  architecture.
• By observing the test results, we realize that
  video quality is influenced by the rate control
  of the front encoder
• Then, we tested and implemented several rate
  control algorithms for the front encoder
• TMN5
• TMN8
• ? domain
• Perceptual rate control
• Multiple zone (Activity)

23
H.263 Video Transcoder

• In our transcoder architecture, every frame is
  first transcoded, and then transmitted or
  dropped.
• To avoid the computation of frames that will be
  discarded, we developed a new frame skipping
  policy that predicts the frames to be transcoded
• M. A. Bonuccelli, F. Lonetti, F. Martelli.
  A Fast Skipping Policy for H.263 Video
  Transcoder. In Proceedings of 12th International
  Workshop on Systems, Signals and Image Processing
  (IWSSIP'05). September 22-24, 2005, Chalkida,
  Greece.

24
Finally, H.264

• This codec is the product of the union between
  VCEG (H.263) and MPEG, forming the Joint
  VideoTeam (JVT)
• It includes all benefits of previous standards in
  order to achieve good quality performance even at
  low bit-rate
• We worked in two directions
• Optimizing the codec, to speed up it
• Implementing a transcoder

25
Optimizing H.264 encoder

• We operated some modifications to the reference
  software in order to obtain acceptable encoding
  times
• instead of computing all half and quarter pixels
  in two rounds, we compute them in only one round
• fast way for choosing the optimal partitioning
  instead of using the SAD (Sum of Absolute
  Differences) measure as decision parameter, we
  use other metrics
• the number of differences in terms of pixels
• the maximum difference value
• the average difference value
• the most popular difference value
• compared with proper self-adjusting thresholds.

26
H.264 transcoder

• We first implemented the cascade pixel domain
  transcoder by simply concatenating a decoder with
  an encoder, in order to develop the motion vector
  composition algorithms
• Then we are implementing the frame skipping to
  obtain a transcoder that decides the frames to be
  dropped

27
H.264 rate control

• Finally, we are implementing the TMN8 rate
  control algorithm to be used in the front encoder
• We think that, with a rate control algorithm able
  to skip frame in encoding phase, the transcoding
  process may be improved both in terms of quality
  and computation time

28
Master Theses

• Luigi DAmaro. Algoritmi per la transcodifica
  video.
• Gianni Rosa. Transcodifica video per
  comunicazione mobile studio di rate control.
• Luca Leonardi. Transcodifica video temporale
  politiche di frame skipping.
• Marina Paletta. Realizzazione di un
  transcodificatore video temporale H.264 per video
  comunicazione mobile.
• Riccardo Vagli. Implementazione di un transcoder
  video basato sullo standard H.264/AVC.
• Alsona Dema. Rate Control in H.264.

29
Conclusions

• We studied the video transcoding problem in
  real-time communications
• We developed temporal transcoders with MPEG4,
  H.263 and H.264 codecs
• We developed some skipping policies to be used in
  each transcoder
• We developed three MVC algorithms to be used in
  the H.264 transcoder

30
Acknowledgements

• We thank all ERI people who introduced us in this
  research area, for the helpful discussions and
  advices
• We thank PisaTel Lab people
• We thank all students who worked with us in this
  project