The SP and SIFrames Design for H'264AVC

1
The SP- and SI-Frames Design for H.264/AVC

• Marta Karczewicz and Ragip Kurceren

(Nokia Research Center)
IEEE Trans. on Circuit and System for Video
Technology, Vol.13, No. 7, July 2003
2
Outline

• Whats SP- and SI-Frames
• Motivation
• Decoding and Encoding Processes for SP- and
  SI-Frames
• Results

3
Whats SP- and SI-Frames (1/2)

• SP-frames make use of motion compensated
  predictive coding to exploit temporal redundancy
  in the sequence similar to P-frames.
• The difference between SP- and P-frames is that
  SP-frames allow identical frames to be
  reconstructed even when they are predicted using
  different reference frames.

4
Whats SP- and SI-Frames (2/2)

• SP-frames can be used instead of I-frames in such
  applications as bitstream switching, splicing,
  random access, fast forward, fast backward, and
  error resilience/recovery.
• SP-frames require significantly fewer bits than
  I-frames to achieve similar quality.
• In some of the mentioned applications, SI-frames
  are used in conjunction with SP-frames. An
  SI-frame uses only spatial prediction as an
  I-frame and still reconstructs identically the
  corresponding SP-frame, which uses
  motion-compensated prediction.

5
Frame types in the existing standards

• I- , P-, and B-Frames

I
P
P
P
P
P
I
P

I
B
B

I
B
P
P
B
6
Applications for SP- and SI-Frames(1)

• Bitstream switching

P2, n-2
P2, n-1
S2, n
P2, n1
P2, n2
Bitstream 2
S 12, n
Secondary SP-frame
P1, n-2
P1, n-1
S1, n
P1, n1
P1, n2
Bitstream 1
7
Applications for SP- and SI-Frames(2)

• Splicing and Random Access

P2, n-2
P2, n-1
S2, n
P2, n1
P2, n2
Bitstream 2
SI 2, n
P1, n-2
P1, n-1
S1, n
P1, n1
P1, n2
Bitstream 1
8
Applications for SP- and SI-Frames(3)

• Error Recovery/Resiliency

P1, n-2
P1, n-1
S1, n
P1, n1
P1, n-3
S 21, n
SI1, n
9
Applications for SP- and SI-Frames(4)

• Video Redundancy Coding
• Sync frame

1
2
3
4
5
6
7
8
9
10
11
12
13
3
4
5
6
7
8
9
10
11
12
13
14
15
5
6
7
8
9
10
11
12
13
14
15
16
17
10
Decoding and Encoding Processes for SP- and
SI-Frames
Inverse Quantization
Inverse Transform
PQP
Frame Memory
MC Prediction
Demultiplexing
Intra Predition
Motion Information
Intra Prediction Mode
Generic block diagram of decoding process.
11
Generic block diagram of decoding process for
secondary SP- and SI-frames
Inverse Quantization
Inverse Transform
lc
drec
lrec
lerr
lpred
SPQP
Frame Memory
Quantization
R
MC Prediction
P
Demultiplexing
SPQP
Transform
Intra Predition
Motion Information
Intra Prediction Mode
12
Generic block diagram of decoding process for
primary SP-frames
Inverse Quantization
Quantization
Inverse Quantization
crec
derr
lrec
lerr
cpred
SPQP
PQP
SPQP
drec
Inverse Transform
lc
Demultiplexing
Transform
Frame Memory
R
MC Prediction
P
Motion Information
13
Generic block diagram of encoding process for
nonintra blocks in SP-frames
Transform
corig
Quantization
cerr
lerr
-
PQP
Inverse Quantization
dpred
Inverse Quantization
derr
PQP
cpred

SPQP
lpred
Quantization
Multiplexing
Quantization
SPQP
lrec
Inverse Quantization Inverse Transform
Transform
Motion Estimation
Frame Memory
P
R
Motion Information
14
To generate secondary representation of the
primary SP-frame lc
Inverse Quantization
Quantization
Inverse Quantization
crec
derr
lrec
lerr
cpred
SPQP
PQP
SPQP
drec
Inverse Transform
lc
Demultiplexing
Transform
Frame Memory
F1,n gt F2,n
R
MC Prediction
P
Motion Information
lerr,2 lrec lpred,2
15
Generic block diagram of encoding process for
nonintra blocks in SP-frames
lerr,2
Transform
corig
Quantization
cerr
lerr
-
PQP
Inverse Quantization
dpred
Inverse Quantization
derr
PQP
cpred

SPQP
lpred
lpred,2
Quantization
Multiplexing
Quantization
SPQP
lrec
Inverse Quantization Inverse Transform
Transform
lc
Motion Estimation
Frame Memory
P
R
lerr,2 lrec lpred,2
Motion Information
16
Generic block diagram of decoding process for
secondary SP- and SI-frames
lerr,2
lc
Inverse Quantization
Inverse Transform
lrec
lerr
drec
lpred
SPQP
Frame Memory
F2,n
lpred,2
Quantization
R
MC Prediction
P
Demultiplexing
SPQP
Transform
Intra Predition
Motion Information
Intra Prediction Mode
lerr,2 lrec lpred,2
lerr,2 lpred,2 (lrec lpred,2) lpred,2
lrec
17
Results coding efficiencies
18
Results coding efficiencies when inserted
periodically
19
Results Comparison with S-Frame
(Drifting effect)
20
Results Comparison with S-Frame (Size)
6.2 times
The SP-frame is 3.4 times smaller than the
S-frame in average.
21
Results PSNR and Total bits over 100 frames (4
switches)
SP or I Frame rate 1fs
22
Results PSNR and Total bits over 100 frames (no
switches)
SP or I Frame rate 1fs
23
The End

• Thank you!!