Title: Distributed Video Coding for Wireless Visual Sensor Networks
1Distributed Video Coding for Wireless Visual
Sensor Networks
2Outline
- Introduction
- Distributed Source Coding (DSC)
- Distributed Video Coding (DVC)
- DVC for Wireless Visual Sensor Networks (WVSN)
- Concluding Remarks
- References
3Introduction
- Conventional video coding
- MPEG-1/2/4, H.261, H.263, H.26L, H.264/AVC
- Interframe predictive coding
- Encoder is 5-10 times more complex than decoder
- Suitable for video down-link
Xi-1
Girod, 2002
4Conventional Video Coding
Aramvith
5Conventional Video Coding
Lin, NTHU, 2007
6Transformation and Quantization
Lin, NTHU, 2007
7Interframe Predictive Video Coding
Lin, NTHU, 2007
8Motion Estimation
Lin, NTHU, 2007
9Motion Estimation
Lin, NTHU, 2007
10Motion Compensated Prediction
Lin, NTHU, 2007
11Applications of Conventional Video Coding
Pereira, 2007
12Introduction
- Problem low-complexity video encoding for
resource-limited video devices - DSC approach Wyner-Ziv video coding with
low-complexity intraframe encoding and possibly
high-complexity interframe decoding with side
information only available at decoder
Side Information
Girod, 2002
13Applications of Low-Complexity Video Coding
- Wireless video cameras
- Wireless low-power surveillance
- Mobile document scanner
- Video conferencing with mobile devices
- Mobile video mail
- Disposable video cameras
- Wireless Visual Sensor Networks
- Networked camcorders
- Distributed video streaming
- Multiview video entertainment
- Wireless capsule endoscopy
Pereira, 2007
14Applications of Low-Complexity Video Coding
Pereira, 2007
15Applications of Low-Complexity Video Coding
Pereira, 2007
16Wireless Visual Sensor Networks
Akyildiz, 2007, and Pereira, 2007
17Wireless Visual Sensor Networks
Akyildiz, 2002
18Introduction
- Requirements of wireless visual sensor networks
- low-complexity video encoder
- high compression efficiency
- Current approaches
- distributed video coding (DVC) based on
distributed source coding (DSC) - collaborative image coding and transmission
- hybrid approach (proposed approach)
19Distributed Source Coding (DSC)
- Lossless DSC, Slepian and Wolf, 1973
- Lossy DSC, Wyner and Ziv, 1976
- Distributed video coding (DVC) based on DSC
- Girod, Stanford University, 2002
- B. Girod, A. M. Aaron, S. Rane, and D.
Rebollo-Monedero, Distributed video coding,
Proceedings of the IEEE, vol. 93, no. 1, pp.
71-83, Jan. 2005. - Special session on Distributed video coding, 2005
IEEE International Conference on Image Processing
(ICIP2005), Italy, Sept. 2005 - Ramchandran, Berkeley, 2002
- R. Puri, A. Majumdar, and K. Ramchandran, PRISM
a video coding paradigm with motion estimation at
the decoder, IEEE Trans. on Image Processing,
vol. 16, no. 10, pp. 2436-2448, Oct. 2007. - R. Puri, A. Majumdar, P. Ishwar, and K.
Ramchandran, Distributed video coding in
wireless sensor networks, IEEE Signal Processing
Magazine, vol. 23, no. 4, pp. 94-106, July 2006.
20Distributed Source Coding
- DISCOVER (Distributed Coding for Video Services)
- 2005
- F. Pereira, L. Torres, C. Guillemot, T. Ebrahimi,
R. Leonardi, and S. Klomp, Distributed video
coding selecting the most promising application
scenarios, to appear in Signal Processing Image
Communication. - C. Guillemot, F. Pereira, L. Torres, T. Ebrahimi.
R. Leonardi, J. Ostermann, Distributed monoview
and multiview video coding basics, problems and
recent advances, IEEE Signal Processing
Magazine, special issue on signal processing for
multiterminal communication systems, vol. 24, no.
5, pp. 67-76, Sept. 2007. - M. Maitre, C. Guillemot, and L. Morin, 3-D
model-based frame interpolation for distributed
video coding of static scenes, IEEE Trans. on
Image Processing, vol. 16, no. 5, pp. 1246-1257,
May 2007. - Six European major universities UPC, IST, EPFL,
UH, INRIA, UNIBS - Special session on Distributed source coding,
2007 IEEE International Conference on Image
Processing (ICIP2007), USA, Sept. 2007 - DISCOVER Workshop on Recent Advances in
Distributed Video Coding, Lisbon, Portugal, Nov.
2007 - http//www.discoverdvc.org/
21Distributed Source Coding
- X?Y in S 000, 001, 010, 011, 100, 101, 110,
111 - H(X) H(Y) 3
- If d(X, Y) 1, H(X) may be reduced to H(XY) 2
- For example, if Y 000 and d(X, Y) 1, the
possible X gt - X in 000, 001, 010, 100 gt H(XY) 2
- A possible solution
- S can be divided into the four disjoint sets
based on d(X, Y) 1 - 000, 111, 100, 011, 010, 101, 001, 110
- At the encoder, if X 100, H(XY) 2 denotes X
in 100, 011 - At the decoder, X 100 can be correctly decoded
based on Y 000 and the correlation between X
and Y, d(X, Y) 1 - X source data to be encoded, Y the side
information of X
22Distributed Source Coding
Slepian-Wolf Theorem, 1973
Encoder
Statistically dependent
Decoder
Encoder
Wyner-Ziv Theorem, 1976
Encoder
Decoder
Girod, 2002
23Distributed Source Coding
Slepian-Wolf Theorem, 1973
Girod, 2002
24Conventional Video Coding
PredictiveInterframe Encoder
PredictiveInterframe Decoder
X
X
Side Information
Girod, 2006
25Distributed Video Coding based on Wyner-Ziv
Theorem
Wyner-ZivIntraframe Encoder
Wyner-ZivInterframe Decoder
X
X
Girod, 2006
26Wyner-Ziv Video Coding
- K key frame, conventional intraframe encoding
- X Wyner-Ziv frame, Wyner-Ziv video encoding
- The corresponding side information Y of X is
generated at decoder based on interpolation of
the previous decoded frames
Girod, 2003
27Side Information Generation
Ebrahimi, 2006
Guo, 2006
28Wyner-Ziv Video Coding
(a)
(b)
(a) The original frame (X) (b) the corresponding
side information (Y) generated at the decoder.
Girod, 2003
29Wyner-Ziv Video Coding
Wyner-Ziv Encoder
Wyner-Ziv Decoder
Minimum distortion Reconstruction
Channel Encoder
Channel Decoder
Quantizer
Correlation channel
Girod, 2002
30Pixel-domain Wyner-Ziv Video Coding
Girod, 2003
31Scalar Quantization
(a)
(b)
(a) The original frame (b) the corresponding 16
gray level quantized frame.
- Scalar quantization in pixel domain
Girod, 2003
32Turbo Encoder
bits output
- For each input block of n 1 bits, the turbo
encoder produces codewords of length n composed
of the actual input bits and one parity bit
Girod, 2002
33Turbo Decoder
Girod, 2002
34Simulation Results
After Wyner-Ziv decoding
Side information
16-level quantization
Girod, 2003
35Simulation Results
Girod, 2003
36Transform-domain Wyner-Ziv Video Coding
WZ frames
Decoded WZ frames
W
W
Interframe Decoder
Intraframe Encoder
IDCT
Xk
Xk
qk
qk
Reconstruction
Request bits
bit-plane Mk
Side information
Yk
For each transform band k
DCT
Y
Interpolation/ Extrapolation
Interpolation/ Extrapolation
Key frames
Conventional Intraframe decoding
Conventional Intraframe coding
K
K
Girod, 2004
37Transform-domain Wyner-Ziv Video Coding
- Each coefficient band is quantized using a scalar
quantizer with 2M levels.
WZ frame
W
- Mk number of bit planes for kth coefficient
- band
4x4 DCT
Xk
For each transform band k
- Combination of quantizers determines the bit
allocation across bands.
level Quantizer
qk
Sample quantizers Values represent number
quantization levels for coefficient band
Girod, 2004
38Transform-domain Wyner-Ziv Video Coding
Turbo Encoder
Turbo Decoder
Extract bit-planes
qk
qk
Buffer
Request bits
bit-plane Mk
Yk
- Bit planes of coefficients are encoded
independently but decoded successively - Rate-compatible punctured turbo code (RCPT)
- Flexibility for varying statistics
- Bit rate controlled by decoder through feedback
channel - Turbo decoder can perform joint source channel
decoding
Girod, 2004
39Simulation Results
Side information
Wyner-Ziv Coding 370 kbps
Girod, 2004
40Simulation Results
H263 Intraframe Coding 330 kbps, 32.9 dB
Wyner-Ziv Coding 274 kbps, 39.0 dB
Girod, 2004
41Simulation Results
H263 interframe coding 145 kbps, 40.4 dB
Wyner-Ziv Coding 156 kbps, 37.5 dB
Girod, 2004
42Simulation Results
Girod, 2004
43DISCOVER DVC Codec
- Based on the feedback channel solution from
Stanford Univ. - Based on a split between Wyner-Ziv (WZ) and key
frames - Key frames used with a regular (GOP size) or
dynamic periodicity - Key frames coded with H.264/AVC Intraframe
encoding
Pereira, 2007
44Simulation Results
Pereira, 2007
45DVC for Wireless Visual Sensor Networks (WVSN)
46Conventional Multiview Video Coding
Multiview video coding structure combining
inter-view and temporal prediction
Kubota, 2007
47Global Motion Estimation
Ebrahimi, 2007
Lin, NTHU, 2007
48Multiview Distributed Video Coding
Ebrahimi, 2006
49Multiview Distributed Video Coding
Temporal side information
Inter-view side information
Ebrahimi, 2007
50Simulation Results
Ebrahimi, 2007
51Collaborative Image Coding and Transmission
1 M. Wu and C. W. Chen, Collaborative image
coding and transmission over Wireless Sensor
Networks, EURASIP Journal on Advances in Signal
Processing, special issue on Visual Sensor
Networks, 2007. 2 K. Y. Chow, K. S. Lui, and E.
Y. Lam, Efficient on-demand image transmission
in visual sensor networks, EURASIP Journal on
Advances in Signal Processing, special issue on
Visual Sensor Networks, 2007.
52Proposed Multiview DVC
- The proposed low-complexity video codec is based
on - the motion estimation is shifted to the decoder
- the low-complexity image matching is performed at
the encoder based on image warping and robust
media hashing - L. W. Kang and C. S. Lu, Low-complexity
power-scalable multi-view distributed video
encoder, in Proc. of 2007 Picture Coding
Symposium, Lisbon, Portugal, Nov. 2007. - L. W. Kang and C. S. Lu, Multi-view distributed
video coding with low-complexity inter-sensor
communication over wireless video sensor
networks, in Proc. of 2007 IEEE Int. Conf. on
Image Processing, special session on Distributed
source coding II Distributed video and image
coding and their applications, San Antonio, TX,
USA, Sept. 2007, vol. 3, pp. 13-16 (invited
paper). - L. W. Kang and C. S. Lu, Low-complexity
Wyner-Ziv video coding based on robust media
hashing, in Proc. of IEEE Int. Workshop on
Multimedia Signal Processing, Victoria, BC,
Canada, Oct. 2006, pp. 267-272. -
- P.S. Co-author Prof. Chun-Shien Lu (??? ??,
??????????)
53Robust Media Hashing
- A compact representation for a frame
54Robust Media Hashing
Structural digital signature (SDS)
Only the parent-child pair with the maximum
magnitude difference (Diff) among those of the
four pairs in a parent-four children pair will
be selected
A parent and its four child nodes.
C. S. Lu and H. Y. M. Liao, Structural digital
signature for image authentication an incidental
distortion resistant scheme, IEEE Trans. on
Multimedia, vol. 5, no. 2, pp. 161-173, June
2003.
55Robust Media Hashing
- Labeling an SDS
- the signature symbol sym(p,c) of a parent-child
pair (p, c) can be defined as follows - each parent-four children pair will be
represented by a symbol sym(p,c), where the pair
(p, c) is with maximum magnitude difference
56Proposed Single-view DVC
An illustrated example for encoding with GOP 4
L. W. Kang and C. S. Lu, Low-complexity
Wyner-Ziv video coding based on robust media
hashing, in Proc. of 2006 IEEE Int. Workshop on
Multimedia Signal Processing, Victoria, BC,
Canada, Oct. 2006, pp. 267-272 (MMSP2006).
57Proposed Multiview DVC
- Consider several adjacent VSNs observing the same
target scene in a WVSN - For each VSN, Vs, an input video sequence is
divided into several GOPs, in which a GOP
consists of a key frame, Ks,t, followed by
several non-key frames, Ws,t
58Key Frame Encoding
- Key frames
- each key frame is encoded using the H.264/AVC
intra-frame encoder first - The global motion estimation between the key
frames from adjacent VSNs will be performed at
the decoder (RCU) - The estimated motion parameters between each pair
of the key frames from adjacent VSNs will be
sent back to the corresponding VSNs via feedback
channel
59Global Motion Estimation between the Key Frames
from Adjacent VSNs
60Key Frame Encoding
61Non-key Frame Encoding
- Based on hash comparisons
- Block coding mode selection (Intra, Inter, or
Skip) - for each frame, all the blocks are sorted in an
increasing order based on their PSNR values
(calculated with their co-located blocks in the
reference frame from the same VSN)
T2
T1
B(1)
B(2)
B(i)
B(i1)
B(i2)
B(j)
B(j1)
B(k)
PSNR(1) PSNR(2) PSNR(i1)
PSNR(k)
Blocks with Intra mode (H.264/AVC intra-frame
encoding)
Blocks with Inter mode (SDS extraction and
comparison)
Blocks with Skip mode
62Non-key Frame Encoding for Blocks with Inter Mode
63Simulation Results
64Concluding Remarks
- Low-complexity video coding becomes a very hot
research topic - Distributed video coding (DVC) based on
distributed source coding (DSC) becomes a new
paradigm of low-complexity video coding - Further researches
- side information generation
- transformation and quantization
- channel coding
- rate control
- Other DSC-related applications
- multimedia authentication
- biometrics security
- layered video coding
- Error resilience for standard video coding
- other low-complexity video coding architectures
65References
- 1 F. Pereira, L. Torres, C. Guillemot, T.
Ebrahimi, R. Leonardi, and S. Klomp, Distributed
video coding selecting the most promising
application scenarios, to appear in Signal
Processing Image Communication. - 2 C. Guillemot, F. Pereira, L. Torres, T.
Ebrahimi. R. Leonardi, J. Ostermann, Distributed
monoview and multiview video coding basics,
problems and recent advances, IEEE Signal
Processing Magazine, vol. 24, no. 5, pp. 67-76,
Sept. 2007. - 3 M. Maitre, C. Guillemot, and L. Morin, 3-D
model-based frame interpolation for distributed
video coding of static scenes, IEEE Trans. on
Image Processing, vol. 16, no. 5, pp. 1246-1257,
May 2007. - 4 R. Puri, A. Majumdar, and K. Ramchandran,
PRISM a video coding paradigm with motion
estimation at the decoder, IEEE Trans. on Image
Processing, vol. 16, no. 10, pp. 2436-2448, Oct.
2007. - 5 R. Puri, A. Majumdar, P. Ishwar, and K.
Ramchandran, Distributed video coding in
wireless sensor networks, IEEE Signal Processing
Magazine, vol. 23, no. 4, pp. 94-106, July 2006. - 6 B. Girod, A. M. Aaron, S. Rane, and D.
Rebollo-Monedero, Distributed video coding,
Proceedings of the IEEE, vol. 93, no. 1, pp.
71-83, Jan. 2005. - 7 X. Artigas, J. Ascenso, M. Dalai, S. Klomp,
D. Kubasov, and M. Ouaret, The DISCOVER codec
architecture, techniques and evaluation, in
Proc. of 2007 Picture Coding Symposium, Lisbon,
Portugal, Nov. 2007.
66Our Preliminary Publications
- 1 L. W. Kang and C. S. Lu, Low-complexity
power-scalable multi-view distributed video
encoder, in Proc. of Picture Coding Symposium,
Lisbon, Portugal, Nov. 2007 (PCS2007). - 2 L. W. Kang and C. S. Lu, Multi-view
distributed video coding with low-complexity
inter-sensor communication over wireless video
sensor networks, in Proc. of IEEE Int. Conf. on
Image Processing, special session on Distributed
Source Coding II Distributed Image and Video
Coding and Their Applications, San Antonio, TX,
USA, Sept. 2007, vol. 3, pp. 13-16 (ICIP2007,
invited paper). - 3 L. W. Kang and C. S. Lu, Low-complexity
Wyner-Ziv video coding based on robust media
hashing, in Proc. of IEEE Int. Workshop on
Multimedia Signal Processing, Victoria, BC,
Canada, Oct. 2006, pp. 267-272 (MMSP2006). - 4 L. W. Kang and C. S. Lu, Wyner-Ziv video
coding with coding mode-aided motion
compensation, in Proc. of IEEE Int. Conf. on
Image Processing, Atlanta, GA, USA, Oct. 2006,
pp. 237-240 (ICIP2006).