Distributed Video Coding for Wireless Visual Sensor Networks - PowerPoint PPT Presentation

1 / 66
About This Presentation
Title:

Distributed Video Coding for Wireless Visual Sensor Networks

Description:

Title: Mobile, Multimedia and Beyond Author: Westrich Last modified by: lwkang Created Date: 5/5/1999 5:00:09 PM Document presentation format: A4 (210x297 ) – PowerPoint PPT presentation

Number of Views:323
Avg rating:3.0/5.0
Slides: 67
Provided by: West162
Category:

less

Transcript and Presenter's Notes

Title: Distributed Video Coding for Wireless Visual Sensor Networks


1
Distributed Video Coding for Wireless Visual
Sensor Networks
2
Outline
  • Introduction
  • Distributed Source Coding (DSC)
  • Distributed Video Coding (DVC)
  • DVC for Wireless Visual Sensor Networks (WVSN)
  • Concluding Remarks
  • References

3
Introduction
  • Conventional video coding
  • MPEG-1/2/4, H.261, H.263, H.26L, H.264/AVC
  • Interframe predictive coding
  • Encoder is 5-10 times more complex than decoder
  • Suitable for video down-link

Xi-1
Girod, 2002
4
Conventional Video Coding
Aramvith
5
Conventional Video Coding
Lin, NTHU, 2007
6
Transformation and Quantization
Lin, NTHU, 2007
7
Interframe Predictive Video Coding
Lin, NTHU, 2007
8
Motion Estimation
Lin, NTHU, 2007
9
Motion Estimation
Lin, NTHU, 2007
10
Motion Compensated Prediction
Lin, NTHU, 2007
11
Applications of Conventional Video Coding
Pereira, 2007
12
Introduction
  • Problem low-complexity video encoding for
    resource-limited video devices
  • DSC approach Wyner-Ziv video coding with
    low-complexity intraframe encoding and possibly
    high-complexity interframe decoding with side
    information only available at decoder

Side Information
Girod, 2002
13
Applications of Low-Complexity Video Coding
  • Wireless video cameras
  • Wireless low-power surveillance
  • Mobile document scanner
  • Video conferencing with mobile devices
  • Mobile video mail
  • Disposable video cameras
  • Wireless Visual Sensor Networks
  • Networked camcorders
  • Distributed video streaming
  • Multiview video entertainment
  • Wireless capsule endoscopy

Pereira, 2007
14
Applications of Low-Complexity Video Coding
Pereira, 2007
15
Applications of Low-Complexity Video Coding
Pereira, 2007
16
Wireless Visual Sensor Networks
Akyildiz, 2007, and Pereira, 2007
17
Wireless Visual Sensor Networks
Akyildiz, 2002
18
Introduction
  • Requirements of wireless visual sensor networks
  • low-complexity video encoder
  • high compression efficiency
  • Current approaches
  • distributed video coding (DVC) based on
    distributed source coding (DSC)
  • collaborative image coding and transmission
  • hybrid approach (proposed approach)

19
Distributed Source Coding (DSC)
  • Lossless DSC, Slepian and Wolf, 1973
  • Lossy DSC, Wyner and Ziv, 1976
  • Distributed video coding (DVC) based on DSC
  • Girod, Stanford University, 2002
  • B. Girod, A. M. Aaron, S. Rane, and D.
    Rebollo-Monedero, Distributed video coding,
    Proceedings of the IEEE, vol. 93, no. 1, pp.
    71-83, Jan. 2005.
  • Special session on Distributed video coding, 2005
    IEEE International Conference on Image Processing
    (ICIP2005), Italy, Sept. 2005
  • Ramchandran, Berkeley, 2002
  • R. Puri, A. Majumdar, and K. Ramchandran, PRISM
    a video coding paradigm with motion estimation at
    the decoder, IEEE Trans. on Image Processing,
    vol. 16, no. 10, pp. 2436-2448, Oct. 2007.
  • R. Puri, A. Majumdar, P. Ishwar, and K.
    Ramchandran, Distributed video coding in
    wireless sensor networks, IEEE Signal Processing
    Magazine, vol. 23, no. 4, pp. 94-106, July 2006.

20
Distributed Source Coding
  • DISCOVER (Distributed Coding for Video Services)
  • 2005
  • F. Pereira, L. Torres, C. Guillemot, T. Ebrahimi,
    R. Leonardi, and S. Klomp, Distributed video
    coding selecting the most promising application
    scenarios, to appear in Signal Processing Image
    Communication.
  • C. Guillemot, F. Pereira, L. Torres, T. Ebrahimi.
    R. Leonardi, J. Ostermann, Distributed monoview
    and multiview video coding basics, problems and
    recent advances, IEEE Signal Processing
    Magazine, special issue on signal processing for
    multiterminal communication systems, vol. 24, no.
    5, pp. 67-76, Sept. 2007.
  • M. Maitre, C. Guillemot, and L. Morin, 3-D
    model-based frame interpolation for distributed
    video coding of static scenes, IEEE Trans. on
    Image Processing, vol. 16, no. 5, pp. 1246-1257,
    May 2007.
  • Six European major universities UPC, IST, EPFL,
    UH, INRIA, UNIBS
  • Special session on Distributed source coding,
    2007 IEEE International Conference on Image
    Processing (ICIP2007), USA, Sept. 2007
  • DISCOVER Workshop on Recent Advances in
    Distributed Video Coding, Lisbon, Portugal, Nov.
    2007
  • http//www.discoverdvc.org/

21
Distributed Source Coding
  • X?Y in S 000, 001, 010, 011, 100, 101, 110,
    111
  • H(X) H(Y) 3
  • If d(X, Y) 1, H(X) may be reduced to H(XY) 2
  • For example, if Y 000 and d(X, Y) 1, the
    possible X gt
  • X in 000, 001, 010, 100 gt H(XY) 2
  • A possible solution
  • S can be divided into the four disjoint sets
    based on d(X, Y) 1
  • 000, 111, 100, 011, 010, 101, 001, 110
  • At the encoder, if X 100, H(XY) 2 denotes X
    in 100, 011
  • At the decoder, X 100 can be correctly decoded
    based on Y 000 and the correlation between X
    and Y, d(X, Y) 1
  • X source data to be encoded, Y the side
    information of X

22
Distributed Source Coding
Slepian-Wolf Theorem, 1973
Encoder
Statistically dependent
Decoder
Encoder
Wyner-Ziv Theorem, 1976
Encoder
Decoder
Girod, 2002
23
Distributed Source Coding
Slepian-Wolf Theorem, 1973
Girod, 2002
24
Conventional Video Coding
PredictiveInterframe Encoder
PredictiveInterframe Decoder
X
X
Side Information
Girod, 2006
25
Distributed Video Coding based on Wyner-Ziv
Theorem
Wyner-ZivIntraframe Encoder
Wyner-ZivInterframe Decoder
X
X
Girod, 2006
26
Wyner-Ziv Video Coding
  • K key frame, conventional intraframe encoding
  • X Wyner-Ziv frame, Wyner-Ziv video encoding
  • The corresponding side information Y of X is
    generated at decoder based on interpolation of
    the previous decoded frames

Girod, 2003
27
Side Information Generation
Ebrahimi, 2006
Guo, 2006
28
Wyner-Ziv Video Coding
(a)
(b)
(a) The original frame (X) (b) the corresponding
side information (Y) generated at the decoder.
Girod, 2003
29
Wyner-Ziv Video Coding
Wyner-Ziv Encoder
Wyner-Ziv Decoder
Minimum distortion Reconstruction
Channel Encoder
Channel Decoder
Quantizer
Correlation channel
Girod, 2002
30
Pixel-domain Wyner-Ziv Video Coding
Girod, 2003
31
Scalar Quantization
(a)
(b)
(a) The original frame (b) the corresponding 16
gray level quantized frame.
  • Scalar quantization in pixel domain

Girod, 2003
32
Turbo Encoder
bits output
  • For each input block of n 1 bits, the turbo
    encoder produces codewords of length n composed
    of the actual input bits and one parity bit

Girod, 2002
33
Turbo Decoder
Girod, 2002
34
Simulation Results
After Wyner-Ziv decoding
Side information
16-level quantization
Girod, 2003
35
Simulation Results
Girod, 2003
36
Transform-domain Wyner-Ziv Video Coding
WZ frames
Decoded WZ frames
W
W
Interframe Decoder
Intraframe Encoder
IDCT
Xk
Xk
qk
qk
Reconstruction

Request bits
bit-plane Mk
Side information
Yk
For each transform band k
DCT
Y
Interpolation/ Extrapolation
Interpolation/ Extrapolation
Key frames
Conventional Intraframe decoding
Conventional Intraframe coding
K
K
Girod, 2004
37
Transform-domain Wyner-Ziv Video Coding
  • Each coefficient band is quantized using a scalar
    quantizer with 2M levels.

WZ frame
W
  • Mk number of bit planes for kth coefficient
  • band

4x4 DCT
Xk
For each transform band k
  • Combination of quantizers determines the bit
    allocation across bands.

level Quantizer
qk
Sample quantizers Values represent number
quantization levels for coefficient band
Girod, 2004
38
Transform-domain Wyner-Ziv Video Coding
Turbo Encoder
Turbo Decoder
Extract bit-planes
qk
qk
Buffer

Request bits
bit-plane Mk
Yk
  • Bit planes of coefficients are encoded
    independently but decoded successively
  • Rate-compatible punctured turbo code (RCPT)
  • Flexibility for varying statistics
  • Bit rate controlled by decoder through feedback
    channel
  • Turbo decoder can perform joint source channel
    decoding

Girod, 2004
39
Simulation Results
Side information
Wyner-Ziv Coding 370 kbps
Girod, 2004
40
Simulation Results
H263 Intraframe Coding 330 kbps, 32.9 dB
Wyner-Ziv Coding 274 kbps, 39.0 dB
Girod, 2004
41
Simulation Results
H263 interframe coding 145 kbps, 40.4 dB
Wyner-Ziv Coding 156 kbps, 37.5 dB
Girod, 2004
42
Simulation Results
Girod, 2004
43
DISCOVER DVC Codec
  • Based on the feedback channel solution from
    Stanford Univ.
  • Based on a split between Wyner-Ziv (WZ) and key
    frames
  • Key frames used with a regular (GOP size) or
    dynamic periodicity
  • Key frames coded with H.264/AVC Intraframe
    encoding

Pereira, 2007
44
Simulation Results
Pereira, 2007
45
DVC for Wireless Visual Sensor Networks (WVSN)
46
Conventional Multiview Video Coding
Multiview video coding structure combining
inter-view and temporal prediction
Kubota, 2007
47
Global Motion Estimation
Ebrahimi, 2007
Lin, NTHU, 2007
48
Multiview Distributed Video Coding
Ebrahimi, 2006
49
Multiview Distributed Video Coding
Temporal side information
Inter-view side information
Ebrahimi, 2007
50
Simulation Results
Ebrahimi, 2007
51
Collaborative Image Coding and Transmission
1 M. Wu and C. W. Chen, Collaborative image
coding and transmission over Wireless Sensor
Networks, EURASIP Journal on Advances in Signal
Processing, special issue on Visual Sensor
Networks, 2007. 2 K. Y. Chow, K. S. Lui, and E.
Y. Lam, Efficient on-demand image transmission
in visual sensor networks, EURASIP Journal on
Advances in Signal Processing, special issue on
Visual Sensor Networks, 2007.
52
Proposed Multiview DVC
  • The proposed low-complexity video codec is based
    on
  • the motion estimation is shifted to the decoder
  • the low-complexity image matching is performed at
    the encoder based on image warping and robust
    media hashing
  • L. W. Kang and C. S. Lu, Low-complexity
    power-scalable multi-view distributed video
    encoder, in Proc. of 2007 Picture Coding
    Symposium, Lisbon, Portugal, Nov. 2007.
  • L. W. Kang and C. S. Lu, Multi-view distributed
    video coding with low-complexity inter-sensor
    communication over wireless video sensor
    networks, in Proc. of 2007 IEEE Int. Conf. on
    Image Processing, special session on Distributed
    source coding II Distributed video and image
    coding and their applications, San Antonio, TX,
    USA, Sept. 2007, vol. 3, pp. 13-16 (invited
    paper).
  • L. W. Kang and C. S. Lu, Low-complexity
    Wyner-Ziv video coding based on robust media
    hashing, in Proc. of IEEE Int. Workshop on
    Multimedia Signal Processing, Victoria, BC,
    Canada, Oct. 2006, pp. 267-272.
  • P.S. Co-author Prof. Chun-Shien Lu (??? ??,
    ??????????)

53
Robust Media Hashing
  • A compact representation for a frame

54
Robust Media Hashing
Structural digital signature (SDS)
Only the parent-child pair with the maximum
magnitude difference (Diff) among those of the
four pairs in a parent-four children pair will
be selected
A parent and its four child nodes.
C. S. Lu and H. Y. M. Liao, Structural digital
signature for image authentication an incidental
distortion resistant scheme, IEEE Trans. on
Multimedia, vol. 5, no. 2, pp. 161-173, June
2003.
55
Robust Media Hashing
  • Labeling an SDS
  • the signature symbol sym(p,c) of a parent-child
    pair (p, c) can be defined as follows
  • each parent-four children pair will be
    represented by a symbol sym(p,c), where the pair
    (p, c) is with maximum magnitude difference

56
Proposed Single-view DVC
An illustrated example for encoding with GOP 4
L. W. Kang and C. S. Lu, Low-complexity
Wyner-Ziv video coding based on robust media
hashing, in Proc. of 2006 IEEE Int. Workshop on
Multimedia Signal Processing, Victoria, BC,
Canada, Oct. 2006, pp. 267-272 (MMSP2006).
57
Proposed Multiview DVC
  • Consider several adjacent VSNs observing the same
    target scene in a WVSN
  • For each VSN, Vs, an input video sequence is
    divided into several GOPs, in which a GOP
    consists of a key frame, Ks,t, followed by
    several non-key frames, Ws,t

58
Key Frame Encoding
  • Key frames
  • each key frame is encoded using the H.264/AVC
    intra-frame encoder first
  • The global motion estimation between the key
    frames from adjacent VSNs will be performed at
    the decoder (RCU)
  • The estimated motion parameters between each pair
    of the key frames from adjacent VSNs will be
    sent back to the corresponding VSNs via feedback
    channel

59
Global Motion Estimation between the Key Frames
from Adjacent VSNs
60
Key Frame Encoding
61
Non-key Frame Encoding
  • Based on hash comparisons
  • Block coding mode selection (Intra, Inter, or
    Skip)
  • for each frame, all the blocks are sorted in an
    increasing order based on their PSNR values
    (calculated with their co-located blocks in the
    reference frame from the same VSN)

T2
T1
B(1)
B(2)
B(i)
B(i1)
B(i2)
B(j)
B(j1)
B(k)



PSNR(1) PSNR(2) PSNR(i1)

PSNR(k)
Blocks with Intra mode (H.264/AVC intra-frame
encoding)
Blocks with Inter mode (SDS extraction and
comparison)
Blocks with Skip mode
62
Non-key Frame Encoding for Blocks with Inter Mode
63
Simulation Results
64
Concluding Remarks
  • Low-complexity video coding becomes a very hot
    research topic
  • Distributed video coding (DVC) based on
    distributed source coding (DSC) becomes a new
    paradigm of low-complexity video coding
  • Further researches
  • side information generation
  • transformation and quantization
  • channel coding
  • rate control
  • Other DSC-related applications
  • multimedia authentication
  • biometrics security
  • layered video coding
  • Error resilience for standard video coding
  • other low-complexity video coding architectures

65
References
  • 1 F. Pereira, L. Torres, C. Guillemot, T.
    Ebrahimi, R. Leonardi, and S. Klomp, Distributed
    video coding selecting the most promising
    application scenarios, to appear in Signal
    Processing Image Communication.
  • 2 C. Guillemot, F. Pereira, L. Torres, T.
    Ebrahimi. R. Leonardi, J. Ostermann, Distributed
    monoview and multiview video coding basics,
    problems and recent advances, IEEE Signal
    Processing Magazine, vol. 24, no. 5, pp. 67-76,
    Sept. 2007.
  • 3 M. Maitre, C. Guillemot, and L. Morin, 3-D
    model-based frame interpolation for distributed
    video coding of static scenes, IEEE Trans. on
    Image Processing, vol. 16, no. 5, pp. 1246-1257,
    May 2007.
  • 4 R. Puri, A. Majumdar, and K. Ramchandran,
    PRISM a video coding paradigm with motion
    estimation at the decoder, IEEE Trans. on Image
    Processing, vol. 16, no. 10, pp. 2436-2448, Oct.
    2007.
  • 5 R. Puri, A. Majumdar, P. Ishwar, and K.
    Ramchandran, Distributed video coding in
    wireless sensor networks, IEEE Signal Processing
    Magazine, vol. 23, no. 4, pp. 94-106, July 2006.
  • 6 B. Girod, A. M. Aaron, S. Rane, and D.
    Rebollo-Monedero, Distributed video coding,
    Proceedings of the IEEE, vol. 93, no. 1, pp.
    71-83, Jan. 2005.
  • 7 X. Artigas, J. Ascenso, M. Dalai, S. Klomp,
    D. Kubasov, and M. Ouaret, The DISCOVER codec
    architecture, techniques and evaluation, in
    Proc. of 2007 Picture Coding Symposium, Lisbon,
    Portugal, Nov. 2007.

66
Our Preliminary Publications
  • 1 L. W. Kang and C. S. Lu, Low-complexity
    power-scalable multi-view distributed video
    encoder, in Proc. of Picture Coding Symposium,
    Lisbon, Portugal, Nov. 2007 (PCS2007).
  • 2 L. W. Kang and C. S. Lu, Multi-view
    distributed video coding with low-complexity
    inter-sensor communication over wireless video
    sensor networks, in Proc. of IEEE Int. Conf. on
    Image Processing, special session on Distributed
    Source Coding II Distributed Image and Video
    Coding and Their Applications, San Antonio, TX,
    USA, Sept. 2007, vol. 3, pp. 13-16 (ICIP2007,
    invited paper).
  • 3 L. W. Kang and C. S. Lu, Low-complexity
    Wyner-Ziv video coding based on robust media
    hashing, in Proc. of IEEE Int. Workshop on
    Multimedia Signal Processing, Victoria, BC,
    Canada, Oct. 2006, pp. 267-272 (MMSP2006).
  • 4 L. W. Kang and C. S. Lu, Wyner-Ziv video
    coding with coding mode-aided motion
    compensation, in Proc. of IEEE Int. Conf. on
    Image Processing, Atlanta, GA, USA, Oct. 2006,
    pp. 237-240 (ICIP2006).
Write a Comment
User Comments (0)
About PowerShow.com