Distributed Source Coding Using Syndromes DISCUS: Design and Construction

1
Distributed Source Coding Using Syndromes
(DISCUS) Design and Construction

• S.Sandeep Pradhan, Kannan Ramchandran
• IEEE Transactions on Information Theory,
• vol. 49, no.3, pp.626-643, Mar 2003

2
Outline

• Introduction
• Preliminaries
• Encoding with a Fidelity Criterion
• Problem Formulation
• Design Algorithm
• Constructions based on Trellis Codes
• Simulation Results
• Conclusion

3
Introduction

• Slepian-Wolf theorem
• By knowing joint distribution of X and Y,
  without explicitly knowing Y, encoder of X can
  perform as well as encoder who knows Y.

• Both encoder and decoder have access to side
  information Y
• Only decoder has access to side information Y

4
Introduction

• Wyner-Ziv Problem
• If decoder knows Y, then the information-theoretic
  rate-distortion performance for coding X is
  identical, no matter encoder knows Y or not.(X Y
  are Gaussian.)
• Prior work on source quantizer design.
• Contributions
• Construction of a framework resting on algebraic
  channel coding principles
• Performance analysis on Gaussian signals.

Source discrete-alphabet ? continuous-valued Comp
ression lossless ? lossy
5
Outline

• Introduction
• Preliminaries
• Encoding with a Fidelity Criterion
• Problem Formulation
• Design Algorithm
• Constructions based on Trellis Codes
• Simulation Results
• Conclusion

6
Preliminaries

• Example
• X, Y equiprobable 3-bit binary words
• Hamming distance is no more than 1.
• Y is available to decoder.
• Solution?
• Cosets 000,111,100,011,010,101,001, 110
• Only transmit coset index/syndrome.

7
Preliminaries

• Quantization
• Digitizes an analog signal.
• Two parameters a partition and a codebook.
• Codebook -2, 0.4, 2.3, 6

8
Preliminaries

• Lloyd Max Quantization
• partition ai are midpoints.
• codebook yi are centroids.
• Optimal scalar quantization.

9
Preliminaries

• Trellis Coded Quantization (TCQ)24
• Dual of TCM
• Example
• Uniformly distributed source in -A, A
• Implemented by Viterbi algorithm
• 24 M.W. Marcellin and T. R. Fischer, Trellis
  coded quantization of memoryless and Gauss-Markov
  sources, IEEE Trans. Commun., vol. 38, pp.8293,
  Jan. 1990.

10
Outline

• Introduction
• Preliminaries
• Encoding with a Fidelity Criterion
• Problem Formulation
• Design Algorithm
• Constructions based on Trellis Codes
• Simulation Results
• Conclusion

11
Encoding with a Fidelity Criterion

• Problem Formulation
• X, Y correlated, memoryless, i.i.d distributed
  sequences
• Yi Xi Ni
• Xi, Yi, Ni continuous-valued
• Ni i.i.d distributed, independent from X
• Xi, Ni zero-mean Gaussian random variables with
  known variance
• Decoder alone has access to Y.
• Goal Form best approximation to X given R bits
  per sample
• Encoding in blocks of length L
• Distortion measure
• Min R, s.t. reconstruction fidelity is less than
  given value D.

12
Encoding with a Fidelity Criterion
System Model encoder and decoder. Interplay of
source coding, channel coding and estimation
13
Encoding with a Fidelity Criterion

• Design Algorithm
• Source Coding (M1, M2)
• Partition source space
• Defining source codebook (S)
• Characterizing active codeword by W (r.v.)
• Estimation (M3)
• Get best estimate of X (minimizing distortion)
  conditioned on outcome of Y and the element in
  .
• Channel Coding (M4, M5)
• Transmit over an error-free channel with rate R
  (less than Rs)
• Doable I(WY) gt 0, so H(WY) H(W) I(WY)
• Build channel code with rate Rc on channel P(YW)
• R Rs Rc.

14
Encoding with a Fidelity Criterion

• Summary of Design Algorithm
• M1 and M3
• minimize Rs, s.t. reconstruction distortion
  within given criterion.
• M2 maximize I(WY).
• M4
• maximize Rc, s.t. error probability meets a
  desired tolerance level.
• M5 minimize decoding computational complexity.

15
Encoding with a Fidelity Criterion

• Scalar Quantization and Memoryless Coset
  Construction (C1)
• Lloyd-Max (memoryless) quantizer
• Memoryless coset partition (M4)
• Example
• L1, (sample by sample)
• Quantization codebook r0, r1, , r7, (Rs 3)
• Channel coding codebook r0, r2, r4, r6, r1,
  r3, r5, r7. (Rc 2)
• R Rs Rc 1 bit/sample.

16
Encoding with a Fidelity Criterion

• Scalar Quantization and Trellis-Based Coset
  Construction (C2)
• Scalar quantizer for Xii1L
• Coset partition (M4) by trellis code.

Codebook (size of 8L), Rs 3 bits/sample, two
cosets
17
Encoding with a Fidelity Criterion

• Example
• Computing syndrome (Rs 3, Rc 2)
• outcome of quantization be 7, 3, 2, 1, 4.
• L 5,
• Syndrome is given by 10110 for 5 samples.

18
Encoding with a Fidelity Criterion

• Trellis-Based Quantization and Memoryless Coset
  Construction (C3)
• Trellis coded quantizer
• Memoryless coset partition
• Example
• Quantization codebook Rs 2
• D0r0, r4, D1r1, r5, D2r2, r6, D3r3,
  r7.
• Memoryless channel code Rc 1
• 1 coded bit with another 1 uncoded bit (from Y)
  to recover Di.

19
Encoding with a Fidelity Criterion

• Trellis-Based Quantization and Trellis-Based
  Coset Coset Construction (C4)
• Trellis coded quantizer
• Trellis coded coset partition

Comparison between C3 and C4.
20
Encoding with a Fidelity Criterion

• Distance Property
• Given a uniform partition, four cases of coset
  constructions have same distance property.
• Non-uniform quantizer, analyze performance by
  simulations.

21
Outline

• Introduction
• Preliminaries
• Encoding with a Fidelity Criterion
• Problem Formulation
• Design Algorithm
• Four Constructions
• Simulation Results
• Conclusion

22
Simulation Results
Correlation -SNR ratio of Xs variance and Ns
variance.

• Quantization levels decrease distortion. (C1)

23
Simulation Results
Correlation -SNR ratio of Xs variance and Ns
variance.
Quantization levels increase prob. Of error. (C1)
24
Simulation Results
Correlation -SNR ratio of Xs variance and Ns
variance.
Error probability comparison of C1 and C2 (3-4dB
gain)
25
Simulation Results
Correlation -SNR ratio of Xs variance and Ns
variance.
Error probability of C4 codes.
26
Conclusions

• Constructive practical framework based on
  algebraic trellis codes.
• Promising performance.