MultiView Image Coding with DisparityCompensated Lifted Wavelets

1
Multi-View Image Coding with Disparity-Compensated
Lifted Wavelets

• Steven Lansel
• Nandhini Nandiwada Santhanam
• EE 398 Image and Video Compression
• Winter 2006

2
Overview

• Problem Overview
• Disparity Vector Selection
• Transform Explanation
• Pareto-like Condition
• Overall R-D Curve
• Optimal Bit Allocation
• Conclusions

3
Problem Overview

• Images are taken of a scene from multiple angles
• Lifted Haar scheme is used to encode two images
• Transform uses disparity vectors to match similar
  blocks in the two images
• Optimal allocation of bits between disparity
  vectors and L/H images is desired

SPIHT
L
-
P
U
P-1
U-1
-
H
SPIHT
4
Disparity Vector Selection

• For each block in image B, the best block in
  image A is found
• Disparity vectors for each block are chosen to
  minimize JD?DVR where D is the sum of squared
  errors and R is the bit rate for the vector
• Bit rate for vectors follows Laplacian model

5
Transform Explanation
Input B
Input A
a is percent of connected pixels in A
Output H
6
Transform Explanation (2)
Input B
Input A
For each pixel in L use the pixel in the same
location in A and the first pixel in B that is
mapped to that pixel in A.
Output L
Output H
Not used for unconnected pixels
7
Inverse Transform and Distortion Equations

• Inverse transform equations
• Distortion equations for non-orthonormal
  transform

Not used for unconnected pixels
8
Transform Characteristics based on Lambda for DVs
9
Rate-Distortion Curves when Compressing L or H
Image
10
Optimal Bit Allocation using a Given Set of
Disparity Vectors

• Total Distortion
• Cost Function
• Pareto-like Solution
• Total Rate Constraint

11
R-D Dependence on Lambda for DVs
Enlarged region from left plot
12
Overall Rate-Distortion Curve
13
Optimal Bit Allocation
14
Conclusion

• Disparity-compensation matches similar blocks in
  the two input images
• Resulting transform is not orthonormal
• Disparity vectors must be determined by finding
  the best ? for a given total rate
• Allocation of bits between L and H images is
  achieved by satisfying weighted equation and
  total bit rate constraint

15
References

• Aditya Atul Mavlankar, Robust Video
  Communication using Motion-Compensated Lifted 3-D
  Wavelet Coding, Masters Thesis, Technische
  Universität München Lehrstuhl für
  Kommunikationsnetze Fachgebiet Medientechnik,
  September 2004.
• B. Pesquet-Popescu and V. Bottreau,
  "Three-Dimensional Lifting Schemes for Motion
  Compensated Video Compression," Proceedings of
  the IEEE International Conference on Acoustics,
  Speech and Signal Processing, Salt Lake City, UT,
  May 2001, pp. 1793-1796.
• Chuo-Ling Chang, Xiaoqing Zhu, Prashant
  Ramanathan, and Bernd Girod, "Inter-view Wavelet
  Compression of Light Fields with
  Disparity-Compensated Lifting," Proc. SPIE Visual
  Communications and Image Processing, VCIP-03,
  Lugano, Switzerland, July 2003, pp. 14-22.