Shape Adaptive Coding Of Multiple Objects and an Application to Interactivity

1
Shape Adaptive Coding Of Multiple Objects and an
Application to Interactivity

• Eric Lin Shaan Patel Hyungsik Shin

2
Outline

• Motivation for Object Coding
• Algorithms
• Experimental Setup
• Results
• Interactivity
• Application/DEMO

3
Motivation

• Compression Efficiency
• Statistics of Objects Isolated
• Reduce High Frequencies
• More Accurate Motion Estimation
• Better Scanning and Entropy Coding
• Interactivity

4
Shape Preserving Property

• Definition
• Exactly the same number of transform coefficients
  as pixels within the original block of an
  arbitrary shape

5
Standard SA - DCT

• Pros Good Compression Efficiency
• Cons K-Point DCT, Mean Weighting Defect
• Sikora 95

6
Low-Pass Extrapolation

• Steps
• Find Mean of Object
• Apply Mean to Non-Object
• Low-Pass Filter Non-Object
• Apply 2-D N-Point DCT

• Pros Simplest, Cheapest Computationally, 2D
  N-Point DCT
• Cons No Shape Preserving Property, Poor
  Compression Efficiency

7
Extension Interpolation

• Steps
• Apply K-point 1-D DCT to each column
• Append N-K zeros at the end of the DCT
  coefficient vector
• Apply N-point 1-D IDCT to new, padded coefficient
  vector. (Do for all columns)
• N x N 2-D DCT is performed on the N x N block

• Pros Steps 1) 3) can be implemented together
  via a multiplication matrix of dimension N x K
• Cons Computationally demanding and de-padding
  of the same complexity at the decoder as the
  encoder
• Yi 98

8
NewPad DCT

• Pros Zero forcing of high frequency components,
  shape preserving properties
• Cons Algorithm Complexity
• Shen 01

9
Comparison of the 4 algorithms

• Shen 00

10
Shape Coding

• Need to transmit shape information to the decoder
• Two different kinds of shape information Shape
  and Shape-Residual
• Shape 0 or 1 , Shape-Residual 0, 1, or -1
• Use run-length coding

11
Sequence
12
More Objects
13
Quantified More Objects
14
Intra Results
15
Inter Results
16
Interactivity/Application

• Can place focus on particular object by coding it
  with better quality
• Allows for users to select, manipulate and change
  attributes of VO
• Can compose scenes with a mixture of objects from
  different sources
• Better customization can enhance user experience
• 
  Nunes, P.Correia,
  F.Pereira, 97

17
Potential Features

• Dragging an Object
• Enabling/Disabling visibility of object
• Swapping the background
• Varying the zoom level of a particular object

18
DEMO
19
Results
20
Conclusion

• Compression Efficiency
• Low Rates Shape Information Large Overhead
• High Rates Shape Adaptive Coding Gains
• Interactive Application
• Features
• Dragging Objects
• Change Visibility of Object
• Change Zoom Level of Object
• Cost
• Potentially Higher Rates
• Complexity

21
References

• 1 T. Sikora, Trends and Perspectives in Image
  and Video Coding, Proc. IEEE, vol. 93, no. 1,
  Jan. 2005, pp. 6-17.
• 2 E. Lin, DEMO. May 2007, lthttp//www.stanford.e
  du/avancal/EE398B/demo.htmlgt
• 3 T. Sikora and B. Makai, "Shape-adaptive DCT
  for generic coding of video," IEEE Transactions
  on Circuits and Systems for Video Technology
  (CSVT), vol. 5, no. 1, pp. 59-62, Feb. 1995.
• 4 P. Kauff and K. Schuur, Shape-adaptive DCT
  with block-based DC separation and Delta DC
  correction, IEEE Trans. Circuits Syst. Video
  Technol., vol. 8, pp. 237242, June 1998.
• 5 M. Yi, S. J. Cho, W. J. Kim, S. D. Kim, and
  S. J. Lee, A new coding algorithm for
  arbitrarily shaped image segments, Signal
  Process. Vol. 12, pp. 231242, 1998.
• 6 Shen, G., Zeng, B., Ming Lei Liou,
  Arbitrarily Shaped Transform Coding Based on a
  New Padding Technique, Circuits and Systems for
  Video Technology, IEEE Transactions Volume
  11, Issue 1, Jan. 2001 Page(s) 67 79
• 7 Y. Wang, MPEG-4 Video Samples.
  lthttp//eeweb.poly.edu/yao/VideobookSampleData/do
  c/sample/MPEG4.htmgt
• 8 G. Shen, B. Zeng and M. L. Liou, "An
  efficient hybrid arbitrarily shaped object
  coding technique," IEEE International Symposium
  on Circuits and Systems (ISCAS), Geneva,
  Switzerland, May 2000.
• 9 Low-Complexity MPEG-4 Shape Encoding towards
  Real-time Object-Based Applications Euee Seon
  Jang, ETRI Journal, vol.26, no.2, Apr. 2004,
  pp.122-135.

22
Thank You

• Questions?