Shortterm CV Motion Prediction and its use in long term motion compensation

1
Short-term CV Motion Predictionand its use in
long term motion compensation

• Stanford University
• EE398 Final Project
• Zhengyun Zhang, Dileep George
• Winter 2005

2
Overview

• Problem Formulation
• Short-term Constant Velocity Predictor
• Results
• Conclusions

3
Problem Formulation

• New codecs (e.g. H.264/AVC) can use long-term
  motion compensation to increase coding gain
• Computationally expensive, O(NxM2xD)
• N macro-blocks
• M length of average motion vector
• D frames to search
• Can we find a more efficient algorithm?

4
Short-term Constant Velocity Model

• Objects in motion tend to not change in velocity
  drastically
• Short term velocity approximately constant
• Predictor can follow this model

5
Short-term CV First Step Predictor
t-2
Desired predictor from past
t-1
t
x
x-1
x1
6
Short-term CV Second Step Predictor
t-N
t-N1
Predictor
b
t-N2
c2b-a
c
a
...
t
x
x-1
x1
7
Short-term CVMotion Search

• Search fixed window centered on predictor (x,y)
• Same as brute-force search, except brute-force
  search always predicts (0,0)
• Short term CV also keeps track of best predictor
  for each frame in the backbuffer while searching

8
ResultsForeman (frames 1-399)
9
ResultsMobile (frames 1-299)
10
Conclusions

• Works comparatively well for small search ranges,
  but not for larger ones
• Good predictor, since its closer to optimal for
  very small search ranges
• Combining with a better per-frame search
  algorithm should get better gain
• e.g. steepest descent
• Combining with decision-tree structure can reduce
  errors with large search ranges

11
References

• H. Chung, A. Ortega, A. A. Sawchuk, "Low
  complexity motion estimation for long term memory
  motion compensation," Proc. Visual Communications
  and Image Processing, VCIP 2002, San Jose, CA,
  January 2002.
• A. M. Tourapis, Enhanced predictive zonal search
  for single and multiple frame motion estimation,
  VCIP 2002, pp.1069-1079
• T. Wiegand, G. J. Sullivan, G. Bjontegaard, A.
  Luthra, Overview of the H.264/AVC Video Coding
  Standard, IEEE Trans. CSVT, Vol. 13, No. 7,
  pp.560-576, July, 2003