Forward-Backward Correlation for Template-Based Tracking

1
Forward-Backward Correlation for Template-Based
Tracking

• Xiao Wang
• ECE Dept.
• Clemson University

2
Introduction

• Object tracking An important computer vision
  problem
• Security and surveillance
• Medical therapy
• Retail space instrumentation
• Video abstraction
• Traffic Management
• Video editing
• Template-Based Tracking
• A classic technique
• Idea of template-based tracker

3
Related Work

• Jepson et al. Robust Online Appearance Models for
  Visual Tracking, CVPR 2001
• Ho et al. Visual Tracking Using Learned
  Subspaces, CVPR 2004
• Davis et al. Tracking Rigid Motion using a
  Compact-Structure Constraint, ICCV 1999
• Avidan et al. Ensemble Tracking, CVPR 2005

4
Overview of the Approach
Forward Correlation Module
Textured Background?
Next Frame
Yes
No
5
Template-Based Tracking

• Template Selection first frame vs. previous
  frame
• Motion Model
• Similarity transformation

scaling
displacement
6
Template-Based Tracking

• Cross Correlation SSD

reference image
displacement
search image
7
Template-Based Tracking

• Similarity measure s(?x, ?y)
• Correlation Coefficient c(?x, ?y)

Mean of template
Mean of image region
8
Forward Correlation

• Forward Correlation
• Reference frame previous frame
• Goal find transformation vector (dx, dy, a)
• Approach cross-correlation

Put into correlation coefficient framework

• Template Update

9
Forward Correlation
Out-of-plane rotation

• Drifting Problem
• Forward correlation approximates rotation with
  translation.
• Forward correlation does not check the
  reliability of the template.
• We need a mechanism to question the assumption of
  forward correlation.

Previous frame
Current Frame
10
Backward Correlation

• Consider our problem as motion segmentation
• Goal of motion segmentation
• Why is motion segmentation of video sequences
  difficult?
• Under-constrained
• Occlusion Disocclusion
• Image noise
• A two-step procedure
• Determine the motion vectors associate with each
  pixel or feature point.
• Group pixels or feature points that perform
  common motion.

11
Backward Correlation

• Kanade-Lucas-Tomasi (KLT) feature tracker
• Idea minimize the dissimilarity of feature
  windows in two images
• Assumption mutual correspondence

12
Backward Correlation

• Now consider the dissimilarity under the template
  window.
• Decompose the template window into 2 partitions

foreground
background

• Rewrite dissimilarity as

high
low
13
Backward Correlation

• Background is moving at a different velocity than
  the foreground.
• Foreground pixels have similar velocity and
  generate low SSD error.
• Correlation between background pixels using
  foreground velocity generates high SSD error.
• Goal group foreground pixels which are moving at
  similar velocities

Reference frame I(x)
Difference image D(x)I(x)-J(xd)2
Current image J(x)
14
Backward Correlation

• Formulations for backward correlation

Set of template candidates
Correlation coefficient (likelihood)
15
Untextured Backgrounds

• Limitation of backward correlation
• Fails if background has little texture.
• Why? --- Examine the assumption.
• Backward correlation has no reason to prefer the
  foreground to the background which is untextured.

low
Also low if untextured
16
Untextured Backgrounds

• Likelihood of backward correlation textured vs.
  untextured

Foreground
Template containing background pixels
Textured background
Untextured background
17
Gradient Module

• Motivation
• Seek a module focusing on the boundary of the
  target being tracked.
• An edge-based segmentation problem.
• Prior information an ellipse model.
• Gradient Module

Unit vector normal at pixel i
Intensity gradient
18
Combining Modules

• Gradient module and backward correlation module
  have orthogonal failure modes.
• Textured or Untextured?
• Use sum of the gradient magnitude of the
  neighborhood region.
• Combination of forward correlation module and
  backward correlation module is straightforward.
• Combination of forward correlation module and
  gradient module requires the normalization of the
  matching scores.

19
Combining Modules

• Normalize the matching score (likelihood)

• Finial state is decided by

20
Adaptive Scale

• Vary the scale by 10 percent during search
  process.
• Filter the result to avoid oversensitive scale
  adaptation.
• Comaniciu et al. Kernel-based object tracking,
  TPAMI 2003

Size of the best state given by the alg.
Size of the object in the previous frame
21
Experimental ResultsCluttered Background

• Traditional template-based tracker slides off
  target

22
Experimental ResultsCluttered Background

• Our algorithm remains locked onto target

23
Experimental ResultsCluttered Background

• Tracking error plot
• Our algorithm (blue, solid) vs. traditional
  template-based tracker (red, dashed)

Error in x direction
Error in y direction
24
Experimental ResultsUntextured Background

• Tracking results of traditional template-based
  tracker

25
Experimental ResultsUntextured Background

• Tracking results of our algorithm

26
Experimental ResultsOcclusion
27
Experimental ResultsTracking a vehicle
28
Conclusion

• Presented an extension to template-based
  tracking.
• Achieved robustness to out-of-plane rotation.
• Effective tracking in both textured and
  untextured environment.
• Remaining challenges
• Robustness when scale changes.
• Use motion discontinuities to improve
  performance.
• Analysis of parameter sensitivity for untextured
  backgrounds.

29

• Questions?

30

• Thank You!