Motion based Correspondence for Distributed 3D tracking of multiple dim objects

1
Motion based Correspondence for Distributed 3D
tracking of multiple dim objects

• Ashok Veeraraghavan

2
Problem Setting
3
Constraints
R, T ??
R, T ??
R, T ??
Bandwidth lt W
Bandwidth lt W
R, T ??
4
Outline

• Tracking Algorithm
• Implemented at each camera node.
• Correspondence problem for dim targets.
• Motion-Based Correspondence Algorithm
• Implemented at central processor
• Recovering Camera Position and Orientation
• Recovering 3D tracks using triangulation.

5
Experimental Setup

• Objective
• Reconstruct the 3D trajectories of the bees so as
  to study the response of bees to visual stimuli.
• Outdoor Bee Tunnel with the surrounding walls
  texture systematically varied
• Study relationship of flight patterns to visual
  stimulii.
• Two Fixed Cameras.
• Free Flying bees are the targets to be tracked.
• Typically the bees are about 20-50 meters away
  from the camera.
• Multiple Targets On average each frame contains
  about 6-8 bees.
• Occupy about 5-10 pixels at closet range Low SNR
• Objective Reconstruct the 3D trajectories of
  the bees so as to study the response of bees to
  visual stimuli.

6
Tracking Algorithm

• Background Subtraction
• Background variations are assumed to be much
  slower than the target.
• Dynamic background estimated using a temporal low
  pass filter for each pixel.
• Connected Component Analysis
• Morphological processing to connect pixels
  belonging to same target.
• Probabilistic Data Association
• Blob Tracking algorithm.

7
Background Subtraction and Connected Component
Analysis
8
Background Subtraction and Connected Component
Analysis
9
Adaptive Velocity Motion Model
r
v
10
Correspondence Problem for Dim Targets

• Correspondence across camera Views
• Associating the objects found in various views
• Especially tricky for multiple dim objects
• Dim Targets
• Low SNR
• Very Small Targets (order of few pixels )
• Features extraction unreliable
• Appearance based correspondence
• Appearance varies with view
• Unreliable for dim targets

11
Motion Based Correspondence

• Rubin and Richards (1985)

• Rao, Yilmaz and Shah (2002)-
• Maxima of spatio-temporal curvature as Dynamic
  Instants

Courtesy Rao2002
12
Dynamic Instants

• Detects any start instant, stop instant,
  non-smooth change in speed, maximal curvature of
  3D tracks. Eg., Start Instants

Courtesy Rao2002
13
Detected Dynamic Instants
Courtesy Rao2002
14
Correspondence Across Views
15
(No Transcript)
16
External Calibration

• Internal Camera parameters known.
• External Orientation of the cameras to be
  estimated from correspondence data obtained by
  matching tracks across views.
• Simple non-linear optimization implemented
  (Levenberg-Marquardt).
• Distance between cameras (Baseline) approximately
  known.
• Optimization is local. Requires good initial
  estimate.

17
3D flight Paths using Triangulation

• Internal camera parameters known.
• External camera calibration parameters estimated
  from point correspondences.
• 3D tracks obtained using Triangulation.

18
3D Flight Paths
19
3D Flight Paths
20
Future Work

• Human Surveillance.
• Work with multiple (more than 2 cameras) cameras.
• Study the trade-off between bandwidth and
  efficiency.
• Especially can we also add some appearance
  information to each target so that limited view
  reconstruction of target is possible?

21
(No Transcript)
22
Thank You