Motion Detection and Covariance Tracking of Multiple Objects

1
Motion Detection and Covariance Tracking of
Multiple Objects
Finding and tracking moving objects in
surveillance style video
2
Motion Detection and Initialization

• For the motion detection, use a simple Image
  differencing to create MEI
• Clean up the image and close the regions with
  bwmorph
• Eliminate regions that are smaller than 20
  (4-connected) pixels
• Label separate objects with bwlabel
• Create images that contain useful data for
  identification

3
Initialization
Step 1
Step 3
Step 2
4
Covariance Initialization

• Save RGB and grayscale image
• Find the bounding box for each blob of motion
  with regionprops, this bounding box defines the
  regions
• In these regions, identify possible features for
  the covariance matrices
• Pixel Features tested were
• Values from background subtraction II
• Motion detection (image differencing with
  intensity saved)?
• RGB values
• Grayscale intensity value
• Gradient values
• The Best results came from using only the
  background subtraction II results and the Image
  differencing results in a rotationally invariant
  covariance matrix

5
Possible Covariance Matrix Features
6
Covariance Tracking

• Compare candidate covariance matrices to model
  covariance matrix with distance metric
• Minimize the distance metric (between the
  candidates and the model to select the closest
  match

7
(No Transcript)
8
Unsmoothed Trajectories
9
Model Update
10
Problems/Issues

• Size of the objects, along with the variance of
  the background, made some features harder to use
  (gradient, RGB)?
• If there are many objects, or objects are
  relatively large, this gets computationally
  intensive (not real time)?
• When similar objects crossed paths, occasionally
  boxes switched to the wrong person
• Model update was computationally intensive, and
  hurt the results (my belief is that this is
  because the characteristics I used were
  relatively consistent as the object moved), and
  was omitted from the code

11
Possible Future Work

• Monitor behavior of individuals (suspicious
  activities etc.)?
• Create a better model update that allows
  different features to be weighted by their
  similarity
• Create a model of normal movement within the
  scene, and identify deviations (i.e. common paths
  for certain objects like cars on the road)?

12
Object Tracking using Motion Prediction and
Object Recognition
Tim Sprague
13
Methods Used

• Kalman Filter
• State of x y dx dyT
• Co-Variance Matching
• Settled on Feature Vector of x y Fx FyT
• Averaged several person covariances together for
  average person

14
Problems Encountered

• Determining What Characteristics to use for
  Covariance Matching
• Standard Deviation for Smoothing and Edge
  Detection for Fx and Fy
• Process and Observation Noise
• Low Resolution

15
Outcomes
16
Outcomes Cont'd

• Distance to
• People Car Fire Hydrant Manhole
• 3.1072 3.7136 1.8054 5.4393

17
Outcomes Cont'd

• Distance to
• People Car Fire Hydrant Manhole
• 1.5514 2.3988 2.0804 3.7495

18
Outcomes Cont'd

• Distance to
• People Car Fire Hydrant Manhole
• 3.4571 4.0773 5.6102 3.9822

19
Lessons Learned

• It's not fun labeling by hand
• People look like fire hydrants at a distance to
  co-variance matching
• Low resolution makes distance matching difficult

20
Possible Future Work

• Determine which Features Maximize the Object
  Matching
• Refine Kalman Filtering
• Better State
• Predict further into future
• Larger database of known images

21
Work Allocation

• Tim
• Trajectory Prediction
• Object Matching
• Ryan
• Motion Detection
• Covariance Tracking
• Both
• Data Collection