Summary of: Tracking and Object Classification for Automated Surveillance

1
Summary ofTracking and Object Classification
for Automated Surveillance

• By Omar Javed and Mubarak Shah
• http//www.cs.ucf.edu/vision/projects/Knight/Resu
  lts.html
• ECE 285 Presentation
• Jeff Ploetner
• 1-10-2005

2
Overview

• Intro and Motivation
• Typical Tracking Problems
• Lighting, Background, Shadows, Occlusions,
  Entries, Exits
• Major Components
• Specific Approach Taken
• Background Subtraction
• Shadow Removal
• Motion Correspondence
• Object Classification
• Recurrent Motion Image (RMI)
• Classification of RMIs
• Carried Object Detection
• Activity Analysis
• Results
• Related Work

3
Motivation

• Automated Surveillance Systems
• Track Objects
• Classify Objects
• Detect Activities
• Minimize Human Involvement
• Human monitors are expensive
• Have short attention spans

4
Tracking Problems

• Background Subtraction
• Changing Lighting Conditions
• Moving Background Objects
• Shadows
• Occlusion
• Inter-object Occlusion
• Thin Scene Structures
• Large Structures
• Object Exits and Entries from Scene

5
Major Components

• Background Subtraction Module
• Object Tracking Module
• Object Classification Module
• Activity Detection Module

6
Approach Taken

• Background Subtraction
• Mixture of Gaussian approach by Stauffer and
  Grimson
• C. Stauffer and W. E. L. Grimson. Learning
  patterns of activity using real-time tracking.
  IEEE Trans. On PAMI, 22(8)747757, Aug 2000.

7
Shadows

• Shadow region darker than reference background
• Some texture and color information of underlying
  surface usually retained
• Many Approaches
• (Aside)The HSV Color Space

http//en.wikipedia.org/wiki/ImageHSV_cylinder.jp
g
8
Shadow Suppression

• Extract regions that are darker than the
  background reference
• Perform color segmentation on each potential
  shadow region
• Use K-means approximation of the EM algorithm

9
Shadow Suppression

• Take Gradient Direction
• Correlate the gradient directions between image
  and background
• If correlation gt 0.75, assume its a shadow and
  remove it
• Else, assume its foreground

10
Shadow Suppression Results
11
Other Results
After
Before
Predicted Shadows
Suppressed Shadows
Thresholded
Thresholded
Morphologically Closed
Morphologically Closed
12
Motion Correspondence

• Correspond regions, not pixels
• Regions have shape and size to compare for better
  correspondence
• Can add parameters to improve robustness
• Minimum Initial Observation Parameter
• Prevents false positives
• Maximum Missed Observation Parameter
• Prevents false negatives
• Established Regions have extra useful information
  to predict future
• Velocity, Projected change in size

13
Motion Correspondence

• Minimize cost function to establish
  correspondence between regions of different
  frames
• Special cases
• Occlusions predict based on velocity
• Inter-object Occlusion
• Thin Scene Structures
• Large Structures
• Object Exits and Entries
• Details in paper

14
Object Classification

• Three object classes
• Single person
• Group of people
• Vehicle
• Detect repetitive changes in shape of object
• Relies on accurate tracking to give
• Bounding box, centroid, and correspondence of
  each object over the frames
• Compensate for translation and scale
• Align object along its centroid
• Scale to keep vertical height constant
• Assume only distance from camera changes vertical
  height
• Use Recurrent Motion Image to classify

15
Recurrent Motion Image
Two Seconds of Frames
Previous Frame
Current Frame
Changed Pixels

• Detects repeated change of pixels
• Sum two seconds of exclusive-or operations
  (changed pixels) between binary silhouettes of
  subsequent frames
• Assume person will take at least one step per
  second
• Yields high values at those pixels at which
  motion occurred repeatedly and low values where
  there is little motion
• Note This is Fast - No image history is needed,
  since its a running sum

16
Recurrent Motion Image

• Threshold the RMI
• If no recurring motion, its a vehicle
• If recurring motion in middle or bottom, its a
  person or a group of people
• Next, look at top part of RMI
• If top part of RMI is stable, its a single
  person
• If top part of RMI is busy, its a group of
  people
• Or directly detect number of heads using shape
  cues

17
Carried Object Detection

• Asymmetric sub regions of human silhouettes not
  undergoing recurrent motion are usually carried
  objects

18
Activity Detection

• Videos from website

19
Results

• Shadow removal algorithm only kicks in if 30 of
  silhouette is classified as a dark region
• Where it does kick in, removes shadows in 70 of
  frame. In 25 if didnt remove shadows, and in
  5 segments belonging to objects were removed
  (mostly due to self-shadow).
• Implemented in C
• Runs at 12-15 FPS on P4-1.7 GHz

20
Related Work

• KNIGHT A REAL TIME SURVEILLANCE SYSTEM FOR
  MULTIPLE OVERLAPPING
• Expands upon this approach to use multiple
  cameras
• A Hierarchical Approach to Robust Background
  Subtraction using Color and Gradient Information
• Goes into more detail about background
  subtraction problems and potential solutions