Activity and Motion Detection in Videos

1
Activity and Motion Detection in Videos

• Longin Jan Latecki and Roland Miezianko, Temple
  University
• Dragoljub Pokrajac, Delaware State University

Dover, August 2005
2
Definition of Motion Detection

• Action of sensing physical movement in a give
  area
• Motion can be detected by measuring change in
  speed or vector of an object

3
Motion Detection

• Goals of motion detection
• Identify moving objects
• Detection of unusual activity patterns
• Computing trajectories of moving objects
• Applications of motion detection
• Indoor/outdoor security
• Real time crime detection
• Traffic monitoring
• Many intelligent video analysis systems are based
  on motion detection.

4
Two Approaches to Motion Detection

• Optical Flow
• Compute motion within region or the frame as a
  whole
• Change detection
• Detect objects within a scene
• Track object across a number of frames

5
Background Subtraction

• Uses a reference background image for comparison
  purposes.
• Current image (containing target object) is
  compared to reference image pixel by pixel.
• Places where there are differences are detected
  and classified as moving objects.

Motivation simple difference of two images
shows moving objects
6
b. Same scene later
a. Original scene
Subtraction of scene a from scene b
Subtracted image with threshold of 100
7
Static Scene Object Detection and Tracking

• Model the background and subtract to obtain
  object mask
• Filter to remove noise
• Group adjacent pixels to obtain objects
• Track objects between frames to develop
  trajectories

8
Background Modelling by Michael Knowles
9
Background Model
10
After Background Filtering
11
Approaches to Background Modeling

• Background Subtraction
• Statistical Methods (e.g., Gaussian Mixture
  Model, Stauffer and Grimson 2000)
• Background Subtraction
• Construct a background image B as average of few
  images
• For each actual frame I, classify individual
  pixels as foreground if B-I gt T (threshold)
• Clean noisy pixels

12
(No Transcript)
13
Background Subtraction
Background Image
Current Image
14
Statistical Methods

• Pixel statistics average and standard deviation
  of color and gray level values (e.g., W4 by
  Haritaoglu, Harwood, and Davis 2000)
• Gaussian Mixture Model (e.g., Stauffer and
  Grimson 2000)

15
Gaussian Mixture Model

• Model the color values of a particular pixel as
  a mixture of Gaussians
• Multiple adaptive Gaussians are necessary to cope
  with acquisition noise, lighting changes, etc.
• Pixel values that do not fit the background
  distributions (Mahalanobis distance) are
  considered foreground

16
Gaussian Mixture Model
Block 44x42 Pixel 172x165
R-G Distribution
R-G-B Distribution
17

• VIDEO

18
Proposed ApproachMeasuring Texture Change

• Classical approaches to motion detection are
  based on background subtraction, i.e., a model of
  background image is computed, e.g., Stauffer and
  Grimson (2000)
• Our approach does not model any background image.
• We estimate the speed of texture change.

19
In our system we divide video plane in disjoint
blocks (4x4 pixels), and compute motion measure
for each block.
mm(x,y,t) for a given block location (x,y) is a
function of t
20
8x8 Blocks
21
Block size relative to image size
Block 24x28 1728 blocks per frame Image
Size 36x48 blocks
22
Motion Measure Computation

• We use spatial-temporal blocks to represent
  videos
• Each block consists of NBLOCK x NBLOCK pixels
  from 3 consecutive frames
• Those pixel values are reduced to K principal
  components using PCA (Kahrunen-Loeve trans.)
• In our applications, NBLOCK4, K10
• Thus, we project 48 gray level values to a
  texture vector with 10 PCA components

23
3D Block Projection with PCA (Kahrunen-Loeve
trans.)
Motion Measure Computation
24
Texture of spatiotemporal blocks works better
than color pixel values

• More robust
• Faster

We illustrate this with texture trajectories.
25
499
624
863
1477
26
Trajectory of block (24,8) (Campus 1 video)
Moving blocks corresponds to regions of high
local variance, i.e., higher spread
Space of spatiotemporal block vectors
27
Comparison to the trajectory of a pixel inside
block (24,8)
Campus 1 video block I24, J28
Standardized PCA components of RGB pixel values
at pixel location (185,217) that is inside of
block (24,28).
28
Detection of Moving Objects Based on Local
Variation

• For each block location (x,y) in the video plane
• Consider texture vectors in a symmetric window
  t-W, tW at time t
• Compute the covariance matrix
• Motion measure is defined as the largest
  eigenvalue of the covariance matrix

29
Feature Vectors in Space
Feature vectors
4.2000 3.5000 2.6000 4.1000
3.7000 2.8000 3.9000 3.9000 2.9000
4.0000 4.0000 3.0000 4.1000 3.9000
2.8000 4.2000 3.8000 2.7000
4.3000 3.7000 2.6500
Covariance matrix
Current time
0.0089 -0.0120 -0.0096 -0.0120
0.0299 0.0201 -0.0096 0.0201 0.0157
Motion Measure
Eigenvalues
0.0499 0.0035 0.0011
0.0499
30
Feature Vectors in Space
Feature vectors
4.3000 3.7000 2.6500 4.4191
3.5944 2.4329 4.1798 3.8415 2.6441
4.2980 3.6195 2.5489 4.2843 3.7529
2.7114 4.1396 3.7219 2.7008
4.3257 3.6078 2.8192
Covariance matrix
0.0087 -0.0063 -0.0051 -0.0063
0.0081 0.0031 -0.0051 0.0031 0.0154
Current time
Motion Measure
Eigenvalues
0.0209 0.0093 0.0020
0.0209
31
Graph of motion measure mm(24,8,) for Campus 1
video
32
Graph of motion measuremm(40,66) of Sub_IR_2
video
Motion Measure Detected Motion
33
Dynamic Distribution Learning and Outlier
Detection