Survey of Video Surveillance Technology

1
Survey of Video Surveillance Technology

• Yu Hen Hu

2
The evolution of video surveillance an overview

• Niels Haering Péter L. Venetianer Alan Lipton
• Machine Vision and Applications, 2008 special
  issue, springer.

3
Problem Information Overload

• Cost for spotting relevant content from large
  number of continuous video feeds is too high to
  be practical for human operators.
• Cost for deployment and configuration of a video
  surveillance is still too high and not fully
  automated.

4
Design Issues

• Sensor calibration
• Cost too high. Need to develop low cost work
  around
• System integration
• Video surveillance is part of a comprehensive
  security system
• Sensor modality
• Color, near IR, omni-directional (fish eye),
  thermal
• Background characteristics
• E.g. water waves,
• System functionalities
• Detecting motion ? recognizing/tracking objects ?
  recognizing event ? autonomous learning of events
• Video meta-data and event languages
• Content representation

5
Calibration issue

• If calibration is not available, size filters
  specifying the minimum and maximum allowed size
  of an object nearfield and farfield help
  eliminate several false alarms

6
Sensing Modality

• (from upper left corner, clockwise) Near
  infrared camera, color camera, thermal image,
  omni-directional camera

7
Environment (Background) Factor

• Water background causes false alarm due to moving
  wave, reflections, etc. and need domain specific
  processing to mitigate problem

8
Major Functions of a Video Analysis Engine
9
System Architecture of a Commercial Video
Surveillance system
10
Detecting Illegal Left Turn

• Left turn Same vehicle, passing through two
  trip-wires one after the other during short
  durations.
• Need machine readable language to describe the
  events

11
Object Video VEW System Overview

• builds a statistical background model,
• detects foreground pixels,
• combines them into blobs, and then
• tracks those to detect targets.
• These targets are then classified based on
  various properties.
• Example Detected face (top left) and best face
  shot (top right)

12
Rule-based Event Detection
moving in an illegal direction (flow control)
virtual tripwires
object left behind
object inside AOI
person counting
crowded density
13
Future Directions

• Unusual event detection
• Learned from operators annotation
• Target property map
• PTZ camera support
• Leader/follower system, scanning camera
• Automatic scene understanding
• Using domain/application specific meta knowledge
• Multi-camera network
• Cross camera tracking
• Self-calibrating sensor network

14
Glimpse of Surveillance Data
15
Multi-Camera Video Surveillance

• Ellis02 T. Ellis, "Multi-camera video
  surveillance," Proc. Int'l Conf. Security
  Technology, 2002, pp. 228-233.
• black01 J. Black, T. Ellis, "Multi Camera Image
  Tracking", Proceedings of the Second
  International Workshop on Performance Evaluation
  of Tracking and Surveillance, December, Kauai,
  Hawaii, USA, (2001)
• black02 J. Black, T. Ellis, P Rosin, "Multi
  View Image Surveillance and Tracking", IEEE
  Workshop on Motion and Video Computing, December,
  Orlando, USA, pp. 169-174. (2002)

16
Overview

• Development of a multi-view surveillance system
  using calibrated cameras
• Developing algorithms to detect and track objects
  (pedestrians, cyclists, vehicles) in an outdoor
  environment.
• Need to adapt to wide varying illumination
  variations, spurious motion of non-objects, and
  interaction between objects and scene.

17
Outline of Approach

• Temporal alignment
• View point integration
• Homography mapping and homography based matching
• Camera calibration and measurement uncertainty
  modeling
• State space model based object tracking

18
Correspondence

• Finding correspondence using Least Median of
  squares (LMS) search

19
Dynamic Occlusion Handling

• The top, and bottom images show two objects
  before and after a dynamic occlusion. The correct
  labels are still assigned after the occlusion.

20
Object Tracking

• Tracking objects between non-overlapping views
  using 3D trajectory prediction.

21

• M. Valera and S.A. Velastin, Intelligent
  distributed surveillance systems a review, IEE
  Proceedings, 2005

22
Three generations of Video Surveillance Systems

• Generation I. video monitoring
• analog CCTV system
• Manual monitoring of multiple video feeds
• Generation II. automated video surveillance
• CCTV computer vision
• Object detection, scene adaptation, behavioral
  modeling and understanding
• Generation III. Wide area surveillance
• Cooperating multiple non-overlapping FOV cameras
  and multi-modality sensors
• Distributed intelligence, fusion of knowledge

23
Applications

• Transport applications
• airports 14, 15, maritime environments 16,
  17, railways, underground 12, 13, 1921, and
  motorways to survey traffic 2226.
• Public places
• banks, supermarkets, homes, department stores
  2731 and parking lots 3234.
• Remote surveillance of human activities
• Attendance at football matches 35 or other
  activities 3638.
• Surveillance to obtain certain quality control in
  many
• industrial processes,
• surveillance in forensic applications 39 and
• remote surveillance in military applications

24
Surveillance Techniques
Traditional flow of processing in visual
surveillance systems
25
Object detection
Temporal difference method
Background subtraction method
26
Recognition, Tracking

• Model
• 2D model with or w/o shape model or 3D model
• Prior knowledge Object appearance or behavior
• Adapt to varying illumination, occlusion,
• Recognition
• Using model and prior knowledge to recognize
  object
• Tracking
• Global Kalman filter, particle filter, HMM
• Local Bounding box locking

27
Behavioral Analysis

• Temporal pattern recognition
• Matching sequence of motion primitives to known
  templates of behavior motion
• Tools
• HMM Hidden Markov model
• DTW dynamic time warpping

28
System architecture
29
Commercial Surveillance System DETER (detection
of events for threat evaluation and recognition

• reporting unusual moving patterns of pedestrians
  and vehicles in outdoor environments such as car
  parks.
• Computer vision module fuses the views of
  multiple cameras into one view and then performs
  tracking of the objects
• threat assessment module feature assembly or
  high-level semantic recognition, the off-line
  training and the on-line threat classifier

30

• PRISMATICA
• pro-active integrated systems for security
  management by technological institutional and
  communication assistance)
• wide-area multi-sensor distributed system,
  receiving inputs from CCTV, local wireless camera
  networks, smart cards and audio sensors

central server module
31
ADVISOR (annotated digital video for intelligent
surveillance and optimized retrieval)
It consists of a network of ADVISOR units, each
of which is installed in a different underground
station and consists of an object detection and
recognition module, tracking module, behavioral
analysis and database module.