Monitoring Human Behavior in an Office Environment - PowerPoint PPT Presentation

1 / 35
About This Presentation
Title:

Monitoring Human Behavior in an Office Environment

Description:

Monitoring Human Behavior in an Office Environment. Douglas Ayers and Mubarak Shah ... Recognize human actions in a room for which prior knowledge is available. ... – PowerPoint PPT presentation

Number of Views:183
Avg rating:3.0/5.0
Slides: 36
Provided by: dougla148
Category:

less

Transcript and Presenter's Notes

Title: Monitoring Human Behavior in an Office Environment


1
Monitoring Human Behavior in an Office Environment
Douglas Ayers and Mubarak Shah Research
Conducted at the University of Central
Florida Now at Harris Corporation
2
Goals of the System
  • Recognize human actions in a room for which prior
    knowledge is available.
  • Handle multiple people
  • Provide a textual description of each action
  • Extract key frames for each action
  • Recognize all actions simultaneously for a given
    field of view

3
Possible Actions
  • Enter
  • Leave
  • Sitting or Standing
  • Picking Up Object
  • Put Down Object
  • Remove Object
  • Open or Close
  • Use Terminal

4
Prior Knowledge
  • Spatial layout of the scene
  • Location of entrances and exits
  • Location of objects and some information about
    how they are use
  • Context can then be used to improve recognition
    and save computation

5
Layout of Scene 1
6
Layout of Scene 2
7
Layout of Scene 3
8
Layout of Scene 4
9
Major Components
  • Skin Detection
  • Tracking
  • Scene Change Detection
  • Action Recognition

10
Flow of the System
Skin Detection
Track people and Objects for this Frame
Determine Possible Interactions Between People
and Objects
Scene Change Detection
Update States, Output Text, Output Key Frames
11
Skin Detection
  • Uses a modified version of Kjeldsen and Kender
    method
  • Uses Y'CbCr color space instead of HSV
  • 3D Histogram based approach
  • Training phase and detection phase
  • Tracking initialized with a window containing the
    largest connected region

12
Tracking
  • Tracking is done on heads and movable objects
  • Code Provided by Fieguth and Terzopoulos
  • Uses Y'CbCr colorspace
  • Tracks multiple people and objects
  • Simple non-linear estimator
  • Goodness of Fit Equation

13
Scene Change Detection
  • Similar to method used by Sandy Pentlands group
    in Pfinder
  • Uses brightness normalized chromiance
  • Takes a .5 sec clip to build model
  • Determines change based on Mahalonobis distance

14
Possible Actions
  • Enter significant skin region detected for
    several frames
  • Leave tracking box is close to edge of
    entrance area
  • Sitting or Standing significant increase or
    decrease in tracking boxs y-component
  • Picking Up Object after the person has moved
    away from the objects initial position, a scene
    change is detected

15
Possible Actions
  • Put Down Object person moves away from the
    object
  • Remove Object object close to edge of entrance
    area
  • Open or Close increase or decrease in the
    amount of scene change
  • Use Terminal person is sitting and a scene
    change in mouse, but not behind it

16
State Model For Action Recognition
Start
Enter
Sit
Leave
End
Standing
Sitting
Stand
Stand / 0
Sit / 0
Near Phone
Near Cabinet
Near Terminal
Pick Up Phone
Open / Close Cabinet
Use Terminal
Talking on Phone
Opening/Closing Cabinet
Put Down Phone
Using Terminal
Hanging Up Phone
17
Key Frames
  • Why get key frames?
  • Key frames take less space to store
  • Key frames take less time to transmit
  • Key frames can be viewed more quickly
  • We use heuristics to determine when key frames
    are taken
  • Some are taken before the action occurs
  • Some are taken after the action occurs

18
Key Frames
  • Enter key frames as the person leaves the
    entrance/exit area
  • Leave key frames as the person enters the
    entrance/exit area
  • Standing/Sitting key frames after the tracking
    box has stopped moving up or down respectively
  • Open/Close key frames when the of changed
    pixels stabilizes

19
Demo Sequences
  • Two people opening cabinet, closing cabinet,
    sitting and standing
  • One person picking up phone and putting down
    phone
  • One person moving file folder
  • One person sitting and using a terminal

20
Sequence 1
21
Key Frames Sequence 1 (350 frames), Part 1
22
Key Frames Sequence 1 (350 frames), Part 2
23
Sequence 2
24
Key Frames Sequence 2 (200 frames)
25
Sequence 3
26
Key Frames Sequence 3 (200 frames)
27
Sequence 4
28
Key Frames Sequence 4 (399 frames), Part 1
29
Key Frames Sequence 4 (399 frames), Part 2
30
Sequence 5
31
Sequence 6
32
Summary
  • Successfully recognizes a set of human actions
  • Uses prior knowledge about scene layout
  • Handles multiple people simultaneously
  • Generates key frames and textual descriptions

33
Limitations
  • Limited by
  • Skin detection
  • Tracking
  • Scene change detection
  • Some actions difficult to model
  • Limited by prior knowledge
  • Limited by field of view

34
Future Work
  • Increase action vocabulary
  • Determine peoples identity
  • Improve field of view
  • Wide angle lens
  • Multiple cameras
  • Implement in real-time

35
(No Transcript)
Write a Comment
User Comments (0)
About PowerShow.com