PEMID: Identifying People by GaitMatching using Cameras and Wearable Accelerometers

1
PEM-ID Identifying People by Gait-Matching
using Cameras and Wearable Accelerometers

• Thiago Teixeira, Deokwoo Jung,
• Gershon Dublon, Andreas Savvides
• Yale ENALAB

2
Introduction

• Can we uniquely identify people in camera
  networks?(in cooperative enviroments)
• Motivation
• Assisted Livingidentify people in a home
• Securitylocate personnel
• Corporate environmentstrack facility usage

• Plus, obtaining data traces for research
• Yale BehaviorScope project

3
Main Idea

• Equip each person of interest with a wearable
  accelerometer node (with known ID)
• Extract motion signature from
• each accelerometer ? unique ID
• each track ? Position
• Find pairs of matching signatures to obtain
  IDPosition

4
Problem Statement

• Given a set SAi of accelerometer signals and a
  set SCj of tracks extracted from a camera
  network
• Find the match matrix ? which globally maximizes
  the similarity between pairs of signals SAi and
  SCj
• Main assumptions
• Tracker provides correct tracks in segments ? 4
  steps
• Camera placement oblique from top (typical CCTV)
• Occlusions short-lived

5
Challenge motion signature

• Motion paths can be subdivided into two types
• Transition motion
• Starting, stopping, turning, changing speed
• Large changes in tangential and centripetal
  acceleration
• Cruising motion
• Approximately same-speed linear motion
• Only small-scale changes in acceleration ? Gait
• Comprises majority of time
• Intuition to ID people most of the time, use
  gait
• Challenge Nodes are not time-synchronized, have
  limited processors and low bandwith

6
Correlating Gait Signals from Asynchronous
Sources

• Sample-oriented methods are unsuitable for
  WSNs(eg. Pearson's corr. coefficient, mutual
  information)
• Fail given time synchronization offsets (or must
  slide one of the signals and recalculate)
• Require a large number of samples to converge
• Requires resampling/interpolation if signals have
  different sampling frequencies and/or phases
• We can do better, using gait frequency and phase

7
Timestamps of Gait Landmarks

• Idea Compare timestamps of heel-strike and
  midswing moments of gait
• H (tH0, tH1, )
• M (tM0, tM1, )
• From accels., and cameras
• SAi HAi, MAi
• SCj HCj, MCj
• Next step define time-noise independent metric
  (offset and jitter)

8
Distance metric

• Define distance from timestamp to sequence
• Then from sequence to sequence
• Then two metrics describing time offset and
  jitter

9
Global Optimization

• Invariance to time offset, timestamp noise

• Global Optimization

10
Multiple-Person Simulations

• We recorded 24 one-person traces
• 12 walking straight in different directions
• 12 walking and turning in different directions
• We overlapped multiple single-person traces with
  random time offsets (up to 1s) to simulate
  multiple-person scenarios

11
Three-Person Experiments

• Three people walking through FOV
• One person wearing an accelerometer? Average
  recognition rate 87.5

http//enaweb.eng.yale.edu/drupal/PEM-ID-videos
12
Conclusion

• Presented a method to ID people in videos using
  accelerometers
• Accuracy gt 83, for up to 10 people 10 accels
• Currently adapting for indoor use
• Much smaller FOV ? multiple cameras
• Occlusions ? use additional features

13
Thank you.Questions?

• BehaviorScope
• http//www.eng.yale.edu/enalab/behaviorscope.htm
• Videos
• http//enaweb.eng.yale.edu/drupal/PEM-ID-videos