Towards Cooperative Localization of Wearable Sensors using Accelerometers and Cameras

1
Towards Cooperative Localization of Wearable
Sensors using Accelerometers and Cameras

• Deokwoo Jung
• (with Thiago Teixeira and Andreas Savvides)
• deokwoo.jung_at_yale.edu
• Embedded Networks Applications Lab
• Yale University

2
Indoor Localization

• Indoor localization is an essential technology
  for many applications
• Security Application, Assisted Living, Life
  logging System etc
• Indoor localization comparison

Cricket Priyantha.et.al ,00,04 RADAR Bahl.et.al,00 Surroundsense Azizyan.et.al,09 Our System
Precision Physical Location (cm) Physical Location (lt5 meter) Logical Location (gt5 meters) Physical Location (cm)
Mobile device Customized Sensor Node WLAN Card - Laptop Mobile Phone Mobile Phone/ Wearable Sensor
Infra- structure Ultrasound Beacon Nodes on Ceiling WLAN APs RF fingerprint database GSM Network Ambient signal database Networked cameras
Sensing modality Ultrasound RF signal Ambient signals- Light, sound, color Human walking motions
3
Cooperative Localization Approach

• Our Approach
• Localizing mobile phones by combining their
  built-in accelerometer (human motion) and
  infrastructure camera (human centroids)
• Why Human Centroids and Human Motion ?
• They are Complementary to each other

Wearable inertial sensors (Human Motion) Camera (Human Centroid)
ID tracking Accurate - node address Difficult feature extraction
Location Positioning Difficult walking orientation and distance estimation Accurate background subtraction
4
Sensing Modeling and Approach

• Human Walking Model
• ? ?Accel. ? Walking distance
• The law of movement of human body by complex
  kinetics
• Inverted pendulum model of human gait.
• The body center of mass (BCOM) oscillates in the
  z direction
• as the person moves forward (y direction).

5
 Statistical Analysis of Sensor Data

• Intuition BCOM follows a sinusoidal pattern
• Velocity of Body a Standard Deviation of Vertical
  Acceleration
• A correlation coefficient for the similarity
  measure between accelerometer and camera data
• Experiment

6
Tracking Algorithm

• A camera extracts only centroid information
• Privacy Preserving and Cheap
• A simple tracking algorithm computes a speed of
  anonymous centroids
• associates human centroids in consecutive frames
  based on their distances.
• Problem Many possible ambiguous associations

7
Path Disambiguation Problem

• Path Ambiguity Problem in Human Centroid Tracking
• A tracker associates one object with more than
  two objects in two consecutive image frames when
  two or more objects come close to each other.

8
Disambiguation Algorithm

• Path Disambiguation as Non-linear Optimization
  Problem
• Find a set of association hypotheses to maximize
  a matching rate,
• The number of correct ID matchings between
  accelerometers and centroids
• Develop a search algorithm in a tree structure
• A leaf node a hypothesis of path segmentations
• Three stage pruning algorithm
• Sub-tree evaluation,
• Classification and Pruning,
• Reconstruction

9
Clustering and Pruning in Hypothesis Tree

• Hypothesis Quality Metric
• how credible a given path hypothesis is compared
  to others?
• Correlation Coefficient Distance metric
• D(?H) E(?, e0H) - E(?, e1H)

Wrong Hypothesis
Correct Hypothesis
10
Clustering and Pruning in Hypothesis Tree

• Leaf Clustering, Pruning, and Path Reconstruction
• Clusters the leaf nodes into groups and prunes
  the subset of groups with lower metric values.
• When only one leaf is left, reconstructs the
  matching sequence

11
Performance Evaluation via Experiment Simulation

• Experiment Setup
• A ceiling mounted camera (12ft) with a Intel
  iMote2 node
• Computes the centroid position of a person, 15
  times per second.
• A wearable sensor node with an Analog Devices
  ADXL330 accelerometer on the persons waist
• Collecting body acceleration data with 15Hz
  sampling rate.
• Transmitting its measurements to a computer
  (fusion center) via a Zigbee wireless link.
• People walk for 1 minute in a 5.4 m2 space.

12
Experiment Dataset

• Walking trajectory of 12 people collected from
  camera

13
Similarity Metric Performance

• 100 matching rate without path ambiguity

14
Disambiguation Algorithm Performance

• The performance depends on the level of crowd in
  camera field of view.
• Evaluate the performance using crowd density
  metric, the number of pedestrians per area, m2,
  Abishai.et.al, Pedestrian flow and level of
  service
• Crowd Density Scenario

Scenario A Normal flow B Restricted Flow C Dense Flow D Very Dense Flow
Crowd Density office building in business hour crowded shopping mall in weekend Crowded weekend party Subway station in Manhattan during the rush hour
People / m2 lt0.5 0.50.8 0.811.26 1.272

• If the crowd density gt 2, the pedestrian flow is
  jammed, i.e. practically peoples movement
  appears to be static
• Our system is mainly targeting for the scenario A
  (free flow), i.e. people can walk around without
  much interaction.

15
Performance over complexity of scenario

• The number of tracking errors grows with
  polynomial order with crowd density (left)
• The matching performance of disambiguation
  algorithm for different crowd densities (right)
• The performance gap is widening as crowd density
  increases.
• The performance becomes twice in the scenario D.

16
Performance over disambiguation stages

• Matching rate improvement by disambiguation
  algorithm

17
Localization System Demonstration

• Controlled experiments with 10 people walking
  scenario.
• The performance of disambiguation algorithm
  (right) is compared to the tracker-only
  localization (left).

18
Conclusion

• We presented a hybrid localization system using
  accelerometers and cameras.
• The proposed disambiguation algorithm operates
  reliably, degrading gracefully even for crowded
  scenarios
• The constraint of accelerometer position (waist)
  can be relaxed using additional inertial
  measurement sensors.
• Future work is to have a complete system
  implementation running on a mobile phone More
  sensors

19
QUESTION ?
Thanks for your interest!
For more information, please visit
http//pantheon.yale.edu/dj92/