Elaborating Sensor Data using Temporal and Spatial Commonsense Reasoning Mining Models of Human Act - PowerPoint PPT Presentation

Loading...

PPT – Elaborating Sensor Data using Temporal and Spatial Commonsense Reasoning Mining Models of Human Act PowerPoint presentation | free to download - id: 95900-ZmMwZ



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Elaborating Sensor Data using Temporal and Spatial Commonsense Reasoning Mining Models of Human Act

Description:

... al., 'Mining Models of Human Activities from the Web,' WWW ... Human activity-trace recognition. Activities of Daily Living (ADLs) Inter-corpus consistency ... – PowerPoint PPT presentation

Number of Views:32
Avg rating:3.0/5.0
Slides: 22
Provided by: sungs5
Learn more at: http://sclab.yonsei.ac.kr
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Elaborating Sensor Data using Temporal and Spatial Commonsense Reasoning Mining Models of Human Act


1
?? ???Elaborating Sensor Data using Temporal
and Spatial Commonsense ReasoningMining Models
of Human Activities from the Web
  • ?? ?? ??? ??
  • 2006. 11. ???

2
Agenda
  • B. Morgan and P. Singh, Elaborating Sensor Data
    using Temporal and Spatial Commonsense
    Reasoning, BSN 2006.
  • The Problem Space
  • LifeNet A First-Person Model
  • The Plug Sensor Network
  • M. Perkowitz, et al., Mining Models of Human
    Activities from the Web, WWW 2004.
  • Introduction
  • Proposed Technique
  • Evaluation
  • Summary and Future Work

3
The Problem Space
  • Two distinct directions for research
  • Human-out (This paper)
  • Telephone
  • Technology-in (Much sensor network research)
  • Text messaging on cell phones
  • Three topics
  • LifeNet probabilistic human model
  • The Plug sensor network
  • An experimental design for evaluation of the
    LifeNet learning method

4
LifeNet A First-Person Model
  • First-person common-sense inference model
  • OpenMind Common Sense, ConceptNet, The PlaceLab
    data, Hondas indoor common sense data
  • Attempts to anticipate and predict what humans do
    in the world
  • All of the reasoning in LifeNet is based on
    probabilistic propositional logic
  • I am washing my hair before my hair is clean

5
The Plug Sensor Network
  • Using for both learning common sense and for
    recognizing and predicting human behavior
  • Using this sensor network to monitor how
    individuals interact with their physical
    environment
  • Nine sensor modalities sound, vibration,
    brightness, current, wall voltage, acceleration

6
Agenda
  • B. Morgan and P. Singh, Elaborating Sensor Data
    using Temporal and Spatial Commonsense
    Reasoning, BSN 2006.
  • The Problem Space
  • LifeNet A First-Person Model
  • The Plug Sensor Network
  • M. Perkowitz, et al., Mining Models of Human
    Activities from the Web, WWW 2004.
  • Introduction
  • Proposed Technique
  • Evaluation
  • Summary and Future Work

7
Introduction Recognize Humans Activities
  • Applications include activity-based actuation
  • Dimming lights when a video is being watched
  • Providing directions for someone using
    unfamiliar facilities
  • etc.
  • Ubiquitous, proactive, disappearing computing
  • Computers have to understand peoples needs by
    observing their physical activities (and to act
    autonomously)
  • The cost of developing recognition infrastructure
    is too high
  • Even small classes of activities is hard to
    recognize
  • A broadly applicable system should be
    general-purpose and easy to use

8
Motivation
Introduction
  • Vision based systems
  • None have reported detecting more than tens of
    activities in practice
  • The features robustly detectable from vision are
    coarse
  • Represent the relationships between blobs in
    the image rather than specific objects
  • Each activity is expensive to model
  • Learning of the models
  • The developers define the structure of the
    possible models
  • System tunes the parameters of the model based on
    examples from the user
  • The user is expected to label the patterns
  • The variety of activities is quite restricted

9
Proposed Technique
  • RFID (Radio Frequency Identification)
  • Cheap Postage-stamp sized, forty-cent
  • Wireless and battery free
  • Activity modeling
  • Define an activity in terms of the probability
    and sequence of the objects
  • Generate the models by translating textual
    definitions
  • Structured like recipes
  • Produced automatically by mining appropriate web
    sites
  • Mining models is part of a larger activity
    recognition system, PROACT (Proactive Activity
    Toolkit)

10
Usage Model
Proposed Technique
  • Assumes that interesting objects in the
    environment contain RFID tags (tens hundreds)
  • Making a database entry mapping the tag ID to a
    name
  • Within a few years, many household objects may be
    RFID-tagged before purchase, thus eliminating the
    overhead of tagging
  • Medium-range readers (Tag-detecting Gloves)
    andLong-range readers (Run robots, Carts, )
  • PROACT uses the sequence and timing of object to
    deduce what activity is happening
  • Likelihood of various activities, details of
    those activities, degree of certainty, etc

11
System Overview
Proposed Technique
  • PROACT provides an activity viewer for debugging
  • Real-time view of activities in progress
  • The sensor data seen
  • Changing of belief in each activity with the data
  • Inference Engine converts the activity models
    produced by the mining engine into Dynamic
    Bayesian Networks
  • D. Patterson, L. Liao, D. Fox, H. Kautz,
    Inferring High-Level Behavior from Low-Level
    Sensors, Ubicomp 2003.

12
Sensors and Models
Proposed Technique
  • Sensors
  • Use two different kinds of RFID readers
  • Long-range reader (mobile robot) map the
    location of objects
  • Short-range reader (glove) determine the objects
    that are touched
  • Models
  • Each model (activity) is composed of a sequence
    (step) s1 sn
  • Each step si has optional duration ti and object
    oij involved along with the probability pij

13
The Model Extractor
Proposed Technique
  • Builds formal models of activities using
    directions
  • Directions are written in natural language by
    human
  • How-to (ehow.com), recipes (epicurious.com),
    training manuals, protocols, etc.
  • Syntactic structure of directions
  • A title t for the activity
  • A textual list r1rm, Each step ri has
  • Possibly a special keyword delimiting duration di
  • What to do during the step subset of the objects
    and duration

14
Converting Directions to Activity Models
Proposed Technique
  • Key steps
  • Labeling
  • Set label of the mined model to title of the
    directions
  • Parsing steps
  • Duration Gaussian with mean d, stdev S(d, i,
    l )
  • Object Oi and Probability P
  • Tagged object filtering
  • Functions
  • Object
  • Object extraction WordNet ontology
  • Noun-phrase extraction QTag tagger
  • Probability
  • Fixed probabilities
  • Google conditional probabilities (GCP)

15
Example
Proposed Technique
16
Evaluation
  • Mined models
  • ehow.com 2300 directions
  • ffts.com 400 recipes
  • epicurious.com 18,600 recipes
  • Three strategies to approximate comprehensive
    evaluation
  • Human activity-trace recognition
  • Activities of Daily Living (ADLs)
  • Inter-corpus consistency
  • Making cookies recipes
  • Intra-corpus distinguish-ability
  • Distinguish-ability within activity domains

17
Distinguish-ability
18
Human and inter-corpus trace recognition
Evaluation
  • ADLs domain
  • Many objects were not tagged, missed, and
    interleaved
  • Models were not perfect
  • Cookie domain
  • The identical recipe can have quite different
    structure
  • For some of the recipes, there is no counterpart
    in the other corpus

19
Impact of techniques on accuracy
Evaluation
  • ADLs
  • Domain is fairly sparse, with many activities
    involving only few object
  • Cookie domain
  • Each activity model involves many more objects

20
Impact of techniques on compactness
Evaluation
21
Summary and Future Work
  • An introduction to the idea of mining activity
    detection from the web
  • Future work
  • Perform a more comprehensive evaluation
  • Improving the effectiveness of mined models
  • Include location
  • Synonymous words
  • Synsets (collections of synonymous words) can be
    extractedfrom WordNet
About PowerShow.com