Creating Dynamic Social Network Models from Sensor Data - PowerPoint PPT Presentation

Loading...

PPT – Creating Dynamic Social Network Models from Sensor Data PowerPoint presentation | free to download - id: 9bf46-MWUyZ



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Creating Dynamic Social Network Models from Sensor Data

Description:

1973 Mark Granovetter strength of weak ties ... iPAQ computes audio features and WiFi node identifiers and signal strength ... – PowerPoint PPT presentation

Number of Views:68
Avg rating:3.0/5.0
Slides: 34
Provided by: csRoch
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Creating Dynamic Social Network Models from Sensor Data


1
Creating Dynamic Social Network Models from
Sensor Data
  • Tanzeem ChoudhuryIntel Research / Affiliate
    Faculty CSE
  • Dieter Fox
  • Henry KautzCSE
  • James KittsSociology

2
  • What are we doing?
  • Why are we doing it?
  • How are we doing it?

3
Social Network Analysis
  • Work across the social physical sciences is
    increasingly studying the structure of human
    interaction
  • 1967 Stanley Milgram 6 degrees of separation
  • 1973 Mark Granovetter strength of weak ties
  • 1977 International Network for Social Network
    Analysis
  • 1992 Ronald Burt structural holes the social
    structure of competition
  • 1998 Watts Strogatz small world graphs

4
Social Networks
  • Social networks are naturally represented and
    analyzed as graphs

5
Example Network Properties
  • Degree of a node
  • Eigenvector centrality
  • global importance of a node
  • Average clustering coefficient
  • degree to which graph decomposes into cliques 
  • Structural holes
  • opportunities for gain by bridging disconnected
    subgraphs

6
Applications
  • Many practical applications
  • Business discovering organizational bottlenecks
  • Health modeling spread of communicable diseases
  • Architecture urban planning designing spaces
    that support human interaction
  • Education understanding impact of peer group on
    educational advancement
  • Much recent theory on finding random graph models
    that fit empirical data

7
The Data Problem
  • Traditionally data comes from manual surveys of
    peoples recollections
  • Very hard to gather
  • Questionable accuracy
  • Few published data sets
  • Almost no longitudinal (dynamic) data
  • 1990s social network studies based on
    electronic communication

8
Social Network Analysis of Email
  • Science, 6 Jan 2006

9
Limits of E-Data
  • Email data is cheap and accurate, but misses
  • Face-to-face speech the vast majority of human
    interaction, especially complex communication
  • The physical context of communication useless
    for studying the relationship between environment
    and interaction
  • Can we gather data on face to face
    communication automatically?

10
Research Goal
  • Demonstrate that we can
  • Model social network dynamics by gathering large
    amounts of rich face-to-face interaction data
    automatically
  • using wearable sensors
  • combined with statistical machine learning
    techniques
  • Find simple and robust measures derived from
    sensor data
  • that are indicative of peoples roles and
    relationships
  • that capture the connections between physical
    environment and network dynamics

11
Questions we want to investigate
  • Changes in social networks over time
  • How do interaction patterns dynamically relate to
    structural position in the network?
  • Why do people sharing relationships tend to be
    similar?
  • Can one predict formation or break-up of
    communities?
  • Effect of location on social networks
  • What are the spatio-temporal distributions of
    interactions?
  • How do locations serve as hubs and bridges?
  • Can we predict the popularity of a particular
    location?

12
Support
  • Human and Social Dynamics one of five new
    priority areas for NSF
  • 800K award to UW / Intel / Georgia Tech team
  • Intel at no-cost
  • Intel Research donating hardware and internships
  • Leveraging work on sensors localization from
    other NSF DARPA projects

13
Procedure
  • Test group
  • 32 first-year incoming CSE graduate students
  • Units worn 5 working days each month
  • Collect data over one year
  • Units record
  • Wi-Fi signal strength, to determine location
  • Audio features adequate to determine when
    conversation is occurring
  • Subjects answer short monthly survey
  • Selective ground truth on of interactions
  • Research interests
  • All data stored securely
  • Indexed by code number assigned to each subject

14
Privacy
  • UW Human Subjects Division approved procedures
    after 6 months of review and revisions
  • Major concern was privacy, addressed by
  • Procedure for recording audio features without
    recording conversational content
  • Procedures for handling data afterwards

15
Data Collection
  • Intel Multi-Modal Sensor Board

Coded Database
codeidentifier
audiofeatures
Real-time audio feature extraction
WiFistrength
16
Data Collection
  • Multi-sensor board sends sensor data stream to
    iPAQ
  • iPAQ computes audio features and WiFi node
    identifiers and signal strength
  • iPAQ writes audio and WiFi features to SD card
  • Each day, subject uploads data using his or her
    code number to the coded data base

17
Older Procedure
  • Because the real-time feature extraction software
    was not finished in time, the Autumn 2005 data
    collections used a different process (also
    approved)
  • Raw data was encrypted on the SD card
  • The upload program simultaneously unencrypted and
    extracted features
  • Only the features were uploaded

18
Speech Detection
  • From the audio signal, we want to extract
    features that can be used to determine
  • Speech segments
  • Number of different participants (but not
    identity of participants)
  • Turn-taking style
  • Rate of conversation (fast versus slow speech)
  • But the features must not allow the audio to be
    reconstructed!

19
Speech Production
The source-filter Model
Fundamental frequency (F0/pitch) and formant
frequencies (F1, F2 ) are the most important
components for speech synthesis
20
Speech Production
  • Voiced sounds Fundamental frequency (i.e.
    harmonic structure) and energy in lower frequency
    component
  • Un-voiced sounds No fundamental frequency and
    energy focused in higher frequencies
  • Our approach Detect speech by reliably detecting
    voiced regions
  • We do not extract or store any formant
    information. At least three formants are required
    to produce intelligible speech

1. Donovan, R. (1996). Trainable Speech
Synthesis. PhD Thesis. Cambridge University 2.
OSaughnessy, D. (1987). Speech Communication
Human and Machine, Addison-Wesley.
21
Goal Reliably Detect Voiced Chunks in Audio
Stream
22
Speech Features Computed
  • Spectral entropy
  • Relative spectral entropy
  • Total energy
  • Energy below 2kHz (low frequencies)
  • Autocorrelation peak values and number of peaks
  • High order MEL frequency cepstral coefficients

23
Features used Autocorrelation
(a)
(b)
Autocorrelation of (a) un-voiced frame and (b)
voiced frame. Voiced chunks have higher
non-initial autocorrelation peak and fewer number
of peaks
24
Features used Spectral Entropy
FFT magnitude of (a) un-voiced frame and (b)
voiced frame. Voiced chunks have lower entropy
than un-voiced chunks, because voiced chunks have
more structure
25
Features used Energy
Energy in voiced chunks is concentrated in the
lower frequencies
Higher order MEL cepstral coefficients contain
pitch (F0) information. The lower order
coefficients are NOT stored
26
Segmenting Speech Regions
27
Attributes Useful for Inferring Interaction
  • Attributes that can be reliably extracted from
    sensors
  • Total number of interactions between people
  • Conversation styles e.g. turn-taking,
    energy-level
  • Location where interactions take place e.g.
    office, lobby etc.
  • Daily schedule of individuals e.g. early
    birds, late nighters

28
Locations
  • Wi-Fi signal strength can be used to determine
    the approximate location of each speech event
  • 5 meter accuracy
  • Location computation done off-line
  • Raw locations are converted to nodes in a coarse
    topological map before further analysis

29
Topological Location Map
  • Nodes in map are identified by area types
  • Hallway
  • Breakout area
  • Meeting room
  • Faculty office
  • Student office
  • Detected conversations are associated with their
    area type

30
Social Network Model
  • Nodes
  • Subjects (wearing sensors, have given consent)
  • Public places (e.g., particular break out area)
  • Regions of private locations (e.g., hallway of
    faculty offices)
  • Instances of conversations
  • Edges
  • Between subjects and conversations
  • Between places or regions and conversations

31
Non-instrumented Subjects
  • We may recruit additional subjects who do not
    wear sensors
  • Such subjects would allow us to infer information
    about their behavior indirectly, and to appear
    (coded) as a node in our network model
  • E.g., based on their particular office locations
  • Only people who have provided written consent
    appear as entities in our network models

32
Disabling Sensor Units
  • As a courtesy, subjects will disable their units
    in particular classrooms or offices

33
Access to the Data
  • Publications about this project will include
    summary statistics about the social network,
    e.g.
  • Clustering coefficient
  • Motifs (temporal patterns)
  • We will not release the actual graph
  • This is prohibited by our HSD approval
  • We welcome additional collaborators
About PowerShow.com