Title: Desiging a Virtual Information Telescope using Mobile Phones and Social Participation Romit Roy Chou
1Desiging a Virtual Information Telescopeusing
Mobile Phones and Social ParticipationRomi
t Roy ChoudhuryAsst. Prof. (Duke University)
2- A little bit about ourselves
3Webpage
http//synrg.ee.duke.edu
SyNRG
4Our Research
What are the visions (top down)
Ubiquitous Services
Incentives
Application
Privacy
Security
Eavesdropping
Loss Discrimination
Transport
Mobility
Network
Energy Savings
MAC / Link
Spatial Reuse
Interference Mgmt.
PHY
Channel fluctuations
What can be enabled (bottom up)
5Some Ongoing Projects
Information Telescope
Spotlight
MobiSys 2008 Infocom 09
ICNP 2008
Hotnets 2008
Shuffle
SmartCast
6Todays Talk
Information Telescope
Papers Infocom 2009 HotMobile 2009 MobiSys
2008 Posters, Demos MobiCom, Sensys,
MobiSys Mobicom Student Research Contest Award
2008
7Todays Talk
Information Telescope
System and Applications
Ongoing, Future Work
Challenges/Opporunities
Vision
- EnLoc
- SurroundSense
- CacheCloak
8- Virtual Information Telescope
9Context
- Next generation mobile phones will have
- large number of sensors
- Cameras, microphones, accelerometers, GPS,
- compasses, health monitors,
10Context
- Each phone may be viewed as
- a micro lens
- Exposing a micro view of the physical world
- to the Internet
11Context
- With 3 billion active phones
- in the world today
- (the fastest growing comuting platform )
- Our Vision is
12A Virtual Information Telescope
Internet
13- One instantiation of this vision through
- a system called Micro-Blog
- Content sharing - Content querying - Content
floating
14Content Sharing
Virtual Telescope
Web Service
Cellular, WiFi
Visualization Service
People
Phones
Physical Space
15Content Querying
Virtual Telescope
Web Service
Cellular, WiFi
Visualization Service
People
Phones
Physical Space
16Content Floating on physical space
superb sushi
Safe_at_ Nite?
17- If designed carefully, a variety of
- applications may emerge on Micro-Blog
18Applications
- Tourism
- View multimedia blogs query for specifics
- Micro Reporters
- News service with feeds from individuals
- On-the-fly Ride Sharing
- Ride givers advertize intension w/ space-time
sticky notes - Respond to sticky notes once you arrive there
- Virtual order on physical disorder
- Land in a new place, and get step by step
information - RSS Feeds on Location
- Inform me when a live band is playing at the mall
19MiroBlog Prototype
- Nokia N95 phones
- Symbian platform
- Carbide C code
20- Micro-Blog Beta live at
- http//synrg.ee.duke.edu/microblog.html
21Prototype
22Case Studies
- Micro-Blog phones distributed to volunteers
- 12 volunteers
- 4 phones in 3 rounds
- 3 weeks
- Not great UI
- Basic training for users
- Exit interview revealed
- useful observations
23From Exit Interview
- Fun activity for free time
- Needs much cooler GUI
- Privacy control vital, dont care about
incentives - more interesting to reply to questions
interested in knowing who is asking - Voice is personal, text is impersonal
- Easier to correct text audio blogs easier but
- Logs show most blogs between 500 to 900pm
- Probably better for battery usage as well
24Thoughts
- Micro-Blog
- Rich space for applications and services
25- Several research challenges and opportunities
- Energy-efficient localization
- Symbolic localization through ambience sensing
- Location privacy
- Incentives
- Spam
- Information distillation
- User Inerfacing
Our Research
26Disclaimer
- All of our projects are ongoing,
- hence not fully mature
- Todays talk more about the problems
- than about solutions
27- Problem I
- Energy Efficient Localization
- (EnLoc)
28To GPS or not to GPS
- GPS is popular localization scheme
- Good error characteristics 10m
- Apps naturally assume GPS
- Shockingly, first Micro-Blog demo lasted lt 10
hours
29Cost of Localization
- Performed extensive measurements
- GPS consumes 400 mW, AGPS marginally better
- Idle power consumption 55 mW
30Alternate Localization
- WiFi fingerprinting, GSM triangulation
- Place Lab, SkyHook
- Improved energy savings
- WiFi 20 hours
- GSM 40 hours
- At the cost of accuracy
- 40m
- 200m
31Tradeoff Summary
40
20
200
32Formulation
L(t2) L(t3) L(t4)
L(t0)
L(t1)
L(t6)
L(t5)
L(t7)
Error
t0
t1
t2
t3
t4
t5
t6
t7
Given energy budget, E, Trace T, and location
reading costs, egps , ewifi , egsm Schedule
location readings to minimize avg. error
33Dynamic Program
- Minimize the area under the curve
- By cutting the curve at appropriate points
- Number of (GPS WiFi GSM) cuts must cost lt
budget
34- Offline optimal offers lower bound on error
- Online algorithm necessary
- Online optimal difficult
- Need to design heuristics
35Our Approach
- Do not invest energy if you can
- predict (even partially)
36Predictive Heuristics
- Prediction opportunities exist
- Human users are not in brownian motion (exploit
inertia) - Exploit habitual mobility patterns
- Population distribution can be leveraged
- Prediction also incorporated into Dynamic Program
- Optimal computed on a given predictor
Error
Prediction generates different error curve
t0
t1
t2
t3
t4
t5
t6
t6
37Mobility Profiling
- Build logical mobility tree per-user
- Each link an uncertainty point (UP)
- Sample location only when uncertain
- Location predictable between UPs
- Exploit acclerometers
- Predict traffic turns
- Periodically localize to reset errors
38Population Statistics
- Humans may deviate from mobility profile
- Predict based on population statistics
39Buy Accuracy with Energy
- Comparison of optimal with simple interpolation
- GPS clearly not the right choice
40Thoughts
- Localization cannot be taken for granted
- Critical tradeoff between energy and accuracy
- Substantial room for saving energy
- While sustaining reasonably good accuracy
- However, physical localization
- May not be the way to go
- Several motivations to pursue symbolic
localization
41- Problem 2
- Symbolic localization
- (SurroundSense)
42Symbolic Localization
- Services may not care about physical location
- Symbolic location often sufficient
- E.g., coffee shop, movie, park, in-car
- Physical to Symbolic conversion
- Lookup location name based on GPS coordinate
- However, risky
RadioShack
Starbucks
GPS Error range
43Hypothesis
Its possible to localize phones by sensing the
ambience such as sound, light, color, movement,
orientation
44SurroundSense
- Develop multi-modal fingerprint
- Using ambient sound/light/color/movement etc.
Starbucks
RadioShack
Wall
SurroundSense Server
45SurroundSense
- Each individual sensor not discriminating enough
- Together, they are quite unique
- Use Support Vector Machines to identify
uniqueness
Location
Classification Algorithm (SVM)
Fingerprint Database
46Should Ambiences be Unique Worldwide?
GSM provides macro location (mall) SurroundSense
refines to Starbucks
47Why will it work?
- Economics forces nearby businesses to be
different - Not profitable to have 5 chinese restaurants
- with same lighting, music, color, layout, etc.
- SurroundSense exploits this ambience diversity
The Intuition
48Fingerprints
49Fingerprints
50Ambience Fingerprinting
Fingerprint Filtering Matching
Sound
Color/Light
Compass
RF/Acc.
Logical Location
Fingerprint Database
Macro Location
Candidate Fingerprints
51Full System on Nokia N95
- Experimented on 100 stores
- 10 different clusters
- Different parts of Duke campus
- and in Durham city
52Full System on Nokia N95
- Some classifications were incorrect
- But we wanted to know how much incorrect?
- We plotted Top-K accuracy
- Top-3 accuracy proved to be 100 for all stores
53Issues and Opportunity
- Cameras may be inside pockets
- Now, we detect when its taken out
- Activate cameras, and take pictures
- Future phones will be flexible (wrist watch) -
see Nokia Morph - Electroic compasses can fingerprint layout
- Tables and shelves laid out in different
orientations - Users forced to orient in those ways
54Summary
- Ambience can be a great clue about location
- Ambient Sound, light, color, movement
- None of the individual sensors good enough
- Combined they may be unique
- Uniqueness facilitated by economic incentive
- Businesses benefit if they are mutually diverse
in ambience - Ambience diversity helps SurroundSense
55- Problem 3
- Location Privacy
- (CacheCloak)
56Location Privacy
Continuous location exposure a serious threat to
privacy
Preserve privacy without sacrificing the quality
of continuous loc. based apps
57Just Call Yourself Freddy
- Pseudonymns
- Effective only when infrequent location exposure
- Else, spatio-temporal patterns enough to
deanonymize - think breadcrumbs
Leslie
Jack
John
Susan
Alex
Romits Office
58Add Noise
- K-anonymity
- Convert location to a space-time bounding box
- Ensure K users in the box
- Location Apps reply to boxed region
- Issues
- Poor quality of location
- Degrades in sparse regions
- Not real-time
Bounding Box
You
K4
59Confuse Via Mixing
- Path intersections is an opportunity for privacy
- If users intersect in space-time, cannot say who
is who later - Issues
- Users may not be collocated in space and time
- Mixing still possible at the expense of delay
60- Existing solutions seem to suggest
- Privacy and Quality of Localization (QoL)
- is a zero sum game
- Need to sacrifice one to gain the other
61Our Goal
- Break away from this tradeoff
- Target Spatial accuracy
- Real-time updates
- Privacy guarantees
- Even in sparse populations
We design CacheCloak
62CacheCloak Intuition
- Exploit mobility prediction to create
- future path intersections
- Users paths are like crossroads of breadcrumbs
- App knows precise locations, but doesnt know the
user
63CacheCloak
- Assume trusted privacy provider
- Reveal location to CacheCloak
- CacheCloak exposes anonymized location to Loc. App
Loc. App1
Loc. App2
Loc. App3
Loc. App4
CacheCloak
64CacheCloak Design
- User A drives down path P1
- P1 is a sequence of locations
- CacheCloak has cached response for each location
- User A takes a new turn (no cached response)
- CacheCloak predicts mobility
- Deliberately intersects predicted path with
another path P2 - Exposes predicted path to application
- Application replies to queries for entire path
- CacheCloak always knows users current location
- Forwards cached responses for that precise
location
65CacheCloak Design
- Adversary confused
- New path intersects paths P1 and P2 (crossroads)
- Not clear where the user came from or turned onto
- Example
66Example
67Benefits
- Real-time
- Response ready when user
- arrives at predicted location
- High QoL
- Responses can be specific to location
- Overhead on the wired backbone (caching helps)
- Entropy guarantees
- Entropy increases at traffic intersections
- In low regions, desired entropy possible via
false branching - Sparse population
- Can be handled with dummy users
68Quantifying Privacy
- City converted into grid of small sqaures
(pixels) - Users are located at a pixel at a given time
- Each pixel associated with 8x8 matrix
- Element (x, y) probability that user enters x
and exits y - Probabilities diffuse
- At intersections
- Over time
- Privacy entropy
y
x
pixel
69Diffusion
- Probability of users presence diffuses
- Diffusion gradient computed based on history
- i.e., what fraction of users take right turn at
this intersection
Time t1
Time t2
Time t3
Road Intersection
70Evaluation
- Trace based simulation
- VanetMobiSim US Census Bureau trace data
- Durham map with traffic lights, speed limits,
etc. - Vehicles follow Google map paths
- Performs collision avoidance
6km x 6km 10m x 10m pixel 1000 cars
71Results
- High average entropy
- Quite insensitive to user density (good for
sparse regions) - Minimum entropy reasonably high
72Results
- Per-user entropy
- Increases quickly over time
- No user starves of location privacy
73Issues and Limitations
- CacheCloak overhead
- Application replies to lots of queries
- However, overhead on wired infrastructure
- Caching reduces this overhead significantly
- CacheCloak assumes same, indistinguishable query
- If user asks different query at each road segment
- Overhead increases
- Adaptive branching dummy users
- Offer user-specified privacy guarantee
74Closing Thoughts
- Two nodes may intersect in space but not in time
- Mixing not possible
- Mobility prediction allows space-time
intersections - Enables better privacy
75Conclusion
- The Virtual Information Telescope
- A generalization of mobile, location
- based, social computing
- Just developing apps
- Not enough
- Many challenges
- Energy
- Localization
- Privacy
- Incentives, data distillation
Internet
76Conclusion
- Project Micro-Blog
- Addressing the challenges systematically
- Building a fully functional system with
applications - The project snapshot as of today, includes
Micro-Blog Overall system and application EnLoc
Energy Efficient Localization SurroundSense
Context aware localization CacheCloak Location
privacy via mobility prediction
77PhonePoint Pens
- Using phone accelerometers
- To write short messages in the air
78PhonePoint Pen
- Using phone accelerometers
- To write short messages in the air
Contact Sandip Agrawal sandip.agrawal_at_duke.edu
For details, visit synrg.ee.duke.edu
79- Please stay tuned for more at
- http//synrg.ee.duke.edu
- Thank You
80Example
81Results
- High average entropy
- Quite insensitive to user density (good for
sparse regions) - Minimum entropy reasonably high
82Results
- Length of predictions
- Remains reasonably short
- Overhead proportional to this length
83Fingerprints
- Acoustic fingerprinting
- Coffee machine hum in cafes, different from A/C
machine drones in grocery, different from spoon
clink in restaurants - Loudness varies (some places with loud music,
others quiet)
84Fingerprints
- Color and Light Fingerprints
- Stores have a specific décor (color, light)
- Theme reflected in wall color, floor color,
lighting style
85Our Approach
- Build symbolic localization algorithms
- Use low accuracy physical localization as
baseline - Low accuracy conserves energy
86SurroundSense
- Sense ambient light, sound, colors
- Combine sensor readings to generate soft
fingerprint - For localization, gather fingerprint from mobile
- Match with database of fingerprints
- Of course, fingerprint may not be globally unique
- Use rough physical localization as a pre-filter
GSM physical location says you are in the
mall SurroundSense augments that with you are
in Apple Store
87- Sample sound/light shows diversity
- Extract features from signals
- Fingerprint 46 sound 2 light features
- 48 dimensional space
Sound (Frequency domain)
Light intensity
88- Match test fingerprint with trained database
- Use Nearest Neighbor algorithm for
classification
Feature Extract
Location
Feature Extract
48 features
89SurroundSense Design
- Prototype on Tmote Invent sensors
- Sound and light sensors
- Low acoustic frequency range 20, 250 Hz
- Currently porting on Nokia N95 phones
90Fingerprint Extraction
- Light intensity and sound signals recorded
- Fourier transform on sound
- Overlapping frequency blocks generated
- Each block 23 bands, 10 Hz each
- Extract simple features
- Each 10 Hz band one feature
- Variance of each band another feature
- Normalized light intensity another feature
- Total - 48 features
- Train the system with half the data
- 48 dimensional fingerprint
91Fingerprint Generation and Matching
- Match test fingerprint with trained database
- Use Nearest Neighbor algorithm for
classification
Feature Extract
Location
Feature Extract
48 features
92Results
- SurroundSense offers consistent localization
- Database contains nearby shops in Duke campus
- Both sensors gt sound gt light
Pairwise Similarity
93- Symbolic localization can be augmented with
- phone accelerometers
- Additional benefits in activity recognition
94Hypothesis
- Movement partially indicative of location
- People sit in cafes
- Run in gyms
- Walk up/down aisles in grocery stores
- Time duration spent may depend on location
- Augment location accuracy with acc. signatures
- Enable activity recognition as well
- E.g., Advertize shoes to users running in the gym
95AAMPL
- Acclerometer augmented phone localization
- Train database with many acc. signatures
- Mobile phone 3-axis accelerometer in pockets
- Less useful if phone in ladys handbag
- Use acc. signatures from individuals in real time
- Match against database of signatures
- Localize phone
- Predict activity
- Combine accelerometers with sound and light --
fingerprint
96AAMPL Architecture
- Client Server based
- Two-stage classifier
- Phones classify sitting/standing
- Sends to server
- Saves energy
- Server classifies location
- WiFi based basic location
- Augmented by AAMPL
- Server compares classified location with Googles
result
97AAMPL Classification
98Evaluation
- Gathered acc. signatures from many restaurants
- Classified with AAMPL, compared with Google
Store Apple Wholefoods Journeys Solstice
Fast Food Chipotle Jimmy Johns
Restaurant Chais Verde Rockfish
99Evaluation
- AAMPL classified each location
- Compared corresponding location with Google
100Thoughts
- Main Idea is that the surrounding is a
fingerprint. - Effective for separating out nearby contexts.
- In reality,
- Spatially clustered shops are diverse by design
- SurroundSense exploits this diversity
101Mote based Prototype
- Used Tmote sensors on behalf of phones
- Equipped with basic
- light sensor and microphone
- Mobile phones expected to be more powerful
- Better audio sensing (Larger freq. range than
20-250hz) - Better light sensing with camera
102Results on TMotes
- Tested in Duke campus (shops restaurants)
Correct matches
SurroundSense identified each store correctly
1031. Simple Interpolation
- Take GPS reading, followed by WiFi
- Extrapolate GPS in the direction of WiFi
- Reset prediction after threshold time, take GPS
again