LocationAware Applications for Built Environments: Geometric Models in the Real World Seth Teller Co

1
Location-Aware Applicationsfor Built
EnvironmentsGeometric Models in the Real
WorldSeth TellerComputer Graphics GroupMIT
LCS/AI Lab (Some work joint w/ H.
Balakrishnan, E. Demaine, M. Stonebraker, J.
Leonard)
2
New class of location-aware apps/devices
Suppose we had

• A faithful geometric and functional model of our
  architectural environment and its contents, and
• A cheap, accurate, handheld location and
  orientation sensor that worked pervasively
  indoors (i.e., without GPS)

?
3
Fundamental new devices apps
(analogous to the way GPS changed most outdoor
apps)

• Indoor navigation
• Software compass
• Asset annotation and tracking
• Software marker
• Software finder
• Facilities maintenance
• Software flashlight

4
Navigation, Resource Finding

• Hand-held software compass
• Knows its position and orientation
• Enables user to
• Find mapped resources in offices, museums, etc.

LCS Floor 2
LCS Floor 1
5
Asset Annotation

• Software marker
• Model software compass object database
• Unique ID location sensor (moveable objects
  only)
• Enables user to
• Mark and query real-world objects in database

Elevator, Location NE43-2 Maintenance
request Filed by Inspector Button does not
work. Date January 15, 2002
Lock, Location NE43-202 Occupant
Leiserson Keys Leiserson, New keys Oct. 1995
Printer, Location NE43-2 South Owner Graphics
group Subnet 18.24.2. Admin hanna_at_graphics
6
Asset Tracking and Finding

• Software finder
• Unique RF ID location sensor for all valuable
  assets
• Wireless network of readers, DBMS back end for
  tracking
• Enables user to
• Track and locate real-world objects in real world

Where are my keys?
Where is my car parked ?
Where are my proceedings of ?
Where should I reshelve this ?
Your laptop has left the building !
7
Facilities Maintenance
Information!

• Software flashlight
• 3D Model Software compass Projector
• Enables user to
• Project CAD information onto real objects

Inspect Wiring
Install Power outlet
8
Rapid model capture 3 approaches

• 1. Scan 3D environment with one or more
  ordinary hand-held (or robotic) video cameras
• Example LCS 2nd Floor Video Sequence

9
Vision-based capture Challenges

• Address scaling, complexity, varying lighting
• Extract coarse, detailed geometry and texture
• Camera excursions through multiple floors
• Deployment of multiple cameras in parallel
• Robotic (autonomous) environment capture

10
2. Model generation from legacy CAD

• Exploit existing (poor) 2D CAD models

• Compile 2D CAD into well-formed 3Dgeometric
  models

11
Initial 2D floorplan
Layers exterior/interior walls text
icons Physical plant has economic incentive to
maintain
12
Analyze for spaces, room labels, etc.
Now have offices, corridors, interstitial spaces
c
13
Extrude to 3D
Doorjambs, raised floors, dropped ceilings, c.
14
Add vertically connecting elements
Stairs, elevators, airshafts, ducting etc.
15
Add procedural lights, furnishings
Override locally with per-user schema or specifics
16
Getting data CAD-based model capture

• Collected 100 MIT buildings, 900 floorplans
• Registered to common 3D coordinate system
• Extracted named spaces, 2D 3D adjacencies

17
Model population with fast HCI methods

• Assume software compass infrastructure in place
• Human operator moves through environment
• Affixing RF-ID/location tag to valuable objects
• Indicating object type, attributes using speech
• Population app logs object and its metadata
  into DBMS
• In long run tags attached at point of
  manufacture

Laser Pointer, Sonar, Camera, Microphone
18
Variety of application domains
19
Cricket location and orientation mechanism
Beacons on ceiling
SPACENE43-510 ID34 COORD146 272 0
Ultrasound (pulse)
Cricket listener
Multiple ultrasound sensors enableheading
determination
Obtain distances to multiple beacons Solve for
(x, y, z) of hand-held device
Solve for ? of hand-held device
20
Cricket v1 Prototype
http//nms.lcs.mit.edu/cricket
RF module (transmit)
Ultrasonic sensor
RF module (receive)
RF antennas
Listener
Beacon
Atmel processor
RS232 i/f
Host software libraries in Java Linux daemon
(in C) for Oxygen BackPaq handhelds Several
prototype applications
21
Prototype Software Compass
22
Use Differential Distance (Phase) to Determine
Compass Orientation
Assume Device lies in horizontal plane Method
Use multiple ultrasonic sensors estimate
device orientation from measured distances
d1, d2, z Future Add tilt sensor
(relax assumption of horizontality)
Beacon
z
d1
d2
d
S1
L
S2
23
Plans for next 1-5 years

• Prototype beacons listeners
• MOBICOM 00, 01 publications
• Deployed v1 hardware to several LCS floors
• v2 hardware in fabrication
• v3 hardware in design phase
• System development
• Power management (1000s of beacons)
• Geometric self-calibration algorithm
• Integration with GPS coordinates at perimeter
• Applications compass, population, finder
• Scalable location-aware monitoring (w/
  Stonebraker)

24
(No Transcript)
25
Resource location

• Requires location-tagged resources, users
• Example applications
• Lead (visitor) to my office
• Lead (me) to a colleagues office
• Lead (everyone) to a talk about to start
• Lead groups of visitors through demo stations
• Spool my color printout where did it end up?
• Whats the fastest walking/wheelchair route to X?
• (Morning) wheres an empty parking spot?
• (Evening) wheres my (_at_! car?

26
Resource embedding

• Requires sensors throughout environment
• Example applications
• Stream/archive audio/video talks lectures
• Find/show me a free conference room right now
• How long is the line at the trucks?
• Show me who walked out with my laptop
• Coupled with user-specified models
• Archiving what was LCS second floor like
  pre-renovations?what did Baker room 323 look
  like in 1999?

27
Resource activation

• Requires location-dependent actuation
• Examples
• Open the parking garage gate upon my approach
• (On my way in) unlock door, turn on lights
• (On my way out) vice-versa
• (Signage) post/lead the way to imminent talk
• Call the elevator for an approaching way-finder
• Call up my desktop on the nearest wall

28
Resource optimization

• Requires model, DBMS, and simulation
• Example applications
• What is our power consumption at 68 degrees? at
  70? What if the A/C is left on after 10pm?
• How quickly will a fire spread in building? how
  long will it take to evacuate? (real-time) get
  everyone out, avoiding hazards
• Fastest route walking from 32-203 to 3-270?
  with no stairs, fewest elevators, ramps, etc.?
• What were duty cycles of all conference rooms?
  can they be improved by time/group swaps?

29
System ingredients

• Instrumented environment
• Functional geometric model w/ resource info
• Pervasive location, orientation capability
• Object tags, environmental sensors
• Persistent interconnect
• Local computation at sensors
• Wireless interconnect
• DBMS for event logging
• Applications making queries
• One-shot Where is my cell-phone
• Continuous Track my laptop through building
• Historical What is average usage of room X

30
Conclusion several related ideas

• Pervasively instrumented environments
• Functional, populated geometric models
• Location and orientation capability
• Object tags, environmental sensors
• Applications
• New class of devices, tools for use in real world
• Smart(er) environments for better resource usage
• Challenges
• Device, deployment, architecture, scaling

31
(No Transcript)
32
Use Differential Distance (Phase) to Determine
Compass Orientation
Assume Device lies in horizontal plane Method
Use multiple ultrasonic sensors estimate
device orientation from measured distances
d1, d2, z
Beacon
sin ? (d2 - d1) / sqrt (1 - z2/d2) where d
(d1d2)/2
d
z
d1
d2
From range, measure a) (d2 - d1)b) z/d
Future add tilt sensor
S1
L
S2
33
Recent progress

• Longest video sequence in egomotion literature
• Rotation-stabilized video (locked to scene)
• Extracted 3D line cloud (no polygons, texture)

34
4. New approach to as-built CAD

• Fundamental problem in construction
• As-built deviates from as-planned
• Hard to monitor compliance
• Hard to acquire CAD after construction
• Sprinkle cricket-cameras throughout site
• Externally at start internally as building
  proceeds
• Cameras localize, network, self-calibrate
• Vision techniques monitor materials, construction
• Many interesting engineering, research challenges

35
Omni-video sequence 1 Basement

• 2400 NTSC frames _at_ 5Hz 8 minutes
• Total path length 106 meters
• Nav odometry, drift rate 10 degrees/minute
• Ground truth SINAS nav, SICK laser scanner

36
Deployment (LCS 5th floor)
Prototype applications Navigation Resource
location Next steps Deploy to all LCS/AI
(3-4 beacons per room) Self-calibration algm
Marker, flashlight prototype devices
37
Summary
Pursuing three integrated activities

• Rapid creation of environment models
• Vision-based capture, CAD-based generation
• Pervasive location, orientation capability
• Multiple active beacons, passive listeners
• Compelling location-aware applications
• Navigation, asset management, maintenance
• Powerful new (Oxygen) hand-held devices

38
(No Transcript)
39
How are we getting there?

• Vision-based model capture
• Develop scalable computer vision algorithms to
  acquire accurate models of architectural spaces
• CAD-based model generation
• Exploit existing 2D CAD information, and simple
  rules, to generate rich 3D models
• Pervasive location/orientation capability
• Cricket beacons (w/ Hari Balakrishnan)
• Deploy throughout LCS, rest of campus

40
How are we getting there?
Vision-based model capture
Environment model and object metadata
CAD-based model generation
Cricket pervasive location, orientation
Cricket is joint work with Hari Balakrishnan
41
What is location-based computing?

• Application of location-specific information
  about an environment and users within it
• Yes, but

42
Location-based computing requires

• 1) A representation of environment, its contents

2) Locating/orienting resources, self in
representation
3) Operators applications on the representation
43
Location-based computing

• 1) A representation of environment, its contents

2) Locating/orienting entities in representation
3) Operators applications on the representation
44
Capturing architectural spaces

• Robotic scene capture outdoors, indoors
• Research effort likely 5-10 years out

45
Generating architectural spaces procedurally

• Nearer term 1 to 2 years out
• Joint work with Berkeley graphics group
• Idea exploit existing, rich 2D data
  sets(floorplans) maintained by MIT DOF
• Add small amount of extra informationextrude to
  well-formed 3D CAD modelwith interior detail,
  furniture, etc!

46
Generating architectural spaces procedurally

• Collaboration with MIT DOF
• Models are live weekly, batch update
• Provide resulting maps, models to Oxygen
• Substrate for embedded place-specific resources

47
Status and Plans

• Extrude exterior basemap using building heights

48
Status and Plans

• Extrude every floor of every building at MIT

49
Location-based Computing
1) A representation of environment, its contents
2) Locating/orienting yourself in representation
3) Operators applications on the representation
?
50
Locative sensors (Joint work w/ Hari)

• Requires instrumentation of environment
• GPS works OK outdoors badly indoors
• Indoors Haris Crickets (based on RF,US)

51
Status and plans

• Prototype software compass on 5th floor
• Reports users location and bearing in floor
  coords.
• Eventually instrument LCS/AI, Stata, etc.
• Make crickets self-calibrating attach to
  resources
• Seamlessly extend GPS coordinates indoors

52
Location-based Computing
1) A representation of environment, its contents
2) Locating/orienting yourself in representation
3) Operators applications on the representation
53
Example operators, applications

• Resource location
• Resource embedding
• Resource activation

54
Conclusions

• End-to-end description of LBC
• Capture process
• Localization/orientation sensor
• Applications
• Capture, location sensing, application efforts
• Specific data, sensors of use to Oxygen project
• Suggested several example applications

55
(No Transcript)