Active Spaces

1
Active Spaces

• Roy H. Campbell
• rhc_at_cs.uiuc.edu
• Computer Science Department
• University of Illinois at Urbana-Champaign

2

Where virtual reality puts people inside a
computer-generated world, ubiquitous computing
forces the computer to live out here in the world
with people.
Mark Weiser
3
Introduction

• A standard definition for an Active Space does
  not exist, but it is evolving.
• There are different approaches regarding
  functionality required and implementation details.

4
Overview

• Georgia Tech, Microsoft
• Emerging perspectives
• University of Illinois
• Scale
• Software Infrastructure
• Berkeley, M.I.T., Washington
• Comparison

5
Active Spaces Properties

• Some common requirements
• Location Detection
• Contextual Model
• Autonomous and Responsive Behavior
• Correlation of Heterogeneous Input
• Interaction Mechanism

6
Example Implementations Classroom 2000.
Georgia Tech

• Goal Preservation and augmentation of electronic
  notes taken by students and teachers.
• Attach and synchronize audio, video and
  annotations to class notes

7
Example Implementations Classroom 2000.
Georgia Tech

• The architectural scheme must be able to evolve
  with time.
•  
• Scheme phases
• Preproduction Automate tasks to begin class Live
• Capture Record and timestamp all events
  associated to class.
• Postproduction Support development of interfaces
  to integrate and associate related streams.
• Access Provide a universal access method (WEB)

8
Example Implementations Classroom 2000.
Georgia Tech

• First implementation
• Useful as an experimentation prototype.
• Equipment and software not good enough.
• Lessons learned were applied to the next
  implementation.

9
Example Implementations Classroom 2000.
Georgia Tech

• Living Laboratory
• Completely equipped classroom that hosted several
  courses
• Improved software that allowed better media
  integration.
• Software infrastructure did not require as much
  attention from users as in previous
  implementation (invisibility required by
  ubiquitous computing)

10
Example Implementations Classroom 2000.
Georgia Tech

• Their conclusions
• There should be a motivating application.
• Built system should address some notion of scale
• Physical space covered.
• Number of individuals involved.
• Number and variety of devices supported.
• Amount of time over which the application is run.
• System must be subjected to real and everyday
  usage before being subject of authentic
  evaluation.

11
Example Implementations EasyLiving.
Microsoft

• Focuses on physical home and work environments.
• Computing must be as natural as lighting

12
Example Implementations EasyLiving.
Microsoft

• People are the central entities of the Active
  Space.
• Main characteristics of their intelligent
  environment
• Self-awareness
• Casual access
• Extensibility

13
Example Implementations EasyLiving.
Microsoft

• Self-awareness
• EasyLiving spaces must be aware of their own
  activity and contents to allow appropriate
  responses to the movement of people and their
  requests
• Requirements
• Geometry
• People within space
• Peoples actions and preferences
• Resources available

14
Example Implementations EasyLiving.
Microsoft

• Casual Access
• Users should not be required to go to a special
  place to interact with the computer. Nor should
  they be required to wear special devices or
  markers to have the computer know where they are
• The computer is everywhere.

15
Example Implementations EasyLiving.
Microsoft

• Extensibility
• EasyLiving capabilities should grow
  automatically as new hardware is added

16
Example Implementations EasyLiving.
Microsoft

• Design issues
• Sensing and Modeling video cameras
• User interfaces new approaches, migration
• Privacy

17
Example Implementations EasyLiving.
Microsoft

• Design issues
• Architecture for extensibility

Room Server
Device
Device
Central Server
Room Server
Device
Device
18
Active Spaces UIUC

• The future of computing is
• implicit  
• to sense and affect its surroundings
• interconnected and mobile
• nimble and adaptive

19
Vision Pervasive Computing

• Anywhere/anytime collaboration
• Real world environments
• augmented with information spaces
• Mobility and ubiquity
• Applications like design and prototyping

20
Implicit Computing

• Features
• embedded in every day devices
• mobile and interconnected
• pervasive throughout the environment
• coupled to information sources
• active, modifying their environment
• Analogy
• telephone (implicit)
• telegraph (explicit)

21
Research Vision

• Distributed, multimodal collaboration
• situated and mobile
• varying resources
• bandwidth, computing, power, and display
  capabilities
• varying collaboration intensities
• focused, interested, and ambient
• dynamic reconfiguration and adaptation
• resources and modalities

22
Intelligent Spaces

• Smart rooms
• intelligent offices domiciles of mobile users
• conference and seminar rooms multiway
  collaboration
• Virtual environments
• smart device prototyping and information space
  introspection
• Lone rangers
• local, high-bandwidth mobile collaborators
• Road warriors
• remote, low-bandwidth mobile collaborators
• Prototyping laboratories
• smart device stereolithographic fabrication

23
Research Vision

• Three software pillars
• dynamic, interoperable software frameworks
• multimodal, adaptive networking
• controls for adaptive management
• Hardware testbeds
• smart rooms and virtual environments
• mobile users and devices
• device prototyping
• development infrastructure

24
Smart Rooms

• Components
• intelligent physical objects (rocks)
• ambient condition sensors
• user, object, and location identifiers
• X10 automation sensors and controls
• lighting, power, temperature controls
• commercial speech tools
• limited vocabulary voice commands
• voice acknowledgments
• tracking cameras
• distributed collaboration and localization

DARPA CPOF
25
Smart Rooms

• Components (continued)
• smart white boards
• brain storming and digital ink capture
• multiple displays
• conference LCD displays
• high-resolution flat panels
• multiway conferences and digital video
• digital video encoders
• MPEG and HDTV multicast
• multimodal networks
• infrared, spread spectrum radio, Ethernet, and ATM

26
Lone Rangers

• Rationale
• continuous connectivity
• local, mobile collaborators
• drop in for smart room collaborations
• Components
• wearable computers with wireless networks
• multimedia workstations/PCs

27
Road Warriors

• Rationale
• wide-area mobile collaborators
• low bandwidth connectivity
• Components
• personal digital assistants (PDAs)
• Palm Pilot
• palmtop computers
• Toshiba Libretto and/or Windows CE systems
• two-way pagers
• RIM/Bellsouth or Motorola ReFLEX PageWriter

28
Virtual Environments

• Components
• drafting table ImmersaDesks
• gaze tracking hardware
• tactile data gloves
• Continuing EVL collaboration
• virtual environment software
• ImmersaDesk prototypes
• Walls
• 30 ft x 9ft 10 x 3 CPUs/displays

29
Prototyping Laboratory

• Rationale
• prototyping of smart space objects
• physical/virtual interconnection
• Components
• stereolithographic prototyping system
• object scanner

30
Challenge Interoperable Component Architecture

• Scalable
• Flexible
• Pervasive
• Dynamic and adaptive
• Mobile
• Visual and aural

31
Challenge Quality of Service

• Spectrum of networks and computers
• Wealth of multimedia and requirements
• Dynamic requirements and resources

32
Challenge Active Space Analysis and Control

• Modality transduction
• Heterogeneous devices
• Information access metaphors

33
Definitions Active Spaces. UIUC

• An "Active Space" (AS) is a framework which
  models and manages physical environments as well
  as its contents
• "Space" implies that a principle use of the AS
  framework is to manage physical spaces, such as
  campuses, buildings, meeting rooms, cities,
  offices, cars, buses, trains, etc

34
Properties Active Spaces. UIUC

• "Active" notes that the framework is intended to
  support reactive and customized behavior.
• Key properties identified so far for Active
  Spaces
• Export an adaptable interaction model, that
  allows interacting with the active spaces
• Provide a contextual model
• Reflective and autonomous
• Have an associated physical counterpart.

35
Properties Active Spaces. UIUC

• Key properties identified so far for Active
  Spaces
• Contain entities with a well defined behavior
  that can be dynamically created and modified.
• Provide a concept of inside and outside.
• Define protocols to interact with other active
  spaces.
• Autonomous, contextual and can trigger events
  spontaneously

36
Key Technical Requirements Active Spaces.
UIUC

• Key Technical Requirements
• Dynamic Behavior
• Reflection
• Adaptability
• Component Based Infrastructure

37
Example Implementations Active Spaces. UIUC
ARCHITECTURAL REQUIREMENTS
38
Implementation Active Spaces. UIUC

• We provide a generic infrastructure that allows
  implementing different scenarios.
• Every particular scenario defines the nature of
  the entities contained in the AS as well as the
  behavior of the particular AS.
• The great diversity of Active Space scenarios
  makes it impossible to define a single Active
  Space that covers all semantic requirements.

39
Future Example of Interaction

• Smart room configures itself as user enters
• PDA configures itself for room
• User OS manages accessible room resources in
  collaboration with room OS

• Video transferred from smart room HDTV display to
  palm top as user leaves
• User accesses smart room from remote facilities
• User environment follows user movements

40
2K

• User-oriented, network-centric OS
• built from components
• Based on reflective distributed objects
• Targets ubiquitous computing/smart spaces
• Supports mobility and persistence
• leveraging CORBA, DCOM, Java RMI services and
  protocols
• Embedding legacy applications and systems

41
Distributed Objects

• Reflective
• naming, location, protocol, and implementation
• Dynamic
• dependency model, programmable reconfiguration
• Self-Aware
• self-monitoring, self-configuring, and dependency
  aware
• Standardized
• Object Request Broker (ORB), OMG, Java RMI, or
  DCOM

42
(No Transcript)
43
Components

• Componentized ORBs dynamic TAO
• concurrency control and thread models
• resource allocation and schedulers
• Microkernel support for ORBs Off
• Corbarized device drivers video, X10
• Corbarized distributed OS 2K
• LegORB 8Kbytes, Palmshell, remote video
• Proxy server Palm video filter
• Persistent store name server nameORB

44
Features Demonstrated

• Configurable ORBs and reflection
• Programming with dependency objects
• Self-awareness in ORBs
• LegORB a thin ORB
• PalmMPEG video proxies for modality
• X10 ORBs ubiquity
• The 2K environment mobile users
• Security UIUC Tiny Sesame

45
Automatic ConfigurationPrerequisites

• Specifies component needs
• HW resources
• HW capacity
• software services required
• Helps implement WYNIWYG
• Video client example
• PC with MPEG decoder card
• 50 of 200 MHz processor required
• CORBA video server

46
Automatic ConfigurationDynamic Dependence
ComponentConfigurator associated with each
relevant component
depends on
depends on
COMPONENT
HOOKS
CLIENTS
Hooked Components
CLIENTS
47
DynamicTAO Dynamically Configurable CORBA ORB

• Reflective ORB
• architecturally self-aware
• dynamic component (un)loading
• Reconfiguration of
• ORB engine
• CORBA applications

48
Configuration Agent Application of dynamic TAO

• A video distribution network
• code distribution, group reconfiguration, state
  inspection
• Configure graph of network
• Agents multicast update
• dynamic TAO dependencies synchronize update
• Future keep servers and clients running as
  permitted by dependencies

49
Context Monitoring
50
PalmPilot Integration in 2K
2k
System Bootstrapping
System Utilization
2
1
3
4
5
6

Camera
7
51
Streaming Video to Palm Pilot
MPEG Stream

• Palm Pilot
• lacks processing power to decode MPEG
• Video proxy
• transforms MPEG streams
• reduces
• frame rate, color depth, size
• sends compressed bitmaps

Video Proxy
Compressed Bitmap Stream
52
QoS Components

• Trading service
• QoS method providers and QoS methods
• transcoders and filters
• Exported service(s)
• QoS proxy resource coordinator
• Resource brokers
• DSRT and communication objects

53
2K QoS Framework
QoS Method Provider
Export
Trading Service
export
import
2
1
client
server
Discovered service(s)
3
6/11
import
5/12
9
QoS Proxy
QoS Proxy
10
Comm. object
Comm. object
13
Net condition
7
Net condition
4
4
7
8
RB
RB
RB
RB
RB
RB
RB Resource Broker
RB Resource Broker
54
Security Active Capabilities

• Mobile security agents
• Secure bootstrapping process
• minimal core security services
• Active capabilities
• application specific access control
• Interoperable security policies
• CORBA compliant
• security services and APIs

55
Policy Classes

• DAC - Discretionary Access Control
• Double DAC
• NDAC - Non ...
• DONDAC, domain-oriented ...
• MAC formed from customized NDAC
• DSP Device Specific Policies
• DANDAC, device aware ...

56
Security Components
Application Client
Application Server
Active Capability/Certificates
Active Capability/Certificates
BOA
Stub
ORB
Dynamic Policies
Security Mechanisms
Network Transport
57
A Light-Weight Security Mechanism Tiny UIUC
SESAME
58
Comparisons

• Berkeley
• M.I.T.
• Washington

59
Berkeley Endeavour

• Funding Darpa Exhibition
• Objective of Program Achieve nothing less than
  radically enhancing human understanding through
  the use of information technology, by making it
  dramatically more convenient for people to
  interact with information, devices, and other
  people

60
Berkeley Endeavour

• Research areas
• Applications
• Information Utility
• Information Devices
• Design Methodologies

61
Berkeley Endeavour

• Key technologies produced
• Integration of MEMS-based sensors, actuators,
  position locators, communicators with imagers
  (cameras), displays, and hand-portable and other
  mobile computing devices into the system
  architecture of the next century.
• Support for fluid software.
• High-speed decision making and education/learning
  spaces.
• Design methodologies.

62
Berkeley Endeavour

• Existing infrastructure
• Interactive University Project
• Millennium
• Tertiary Disk Prototype
• Wireless Network (GSM base transceiver, REFLEX
  two-way paging basestation, Lucent WaveLAN)
• Post-PC infrastructure (IBM Workpads)

63
Washington Portolano

• Funding DARPA Expedition
• Objective of Program
• Invisible User Interfaces
• Universal Connectivity
• Intelligent Services

64
Washington Portolano

• Research areas
• Devices
• Intentional User Interfaces
• Low-power architectures/devices/communication
• Application partitioning for low-power

65
Washington Portolano

• Research areas
• Middleware
• Application development/deployment
• Data-centric networking
• Scalable discovery/directory services

66
Washington Portolano

• Research areas
• Services
• Composable, horizontally-layered services
• Distributed databases and data integration
• Data mining and database layering

67
Washington Portolano

• Existing infrastructure
• Devices used in previous ParcTab experiments.

68
Washington Portolano

• Key technologies produced
• Portolano infrastructure.
• User and usage studies of wide deployment.
• Ubiquitous large displays
• Application development environment
• Creation of a set of office application/device
  suites
• Modular sensor system

69
Washington Portolano

• Key technologies produced
• Location sensor technologies
• Novel automatic identification technologies
• Flexible active network infrastructures
• Sensor fusion
• Extended information-space management
  infrastructure
• Making sense of complex information from
  distributed information sources

70
M.I.T. Oxygen

• Funding DARPA
• Objective of Program Do more by doing less
• Bring new technologies into peoples lives, not
  vice versa.
• Improve human productivity.
• Offer new technology to everybody not only a
  fraction of people.

71
M.I.T. Oxygen

• Research areas
• Hardware
• Speech recognition
• Artificial Inteligence
• Networks
• OSes

72
M.I.T. Oxygen

• Key technologies produced
• Handy 21 multipurpose software configurable PDA.
• Enviro 21Stays attached to the environments
  around the people.
• Net 21 Creates secure collaborative regions
  among Oxygen users.
• Speech understanding
• Knowledge access
• Collaboration
• Automation
• Customization

73
Conclusion

• An Active Space is meaningless without a well
  defined application.
• Correlation of heterogeneous input data is
  required to create a software model of the active
  space.
• Adaptation of applications based on the software
  model of the active space guarantees
  heterogeneous access to the active space.