Title: The%20Endeavour%20Expedition:%2021st%20Century%20Computing%20to%20the%20eXtreme
1The Endeavour Expedition21st Century Computing
to the eXtreme
- Randy H. Katz, Principal Investigator
- EECS Department
- University of California, Berkeley
- Berkeley, CA 94720-1776
2The Endeavour Expedition21st Century Computing
to the eXtreme
- New Ideas
- Systems Architecture for Vastly Diverse
Computing Devices (MEMS, cameras, displays) - Wide-area Oceanic Data Information Utility
- Sensor-Centric Data Management for Capture
and Reuse (MEMS networked storage) - Negotiation Architecture for Cooperating
Components (Composable system architecture) - Tacit Knowledge Infrastructure to support
High-Speed Decision-Making - Information Management for Intelligent
Classroom Environments - Scalable Safe Component-based Design and UI
Design Tools
R. H. Katz, Principal Investigator, University
of California, Berkeley
- Impact
- Enhancing human understanding by making it
dramatically more convenient for people to
interact with information, devices, and other
people - Supported by a planetary-scale Information
Utility, stress tested by applications in
decision making and learning, achieved thru
new methodologies for design, construction,
and administration of systems of
unprecedented scale and complexity
Schedule
Usability Studies Early Tool Design
Implementation of UI Sys Design Tools
Tools Release Final Evaluations
Design Methodologies
Initial Application Implementation Evaluation
Refined Implementation Final Evaluation
Information Applications
Initial Architectural Design Testbeds
Initial Evaluation 2nd Gen Redesign
Final Deployment Evaluation
Information Utility
Initial Architectural Design Document
Initial Experiments Revised Design Doc
Final Experiments Architecture Docs
Jun 99 Start
Jun 00
Jun 01
May 02 End
3Agenda
- Project Motivation and Overview, Katz
- System Architecture for eXtreme Devices, Culler
- Oceanic Data Storage Utility, Kubiatowicz
- Sensor-Centric Data Management, Hellerstein
- Usability and User Interface Design, Landay
- Remaining Options and Wrap-up, KatzQA by DARPA
PMs expected throughout
4Agenda
- Project Motivation and Overview, Katz
- System Architecture for Extreme Devices, Culler
- Oceanic Data Storage Utility, Kubiatowicz
- Sensor-Centric Data Management, Hellerstein
- Usability and User Interface Design, Landay
- Remaining Options and Wrap-up, Katz
5Why Endeavour?
- Endeavour to strive or reach a serious
determined effort (Websters 7th New Collegiate
Dictionary) British spelling - Captain Cooks ship from his first voyage of
exploration of the great unknown of his day the
southern Pacific Ocean (1768-1771) - Brought more land and wealth to the British
Empire than any military campaign - Cooks lasting contribution comprehensive
knowledge of the people, customs, and ideas that
lay across the sea - He left nothing to his successors other than to
marvel at the completeness of his work.
6Expedition Goals
- Enhancing understanding
- Dramatically more convenient for people to
interact with information, devices, and other
people - Supported by a planetary-scale Information
Utility - Stress tested by challenging applications in
decision making and learning - New methodologies for design, construction, and
administration of systems of unprecedented scale
and complexity - Figure of merit how effectively we amplify and
leverage human intellect - A pervasive Information Utility, based on fluid
systems to enable new approaches for problem
solving learning
7Expedition Assumptions
- Human time and attention, not processing or
storage, are the limiting factors - Givens
- Vast diversity of computing devices (PDAs,
cameras, displays, sensors, actuators, mobile
robots, vehicles) No such thing as an average
device - Unlimited storage everything that can be
captured, digitized, and stored, will be - Every computing device is connected in proportion
to its capacity - Devices are predominately compatible rather than
incompatible (plug-and-play enabled by on-the-fly
translation/adaptation)
8Expedition Challenges
- Personal Information Mgmt is the Killer App
- Not corporate processing but management,
analysis, aggregation, dissemination, filtering
for the individual - People Create Knowledge, not Data
- Not management/retrieval of explicitly entered
information, but automated extraction and
organization of daily activities - Information Technology as a Utility
- Continuous service delivery, on a
planetary-scale, on top of a highly dynamic
information base - Beyond the Desktop
- Community computing infer relationships among
information, delegate control, establish
authority
9Driving Factors
- Technology Push
- Accelerating developments at the eXtremes
- Cluster-based compute/storage servers
- MEMS sensor/actuators, CCD cameras, LCD displays,
- User Pull
- More effective community leverage the next power
tool - Desire
- Enhanced interaction, ease of use
- Easier configuration, plug and play
- Less fragile tools, always there utility
functionality
10Computing EvolutionDistribution with Sharing
Increasing Freedom from Colocation Increasing
Sharing Distribution Increasing
Personalization Increasing Ratio of
ComputersUsers
11Computing Revolution Devices in the eXtreme
12Expedition Approach
- Information Devices
- Beyond desktop computers to MEMS-sensors/actuators
with capture/display to yield enhanced activity
spaces - InformationUtility
- InformationApplications
- High Speed/Collaborative Decision Making and
Learning - Augmented Smart Spaces Rooms and Vehicles
- Design Methodology
- User-centric Design withHW/SW Co-design
- Formal methods for safe and trustworthy
decomposable and reusable components
- Fluid, Network-Centric System Software
- Partitioning and management of state between soft
and persistent state - Data processing placement and movement
- Component discovery and negotiation
- Flexible capture, self-organization, and re-use
of information
13High Speed Decision Making
Learning Classroom
E-Book
Vehicles
Applications
Collaboration Spaces
Info Appliances
Human Activity Capture
Generalized UI Support
Event Modeling
Transcoding, Filtering, Aggregating
Statistical Processing/Inference
Proxy Agents
Negotiated APIs
Self-Organizing Data
Information Utility
Interface Contracts
Wide-area Search Index
Nomadic Data Processing
Wide-Area Data Processing
Automated Duplication
Distributed Cache Management
Movement Positioning
Stream- and Path-Oriented Processing Data Mgmt
Non-Blocking RMI
Soft-/Hard-State Partitioning
Laptop
PDA
Wallmount Display
Camera
Information Devices
Smartboard
MEMS Sensor/Actuator/Locator
Handset
14Needed Expedition Expertise
- Today, scientists and adventurers are lured by
exploratory challenges to all regions of the
globe and beyond. The explorer attempts routes of
greater difficulty, the researcher perfects field
techniques in remote locales. All are breaking
new ground in isolated areas of the world usually
under harsh conditions over extended periods of
time. - http//www.expeditionresearch.org/english/
- MEMS and hardware devices
- Scalable computing architectures
- Networked-oriented operating systems
- Distributed file systems
- Data management systems
- Security/privacy
- User interfaces
- Collaboration applications
- Intelligent learning systems
- Program verification
- Methodologies for HW/SW design/evaluation
15Interdisciplinary, Technology-Centered Expedition
Team
- Alex Aiken, PL
- Eric Brewer, OS
- John Canny, AI
- David Culler, OS/Arch
- Joseph Hellerstein, DB
- Michael Jordan, Learning
- Anthony Joseph, OS
- Randy Katz, Nets
- John Kubiatowicz, Arch
- James Landay, UI
- Jitendra Malik, Vision
- George Necula, PL
- Christos Papadimitriou, Theory
- David Patterson, Arch
- Kris Pister, Mems
- Larry Rowe, MM
- Alberto Sangiovanni-Vincentelli, CAD
- Doug Tygar, Security
- Robert Wilensky, DL/AI
16Organization The Expedition Cube
17Base Program Leader Katz
- Broad but necessarily shallow investigation into
all technologies/applications of interest - Primary focus on Information Utility
- No new HW design commercially available
information devices - Only small-scale testbed in Soda Hall
- Fundamental enabling technologies for Fluid
Software - Partitioning and management of state between soft
and persistent state - Data and processing placement and movement
- Component discovery and negotiation
- Flexible capture, self-organization, info re-use
- Limited Applications
- Methodology Formal Methods User-Centered Design
18Base Program Schedule
Year 1
Year 2
Year 3
Design Methodology
Refined Tools Flow
Information Utility
Information Applications
19In-Depth Technical Presentations
- Option 1 Systems Architecture for Vastly
Diverse Computing Devices, David Culler,
Subexpedition Leader - Option 2 Implementation/Deployment of the
Oceanic Data Information Utility, John
Kubiatowicz, Subexpedition Leader - Option 3 Sensor-Centric Data Management for
Capture and Reuse, Joseph Hellerstein,
Subexpedition Leader - Parts of Options 5, 6, 7 UI Design Cross Cut (UI
design tools with applications to Tacit
Information Extraction and Intelligent
Classrooms), James Landay, Subexpedition Leader
20Roll-Up of Remaining Options
- Option 4 A Negotiation Architecture for
Cooperating Components, Robert Wilensky,
Subexpedition Leader - Option 7 Scalable Safe Component-based Design,
Alberto Sangiovanni-Vincentelli, Subexpedition
Leader - Option 8 Scale-Up Field Trials, Randy Katz,
Subexpedition Leader - (Essential elements of Option 5 Tacit
Information Infrastruction and High Speed
Decision Making and Option 6 Information
Management for Intelligent Classroom Environment
covered by James Landay
21Option 4 Negotiation Architecture for
Cooperating Components
- Cooperating Components
- Self-administration through auto-discovery and
configuration among confederated components - Less brittle/more adaptive systems
- Essential for all pieces of the Endeavour Utility
Infrastructure - Negotiation Architecture
- Components announce their needs and services
- Service discovery and rendezvous mechanisms to
initiate confederations - Negotiated/contractural APIs contract designing
agents - Compliance monitoring and renegotiation
- Graceful degradation in response to environmental
changes
22The Problem Configuration Difficulties
- Individual computing components require
considerable manual configuration - OS, software installation
- Local data (solved by Oceanic storage!)
- Configuration to access services
- Today small number of machines per
individual--(manual) configuration limits - State (software/data) is inconsistent across
machines - Manual updating is time-consuming
- Degrades poorly in the presence of failure/change
- Future orders of magnitude more machines per
individual--manual configuration completely
infeasible
23Solution Negotiation Architecture for
Auto-Configuration
- Allow components to dynamically configure
themselves by having components - Specify the potential services they provide, the
terms and conditions, and to whom - Disseminate the availability of these services
- Specify the services they require, and their
terms and conditions - Discover other objects that provide required
services - Allow objects to enter into multi-phase
negotiations of contracts, committing to provide
services under terms and conditions - Provide compliance monitoring services of
contracts - Provide means for dealing with non-performing
confederates
24Plan for Success
- Develop
- Language for specifying services, and their terms
and conditions - Protocol for negotiating contracts between
objects - Infrastructural services, including discovery,
service availability dissemination, and
compliance monitoring services - Means to adapt to a non-performing service
- Emphasis on system architecture/easy of use
- E.g., standard, parameterized boilerplate
contracts between components, with standard
compliance officers - Some related issues
- Can we assure interesting adaptive properties?
- Recent development HPs espeak
25Option 7 SafeComponent Design Leader
Sangiovanni
- Information Appliances as an application of
hardware/software codesign - Specification based on Co-design Finite State
Machines - Exploited in software for eXtreme devices
- Formal methods to verify safety from faults
- Safe partitioning of components into
communicating subcomponents placed into the
wide-area - Component-Based System Design
- Composition of third party components to build
systems - Can such components be trusted?
- Correctness (Necula)
- Security (Tygar)
26Option 7 SafeComponent Design Leader
Sangiovanni
- Formal Specifications and Methods
- Decomposition of components into safe
partitionings of communicating subcomponents
placed in the wide-area - HW/SW Co-design Finite State Machines
- Exploits success in embedded software arena
- Use in software for eXtreme devices
- Compositions of third party components
- JAVA or C/C modules
- Use in Oceanic Store, Sensor-centric Data Mgmt
- Formal methods to verify
- Correctness/safety from faults
- Trust and assurance
27An Essential Problem forComponent-based Fluid
Software
- Cannot be trusted to behave as advertised
- If unknown origin must be assumed to be
malicious - If known origin can be erroneous or even
malicious - Concerned with
- Extrinsic properties (non-semantic properties)
- e.g., author, time of creation, 3rd
party-endorsements, ... - Semantic properties (behaviors)
- e.g., memory safety, lack of information flow,
etc. - Needed
- Safety enforcement technologies
- Design and development methodologies
28Solution Proof Carrying Code
- Safety without sacrificing performance
- Works for low-level languages, machine code,
optimized code - Small trusted code base
- Checking is easier than proving
- No need to use (and trust) a compiler
- Flexible and general (in principle)
- Need a specification that captures the property
of interest - Plus proof of that property for the untrusted
code - If you can prove it, PCC can check it!
- Install one checker for a multitude of policies
- Use tools that certify their output
- Delegate but do not trust
- Effective way to debug the tools themselves!
29Plan for Success
Slow
Quick
- Ratify a broad set of safe programming
practices for component-based systems - Failures point to programming errorsOR tool
suggests convenient stylistic adjustmentsOR tool
inserts run-time checks - Build toolkit for producing provably-safe native
methods with off-the-shelf Java compilers - Build toolkit for certifying type safety of C
programs
30Security and Assurance
- Two issues for apps based on mobile code
- Protecting the remote host from the mobile code
- Protecting the mobile code from the remote host!
- Automatic generation of best security protocol
- Ad hoc and temporal access control
- Access control/security negotiation
- Cryptographic hardware tokens as type of
Information Device - How to evaluate, build, break tamper-resistant
boundaries - Differential power analysis
31Infrastructure Enables
- Microactions/economics for resource control
- Pervasive need for authentication
- Enables resource management based on privileges
- Rights management tagging
- Who can operate on what under what conditions?
- Design for survivability
- Exploit resource control to mitigate denial of
service attacks - All of this with privacy
- Users control when and to whom information is
released - Trade better system support for privacy
32Plan for Success
- One year
- Synthesis of code for optimal security protocols
- Toolkit for cryptographic key management for
mobile code - Design of ad hoc and temporal access control
- Little TEMPEST protection for hardware tokens
- Three year
- Integration with applications across Endeavour
- Privacy analysis for high assurance mechanisms
- Automatic or semi-automatic resource allocation
using micro-auctions. - High survivability mechanisms
33Option 8 Scaled-up Field TrialsLeader Katz
- Testbed Rationale
- Study impact on larger/more diverse user
community - Higher usage levels to stress underlying
architecture - Make commitment to true utility functionality
- Increasing Scale of Testbeds
- Building-Scale
- Order 100s individuals
- Campus-Scale
- Order 1000s individuals
- City-Scale
- Order 100000 individuals
34Experimental Testbeds
Soda Hall
IBM WorkPad
Smart Dust
Velo
Nino
LCD Displays
MC-16
Motorola Pagewriter 2000
CF788
Pager
WLAN / Bluetooth
Smart Classrooms Audio/Video Capture
Rooms Pervasive Computing Lab CoLab
H.323 GW
GSM BTS
Wearable Displays
TCI _at_Home Adaptive Broadband LMDS
Millennium Cluster
CalRen/Internet2/NGI
Millennium Cluster
35Summary Putting It All Together
- 1. eXtreme Devices
- 2. Data Utility
- 3. Capture/Reuse
- 4. Negotiation
- 5. Tacit Knowledge
- 6. Classroom
- 7. Design Methods
- 8. Scale-up
Devices Utility Applications
Component Discovery Negotiation
Fluid Software
Info Extract/Re-use
Self-Organization
Decision Making Group Learning
36Conclusions
- 21st Century Computing
- Making peoples exploitation of information more
effective - Encompassing eXtreme diversity, distribution, and
scale - Computing you can depend on
- Key Support Technologies
- Fluid software computational paradigms
- System and UI support for eXtreme devices
- Pervasive, planetary-scale system utility
functionality - Active, adaptive, safe and trusted components
- New power tool applications that leverage
community activity
37Conclusions
- Commercial spin, but direct relevance for many
DoD future information technology requirements - Survivable, secure communications systems
- System support for pervasive sensor networks
- Fluid infrastructure support for
- CONUS forward basing concepts
- Rapid force deployment
- Coalition leverage of shared/untrusted
infrastructure - Information apps serve are examples for
- Training
- Mission planning
- Battlespace decision making
38Conclusions
- Broad multidisciplinary team spanning the needed
applications, evaluation, and system technology
skills - Builds on many existing DARPA investments
- BARWAN, Digital Libraries, iStore, Marco, MASH,
MEMS, Ninja, Proof Carry Code,Tertiary Disk, ), - Integrates and extends these into a comprehensive
information system architecture for 21st century
computing - History of building large-scale prototypes,
influencing industrial development
39Back-Up
40Technology Evolution versus Revolution
Information Appliances
More
Many people per computer
One person per computer
Scaled down PCs, desktop metaphor
PC Network
Distribution
Many computers per person
WS/Server
Time Sharing
RJE
Less
Batch
Less
More
Personalization
41Option 1 System Architecture for Vastly Diverse
DevicesLeader Culler
- Distributed control resource management data
mvmt transformation, not processing - Path concept for information flow, not the thread
- Persistent state in the infrastructure, soft
state in the device - Non-blocking system state, no application state
in the kernel - Functionality not in device is accessible thru
non-blocking remote method invocation - Extend the Ninja concepts (thin client/fat
infrastructure) beyond PDAs to MEMS devices,
cameras, displays, etc.
42Option 2 Implementation Deploy-ment of Oceanic
Data Info UtilityLeader Kubiatowicz
- Nomadic Data Access serverless, homeless, freely
flowing thru infrastructure - Opportunistic data distribution
- Support for promiscuous caching freedom from
administrative boundaries high availability and
disaster recovery application-specific data
consistency security - Data Location and Consistency
- Overlapping, partially consistent indices
- Data freedom of movement
- Expanding search parties to find data, using
application-specific hints (e.g., tacit
information)
43Option 3 Sensor-Centric Data Management for
Capture/ReuseLeader Hellerstein
- Integration of embedded MEMS with software that
can extract, manage, analyze streams of
sensor-generated data - Wide-area distributed path-based processing and
storage - Data reduction strategies for filtering/aggregatio
n - Distributed collection and processing
- New information management techniques
- Managing infinite length strings
- Application-specific filtering and aggregation
- Optimizing for running results rather than final
answers - Beyond data mining to evidence accumulation
from inherently noisy sensors
44Option 5 Tacit Knowledge Infra-structure/Rapid
Decision MakingLeader Canny
- Exploit information about the flow of information
to improve collaborative work - Capture, organize, and place tacit information
for most effective use - Learning techniques infer communications flow,
indirect relationships, and availability/participa
tion to enhance awareness and support
opportunistic decision making - New collaborative applications
- 3D activity spaces for representing
decision-making activities, people, information
sources - Visual cues to denote strength of ties between
agents, awareness levels, activity tracking,
attention span
45Problem Applications for Ubiquitous Computing
- People are the main knowledge asset in an
organization - How do we design computing tools and work
processes in the age of universal computing? - Study practice look at difficulties of use
identify new opportunities
46Application Remote Interaction
- PRoPs Wireless robot appliances that act as
proxys or avatars - What they could achieve
- Mobility and access to remote workplaces
factories, offices, warehouses - A better level of interpersonal interaction
through non-verbal communication - Recreation when its too far to go
47Application Tacit Information Mining
- Use logs from single or multiple servers to
compute - High level context, current activity
- An organized activity view
- Personal expertise and referrals
- Document authority
- Document history and creation context
- Perspectives on a document or meeting
48Application Bearable Computing
- An exploration of issues in personal, persistent
computing (augmented reality, worn interfaces)
using ordinary laptop computers - Avoid head-mounted displays (expensive and
low-res) head-tracking, and cables - The approach use optics to overlay computer
images on reality, but use laptop or
pocket-mounted displays - Testbed Grad course in HCC this semester
49Option 6 Info Mgmt for Intelligent
ClassroomsLeader Joseph
- Electronic Problem-based Learning
- Collaborative learning enabled by information
appliances - Enhanced Physical and Virtual Learning Spaces
- Wide-area, large-scale group collaboration
- Capture interaction once for replay
- Preference/task-driven information device
selection - Service accessibility
- Device connectivity
- Wide-area support
- Iterative evaluation
50The Problem Configuration and Scaling
- Device/Network-independent People-to-People
Communications - Any-to-Any people-level (not device)
communications - Service Handoff (cross network/device mobility)
- Classroom Learning
- Related option is 6
- Challenge of scaling, while preserving 1-on-1
- Wide-area information mgmt / access
- Related options area 1, 2, 4, 8
- Device/Network-independent People-to-Service
Communication - Flexible consistency, replication, access control
51Solution Service Architecture
- Device/Network-independent People-to-People
Communications - Open arch for device network-independence
- Ninjas Automatic Path Creation
- Icebergs IAPs, PAT, Preference Registry (dyn
rules) - Iceberg testbed Universal Inbox
- Classroom Learning
- Iceberg information dissemination technologies
- InfoCaster, CASA, Secure Service Discovery
Service - Iceberg testbed real-world data
- Wide-area information mgmt / access
- Experience w/ Secure Service Discovery Services
Wide-area information dissemination
52Plan for Success
- Dev./Net.-indep. People-to-People Comm
- Y1 Deploy real-world testbed w/ 1st cut arch
- Y2 Detailed experiments and design of 2nd gen
- Y3 Deployment / measurement of 2nd gen
- Classroom Learning
- Y1 Design classroom experiment, deploy sw/hw
- Y2 Group mtg experiment/large class experiment
- Y3 Larger class?
- Wide-area information mgmt / access
- Y1 Deploy SDS. First-cut info utility svc.
- Y2 Few users of single-node info utility
- Y3 Second version (distrib) w/ real users