The Endeavour Expedition: Computing and Communications at the eXtremes

1
The Endeavour Expedition Computing and
Communications at the eXtremes

• Professor Randy H. Katz
• EECS Y2K Conference
• 10 February 2000
• Sibley Auditorium, Bechtel Center

2
The eXtremes
New System Architectures New Enabled
Applications Diverse, Connected, Physical
3
Presentation Outline

• Expedition Motivation
• The Needed Revolution in Computing and
  Communications
• Packing for the Expedition
• Project Organization
• Base Exploration plus Extended Expeditions
• Summary and Conclusions

4
Background

• PITAC Report Information Technology
  Research--Investing in Our Future
• Create a strategic initiative in long-term
  information technology RD
• Priorities Software, Scalable Information
  Infrastructure, High-End Computing, Socioeconomic
  Impacts
• DARPA (and Industry) as Patron
• Chart potential revolutions in information
  technology, with promise to achieve dramatic
  improvements in computing and applications in
  21st Century
• Technology discontinuities drive new computing
  paradigms, applications, system architectures
• E.g., Project MAC, DARPAnet, Xerox Alto
• What will drive the next discontinuity?

5
The Endeavour ExpeditionMotivation and Goals

• Exploiting IT to enhance understanding
• Make it dramatically more convenient for people
  to interact with information, devices, and other
  people
• Supported by a planetary-scale Information
  Utility
• Stress tested by challenging applications in
  decision making and learning
• New methodologies for design, construction, and
  administration of systems of unprecedented scale
  and complexity
• Figure of merit how effectively we amplify and
  leverage human intellect
• Technical Approach Pervasive Information
  Utility, based on fluid systems, enabling new
  approaches for problem solving learning

6
Why Endeavour?

• To strive or reach a serious determined effort
  (Websters 7th New Collegiate Dictionary)
  British spelling
• Captain Cooks ship from his first voyage of
  exploration of the great unknown of his day the
  southern Pacific Ocean (1768-1771)
• Brought more land and wealth to the British
  Empire than any military campaign
• Cooks lasting contribution comprehensive
  knowledge of the people, customs, and ideas that
  lay across the sea
• He left nothing to his successors other than to
  marvel at the completeness of his work.
• Software and resources as a fluid pools, floods,
  rivers, eddies, containers, ...

7
Signing On to the Expedition

• Difficulties are just things to overcome.
• "Men and Women wanted for Hazardous Journey.
  Small wages, bitter cold, long months of complete
  darkness, constant danger, safe return doubtful.
  Honour and recognition in case of success."
• Sir Ernest Shackleton, Legendary Antarctic
  explorer who lost not a single person on two
  highly perilous expeditions to reach the South
  Pole (which he never reached!)
• Business UNusual Research seminars, experimental
  courses, new synthesis of existing courses

8
Expedition Assumptions

• Human time and attention, not processing or
  storage, are the limiting factors
• Givens
• Vast diversity of computing devices (PDAs,
  cameras, displays, sensors, actuators, mobile
  robots, vehicles) No average device
• Unlimited storage everything that can be
  captured, digitized, and stored, will be
• Every computing device is connected in proportion
  to its capacity
• Devices are predominately compatible rather than
  incompatible (plug-and-play enabled by on-the-fly
  translation/adaptation)
• Tremendous innovations in technology yet a
  comprehensive system architecture is lacking

9
Expedition Challenges

• Managing Attention is the Killer App
• Not corporate processing but management,
  analysis, aggregation, dissemination, filtering
  for the individual
• Create Knowledge, not Data
• Not management/retrieval of explicitly entered
  information, but automated extraction and
  organization of daily activities
• Information Technology as a Utility
• Continuous service delivery, on a
  planetary-scale, on top of a highly dynamic
  information base
• Beyond the Desktop
• Community computing infer relationships among
  information, delegate control, establish
  authority

10
The Coming Revolution
11
Expedition Approach

• Information Devices
• Beyond desktop computers to MEMS-sensors/actuators
  with capture/display to yield enhanced activity
  spaces
• InformationUtility
• InformationApplications
• High Speed/Collaborative Decision Making and
  Learning
• Augmented Smart Spaces Rooms and Vehicles
• Design Methodology
• User-centric Design withHW/SW Co-design
• Formal methods for safe and trustworthy
  decomposable and reusable components

• Fluid, Network-Centric System Software
• Partitioning and management of state between soft
  and persistent state
• Data processing placement and movement
• Component discovery and negotiation
• Flexible capture, self-organization, and re-use
  of information

12
The EndeavournautsInterdisciplinary,
Technology-Centered Expedition Team

• Alex Aiken, PL
• Eric Brewer, OS
• John Canny, AI
• David Culler, OS/Arch
• Michael Franklin, DB
• Joseph Hellerstein, DB
• Michael Jordan, Learning
• Anthony Joseph, OS
• Randy Katz, Nets
• John Kubiatowicz, Arch
• James Landay, UI

• Jitendra Malik, Vision
• George Necula, PL
• Christos Papadimitriou, Theory
• David Patterson, Arch
• Kris Pister, Mems
• Larry Rowe, MM
• Alberto Sangiovanni-Vincentelli, CAD
• Doug Tygar, Security
• Robert Wilensky, DL/AI

Speaking in New Faculty Perspectives Session
Speaking in Keynote Session
13
Organization The Expedition Cube
14
Base Expedition Leader Katz

• Broad but necessarily shallow investigation into
  all technologies/applications of interest
• Primary focus on Information Utility
• No new HW design commercially available
  information devices
• Only small-scale testbed in Soda Hall
• Fundamental enabling technologies for Fluid
  Software
• Partitioning and management of state between soft
  and persistent state
• Data and processing placement and movement
• Component discovery and negotiation
• Flexible capture, self-organization, info re-use
• Limited Applications
• Methodology Formal Methods User-Centered Design

15
System Architecture for Vastly Diverse
DevicesLeader Culler

• Design Issues for Small Device OS
• Current managing address spaces,thread
  scheduling, IP stack, windowing system, device
  drivers, file system, APIs, power management
• How can OSs for tiny devices be made radically
  simpler, manageable, and automatically
  composable?
• Devices of Interest Dust Motes

16
The Large Service-Centric Platform Arch
Infrastructure Services

• Enable distributed creation/deployment of
  scalable, available services
• Service registry, aggregate execution env.,
  transparency
• Persistent distributed data structures
• Massive fluid storage (Oceanic Storage)
• Adaptive high-bandwidth flows (rivers)
• Build infrastructure via composition of services

17
The Small Radically Simple OS for Management
Composition

• Basic Assumptions
• Communication is fundamental
• Direct user interface is the exception not the
  norm
• Critical resource is scheduling data movements,
  not arbitrary threads of computation
• Tiny OS Little more than an FSM
• Commands event stream merged with
  sensor/actuator events
• General thread compiled to sequence of bounded
  atomic xacts
• Constant self-checking and telemetry
• Rely on the infrastructure for complex processing
  
• Correctness-by-construction techniques for
  cooperating FSMs (tie in to HW/SW co-design)

18
Implementation Deployment of Oceanic Data Info
UtilityLeader Kubiatowicz

• Confederations of (Mutually Suspicious) Utilities

• Ubiquitous devices require ubiquitous storage
• Consumers of data move, change access devices,
  work in many different physical places, etc.
• Needed properties
• Strong Security
• Coherence
• Automatic replica management and optimization
• Simple and automatic recovery from disasters
• Utility model

19
OceanStore Architecture/Technology

• Name and Data Location
• Issue Find nearby data without global
  communication
• Approach Data location is aform of
  gradient-search of local pools of data (use of
  attenuated Bloom-filters)
• High Availability and Disaster Recovery
• Issue Eliminate backup as independent/fallible
  technology
• Approach Erasure-codes/mobile replicas provide
  stable storage for archival copies and snapshots
  of live data
• Introspective Monitoring and Optimization
• Issue Optimize performance on a global scale
• Approach Monitoring and analysis of access/usage
  relationships
• Rapid Update in Untrusted Infrastructure
• Issue Updates should not reveal info to
  untrusted servers
• Approach Incremental cryptographic
  techniques/oblivious function techniques to
  perform update

20
Sensor-Centric Data Management for
Capture/ReuseLeader Hellerstein

• Managing Data Floods
• Never ends interactive direction
• Big data reduction/aggregation
• Unpredictable scale of devices and nets not
  behave nicely
• Builds on CONTROL and River/Eddy System
• Early answers, interactivity, online aggregation
• Information processing via massively parallel,
  adaptive dataflows
• Extended to wide-area operator placement,
  reordering

• Telegraph Data Manager
• Distributed Storage Manager based on event flow
  and state machines
• Continuously adaptive dataflow with applications
  to sensor data and streaming media

21
Negotiation Architecture for Cooperating
ComponentsLeader Wilensky

• Cooperating Components
• Self-administration through auto-discovery and
  configuration among confederated components
• Less brittle/more adaptive systems
• Negotiation Architecture
• Components announce their needs and services
• Service discovery and rendezvous mechanisms to
  initiate confederations
• Negotiated/contractural APIs contract designing
  agents
• Compliance monitoring/renegotiation/non-compliance
  recovery
• Graceful degradation in response to environmental
  changes

22
Tacit Knowledge Infrastructureand Collaborative
ApplicationsLeaders Canny/Joseph/Landay

• Exploit information about the flow of information
  to improve collaborative work
• Capture, organize, and place tacit information
  for most effective use
• Learning techniques infer communications flow,
  indirect relationships, availability/participation
  to enhance awareness and support opportunistic
  decision making
• New applications
• 3D activity spaces for representing
  decision-making activities, people, information
  sources
• Visual cues to denote strength of ties between
  agents, awareness levels, activity tracking,
  attention span
• Electronic Problem-based Learning in Enhanced
  Physical and Virtual Learning Spaces

23
User Interfaces and Design ToolsJames Landay and
John Canny

• Future devices wont have PC-style UI
• Extreme range of devices
• Small or embedded in environment
• Often w/ alternative I/O w/o screens
• Special purpose applications, e.g., Info
  Appliances
• Lots of devices per user, all working in concert
• Key Technologies
• Tacit information analysis algorithms
• Design tools that integrate
• Sketching other low-fidelity techniques
• Immediate context tacit information
• Interface models

24
Safe Component Design Leader Sangiovanni

• Correctness by Construction
• Safe partitionings of communicating subcomponents
  placed in wide-area
• Builds on on-going work in embedded systems
  design
• Compositions of Third Party Components
• Safety enforcement technologies
• Design and development methodologies
• Builds on Neculas Proof Carrying Code
• Trust and Assurance
• Integrated use of secure tokens for rights
  management, economic protocols/auctions, support
  for mobile code, ...
• Secure protocol design deployment based on
  super-fast model-checking/automatic generation
  from requirements

25
Experimental Testbeds
Soda Hall
IBM WorkPad
Smart Dust
Velo
Nino
LCD Displays
MC-16
Motorola Pagewriter 2000
CF788
Pager
WLAN / Bluetooth
Smart Classrooms Audio/Video Capture
Rooms Pervasive Computing Lab CoLab
H.323 GW
GSM BTS
Wearable Displays
TCI _at_Home Adaptive Broadband LMDS
Millennium Cluster
CalRen/Internet2/NGI
Millennium Cluster
26
Putting It Together

• 1. eXtreme Devices
• 2. Data Utility
• 3. Capture/Reuse
• 4. Negotiation
• 5. Tacit Knowledge
• 6. Classroom
• 7. Design Methods
• 8. Scale-up

Devices Utility Applications
Component Discovery Negotiation
Fluid Software
Info Extract/Re-use
Self-Organization
Decision Making Group Learning
27
Summary and Conclusions

• 21st Century Computing
• Making peoples exploitation of information more
  effective
• Encompassing eXtreme diversity, distribution, and
  scale
• Computing you can depend on
• Key Support Technologies
• Fluid software computational paradigms
• System and UI support for eXtreme devices
• Pervasive, planetary-scale system utility
  functionality
• Active, adaptive, safe and trusted components
• New power tool applications that leverage
  community activity
• Broad multidisciplinary team spanning the needed
  applications, evaluation, and system technology
  skills
• Culture of large-scale, industry-relevant high
  impact research projects

28
Industrial Collaborators
SRI