Next Generation Distributed Systems: The and dynamicTAO approach

1
Next Generation Distributed Systems The
and dynamicTAO approach

• Fabio Kon
• kon_at_ime.usp.br
• Department of Computer Science
• University of São Paulo, Brazil
• http//www.ime.usp.br/kon

2
Introduction

• Modern Computing Environments
• Hardware diversity embedded systems, PDAs,
  laptops, workstations, supercomputers.
• Software diversity different programming
  languages, component architectures, operating
  systems.
• Mobile computers
• Mobile users (different accounts in different
  systems)

3
Highly-Dynamic Environments

• Frequent changes
• 1. Structural changes
• HW and SW upgrades, OS patches, protocol updates
• 2. Dynamic changes
• availability of memory, CPU, and network
  bandwidth connectivity, physical location

4
Goal of Current Research

• Facilitate management of dynamic, heterogeneous
  computing environments for
• Users
• System administrators
• Developers

5
The Approach

• The Network is the Computer , Sun microsystems.
• Network-Centrism
• user profiles, user environments
• services, applications, components
• WYNIWYG
• (What You Need Is What You Get)
• dynamic instantiation of applications and
  services
• automatic configuration

6
From where can we start?

• Run on multiple hardware platforms
• Run on top of different OSes
• Support different programming languages
• Support dynamism, late binding, components
• Solution
• OMG IDL
• CORBA ORBs
• Standard CORBA Services (Naming, Trading,
  Persistence)

7
But There Was A Problem

• Conventional ORBs were static
• Fixed threading model
• Fixed transport protocol IIOP (over TCP/IP)
• Fixed security strategy
• Fixed scheduling
• Inadequate for a wide range of applications
• Multimedia
• Mobile Computing
• Adaptive Applications

8
Reflective ORB

• Allows inspection and dynamic reconfiguration of
  the ORB internal engine.
• 1. dynamicTAO an extension of the TAO ORB
  Schmidt
• very complete, big
• 2. LegORB (now, UIC ) a component-based ORB
• not complete, but expanding
• very small (minimal client 6K or 20K, minimal
  server 30K)
• 3. OpenORB Blair et al, University Of
  Lancaster
• prototype in Python, implementation in C/COM

9
What is missing?

• We have
• Reflective Middleware layer supporting
  distributed objects in a dynamically configurable
  way.
• Standard services for Naming, Trading, Security,
  Persistence, Transactions, Events.
• We still need
• Support for automatic configuration.
• Dynamic instantiation of user environments.
• Dynamic resource management.

10
2K Services

• Component Repository
• Automatic Configuration
• Distributed Resource Management
• Mobile Configuration Agents
• User Environment Service
• Distributed QoS Compilation Service
• Security, Data Management, ...

11
The 2K Architecture
12
Automatic Configuration Service

• Automatically instantiates applications and
  services by assembling their components.
• Based on
• Prerequisites static representation of
  dependencies.
• ComponentConfigurators dynamic representation
  of dependencies.

13
Prerequisites

• What a component needs to run
• nature of hardware resources
• share of the hardware resources
• software services (i.e., components) it requires
• Video Client example
• PC with Sound card
• 50 of CPU gt300MHz
• CORBA Video Service

14
Automatic Configuration Process

• 1. Fetches component code and prerequisites from
  the Component Repository.
• 2. Dynamically link component code into the
  application address-space.
• 3. Based on the prerequisites, repeats the
  process for other components.

15
Automatic Configuration Architectural Framework
fetch prerequisites
load application
Component Repository
Prerequisite Resolver
fetch components
return reference
Prerequisite Parser
QoS-Aware Resource Manager
Cache
16
Component Configurators

• Reify dynamic inter-component dependencies.
• Created on-the-fly by the Prerequisite Resolver.
• System and application software can inspect and
  reconfigure the Dependence Graph.

17
ComponentConfiguratorFramework

• Allows browsing, inspection, and reconfiguration
• Can be customized through inheritance
• Clear separation of concerns

18
QoS-Aware Distributed Resource Management

• Global Resource Manager (GRM)
• one in each cluster
• maintains an approximate view of the cluster
  resource utilization
• Local Resource Manager (LRM)
• runs in each node
• exports the state of the local resources
• Has a Real-Time Scheduler (DSRT)
• admission control, reservation, and scheduling

19
Loading an Application with the Resource
Management Service

• 1. Client contacts local LRM, giving application
  name and QoS requirements
• 2. LRM performs admission test
• 3. Request forwarded to GRM
• 4. GRM forwards request to best candidate
• 5. Remote LRM performs admission test,
  reservation, and runs AutoConfig.

20
AutoConfig ServiceLoading Several Components
21
The 2K Architecture
22
Dynamically Configurable MiddlewareReflective
ORBs

• Reflective Systems Smith 84
• Meta-Object Protocol Kiczales 91
• Reflective ORBs Singhai and Campbell 97
• The ORB maintains a representation of its own
  internal structure, supporting
• Inspection
• Dynamic Reconfiguration
• Causal Connection

23
dynamicTAO

• Built as an extension of the TAO ORB Schmidt et
  al
• Written in C
• Modular design based on
  object-oriented design patterns
• TAO already supported startup configuration
  configuration file specifies strategies for
• concurrency (threading model)
• request demultiplexing
• scheduling
• connection management

24
Adding Support for Dynamic Configuration

• dynamicTAO exports an interface called
  DynamicConfigurator, supporting
• 1. Transfer of components across the distributed
  system
• 2. Loading and unloading components
• 3. Inspecting and modifying the configuration of
  the ORB (and of applications running on top it)

25
dynamicTAO Architecture
26
Reifying the ORB Structure

• ComponentConfigurator framework
• Stores inter-component dependencies
• Allows browsing, inspection, and reconfiguration
• Can be customized through inheritance

27
dynamicTAO Structure
28
DynamicConfiguratorIDL Interface

• interface DynamicConfigurator
• stringList list_categories ()
• stringList list_implementations (in string
  categoryName)
• stringList list_loaded_implementations ()
• stringList list_hooks (in string
  componentName)
• string get_hooked_comp (in string
  componentName,
• in string
  hookName)
• string get_comp_info (in string
  componentName)
• .
• .
• .

29
Manage Component Implementations loaded in memory

• long load_implementation (in string
  categoryName,
• in string impName,
• in string params,
  ...)
• void hook_implementation (in string
  loadedImpName,
• in string
  componentName,
• in string hookName)
• void suspend_implementation (in string
  loadedImpName)
• void resume_implementation (in string
  loadedImpName)
• void remove_implementation (in string
  loadedImpName)
• void configure_implementation (in string
  loadedImpName,
• in string
  message)

30
Manage the ORB Persistent Component Repository

• void upload_implementation (in string
  categoryName,
• in string impName,
  
• in implCode
  binCode)
• void download_implementation (in string
  categoryName,
• inout string
  impName,
• out implCode
  binCode)
• void delete_implementation (in string
  categoryName,
• in string
  impName)

31
Example of Dynamic Configuration

• 1. myRemoteOrb-gtupload_implementation
  (Security, superSAFE,
• 
  superSAFE_impl)
• 2. newSecurityStrategy
• myRemoteOrb-gtload_implementation
  (Security, superSAFE)
• 3. oldSecurityStrategy
• myRemoteOrb-gtget_hooked_comp
  (dynamicTAO, Security_Strategy)
• 4. myRemoteOrb-gthook_implementation
  (newSecurityStrategy,
• 
  dynamicTAO, Security_Strategy)
• 5. myRemoteOrb-gtremove_implementation
  (oldSecurityStrategy)

32
Consistency

• Dynamic reconfiguration may break the consistency
  of the internal ORB engine.
• Consistency must be ensured by the ORB developer
  and by the component developer.
• Achieved by creating customized subclasses of the
  ComponentConfigurator class
• TAOConfigurator
• Servant1Configurator
• MonitoringStrategyConfigurator
• ...

33
Implementing Reconfigurable ORB Components

• Two major things to consider
• 1. Transferring the state from the old component
  to the new component
• 2. Making sure that no dangling references to the
  old component remain
• Must customize TAOConfigurator or strategy
  configurator (e.g. ThreadPoolConfigurator)

34
Accessing the ORB Reconfiguration Interface

• 1. Local or remote code through IDL
• 2. Telnet
• 3. Java GUI
• 4. Reconfiguration Agents

35
DOCTORDynamic ORB Configuration Tool
36
Mobile Agents

• A mobile agent visits a collection of ORBs.
• In each ORB along its path, it can
• install new components on the disk,
• dynamically link new components,
• inspect the state and configuration of the ORB
  and the applications on top of it,
• reconfigure ORBs and applications.

37
A Flexible Framework

• Different NetworkBrokers support different agent
  flavors. For example
• simple, lightweight, script-based agents
  (carrying data and DCP commands only).
• powerful, heavyweight, Java-based agents
  (carrying data, bytecode, and dynamic state,
  taking autonomous decisions).
• Simple agents are suitable for PDAs, embedded
  systems.

38
Reconfiguration with Mobile Agents

• SysAdmins use a GUI to build agents for
• reconfiguration
• inspection
• GUI is used to
• 1. Build distribution graph
• 2. Select reconfiguration and inspection commands
• 3. Visualize results.

39
Security

• SecureAgentBroker uses the GSS-API and supports
  Role-Based Access Control.
• Agents are signed and transmitted via secure
  connections, using encryption.
• RBAC is used in each ORB to decide which commands
  each agent is allowed to perform.

40
The SecureAgentBroker
41
Open Problems

• Support for fault-tolerance
• fault-recovery when part of the reconfiguration
  process fails within a node
• fault-recovery when the reconfiguration fails in
  part of the distributed system
• atomic transactions
• Deploying agents for (re)configuration of active
  spaces in ubiquitous computing.

42
Applications of Reflective ORBs

• Completed Prototypes
• Flexible Object Monitoring Service
• Dynamic Security Service
• Multimedia applications (Nahrstedt, U. Illinois)
• Ongoing work
• Ubiquitous Computing (Illinois)
• Framework for Adaptive Applications (U. São
  Paulo)

43
Monitoring Distributed Object Interactions

• dynamicTAO shows how to adapt
• Applications also need to know when to adapt
• Monitoring Service
• Can be dynamically loaded and unloaded
• No modifications in the applications
• Totally transparent to applications
• Uses the CORBA request-level interceptor OMG98a

44
Monitoring Service Architecture
45
Monitoring Service Overhead
Client
Server
Fast Ethernet
Ultra-60
Ultra-2

• String getHello ()
• Overhead
• when monitoring getHello 10.1
• with Monitoring Service on, but without
  monitoring getHello 2.0

46
Dynamic Security Service Prototype

• Can be dynamically loaded and unloaded
• Uses
• CORBA interceptors for access control
• Cherubim Security Framework Campbell Qian 98
• Java Active Capabilities flexible dynamic
  policies
• implemented DAC, MAC
• working on RBAC, ABAC (?)

47
Open Problems

• Improving Security Services
• how to provide security for millions of
  distributed objects efficiently?
• Monitoring Service tools
• Specify what should be monitored
• Visualize monitored data graphically

48
The Future

• As computing devices become pervasive in our
  society, we will encounter
• highly dynamic, heterogeneous environments
• complex dependencies
• difficult management
• We need standards and an integrated architecture
  to help manage this complexity in a clean and
  efficient way.

49
End of Part 1

• Questions?

50
Security Architecture

• Java Active Capabilities
• Flexible Security Policies
• Caching of Authorization Decisions
• Auditing

51
Example of ConsistentDynamic Reconfiguration

• Concurrency strategies
• 1. Reactive (single-threaded)
• 2. Thread-per-Connection
• 3. Thread-Pool
• Switching from 1 or 2 to any other OK
• Switching from Thread-Pool problematic