Heterogeneous MultiCluster Networking with the Madeleine III Communication Library

1
Heterogeneous Multi-Cluster Networkingwith
theMadeleine III Communication Library

• Olivier Aumage
• LIP, ENS Lyon

2
Madeleine

• Generic communication interface
• Cluster level communication
• Multi-protocol support
• Main characteristics
• Efficiency
• Portability

3
Current Evolution

• Towards larger structures
• Large scale
• Metacomputing environments
• Loosely coupled environments
• Lab scale
• Multi-cluster computing

4
Multi-cluster Networking

• Communication subsystem requirements
• Efficiency
• Fast cluster level communication
• Fast inter-cluster communication
• Flexibility
• Wide range of configuration shapes
• Partially overlapping clusters
• Indirectly connected clusters
• Heterogeneity
• Architecture
• Network
• Application feedback
• Topology information

5
Extending Madeleine

• Adding multi-cluster support to Madeleine
• Internals modifications and improvements
• Configuration virtualization
• Gateway support
• Application interface
• Topology information system
• Session management
• Configuration description
• Application processes deployment

6
Internals Improvements

• Existing
• Madeleines channel concept
• One network
• One set of processes
• One set of connections
• The channel is the network virtualization

7
Internals Improvements

• Configuration virtualization
• Extension of the channel concept
• Virtual channels
• Built on top of regular channels
• Multi-network virtualization
• Gateway support
• Need for efficient forwardingsupport

8
Internals Improvements

• Gateway support
• Automatic retransmission mechanism
• First prototype presented at HCW01
• New redesigned version
• Dead-lock free
• Gateway number unlimited
• Characteristics
• Generic approach
• Packetized message transmission
• Heterogeneity safe
• Pipelined reception/retransmission
• Message ordering on last gateway
• No extra copy on endpoints

9
Gateway Support Integration

• Madeleines modular internal architecture
• Buffer Management Modules (BMM)
• Transmission Modules (TM)

10
Gateway Support Integration

• Generic forwarding transmission module
• Limitation of code traversal on gateways

11
Functioning

• Automatic multi-network forwarding support
• Multi-threaded handling
• Static routes
• Use of a MTU

12
Application Interface

• The configuration directory
• A flat view of the configuration
• Virtual channels
• Compatibility with existing code
• Quick application development
• An annotated view of the configuration
• Information in the configuration directory
• Application optimization

13
Application Interface

• The configuration directory
• Information about
• Session processes
• Physical nodes
• Drivers availability
• Channels and network mappings
• Virtual channels
• Routing tables
• Direct/indirect virtual connections
• Used by Madeleine during initialization
• Available to the application during the computing
  session

14
Session Management

• A new external tool Léonie
• Sessions
• Configuration files parsing
• Multi-cluster configurations
• Unified process spawning
• Grouped process spawning
• Support for network-specific launchers (e.g.
  bipload)
• Support for optimized process launchers
• Network
• Internal tables building
• Information directory
• Virtual channel routing tables
• Scheduling
• NIC initialization, channel setup

15
Configuration Description
Application
Networks
networks (
application
name myrinet_bip dev bip mandatory_loader
bip_load
networks
hosts (myri1, myri2, sci4)
channels (
,
name c1 net myrinet_bip hosts (myri1,
myri2, sci4)
name sci_sisci dev sisci
,
hosts (sci1, sci2, sci3, sci4)
name c2 net sci_sisci hosts(sci1,
sci2, sci3, sci4)
)
)
vchannels
name virtual channels (c1, c2)



16
Session Deployment
Léonie
17
Session Deployment
Madeleine
Léonie
18
Session Deployment
Madeleine
Léonie
19
Session Deployment
Madeleine
Léonie
20
Session Deployment
Madeleine
Léonie
21
Evaluation

• Testing configuration
• Dual Pentium II 450 MHz processors
• 33 MHz, 32bit PCI bus
• Linux kernel 2.2.13
• 128MB
• One SISCI/SCI network
• D310 NICs
• One BIP/Myrinet network
• LANai 4.3
• 32bit bus
• 1 MB SRAM
• One Ethernet network

22
Multi-cluster
23
Conclusion

• Madeleine III a new multi-cluster communication
  library
• Virtual channels
• Transparent application access to multi-network
  communication
• Immediate adaptation of mono-cluster applications
• Fast gateway support
• Configuration directory
• Source of optimization opportunities for
  applications
• Progressive application improvements
• Léonie session manager
• Flexibility

24
Ongoing/future work

• Providing high bandwidth multi-cluster support to
  high level libraries
• MPICH
• Improving MPICH/Madeleine adaptation
• Preliminary multi-cluster MPICH/Madeleine
  available
• Application feedback about topology
• New communicator construction primitives ?
• PNNL Global Arrays library for distributed
  multi-level arrays
• Importing Madeleine multi-cluster facilities to
  GA
• High level application-oriented benchmarking

25
(No Transcript)