A Portable and Adaptive Multiprotocol Communication Library for Multithreaded Runtime Systems

1
A Portable and Adaptive Multi-protocol
Communication Library for Multithreaded Runtime
Systems

• Olivier Aumage, Luc Bougé, Raymond Namyst
• LIP, ENS-Lyon

2
Objectives

• PM2 Multithreading Library RPC
• Efficiency
• Zero-copy transmission
• Multi-paradigm protocols
• Multiple protocols
• Multiple adapters

Node 1
Network
Node 2
3
Related works

• Standards TCP UDP
• Low level BIP SBP
• High level MPI PVM
• Intermediate level
• Nexus
• Fast messages
• VIA
• Madeleine

4
Existing - Madeleine I

• Portable
• Efficient
• Designed for RPC
• Poor support for multi-protocol

5
Madeleine I
Header
Ack
Allocation
Body
Network
Node 1
Node 2
6
Contribution - Madeleine II

• Removals
• Header/body distinction
• Arbitrary limitations
• Adds
• Semantical approach
• Optimization level control
• Immediate data transfer

7
Interface
Mad_begin_packing
Mad_pack
Mad_end_packing
Mad_begin_unpacking
Mad_unpack
Mad_end_unpacking
8
Packing - Unpacking

• Commands
• Send Mad_pack(cnx, buffer, len, pack_mode,
  unpack_mode)
• Receive Mad_unpack(cnx, buffer, len, pack_mode,
  unpack_mode)

• Pack modes
• send_SAFER
• send_LATER
• send_CHEAPER

• Unpack modes
• receive_EXPRESS
• receive_CHEAPER

9
SAFER vs CHEAPER
Pack
Modification
?
End_packing
Transmitted version
10
Receive Modes
Unpack
After Unpack
Data available
Data available ???
End_packing
Data available
receive_CHEAPER
receive_EXPRESS
11
Buffer Management Layer

• Protocol independent code
• Buffer management modules
• Virtual buffers
• Static
• Dynamic
• Groups
• Buffer aggregation

12
Protocol Management Layer

• Protocol dependant code
• Transfer modules
• Send receive operations
• Aggregated data transfers
• Transfer method selection
• Choice function

13
Transfer Method Selection
Interface
Pack
Buffermanagement
?
Protocolmanagement
14
Results

• Testbed
• Cluster of Pentium II 450MHz/128MB
• Performance measurements over
• SISCI/SCI
• M-VIA/Ethernet 100

15
SCI
16
SCI
17
VIA
18
VIA
19
Conclusion

• Madeleine II available on top of
• SCI, TCP, VIA, MPI, SBP and now BIP
• Multi-level approach
• Advanced management techniques
• Buffers
• Protocols
• Network resources
• Efficiency

20
(Ongoing) Work

• Multi-cluster support
• Collective communication
• Routing
• Dynamicity
• Fault tolerance

21
(No Transcript)
22
Example
Sending side
Receiving side
23
Example
24
Send_LATER
Pack
Modification
End_packing
Transmitted version
25
Nexus

• Communication in Globus
• Point-to-point oriented
• Multi-protocol support
• Direct use is not easy
• Lack of flexibility low performance

26
MPI

• Standard
• Efficient
• Portable
• Threads support ?
• Incremental messages building ?

27
Proposal protocols
Interface
Unpack
Pack
Buffermanagement
Receive
Send
Protocolmanagement
28
Internal Architecture

• BMM buffer management module
• TM transfer module

Interface
BMM
BMM
Buffermanagement
TM
TM
TM
Protocolmanagement
29
Transmission Module

• Protocol dependent
• One for each data transfer method
• VIA driver 3 TM
• SCI driver 3 TM
• Associated to a buffer management module

30
Buffer Management Module

• Buffer kind
• Static/Dynamic
• Aggregate mode
• Without
• Sequential aggregation
• Half-sequential aggregation

31
Aggregates

• Sequential

Flush
Flush
TM1
TM1
TM2
32
Aggregates

• Semi-sequential

Flush
Flush
Common public buffer list
TM 1 private buffer list
TM 2 private buffer list
33
Special cases

• Send_LATER / Receive_CHEAPER
• Automatic aggregate construction

End_packing
Main
Pack
Pack
TM 1
TM 2
34
Special cases

• Send_LATER / Receive_EXPRESS
• Automatic aggregate construction
• Delayed data transfer
• Expensive

Pack
Unpack
Delayed send
Sending side
Receiving side
Expected data
35
Aggregates Shape

• Asymetric

Flush
Flush
Flush
Sending side
Receiving side

• Symetric

Flush
Flush
Flush
Flush
36
TCP
37
TCP
38
VIA
39
VIA
40
VIA
41
Plan

• Introduction
• Objectives
• Related work
• Proposal
• Results
• Conclusion
• Future

42
Proposal Communication

• Channel concept
• Similar to MPI communicators
• A channel is
• One protocol
• One network interface card
• One set of nodes
• One set of point-to-point connections

43
Channels
Node 4
Node 3
SCI
TCP
Node 1
TCP
Node 2