COMP Superscalar: Bringing GRID superscalar and GCM together

1
COMP Superscalar Bringing GRID superscalar and
GCM together
Enric Tejedor Universitat Politècnica de
Catalunya etejedor_at_ac.upc.edu
Rosa M. Badia Barcelona Supercomputing
Center rosa.m.badia_at_bsc.es
V ProActive and GCM User Group, GridCOMP
conference October 21, 2008. INRIA, Sophia
Antipolis, France
2
Outline

• Introduction
• Grid Component Model
• GRID superscalar
• Programming model
• Runtime operation
• Initialisation
• Task processing
• Finalisation and error handling
• Conclusions

3
1. Introduction
4
Basic concepts

• The Grid Component Model (GCM)
• CoreGRID NoE, GridCOMP
• Distributed, based on Fractal, intended for the
  Grid
• GRID superscalar
• Framework to ease the development of Grid-unaware
  applications
• Simple programming model Grid as transparent as
  possible
• Runtime that optimises the performance of the
  application (exploiting possible concurrency)

5
A componentised Grid framework

• GRID superscalar is suitable to benefit from the
  GCM features
• Componentisation process identify the inner
  functionalities of the runtime, and assign each
  one to a separate component
• Result COMP Superscalar, whose runtime gains in
• Reusability
• Flexibility
• Deployability
• Separation of concerns
• Ease of development

6
Componentised runtime
7
2. Programming model
8
Programming model
Parallel Resources (multicore,SMP, cluster, grid)?
Synchronization, results transfer
Resource 1
Resource 2
Task selection parameters direction (input,
output, inout)?
Sequential Application
Resource 3
... for (i0 iltN i) T1 (data1, data2)
T2 (data4, data5) T3 (data2, data5, data6)
T4 (data7, data8) T5 (data6, data8,
data9) ...
. . .
Resource N
Scheduling, data transfer, task execution
Task graph creation based on data precedence
9
COMPSs programming model
initialize(f1) for (int i 0 i lt 2 i)
genRandom(f2) add(f1, f2) print(f2)
Java application
Java interface

• public interface SumItf
• _at_ClassName(example.Sum")
• _at_MethodConstraints(OSType "Linux")
• void genRandom(
• _at_ParamMetadata(type Type.FILE, direction
  Direction.OUT)
• String f
• )
• _at_ClassName(example.Sum")
• ...

Implementation
Task constraints
Parameter metadata
10
Usage example original code

• initialize(f1)
• for (int i 0 i lt 2 i)
• genRandom(f2)
• add(f1, f2)
• print(f1)

11
PM steps (1) selecting the tasks

• public interface SumItf
• _at_ClassName(example.Sum")
• _at_MethodConstraints(OSType "Linux")
• void genRandom(
• _at_ParamMetadata(type Type.FILE, direction
  Direction.OUT)
• String f
• )
• _at_ClassName(example.Sum")
• _at_MethodConstraints(processorArch "Intel",
  processorSpeed 1.8f)
• void add(
• _at_ParamMetadata(type Type.FILE, direction
  Direction.INOUT)
• String f1,
• _at_ParamMetadata(type Type.FILE, direction
  Direction.IN)
• String f2
• )

Implementation
Task constraints
Parameter metadata
12
PM steps (2a) using API calls

• initialize(f1)
• COMPSs css new COMPSsImpl()
• css.start()
• for (int i 0 i lt 2 i)
• genRandom(f2)
• add(f1, f2)
• String resF2 css.openFile(f2, OpenMode.READ)
• print(resF2)
• css.stop(true)

13
PM steps(2b) no changes!

• initialize(f1)
• for (int i 0 i lt 2 i)
• genRandom(f2)
• add(f1, f2)
• print(f1)

14
Custom Java Class Loader
15
3. Runtime operation
16
Implementation details

• Base technologies
• Java as programming language
• ProActive implementation of the GCM model, used
  to build the components
• JavaGAT used for job submission and file transfer

17
Initialization

• Multicast invocation
• Forwarded to all subcomponents

JM
TA
TS
init()
FM
FIP
FTM
18
Initialization

• Multicast invocation
• Reduction of return values

JM
TA
TS
Reduction
FM
Reduction
FIP
FTM
19
Task processing (1)

• Application submits task
• Task Analyser receives the request

TA
TS
JM
executeTask()
FM
FIP
FTM
20
Task processing (2)

• File Information Provider is contacted
• File accesses of the task are registered

JM
TA
TS
executeTask()
FM
FIP
FTM
21
Task processing (3)

• Task Analyser discovers dependencies
• Dependency-free tasks are sent for scheduling

TA
TS
JM
executeTask()
FM
FIP
FTM
22
Task processing (4)

• Task Scheduler decides where to execute tasks
• Job Manager is informed of this decision

Resource capabilities
Task constraints
TS
JM
TA
Scheduling algorithm
executeTask()
FM
FIP
FTM
23
Task processing (5)

• Job Manager checks the necessary file transfers
• File Transfer Manager actually performs them

TA
TS
JM
executeTask()
FM
GAT
FIP
FTM
24
Task processing (6)

• FTM informs of the end of transfers for a task
• Task Grid job, submitted for execution

GAT
TA
TS
JM
executeTask()
FM
FIP
FTM
25
Task processing (7)

• Callback for the end of a job is received
• The notification is forwarded to the Task Analyser

GAT
TA
TS
JM
executeTask()
FM
FIP
FTM
26
Opening a file

• Java Application
• String ren openFile(f)
• print(ren)

A P I
JM
TS
TA
FM
FIP
FTM
Local host
Remote host
f10v6.IT
f10v6.IT
27
Error handling

• The COMPSs runtime could experience errors of
  different kinds (e.g. a job submission failure)
• Default response to an error stopping the
  components
• But a structure of components cannot be stopped
  in any arbitrary order!

stopFc on B is never served!
stopFc()
LCC
A
C
B
28
Safe stopping process for the runtime

• COMPSs subcomponents have data dependencies
• Modification of the Life Cycle Controller
• Components stopped in a safe order TS, TA, JM,
  FTM, FIP
• Otherwise, a deadlock could happen

LCC
TA
TA
TS
JM
FM
FIP
FTM
29
4. Conclusions
30
Conclusions

• COMP Superscalar componentised version of GRID
  superscalar
• GCM-based
• Hierarchical runtime separation of concerns
• Task scheduling, file and resource management
• Straightforward programming model for
  Grid-unaware Java applications
• Feedback about GCM-ProActive
• Component distribution
• Performance no noticeable overhead for a medium
  number of tasks
• Component composition and dynamic
  reconfiguration Grid IDE

31
Thank you!Questions?