Constructing and executing Grid workflow applications by Grid portal technology

1
Constructing and executing Grid workflow
applications by Grid portal technology

• G. Dózsa et al.
• Computer and Automation Research Institute
• Laboratory of Parallel and Distributed
  Systemswww.lpds.sztaki.hu/p-grade

2
Design Layers in P-GRADE
I. Workflow layer
Secure File Transfer (GridFTP)
3
Workflow Layer in P-GRADE
Secure File Transfer (GridFTP)
4
Demo application Ultra-short range weather
forecast (Hungarian Meteorology Service)
Forecasting dangerous weather situations (storms,
fog, etc.), crucial task in the protection of
life and property
25 x
Processed information surface level
measurements, high-altitude measurements, radar,
satellite, lightning, results of previous
computed models
10 x
25 x
5 x

• Requirements
• Execution time lt 10 min
• Higher resolution

5
Monitoring support for the workflow
Monitoring of Workflow
Input files
Secure data transfer
Job
Monitoring of Job
6
Workflow support by the P-GRADE portal
Job Monitoring
Workflow Monitoring
7
Workflow creation
Input files
Job
Secure data transfer
8
loggedIn-jobList
9
Visualisation of workflow execution
Monitoring of Workflow
10
traceApplets
11
P-GRADE workflow portal Live demo at SC2003
Cluster 2003
P-GRADE Workflow portal
12
Applied Grid infrastructure
BUTE cluster 24 processors
SZTAKI cluster58 processzors
UoW cluster32 processzors
P-GRADE GRM / PROVE
Condor DAGman
Condor-G
GLOBUS
GLOBUS
GLOBUS
Sun Grid Engine
Condor
Condor
MPI
MPI
MPI
Mercury
Mercury
Mercury
13
Possible usage of P-GRADE
P-GRADE
PVM
MPI
Workflow
Super- computers
Clusters
Grids
ClusterGrid(Condor)
SuperGrid(Globus-2 Condor-G)
OGSA
14
Integrated technologies
Condor
EU GridLab
GT-2
Condor-G
DagMAN
GridSphere
GridFTP
GSI, GRAM
Mercury
SGE
P-GRADE Workflow
PVM
PBS
MPI
15
Plan for the project P-GRADE portal for OGSA
Grids
P-GRADE portal
PVM
MPI
Workflow
Super- computers
Clusters
Grids
ClusterGrid(Condor)
SuperGrid(Globus-2 Condor-G)
OGSA
16
Plan for the project Integrated technologies
Condor
EU GridLab
Condor-G
DagMAN
GridSphere
Mercury
SGE
P-GRADE Workflow
PVM
PBS
MPI
17
P-GRADE workflow portal on the UK OGSI testbed
P-GRADE Workflow portal
18
Summary Advantages of P-GRADE portal

• Can be used from anywhere
• Fast creation of workflow
• Reusability of sequential, PVM, MPI, P-GRADE
  code
• Hiding of Grid communication functions
• Unified and integrated graphical concept
• Support for hierarchical design
• Even non-professional programmers can use it
• Grid execution mechanism for Globus/Condor/OGSA