OGSITestbed Project

1
Grid Enabling Legacy Applications
Grid Execution Management for Legacy Code
Applications
Tamas Kiss Centre for Parallel ComputingUniversit
y of WestminsterLondon, UK
2
Legacy Applications

• Code from the past, maintained because it works
• Often supports business critical functions
• Not Grid enabled

What to do with legacy codes when utilising the
Grid?

• Bin them and implement Grid enabled applications

• Reengineer them

• Port them onto the Grid with minimum user effort

3
GEMLCA Grid Execution Management for Legacy
Code Architecture
Objectives

• To deploy legacy code applications as Grid
  services without reengineering the original code
  and minimal user effort
• To create complex Grid workflows where components
  are legacy code applications
• To make these functions available from a Grid
  Portal

GEMLCA
GEMLCA PGPortal Integration
4
GEMLCA Concept
5
The GEMLCA-view of a legacy code

• Any code that correspond to the following model
  can be exposed as Grid service by GEMLCA

Legacy code
Input data (files, command line params, env.
vars)
Output data (files)
. . .
Perform computation
Call shared libraries
Communicate over the network
Query databases
6
GEMLCA Concept
Code is deployed statically
Legacycode
Input data is provided dynamically
Access to the code is controlled by the GEMLCA
Grid service
7
Implementing the concept

• The GEMLCA service can be implemented with any
  grid/service-oriented technology E.g
• Globus 3 or 4 ? currently available
  implementations
• Jini
• Web services
• GEMLCA service could invoke legacy codes in many
  different ways. Current implementation
• Submit the legacy code as a batch job to a local
  job manager (e.g. Condor or PBS) through a Grid
  middleware layer (GT3/4)

8
Whats behind the GEMLCA service

• Consequences
• Not only the legacy code and the GEMLCA service,
  but also a local jobmanager and a Grid middleware
  layer must be installed on the hosting system!
• The aim of the code registration process is to
  tell GEMLCA how to submit the legacy code to the
  grid middleware layer

Compute
GEMLCA Service
Gridmiddlewarelayer
Client
Servers
Grid Service
Client
GEMLCA
Job submission and file transfer services
Process
Legacy
Local jobmanager
Code Job
e.g. Condor/Fork

9
Whats the point?

• Heterogeneous codes can be hidden behind the same
  interface (the programming interface of the
  GEMLCA service)
• Different programs can be invoked in the same way
• Extend non grid-aware programs with security
  infrastructure (access enabled through a Grid
  service)
• Share your codes with your colleagues or partner
  institutes
• Expose business logic to your employees or
  customers
• Build customized GEMLCA clients (such as the
  GEMLCA P-GRADE Portal)
• Compose complex processes by connecting multiple
  legacy code grid services together

10
The GEMLCA P-GRADE Portal

• A Web-based GEMLCA client environment

University of Westminster, LondonMTA SZTAKI,
Budapest
11
The aim of the GEMLCA P-GRADE Portal

• To provide graphical clients to GEMLCA with a
  portal-based solution
• To enable the integration of legacy code grid
  services into workflows

12
The GEMLCA P-GRADE Portal
GEMLCAP-GRADE Portal Server
13
The GEMLCA-specific version of the P-GRADE Portal
is different from the original P-GRADE Portal!

• It contains a web page to register legacy codes
  as grid services
• It contains a GEMLCA-specific workflow editor
• Workflow components can be legacy code grid
  services (not only batch jobs)
• It contains a GEMLCA-specific workflow manager
  subsystem
• It can invoke GEMLCA services (not only
  submitting jobs)

14
Legacy code registration page
GEMLCA Administration Tool portlet
15
Legacy code registration page
16
GEMLCA Specific Workflow editor
UoW
Sztaki
UoR
17
GEMLCA workflow editor in a nutshell
Workflow Creation
UoW
Sztaki
UoR
18
Batch components vs. GEMLCA components in P-GRADE
Portal workflows
Batch component
GEMLCA component

• User provides the binary executable
• User provides input data for the executable
• Job produces result files

• Executable is already installed on a machine
• User provides input data for the executable
• Job produces result files

Workflow components must be defined in different
ways
Input files represented by ports
Output files represented by ports
Ports guarantee compatibility ? batch and GEMLCA
components can mutually produce data to each
other!
19
Combining legacy and non-legacy components
Code invocation
Code invocation
Job submission
Code invocation
Job submission
20
Application example
Designing Optimal Periodic Nonuniform Sampling
Sequences
T Factor
Sequential
GEMLCA
18
19min
8min
20
3h 33min
35min
22
41h 53min
7h 23min
21
GEMLCA and Production Grids The problem
22
The solution
23
Centralised GEMLCA solution The GEMLCA
repository (GEMLCA-R)
P-GRADE Portal Server
24
GEMLCA-R
3rd party service provider (UoW)
GEMLCA resource (GT4 GEMLCA classes)
NGS site1 (GT2)
Workflow definition
NGS site2 (GT2)
P-Grade Portal
Central repository legacy code1 legacy
code2 . legacy coden
NGS siten (GT2)
user
job submission
25
Advantages of GEMLCA-R

• legacy codes can be uploaded into a central
  repository and made available for authorised
  users through a Grid portal
• extends the usability of production Grids as
  users utilise others legacy codes stored in the
  repository
• No support needed at the Grid sites

26
GEMLCA on the UK NGSThe P-GRADE NGS GEMLCA Portal

• Portal Website http//www.cpc.wmin.ac.uk/ngsporta
  l/
• Runs both GT4 GEMLCA and GEMLCA-R

27
GEMLCA on the WestFocus GridAlliance Grid

• GT4 testbed for industry and academia
• Connects two 32 machine clusters at Westminster
  and one at Brunel University
• Runs the P-GRADE Grid portal and GEMLCA
• Connected to and interoperable with the UK NGS

28
Connecting Grids with the P-GRADE Portal and
GEMLCA

• Extending current second generation production
  Grids with GT4 resources already in production
  on the UK NGS
• Connecting GT2 Grids (e.g. NGS) to GT4 Grids
  (e.g. OpenScience Grid, WestFocus Grid)
• Connecting GT2 Grids (e.g. NGS) to the
  LCG/gLite-based EGEE Grid

29
Extending the NGS with GT4 Resources
Manchester
Westminster
Oxford
GEMLCA Repository
30
Connecting the NGS to GT4 Grids
NGS / GT2
Manchester
Service Invocation
GEMLCA Repository
31
Connecting the NGS to EGEE
NGS
Manchester
GEMLCA Repository
32
Connecting the NGS to EGEE
GT2 Job submission to Poznan
GEMLCA legacy code submitted to Manchester
GT4 Service Invocation at UoW
GT2 Job submission to Oxford
33
Portal service for connecting all major
production Grids
34
Conclusions

• GEMLCA enables the deployment of legacy code
  applications as Grid services without any real
  user effort.
• GEMLCA is integrated with the P-GRADE portal to
  offer user-friendly development and execution
  environment.
• The integrated GEMLCA P-GRADE solution is
  available for the UK NGS as a service!
• www.cpc.wmin.ac.uk/ngsportal