Multi-Grid and Multi-VO Job Submission based on a Unified Computational Model Krakow Grid Workshop, November 22nd 2005 Trinity College Dublin

1
Multi-Grid and Multi-VO Job Submission based on a
Unified Computational ModelKrakow Grid Workshop,
November 22nd 2005Trinity College Dublin

• Mr. Oliver Lyttleton
• Mr. David O'Callaghan
• Mr. Gabriele Pierantoni
• Dr. Geoff Quigley

2
Introduction

• What is WebComG?
• Problem Statement
• A Key Concept Internal and border regions
• General Architecture
• Implementation and Prototypes
• Constrained Submission
• Full Interoperability
• Future Work

3
WebComG

• Project comprising software to support
  development and execution of Condensed Graphs
  in a Grid environment
• Condensed Graph graph representation of program
  structure
• Instances of WebComG running on different
  machines can communicate with each other
• It provides load-balancing, fault-tolerance,
  resource management, and security

4
Problem Statement

• Our use of heterogeneous grid middlewares within
  a single execution environment relies upon
• A Computational Model capable of expressing the
  workflow of operations which may contain
  interactions with heterogeneous grid middlewares
• Bridge infrastructures able to communicate with
  different grid middlewares
• The Condensed Graph computational model is ideal
  for expressing such complex workflows and its
  implementation (WebComG) allows their execution.
• Web Services, such as those offered by GT4, are
  ideal for the implementation of the Bridge
  Infrastructures.

5
A Key Concept Internal and Border Regions

• To represent the interoperation between WebComG
  and Grid Middleware applications a concept of
  internal and border regions is used
• An internal region is a set of machines that run
  either WebComG or a particular Grid Middleware
  (LCG2 or GT4)
• A border region is a set of machines which must
  host Border Services. They may host WebComG
  technologies and Grid Middleware technologies in
  addition

6
The Border Services

• The Submission Service Running on a GT4
  Container
• The File Closet Service Running on a GT4
  Container
• The Security Service Running on a GT4 Container
• The Logging Service Running on a GT4 Container
• The Client Submission Service(s) Use the above
  Services and submit jobs to the Grid Middleware
  (LCG2 and GT4)

7
Architectural Overview Collapsed Borders for
LCG2(More complex Subsets)
Only two technologies overlap in the simple
border subset
WebCom Region
Border Region
LCG2 Region
All three technologies overlap in the collapsed
border subset
8
Architectural Overview Expanded Borders(Simple
Subsets)
Submission Service in border set
Border Region
WebCom Region
LCG2 Region
GT4 Region
At most two technologies overlap
Grid Middleware Client
Grid Middleware Resource Broker/ GRAM Server
9
Border Services Choreography
GMC Grid Middleware Client Machine SSM
Submission Service Machine GM Grid Machine
Condensed Graph

• CG Submission invokes
• File Staging
• Security
• Logging

• Grid Job Submission invokes
• File Staging
• Security
• Logging

SSM


WCM
GM
GM
GMC
WCM
10
Border Services Choreography
GMC Grid Middleware Client Machine SSM
Submission Service Machine GM Grid Middleware
Condensed Graph
SSM

Middleware Submission Node

WCM
GM
GM
GMC
WCM
11
Border Services Choreography
GMC Grid Middleware Client Machine SSM
Submission Service Machine GM Grid Middleware
Condensed Graph
SSM


Job Description
WCM
GM
GM
GMC
WCM
12
Border Services Choreography
GMC Grid Middleware Client Machine SSM
Submission Service Machine GM Grid Middleware
Condensed Graph
SSM


Job Input Files
WCM
GM
GM
GMC
WCM
13
Border Services Choreography
GMC Grid Middleware Client Machine SSM
Submission Service Machine GM Grid Middleware
Condensed Graph
SSM


User Credential
WCM
GM
GM
GMC
WCM
14
Border Services Choreography
GMC Grid Middleware Client Machine SSM
Submission Service Machine GM Grid Middleware
Condensed Graph
SSM


WCM
GM
Grid Submission
GMC
GM
WCM
15
Border Services Choreography
GMC Grid Middleware Client Machine SSM
Submission Service Machine GM Grid Middleware
Condensed Graph
SSM


WCM
GM
Grid Output
GMC
GM
WCM
16
Border Services Choreography
GMC Grid Middleware Client Machine SSM
Submission Service Machine GM Grid Middleware
Condensed Graph
SSM


Grid Output
WCM
GM
GMC
GM
WCM
17
Future Work

• Complete implementation of full interoperability
• Develop interoperability with other Grid
  Middleware
• Use Grid Filesystem instead of File Closet Service