Grid Interoperability Shootout GridPP and NGS

1
Grid Interoperability ShootoutGridPP and NGS

• UK e-Science All Hands Meeting, Nottingham 2007
• J Jensen, G Stewart, M Viljoen, D Wallom, S Young
• (contact j.dot.jensen.at.rl.dot.ac.dot.uk)

2
Two National Grids

• GridPP UK Grid for particle physics
• And others
• NGS National Grid Service
• Everybody
• Different audience, middleware, aims
• Interoperability is beneficial (isnt it?)

3
Differences

• Physics
• Built for physics UCs
• SL4 (SLC4)
• gLite
• SRM
• ssh

• Non-physics
• Built for non-p UCs
• Various OS
• Globus, Condor,
• SRB
• GSISSH

4
Differences

• Command line
• Supported by specialised groups
• V. big data volumes
• No need for secrecy
• Own or free sw.

• Portals cmd line
• Supported by a single (loose) group
• Not so big data vol.
• Some confidential
• Licensed software

5
Differences dteam vs. ETF

• deployment team
• Deployment ?
• Ops and experts
• Meeting user reps.
• (large expts or UB rep).

• Engineering Task Force
• Evaluations
• Ops and support
• No users?

6
The UI

• Site UIs
• User installable UI
• OS dependent

• Portals
• Specialised portals
• Workflow
• Or use GSISSH
• OS independent

7
Common stuff

• The CA
• MyProxy
• VOMS (sort of)
• BDII GLUE schema

• The big VO, small VO problem

8
Authentication

• Both using e-Science CA
• NGS has made further progress with login
• Training CAs (low assurance certificates)
• Celebrated GSISSH Terminal
• Portable, using certificates to log in
• SSO integration, via site MyProxy
• Shibboleth (prototype portals)

9
RB and LB

• NGS recently started using RB
• Submit to at least core sites
• Depends on information system
• Also tested GridWay

10
RB
gLite RB
You probably cant read this (how to lie with
statistics)
11
Computing Element

• GridPP LCG CE ? gLite CE
• gLite CE
• Submits jobs to Torque, LSF, SGE,
• BLAHP support (parameter passing to batch)

12
MPI

• NGS providing MPI
• Needed, jobs are less embarrassingly parallel
• Took some effort
• NGS benefited from (and worked with) GridIreland
• Select MPI jobs via globus-job-submit
• Still need support in gLite RB

13
Monitoring
INCA, GITS
gstat, SAM MonAMI
NGS
GridPP
TG
LCG
14
Storage Elements

• GridPP using DPM, dCache, CASTOR
• DPM for smaller sites (up to 200 TB?)
• dCache for larger or more complex infra.
• DPM has been evaluated by ETF
• Recommended (but like many parts of gLite not
  very portable (VOMS dep.))

15
Storage SRM and SRB
globus-url-copy
gridftp (Argonne)
SE
SRB
16
Storage SRM and SRB
gLite FTS
gridftp
SRM
SE
SRB
17
Storage SRM and SRB
Pretend SRB is a Classic SE (GridFTP server
with information interface) Still supported via
FTS Transfer files instead of developing interfac
es! Interoperability with no addl effort !!
gLite FTS
gridftp
SRM
SE
SRB
18
Storage SRM and SRB?
lcg-
info
SRM
SE
SRB
19
Storage SRM and SRB?
Interfacing once again brought up
This time an ASGC (Taiwan) project Aiming for a
demo at SC 2007
SRM
SRM2.2
SRM
SRB
20
The big(ger) picture
EGI?
Other National Grids
NGS
GridPP
Astro Grid
Grid Ireland
OMII
TG
OSG
LCG
21
The big(ger) picture
EGI?
Other National Grids
NGS
GridPP
Astro Grid
Grid Ireland
OMII
TG
OSG
LCG
22
The big(ger) picture
EGI?
Other National Grids
NGS
GridPP
Astro Grid
Grid Ireland
OMII
TG
OSG
LCG
23
Another big(ger) picture
gin-data
gin-jobs
OGF
gin-auth
gin-info
24
Opportunities
25

• Need for capability computing
• GridPP has high priority for ops
• Data catalogues, file transfers
• Documentation

26

• Portal work
• GSISSH
• Research of the month
• ETF (or ETFs experience)
• Documentation, Training
• User group meeting

27

• How to make grids interoperate
• Standards
• Strong Expertise in UK(I)
• Influence external projects (priorities)

28

• Externalities
• EGEE
• OMII UK/Europe
• EGI and suchlike

• Standards
• OGF
• W3C
• OASIS
• Specialised industry or academic stds (e.g.
  archives)

29
Boundaries
National
Disciplinary
Technology
30
Boundaries
Broken by GridPP (LCG) and NGS (TeraGrid)
National
TRUST
Disciplinary
Technology
31
Boundaries
National
Broken within Grids (to some extent)
Disciplinary
Technology
32
Boundaries
National
Focus for interoperability
Disciplinary
Technology
33
Recommendations
Job subm.
Job subm.
Mgmt
Mgmt
Storage
Storage
Establish links when it makes sense stronger
in some areas than others
34
Recommendations
RB
RB
Job subm.
Job subm.
CE
monitoring LB
monitoring LB
Catalogues
data transfer
Infosys
Infosys
Storage
Storage
Note differences in software stack link when it
makes sense
35
Conclusions

• Opportunities for collaboration
• Build on common areas
• Many individual areas
• Strong expertise in UK in many fields
• Sometimes there are differences for a reason
  permit site and grid differences