Grid Technologies Intro

1
Grid Technologies Intro
Introduction to Grids and NorduGrid
Ivan Degtyarenko ivan.degtyarenko dog csc dot
fi CSC The Finnish IT Center for Science June
3rd, 2008
2
Contents

• Part 1 Introduction to Grids and NorduGrid
• What is the Grid?
• Role of Middleware
• Accessing the Grid
• Grid reality
• NorduGrid collaboration / M-grid as an example
• Part 2 NorduGrid middleware ARC
• Submitting jobs in NorduGrid, writing job
  description files
• Simple file transfers
• Monitoring jobs using the Grid Monitor graphical
  interface

3
Hierarchy of the IT Resources
international HPC centers
grid infrastructures
regional computer centers
in-house resources
desktops
4
What is the Grid?

• no clear definition Compute Grids, Science
  Grids, Data Grids, Cluster Grids etc.
• must be evaluated in terms of the applications,
  and scientific results that it delivers, not its
  architecture
• the key concept is the ability to negotiate
  resource-sharing arrangements among a set of
  participating parties and then to use the
  resulting resource pool
• a basic definition for grid might look like   "a
  hardware and software infrastructure that
  provides dependable, consistent, pervasive and
  inexpensive access to high-end computational
  capabilities"

5
What is the Grid? (cont.)

• Ian Foster, What is the Grid? A three point
  checklist, Grid Today, 1(6), 2002. A Grid is a
  system that
• coordinates resources that are not subject to
  centralized control a Grid integrates and
  coordinates resources and users that live within
  different control domains
• using standard, open, general-purpose protocols
  and interfaces a Grid is built from
  multi-purpose protocols and interfaces that
  address such fundamental issues as
  authentication, authorization, resource
  discovery, and resource access
• deliver nontrivial qualities of service the
  utility of the combined system is significantly
  greater than that of the sum of its parts
  (response time, throughput, availability,
  security etc.)

6
Role of Grid Middleware
user applications
grid middleware
Grid resource
Grid resource
Grid resource
Grid resource
7
Grid fundamental services implemented in the
middleware

• resource discovery and monitoring
• job submission and management
• brokering and data services
• information services
• resource management

8
Computational Grids resource sharing principle

• Goal is to minimize waste of resources empty
  nodes are always available for use (dynamic
  sharing)

9
Accessing the Grid Authentication
user identification is called authorization

• one needs to be authenticated
• every grid user holds a X.509 certificate issued
  by a Certificate Authority
• being authenticated by a site means that the
  Issuer CA of your credential is trusted by the
  Grid resource
• certificates to be updated annually
• authentication and authorization are decoupled
  processes

10
Accessing the Grid Authorization
granting resource access is called authorization

• users are grouped into Virtual Organizations (VO)
• based on a common research area, country, funding
  agency, project, etc.
• the same user can belong to several VOs
• resource providers grant access to VOs
• effective managing comparing to individual users
• implies trust towards the organization managing
  the VO
• for instance e.g. in the M-grid the users of
  each site form one VO, and we could combine all
  to a larger "M-grid VO" when negotiating with
  external parties

11
Grid Security
WARNING! When working on the Grid, you must
accept that some information on your jobs and on
your Grid identity is made public, i.e your
name / affiliation IP address of your
client computer job names and duration
runtime environment others
12
Grid Reality

• a convenient way of sharing resources
• free computational power available to those
  willing to put in the effort to make use of it
• production grids in place and used
• Nordugrid, EGEE, DEISA
• different middleware, not normally interoperable
• bureaucracy involved
• the grid won't magically bind your software
  together

13
NorduGrid Collaboration

• past Grid pilot project by Nordunet in 2001
• implemented a production Grid system working non
  stop since May 2002
• present a community around open source Grid
  middleware NorduGrid ARC
• national Grids (e.g. M-grid, SweGrid, NorGrid),
  users also outside the Nordic countries
• real users, real applications
• funding for further development (KnowARC, NDGF)
• open for anyone to participate

14
The Finnish Material Sciences Grid (M-grid)

• goal throughput computing capacity mainly for
  the needs of physics and chemistry researchers
• opened to all CSC customers in Nov 2005, not
  limited by the field of research
• joint project between seven Finnish universities,
  Helsinki Institute of Physics and CSC
• partners mainly laboratories and departments, not
  university IT centers
• first large initiative to put Grid middleware
  into production use in Finland
• uses NorduGrid ARC middleware
• platform Linux based PC clusters