Single Point of Access to Resources of HPC-Europa - PowerPoint PPT Presentation

Loading...

PPT – Single Point of Access to Resources of HPC-Europa PowerPoint presentation | free to download - id: 1a40ea-ZDc1Z



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Single Point of Access to Resources of HPC-Europa

Description:

Budget: ~20 mln EURO. 14 Partners across Europe. Project activities ... PBS Pro. UNICORE. HLRS. LoadLeveler. OpenPBS with MAUI. UNICORE. CINECA. LRMS ... – PowerPoint PPT presentation

Number of Views:51
Avg rating:3.0/5.0
Slides: 18
Provided by: silviamo
Learn more at: http://www.cyf-kr.edu.pl
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Single Point of Access to Resources of HPC-Europa


1
Single Point of Access to Resources of HPC-Europa
  • Krzysztof Kurowski, Jarek Nabrzyski, Ariel
    Oleksiak, Dawid Szejnfeld
  • Poznan Supercomputing and Networking Center
  • Cracow Grid Workshop, 14 December 2004

2
Outline
  • HPC-Europa
  • JRA2 Single Point of Access
  • Assumptions Requirements
  • Architecture
  • Components of the SPA
  • Job Submission Interface
  • Conclusion

3
HPC-Europa
  • HPC-Europa - Pan-European Research infrastructure
    on High Performance Computing for the Science of
    21st century
  • Goal to provide advanced computational services
    in an integrated way to the European research
    community
  • Budget 20 mln EURO
  • 14 Partners across Europe
  • Project activities
  • Transnational Access Programme
  • Networking Activities
  • Joint Research Activities (JRA1, JRA2)

4
JRA2 Single Point of Access
  • Motivation
  • To provide a uniform access to all centers
    resources, transparently and regardless of user
    physical location
  • Main objectives
  • ease of use,
  • improve availability,
  • global optimization of resources utilization
  • To achieve these goals
  • JRA2 builds a HPC-Europa portal providing
    uniform, flexible and intuitive user access to
    Grid resources from anywhere
  • Develops and/or adapts tools and services needed
    to establish full operational Grid environment
  • Will develop a meta-broker responsible for users
    access to all consortiums resources

5
Assumptions Requirements
  • User-oriented
  • Uniform, transparent access
  • Easy to use, intuitive interface
  • Increased resource availability and access to all
    centers
  • Provider-oriented
  • Keep centers autonomy
  • To enable it to use its own chosen middleware
  • To allow local policies
  • Not to weaken centers security
  • Optimize global resource usage

6
Accounting Charging Model
  • Allocation Unit (AU)
  • 1TFlop sustained for 1 hour
  • Kept for each resource provider for every single
    HPC system
  • Used for 2 purposes
  • charging the EC for the use of HPC facilities
  • limiting the maximum amount of resources a user
    (or group of users) may utilize
  • The common pool of AUs concept
  • If a center contributes n AUs to the global
    infrastructure, then its users get access to n
    AUs from a common pool

Center C
nC AUs
nC AUs
Common pool of AUs
nA AUs
nB AUs
nA AUs
nB AUs
Center B
Center A
7
Typical Grid Architecture
VO
Grid Portal / Grid Resource Broker
HPC center
HPC center
HPC center
HPC center
8
HPC-Europa SPA
VO
Grid Portal / Grid Resource Broker
HPC center
HPC center
HPC center
Grid Middleware A
HPC center
Grid Middleware C
Grid Middleware B
Grid Middleware D
9
HPC Centers Middleware
Center High-level middleware Underlying Grid technology LRMS
CINECA - UNICORE LoadLeveler OpenPBS with MAUI
HLRS - UNICORE OpenPBS ERS-NQSPBS Pro
CEPBA eNANOS (resource broker) Globus 3.2 LoadLeveler OpenPBS NQS
EPCC JOSH (JOb Scheduling Hierarchically) Globus 3.2 LoadLeveler SGE
NTUA GRIA (Grid Resources for Industrial Applications) Condor
PSNC GRMS (Grid Resource Management System) Pre-WS Globus 3.2 OpenPBS LSF SGE
10
Architecture of the SPA (1st stage)
Authorization Service
Accounting System
SPA
Job submission
Data mngmt
Resource Information
Workflow
Accounting
Sofware Repository
User Manager
Portlets
GridSphere Framework
Information Service
Service Plugins
Multi-Grid Services
JOSH Plugin
GRMS Plugin
eNanos Plugin
UNICORE Plugin
GRIA Plugin
Globus MDS Plugin

Software Repository
Data Management System
11
Architecture of the SPA
Authorization Service
SPA
Accounting System
Job submission
Data mngmt
Resource Information
Workflow
Accounting
Sofware Repository
User Manager
Portlets
GridSphere Framework
Information Service
Meta-broker
Service Plugins
Multi-Grid Services
Software Repository
JOSH Plugin
GRMS Plugin
eNanos Plugin
UNICORE Plugin
GRIA Plugin
Globus MDS Plugin

Data Management System
12
GridSphere
  • GridSphereThe advanced open-source portlet-based
    portal framework (www.gridsphere.org)
  • Features
  • portlet-based (supporting JSR168 standard)
  • customization mechanism for a wide range of end
    users
  • a flexible easy to use interface for users
  • a model to create pluggable and dynamic
    application support for portal developers
  • pluggable access to Grid services using the
    Portlet Services concept
  • uniform access to various Grid systems using the
    Grid Portlets model
  • set of useful core and Grid portlets
  • open source
  • Chosen as the HPC-Europa portal on the basis of
    portals evaluation made

13
Main Services
  • Meta-broker
  • Discovers and choose resources, and submits job
    to HPC centers
  • Replica management system
  • Goal of the HPC-Europa NA3
  • Accounting System
  • Historical information about resource usage
    (including AUs)
  • Information System
  • Central information indexing service
  • Software Repository
  • Logical names, meta-data, binaries compiled for
    various systems, needed environment (preinstalled
    libraries, environment variables, etc.)
  • Authorization Service
  • Authorize users request, check resource limits

14
Multi-Grid Services
  • Extension of the GridSpheres Grid Portlets
    concept by
  • Interface based on standards and functionality of
    several Grid systems
  • Dynamic loading of multiple service
    implementations (on the basis of a given HPC
    center)
  • Remote access
  • Definition of the common interface
  • For each functionality, e.g. job submission,
    resource information etc.
  • Based on standards where possible (e.g. JSDL)
  • Taking into account both gathered requirements
    and the functionality of Grid middleware deployed
    in HPC centers
  • Capability check
  • description of implemented capabilities in the
    form of the XML document
  • used to disable not available options (in portal)
    or to select sites which can accept given job
    description (meta-broker)

15
Job Submission Interface (JSI)
  • Job submission operations
  • submitJob (UNICORE, GRMS, JOSH, eNANOS, GRIA)
  • cancelJob (UNICORE, GRMS, JOSH, eNANOS, GRIA)
  • getJobInfo (UNICORE, GRMS, JOSH, eNANOS, GRIA)
  • findResources (UNICORE, GRMS, eNANOS)
  • submitJobToBestResource (GRMS, JOSH, eNANOS,
    GRIA)
  • findBestResource (JOSH)
  • Data elements descriptions
  • Job Description XML schema
  • Job Information XML schema
  • Auxiliary operations
  • getMiddlewareName
  • getMiddlewareDesc
  • getMiddlewareCaps
  • Job states
  • FINISHED, FAILED, PENDING, QUEUED, RUNNING,
    CANCELLED, SUSPENDED, WAITING

16
Job Description
  • Based on the GGF Job Submission Description
    Language (JSDL) specification
  • Subset of JSDL defines the common interface
  • The specification is still evolving

17
Conclusion
  • Benefits of the SPA
  • Practical
  • enables scientists to solve their real problems
    using HPC resources distributed across Europe in
    a uniform and easy way
  • Research
  • interoperability of various grid technologies and
    middleware,
  • hierarchical model of Grid middleware
  • Open issues
  • Interoperability of different security models
  • Global policies concerning usage of AUs by
    HPC-Europa users
About PowerShow.com