Introduction to GRID computing - PowerPoint PPT Presentation

About This Presentation
Title:

Introduction to GRID computing

Description:

Introduction to GRID computing Introduction GRID Tutorial Jules Wolfrat SARA Definition of Grid From an EU brochure: It doesn t matter if your team is modeling the ... – PowerPoint PPT presentation

Number of Views:132
Avg rating:3.0/5.0
Slides: 30
Provided by: JulesW
Learn more at: http://www.dutchgrid.org
Category:

less

Transcript and Presenter's Notes

Title: Introduction to GRID computing


1
Introduction to GRID computing
  • Introduction GRID Tutorial
  • Jules Wolfrat
  • SARA

2
Definition of Grid
  • From an EU brochure
  • It doesnt matter if your team is modeling the
    Earths atmosphere, designing cars, creating
    animated films or finding new medicines, the
    basic principle is the same your Grid supplies
    all the computing power, software, data and
    knowledge you need in one integrated package, and
    helps project teams work more closely together
  • The analogy with the power grid
  • Like you can plug in anywhere to the power grid
    without knowing where your energy is coming from
    you can plug into the grid without knowing where
    your (computing) resources are coming from.

3
History (1)
  • From a news item in 1991
  • Smarr describes the metacomputer as a network of
    heterogeneous, computational resources linked by
    software in such a way that they can be used as
    easily as a personal computer
  • So the concept was introduced already in the
    early 90s, known as metacomputing.
  • Motivation was the emergence of computer
    networks.

4
Example (1)
  • Following is an example of the kind of
    initiatives started in those years from close by
  • In 1996 a project was started in Amsterdam
  • The Amsterdam Metacomputing project is an ongoing
    effort from the University of Amsterdam (UvA),
    the Free University (VU) and "Academic Computing
    Services Amsterdam" (SARA) to develop a
    Metacomputer environment on the Amsterdam campus.
  • Important components of this environment will be
    automatic distribution and monitoring of jobs
    over a network of computer systems, uniform
    access to files of other users from each place to
    work and to each computer system incorporated in
    the environment, distributed storage of data on
    various fileservers, automatic backup, migration
    and archiving, general availability of both
    commercial and public domain software on software
    servers, and a minimum of system management
    tasks.
  • In this way scientists will be able to devote all
    of their time to their actual task science.

5
Example (2)
  • An extensive package of services will gradually
    be implemented and finally include the following
    components
  • fileservers and distributed, transparent
    file-systems
  • backup, migration and archiving services
  • batch-queueing systems, designed for efficient
    use of local systems, and if desired, of
    computational servers supplied by SARA
  • public domain and specialist (commercial)
    software servers.
  • All components will be accessible from the
    scientist's desktop. A client-server architecture
    will play an important role. Combining components
    will be a relatively easy task, enhancing
    efficiency in terms of man-hours needed to
    accomplish a given task. These pages, as well as
    the Metacomputer are still in a development stage
    ..

6
Example (3)
Systems available at SARA in 1996
CRAY YMP Vector system
Parsytec CC 56 CPUs
IBM SP2 76 CPUs
7
Example (4)
  • SARA news item on 16-6-1998
  • Basis voor meta-omgeving gelegd.
  • Sinds 4 mei maakt SARA's IBM RS/6000 SP
    parallelle supercomputer gebruik van de DCE/DFS
    omgeving, een filesysteem dat een transparante
    computeromgeving mogelijk maakt. Met het nieuwe
    filesysteem zijn bestanden van DCE/DFS gebruikers
    wereldwijd toegankelijk met andere
    computersystemen die beschikken over DCE/DFS,
    waarmee een belangrijke basis is gelegd voor de
    meta-omgeving.
  • Gebruikers aan de VU science faculty hebben nu
    op een uniforme manier toegang tot hun bestanden,
    ongeacht of ze werken op de RS/6000 SP of een
    lokaal workstation. Hetzelfde geldt voor
    gebruikers van het Parsytec CC systeem bij SARA
    vanaf zowel de Parsytec als de RS/6000 SP zijn
    alle bestanden voor de gebruiker direct
    toegankelijk.

8
Example (5)
  • A web interface was developed for submitting jobs
    to the metacomputing environment, also a meta job
    language was used.
  • Also job migration between systems and mpi over
    two systems was investigated
  • First time we heard about globus, one of the well
    known building blocks now for grid
    infrastructures.
  • Network link between systems was a problem, only
    FE link, Gbit not available, HiPPI (800 Mbps) not
    available for Parsytec.

9
Today (1)
  • So what is new today?
  • Scale! Grid infrastructures operate worldwide
  • International infrastructures - EGEE, DEISA,
    Nordugrid, OSG, TeraGrid
  • National NAREGI (Japan), UK-eScience, D-Grid,
    NLGrid
  • Interoperability availability of middleware
    Globus toolkit, UNICORE, NAREGI, schedulers

10
Today (2)
  • Some basic requirements for a grid infrastructure
  • Transparent user administration single sign on
    (single grid identity), authorisation and
    accounting based on grid identity AAA
    facilities
  • Job scheduling which can handle different
    environments
  • Global data access
  • Global information services job information,
    data information, resource information
  • Interoperability!
  • Standards needed for federation of
    infrastructures GGF, IETF.

11
Networking (1)
  • Developments in network connectivity (high
    bandwidths) and tools play an important role
  • 10 Gbps WAN links available today, both shared
    links and dedicated lightpaths (based on lambda
    technology)
  • 1 Gbps network adapters are commodity items on
    systems today and 10GE adapters available

12
Networking (2)
  • GridFTP can use multiple streams in order to take
    full advantage of available bandwidth
  • Parallel files systems can take full advantage of
    underlying high speed networks - throughput can
    be in the order of 100MByte/s and more
  • Tuning of WAN TCP must get attention, e.g.
    latencies are in the order of milliseconds (20
    in Europe), defaults on systems mostly not suited
    for bulk data transports.

13
SURFnet6 DWDM on dark fiber
SURFnet 6 infrastructure
Muenster
14
NetherLight Lightpath connections to the
Netherlands
3rd quarter 2005
15
Global Lambda Integrated Facility (GLIF)World Map
Visualization courtesy of Bob Patterson,
NCSA/University of Illinois at Urbana-Champaign. D
ata compilation by Maxine Brown, University of
Illinois at Chicago. Earth texture from NASA.
www.glif.is
16
GEANT2 topology
17
The EGEE project
  • EGEE
  • 1 April 2004 31 March 2006
  • 71 partners in 27 countries, federated in
    regional Grids
  • Operation of a pan European production Grid
  • EGEE-II
  • 1 April 2006 31 March 2008
  • Expanded consortium
  • 91 partners
  • 11 Joint Research Units
  • Natural continuation of EGEE
  • Emphasis on providing production-level
    infrastructure
  • increased support for applications
  • interoperation with other Grid infrastructures
  • more involvement from Industry

18
The EGEE infrastructure
  • Mission
  • Manage and operate production e-Infrastructure
    open to all user communities and service
    providers
  • Contribute to Grid standardisation and policy
    efforts
  • Infrastructure operation
  • Currently include 200 sites across 39 countries
  • Continuous monitoring of Grid services in a
    distributed global infrastructure
  • Automated site configuration/management
  • Future
  • Expand on interoperability with related
    infrastructures

19
EGEE-II Federations and Countries
20
Operational Organisation
21
User Support Activities
22
User support in NE region
  • NE website http//www.egee-ne.org/operations
  • User support contact user support at local site
    or mail to support_at_egee-ne.org
  • NE uses a ticketing system monitored by different
    partners from our region. In NL NIKHEF, RC-RuG,
    SARA responsible.
  • Tickets from GGUS are also imported in the NE
    system
  • Application support NA4 activity. In NL RC-RuG,
    SARA

23
A Selection of Monitoring tools
1. GIIS Monitor
2. GIIS Monitor graphs
3. GOC Data Base
4. Scheduled Downtimes
5. GridIce VO view
6. Live Job Monitor
24
(No Transcript)
25
The DEISA project
  • Objective To enable Europes terascale science
    by the integration of Europes most powerful
    supercomputing systems.
  • DEISA is an European Supercomputing Service built
    on top of existing national services. This
    service is based on the deployment and operation
    of a persistent, production quality, distributed
    supercomputing environment with continental
    scope.

26
THE DEISA SUPERCOMPUTING GRID
AIX distributed super-cluster
Vector systems (NEC, )
GEANT
Linux systems (SGI, IBM, )
27
Participating Sites
BSC Barcelona Supercomputing Centre
Spain CINECA Consortio Interuniversitario
per il Calcolo Automatico Italy CSC
Finnish Information Technology Centre for
Science Finland EPCC/HPCx University of
Edinburgh and CCLRC
UK ECMWF European Centre for
Medium-Range Weather Forecast UK
(int) FZJ Research Centre
Juelich Germany HLRS High
Performance Computing Centre Stuttgart
Germany IDRIS Institut du
Développement et des Ressources
France en Informatique Scientifique -
CNRS LRZ Leibniz Rechenzentrum Munich
Germany RZG Rechenzentrum Garching of
the Max Planck Society Germany SARA
Dutch National High Performance Computing
The Netherlands and Networking centre

28
DEISA technologies
  • GPFS parallel filesystem for transparent file
    access from all systems dedicated European
    network used for high throughput
  • Loadleveler-MC for job submission on AIX systems
  • UNICORE for job submission to all systems
  • Common Programming Environment (CPE) on all
    systems for DEISA users
  • Single username on all systems

29
Access
  • Users can submit proposals for access to DEISA
    resources through DECI (DEISA Extreme Computing
    Initiative) calls
  • Proposals are evaluated by national committees
    and depending on ranking get access to resources
  • Most partners contribute about 10 of their
    resources for DEISA applications
  • URL www.deisa.org
Write a Comment
User Comments (0)
About PowerShow.com