EU DataGrid progress and EGEE plans - PowerPoint PPT Presentation

1 / 22
About This Presentation
Title:

EU DataGrid progress and EGEE plans

Description:

Continue collaboration with US and across Atlantic testbeds (with DataTAG) ... Most of partners built national and regional Grid federations to participate in EGEE ... – PowerPoint PPT presentation

Number of Views:27
Avg rating:3.0/5.0
Slides: 23
Provided by: fabri61
Category:

less

Transcript and Presenter's Notes

Title: EU DataGrid progress and EGEE plans


1
EU DataGrid progress and EGEE plans
  • Fabrizio Gagliardi
  • EDG Project Leader - EGEE Proposal Coordinator
  • Fabrizio.Gagliardi_at_cern.ch

2
The EU DataGrid Project
  • 9.8 M Euros EU funding over 3 years, twice as
    much from partners
  • 90 for middleware and applications (HEP, Earth
    Obs. and Bio Med.)
  • Three year phased developments demos
    (2001-2003)
  • 2nd annual project review successfully passed in
    Feb 2003!
  • Total of 21 partners
  • Research and Academic institutes as well as
    industrial companies
  • Related projects and activities
  • DataTAG (2002-2003)
  • CrossGrid (2002-2004)
  • GRIDSTART (2002-2004)
  • Grace (2002-2004)

3
Applications
  • Genomic Exploration
  • Earth Observation
  • High Energy Physics

more and more scientists begin to use the EDG
middleware and testbed, relying on Grid
technology to solve huge data challenges
4
Project priorities refocused at EDG5
After initial middleware development and testbed
deployment, effort has been refocused on quality
and stability
  • Quality Policy Statement published
  • http//eu-datagrid.web.cern.ch/eu-datagrid/WP12/de
    fault.htm
  • List of priorities defined at a project retreat
  • http//documents.cern.ch/age?a021130
  • Followed-up at the Budapest project conference
  • http//www.tomiexpress.hu/datagrid/
  • Show-stoppers found by users on the application
    testbed were the highest priority
  • Incremental improvements driven by the needs of
    the applications (HEPCAL, Bio and EO use cases)

5
Atlas CMS Evaluations
  • RESULTS
  • Could distribute and run CMS s/w in EDG
    environment
  • Generated 250K events for physics with 10,000
    jobs in 3 week period
  • OBSERVATIONS
  • Were able to quickly add new sites to provide
    extra resources
  • Fast turnaround in bug fixing and installing new
    software
  • Test was labour intensive (since software was
    developing and the overall system was fragile)
  • EDG 2.0 should fix the major problems providing a
    system suitable for full integration in
    distributed production
  • RESULTS
  • Atlas software was used in the EDG Grid
    environment
  • Several hundred simulation jobs of length 4-24
    hours were executed , data was replicated using
    grid tools
  • Results of simulation agreed with non-Grid
    runs
  • OBSERVATIONS
  • Good interaction with EDG middleware providers
    and with WP6/8
  • With a substantial effort it was possible to
    perform the jobs
  • Showed up bugs and performance limitations (fixed
    or to be fixed in EDG 2.0)
  • We need EDG 2.0 release for use in large scale
    data challenges

6
EDG Application Testbed for the EU review
  • Since Last Year
  • Improved software (EDG 1.4.3).
  • Doubled sites. More waiting
  • Australia, Taiwan, USA (U. Wisc.), UK Sites,
    INFN, French sites, CrossGrid,
  • Significantly more CPU/Storage.
  • Hidden Infrastructure
  • MDS Hierarchy, Resource Brokers, User Interfaces,
    VO Replica Catalogs, VO Membership Servers,
    Certificate Authorities

7
EDG GEANT
  • Connections of the different nodes of the EDG
    testbed are made possible by the EU-funded GEANT
    project
  • connecting more than 30 countries across
    Europe
  • speeds of up to 10 Gbit/s
  • high data throughput
  • Quality of Service
  • EDG and GEANT the first major production
    quality tests of the network
  • speed
  • reliability
  • monitoring capabilities

8
Related Grid Projects

Through links with sister projects, there is
the potential for a truely global scientific
applications grid
9
Current Plans
  • Second EU review successfully passed in February
    2003
  • Major re-orientation of the project towards
    production accepted
  • Need to develop further plans in view of future
    project EGEE
  • Continue collaboration with US and across
    Atlantic testbeds (with DataTAG)
  • Need to accommodate other applications
  • Plan long term support of EDG developments
  • Major proposal for further EU funding (EGEE)

10
EGEE visionEnabling Grids for E-science in
Europe
  • Goal
  • Create a wide European Grid production quality
    infrastructure on top of present and future EU RN
    infrastructure
  • Build on
  • EU and EU member states major investments in Grid
    Technology
  • International connections (US and AP)
  • Several pioneering prototype results
  • Larg Grid development team (gt60 people)
  • Requires major EU funding effort
  • Approach
  • Leverage current and planned national and
    regional Grid programmes (e.g. LCG)
  • Work closely with relevant industrial Grid
    developers, NRENs and US-AP projects

Applications
 
EGEE
Geant network
11
Why EGEE? The Historical Analogy
  • Prior to the EU Geant program ,there was in
    Europe a multitude of exploratory projects in
    networking technology. Geant was truly production
    oriented, and brought European telecom operators
    actively into the picture
  • In a similar way, EGEE can ensure preservation
    of current investments in European Grid RD,
    extending the present infrastructure and
    focussing all activities towards establishing a
    production quality Grid

12
Why EGEE? The Societal Impact
Access to a production quality GRID will change
the way science and much else is done in Europe
An international network of scientist will be
able to model a new flood of the Danube in real
time, using meteorological and geological data
from several centers across Europe.
A team of engineering students will be able to
run the latest 3D rendering programs from their
laptops using the Grid.
A geneticist at a conference, inspired by a talk
she hears, will be able to launch a complex
biomolecular simulation from her mobile phone.
13
Why EGEE? The Political Context
  • Current Grid RD projects run out within 18
    months
  • The EGEE partners have already made major
    progress in aligning national and regional Grid
    RD efforts, in preparation for EGEE
  • Launching EGEE now will preserve the current
    strong momentum of the European Grid community,
    and the enthusiasm of the hundreds of young
    European researchers already involved in EU Grid
    projects (gt150 in EDG only)

14
EGEE Approach
  • Most of partners built national and regional Grid
    federations to participate in EGEE
  • Condition to participate in EGEE is to have
    already an established Grid activity or be an
    established Grid technology centre
  • EGEE overall project funding needs will require
    staggered proposals to respond to several
    separate EU calls
  • EU FP6 Communication Network Development Call
    opened on December 17th, proposal submitted on
    May 6th

15
The EGEE consortium of Grid Federations
Total of 70 full partners covering entire EU and
beyond
16
Integrated Infrastructure Initiative (I3)
  • Response to FP6 call Communication Network
    Development Grids
  • Three lines of EU funding (with current funding
    breakdown)
  • Networking Activities
  • Management coordination (7 of total funding)
  • Dissemination and outreach (7 of total funding)
  • Application support (10 of total funding)
  • Specific Service Activities
  • Provision and procurement of Grid services (52
    of total funding)
  • Joint Research Activity
  • Engineering development to improve the services
    provided by the Grid infrastructure (24 of total
    funding)

17
Networking activities
General management and coordination of the
project User groups, Industry Forum Dissemination,
outreach, training and pilot applications
18
Networking activities
  • User Training and Induction User Support and
    Consultancy Applications Interface. The lead
    partner for the Training activity is the UK
    (Edinburgh)
  • Dissemination and Outreach The lead partner for
    the Dissemination activity is the European
    Research Network organisation Terena (Amsterdam)
  • Both lead partners will rely on a supporting
    network in the partner regions, partially funded
    by the project. The User Support and Consultancy
    is envisaged as a distributed effort

19
Networking activities
  • The Applications Interface Activity has three
    components
  • two Pilot Application Centres high energy
    physics (LCG www.cern.ch/lcg ) and bio/medical
  • one more generic component dealing with the
    longer term recruitment and support of other
    communities
  • The Pilot Application Centres are based at CERN
    and in France, with participation to their
    activities by other partners envisaged
  • The third, more generic component is distributed
    more evenly among the partners, in order to
    ensure as broad a contact with scientific
    communities as possible

20
Specific service activities
  • Integration of national and international Grid
    infrastructures

21
Specific service activities
  • The structure of the Grid services will
    comprise
  • EGEE Operations Management at CERN
  • EGEE Core Infrastructure Centres in the UK,
    France, Italy, Germany and CERN (leveraging LCG
    at the start), responsible for managing the
    overall Grid infrastructure
  • Regional Operations Centres, responsible for
    coordinating regional resources, regional
    deployment and support of services in all other
    countries

22
Joint research activity
  • Hardening and re-engineering of Middleware

23
Joint research activity
  • Middleware Centres for key services Resource
    Access (Italy) Data Management (CERN)
    Information Collection and Accounting (UK)
    Resource Brokering (Italy) Closely connected to
    this middleware development is a Quality
    Assurance team (France) and a Grid Security team
    (Northern Europe Consortium). A Middleware
    Integration team and Middleware Testing Centre
    will be located at CERN.
  • Middleware services and related support teams
    require critical mass. With matching funding
    from the host institutions, this translates to
    over 40 FTEs for all middleware activities
    (compare with about 60 FTEs in EDG).

24
Summary of budget requirements for EGEE
  • EGEE will integrate existing computer resources,
    no hardware requests. The bulk of the budget is
    for human resources
  • Building strong teams where already existing
    expertise, to progress as quickly as possible.
    Host partners for these teams will provide
    matching funding and demonstrate adequate
    expertise for the job
  • The partners estimated a total funding need of
    about 100 M corresponding to about 33 M EU
    funding. Not included in this are US and Russian
    contributions (4 M ), and overall project
    management, corresponding to 7 of the EU funding
    over two years

25
EGEE and Industry
  • Industrial participation encouraged both as
    potential end-users and IT technology and service
    suppliers
  • Normally through national and regional Grid EGEE
    federations
  • EGEE will maintain an Industry Forum to keep
    selected Industrial and Commercial interested
    parties in close contact
  • Services developed in first EGEE 2 years phase
    (2004-5) might be tendered to Industry in second
    phase (2006-7)

26
Conclusions
  • The EU DataGrid project has successfully
    fulfilled its role of EU Grid flagship project in
    collaboration with several other EU and
    international projects
  • Essential to keep the momentum and the current
    lead in production Grids in Europe
  • Important to build an international cooperation
    between European and US/AP Grid infrastructure
    projects
  • The scientific user communities are already
    international (HEP is an excellent example) and
    so the computing resources and most of the
    experimental instruments
  • EGEE proposes the right framework and plans to
    accomplish the above objectives
  • EGEE executive summary document linked to the
    last project conference agenda (http//agenda.cern
    .ch/fullAgenda.php?idaa03816)
Write a Comment
User Comments (0)
About PowerShow.com