Title: Fabrizio Gagliardi EU DataGrid Project Leader EGEE Project Coordinator f'gagliardicern'ch
1Fabrizio GagliardiEU DataGrid Project
LeaderEGEE Project Coordinatorf.gagliardi_at_cern.c
h
NORDUnet 2003 - Reykjavík, Iceland 24/27 August
2003
From Grid Prototypes to Grid Infrastructures
EGEE is proposed as a project funded by the
European Union under contract IST-2003-508833
2Introduction
- International computing networks are major
enablers of new computing models - GRID computing is one of them, probably the most
promising - Effective implementation of a truly distributed
computing model across different non uniform
computing administrative domains - Many prototype projects based on RN Geant and
similar international initiatives
3Some DataGrid Projects
Through collaborative Grid projects, there is
the potential for a truely global scientific
applications grid
4EDG GEANT
- Connections of the different nodes of the EDG
testbed are made possible by the EU-funded GEANT
project
- connecting more than 30 countries across
Europe - speeds of up to 10 Gbit/s
- high data throughput
- Quality of Service
- EDG and GEANT the first major production
quality tests of the network
- speed
- reliability
- monitoring capabilities
5Applications
- Genomic Exploration
- Earth Observation
- High Energy Physics
more and more scientists begin to use the Grid
computing model and existing Grid testbeds,
relying on Grid technology to solve huge data
challenges
6LHC Resources Requirements
Storage Raw recording rate 0.1 1 GByte/sec
Accumulating data at 5-8 PetaBytes/year
10 PetaBytes of disk
Processing 100,000 of todays fastest PCs
7DataGrid prototypes DataGrid (I)
- 9.8 M Euros EU funding over 3 years (twice as
much from partners) - 90 for middleware and applications (High Energy
Physics, Earth Observation, Genomic Exploration) - Total of 21 partners, over 150 programmers from
research and academic institutes as well as
industrial companies - Three year phased developments demos
(2001-2003) - Several improved versions of middleware software
(final release end 2003) - Several components of software integrated in the
large Particle Physics Production LHC Computing
Project (LCG) - Software used by partner projects DataTAG,
CROSSGRID, GRACE
8DataGrid prototypes DataGrid (II)
- DataGrid testbed more than 1000 CPUs at more
than 15 sites (up to 40) - Connections made possible by the EU-funded GEANT
project - connecting more than 30 countries across Europe
- speeds of up to 10 Gbit/s
- high data throughput
- quality of Service
9The next step go production
- Similarly to research networks few years ago,
after many prototype projects we need to go
production - Major issues Security, (im)maturity of M/Ware
toolkits, difficult user interface for average
use, cost and complexity of operations, etc
10EGEE visionEnabling Grids for E-science in
Europe
- Goal
- Create a wide European Grid production quality
infrastructure on top of present and future EU RN
infrastructure - Build on
- EU and EU member states major investments in Grid
Technology - International connections (US and AP)
- Several pioneering prototype results
- Larg Grid development team (gt60 people in EDG)
- Requires major EU funding effort
- Approach
- Leverage current and planned national and
regional Grid programmes - Work closely with relevant industrial Grid
developers, NRENs and US-AP projects
Applications
EGEE
Geant network
11Why EGEE? Impact on Society
- Access to a production quality grid will change
the way science and business is done in Europe
An international network of scientist will be
able to model a new flood of the Danube in real
time, using meteorological and geological data
from several centers across Europe
A team of engineering students will be able to
run the latest 3D rendering programs from their
laptops using the Grid
A geneticist at a conference, inspired by a talk
she hears, will be able to launch a complex
biomolecular simulation from her mobile phone
12Why EGEE? Political context
- Current Grid RD projects run out within few
months - The EGEE partners have already made major
progress in aligning national and regional Grid
RD efforts, in preparation for EGEE - EGEE will preserve the current strong momentum of
the European Grid community, and the enthusiasm
of the hundreds of young European researchers
already involved in EU Grid projects (gt150 in EDG
only)
13Why EGEE? Historical analogy
- Prior to the EU Geant programme, there was in
Europe a multitude of exploratory projects in
networking technology. Geant was truly production
oriented, and brought European telecom operators
actively into the picture - In a similar way, EGEE can ensure preservation of
current investments in European Grid RD,
extending the present infrastructure and
focussing all activities towards establishing a
production quality Grid
14EGEE Partner Federations
- Integrate regional Grid efforts
- Represent leading grid activities in Europe
15EGEE Activity Areas
- Services
- Deliver production level grid services
(manageable, robust, resilient to failure) - Ensure security and scalability
- Middleware
- Professional Grid middleware re-engineering
activity in support of the production services - Networking
- Proactively market Grid services to new research
communities in academia and industry - Provide necessary education
16EGEE Operations Structure
Operations Center
Infrastructure
Regional Support Center (Support for
Applications Local Resources)
Resource Center (Processors, disks)
Grid server Nodes
17EGEE Service Activity
- Create, operate, support and manage a production
quality infrastructure - Structure
- EGEE Operations Management at CERN
- EGEE Core Infrastructure Centres in the UK,
France, Italy and CERN (leveraging HEP LCG at the
start), responsible for managing the overall Grid
infrastructure - Regional Operations Centres, responsible for
coordinating regional resources, regional
deployment and support of services in all other
countries - Offered services
- Middleware deployment and installation
- Software and documentation repository
- Grid monitoring and problem tracking
- Bug reporting and knowledge database
- VO services
- Grid management services
18EGEE Networking Activity
- Dissemination and outreach
- User training and induction
- Application identification and support
- Two pilot application centers (for high energy
physics and biomedical grids) - One more generic component dealing with longer
term recruitment and support of other communities - Policy and International cooperation
rely on a supporting network in the partner
regions
19EGEE Middleware Activity
- Hardening and re-engineering of existing
middleware functionality, leveraging the
experience of partners - Activity concentrated in few major centers
- Key services Resource Access
- Data Management
- Information Collection and Accounting
- Resource Brokering (Italy)
- Quality Assurance
- Grid Security
- Middleware Integration
- Middleware Testing
20EGEE Industry
- Industrial participation encouraged both as
potential end-users and IT technology and service
suppliers - Normally through national and regional Grid EGEE
federations - EGEE will maintain an Industry Forum to keep
selected Industrial and Commercial interested
parties in close contact - Services developed in first EGEE 2 years phase
(2004-5) might be tendered to Industry in second
phase (2006-7)
21EGEE Timeline
- May 2003 proposal submitted
- July 2003 proposal accepted
- September 2003 start negotiation
- April 2004 start project
22Conclusions
- The EU DataGrid project has successfully
fulfilled its role of EU Grid flagship project in
collaboration with several other EU and
international projects - Essential to keep the momentum and the current
lead in production Grids in Europe - Important to build an international cooperation
between European and US/AP Grid infrastructure
projects - The scientific user communities are already
international (HEP is an excellent example) and
so the computing resources and most of the
experimental instruments - EGEE proposes the right framework and plans to
accomplish the above objectives leveraging
present and future international research networks
23More information
- More information on the EU DataGrid project on
- www.edg.org
- More information on EGEE on
- www.cern.ch/egee
- Or mail me at fabrizio.gagliardi_at_cern.ch