EELAs:the seed LGI NGIs:the goal

1
EELAs the seedLGI / NGIs the goal

• Bernard M. Marechal (EELA-2 Project Coordinator)
• CETA-CIEMAT UFRJ (Spain Brazil)
• CONAIS 2009
• Villahermosa (Mexico), 23.09.2009

2
Outline

• Short EELA EELA-2 story
• The EELA-2 model for long-term sustainability
• Proposal for future activities

3
Computationally intensive research
4
Computationally intensive research
5
Computationally intensive research

• Virtual Organisations

• E-Infrastructure, i.e.
• DCI (Grid, Clouds,...)

• Applications
• Data
• Instruments

6
The beginning

• Several meetings (Rio de Janeiro, CERN,
  Itacuruca) in 2004, foreseeing a EULA Grid
  Project supported by EC
• EELA proposal prepared, submitted and approved in
  2005
• EELA worked hard and well during 2 years (2006
  2007)
• EELA-2 prepared submitted in 2007 and approved
  in 2008
• EELA-2 will end on 31 March 2010
• EC support fade out in the future
• LGI / NGIs fade in mandatory to support
  e-Science in LA on the long term

7
Projects in numbers
EELA (SSA under EU FP6)
EELA-2 (CP-CSA under EU FP7)

• E-infrastructure shared between Europe and Latin
  America
• EC support 1.7 M
• CIEMAT extra support 0.4 M
• 10 Countries (3 in Europe)
• 2 International Organisations
• 20 Members (7 in Europe)

• E-science grid facility for Europe and Latin
  America
• EC support 2.1 M
• CETA-CIEMAT extra funds 0.3 M
• Currently 16 Countries (11 in LA)
• 2 new countries (Panama Uruguay)
• 1 International Organisation (CLARA)
• Currently 78 Members (62 in LA)
• 32 Institutions joining (31 in LA)
• Currently 13 JRUs (9 in LA)
• 4 new JRUs (3 in LA)

At the final review EELA was awarded the highest
EC rank Good to excellent project
8
Countries and Resources Centres
9
Joint Research Units in EELA-2

• A Joint Research Unit (JRU) is a partnership for
  e-Science Collaboration, with no legal status as
  such, formed between entities of the same
  nationality and meeting the following conditions
• scientific and economic unity (internal MoU in
  place)
• lasting a certain length of time (not linked to a
  particular project)
• recognised by a public authority (minister,
  secretary of state, etc.).

No JRUs were established in LA during EELA
One of the major successes of EELA-2
10
EELA EELA-2 objectives

• EELA-2
• Provide an empowered Grid Facility with versatile
  services fulfilling application requirements
• Ensure production quality services
• Ensure the long term sustainability of the
  e-Infrastructure beyond the term of the project
• Expand the current EELA e-Infrastructure
• Look for new communities outside academia
  (Industry and Business)

• EELA
• Build a bridge between consolidated
  e-Infrastructure initiatives in Europe and
  emerging ones in Latin American
• Create a collaboration network to deploy a large
  portfolio of scientific applications on a well
  supported Pilot Test-bed
• Care in parallel of the training in grid
  technologies and of the knowledge dissemination
  and outreach

11
More Numbers
EELA
EELA-2

• At the end of the Project
• Sites 19 (13 in Production)
• Cores 2700
• Storage 100 TB
• Jobs 430,000
• Scientific domains 4
• Applications 18
• 12 in production
• 6 gridified
• Conferences 3
• Workshops 8
• Tutorials 18
• Trained people 733
• Grid Schools 2

• After 1 year
• Sites 22
• Cores ? 5800
• Storage ? 190 TB
• Jobs ? 1,000,000
• Scientific domains 9
• Applications 55
• 12 in production
• 15 Gridified
• Conferences 1
• Workshops 4
• Tutorials 8
• Trained people 158
• Grid Schools 1

12
EELA-2 achievements - NA2
NA2 Dissemination Training Much more training
effort delivered than pledged
13
EELA-2 achievements - NA3
NA3 Application Support The EELA-2
gridification universe
14
EELA-2 achievements - NA3
Applications distribution per country, continent
and scientific domain (Y1)
2
14
15
EELA-2 achievements - SA1
SA1 Infrastructure Services Every EELA-2
country has access to the LA PKI
16
EELA-2 achievements - SA1
Resource Centres Map (Y1)
17
EELA-2 achievements - SA1
Infrastructure Usage - all VOs (Y1)
About 1 million jobs executed
Resource Centres
18
EELA-2 achievements - SA2
SA2 Network Resource Provision EELA-2 Network
Monitoring

• Deployment of a sustainable perfSONAR-based
  multi-domain network monitoring service
• 3 measurement points on EELA-2 RCs (Overlay
  network monitoring) already installed
• Integration with available monitoring services in
  the domains (LA and EU-NRENs, RedCLARA, GÉANT2,
  EGEE-III, etc.)

• Multi-domain monitoring

• Overlay network monitoring

19
EELA-2 achievements - JRA1

• JRA1 Development of Services for Applications
  and Infrastructure
• Activity Goals
• Making the e-Infrastructure more useful and wide
  spread to increase its reach
• Execution of grid middleware on top of different
  platforms, including Microsoft Windows
• Scavenging of idle resources, particularly in
  shared desktops
• and its usability
• New application-oriented grid services
• Support for the integration of a lightweight
  opportunistic grid middleware (OurGrid) whose
  development is led by one of the consortium
  members
• With the effect of helping in fostering the
  sustainability of the e-Infrastructure

20
EELA-2 achievements - JRA1
Highlights of the 1st Year

• Interoperability between OurGrid and gLite,
  planned for M16, just available (see Hot News
  in the EELA-2 Website homepage)
• MoU signed with other projects working on similar
  goals (EDGeS and SEE-GRID-SCI)
• Initial assessment of the possibility of using
  EDGeS G3 indicates the feasibility of using
  technology that has already been successfully
  used in a similar setting
• Initial evaluation of the innovative co-existence
  approach reveals advantages that had not been
  foreseen at the time the project was proposed
• Possibility of draining BoT applications out of
  the service grid, improving the performance of
  tightly-coupled applications

21
The global Grid coverage
How to take care of the future of
e-infrastructures?
22
Grid Ecosystem evolution in Europe
23
and in Latin America ?
24
Some considerations

• EELA-2 will end up on 01/04/2010
• Will 6-year of successful efforts to
• Set up a production quality Latin American
  e-Infrastructure at the level of existing
  European Grids
• Acquire the various Grid technology skills to
  autonomously support the Latin American
  e-Infrastructure on the long term (about 1000 LA
  people trained and 10s of local tutor formed)
• Allow LA scientific communities to
• Strengthen their active participation in EU LA
  collaborations by satisfying the required
  contributions to computing resources
• Enter new international collaborations to take
  part in new research field or attract new
  partners in their own field of research
• Altogether contribute to amplify the relevance
  of e-Infrastructures, blazing the trail towards
  Latin American e-Science Initiatives via National
  Grid Initiatives (NGIs) created in cooperation
  with existing NRENs and RedCLARA (EELA-2
  proposal)
• just be wasted ???

24
25
The EELA-2 proposal

• A 3-level organisation is proposed
• Local (RCs ? LAN/MAN)
• National (NGIs ? NRENs)
• Continental (LGI ? CLARA)
• Multi-middleware (gLite, OurGrid, OSG, )
  structures NGIs and LGI must not be middleware
  dependent
• Close synergy with EGI that foresees to support
  three middleware (ARC, gLite and UNICORE)
• Important role of both the LA NRENs and CLARA
  fully justified by their participation in EELA
  and EELA-2
• Even more e-Infrastructures (Grid) should be
  part of the NRENs and CLARA structure (cost
  effective, technically correct, politically
  easier)

25
26
The Latin American Grid Initiative
(LGI)(http//documents.eu-eela.org/record/1119/fi
les/)
RC Resource Centre, NGI National Grid
Initiative, GOC Grid Operation Centre,
NREN National Research Education Network, NOC
Network Operation Centre, LGI Latin
American Grid Initiative, CLARA Cooperación
Latino Americana de Redes Avanzadas, GSC Grid
Support Centre, NSC Network Support Centre
27
The Latin American Grid Infrastructure - GridLGI

• GridLGI is composed, from bottom to top of three
  layers, each of them including Grid and Network
  components
• At the lower Institute level
• The Resource Centre (RC) is the actual site of
  the Grid computing resources
• The backbone LAN or MAN network connects the
  Computing and Storage Elements (CE SE) at the
  RC site
• At the intermediate national level
• The National Grid Initiative (NGI) integrates all
  Grid Institutions in the country and federates
  under its Grid Operation Centre (GOC) all their
  RCs in a National Grid (GridNGI)
• The underlying National Research Education
  Network (NREN) links country-wide the Research
  and Education Institutes and optimizes / monitors
  their network activity via its associated Network
  Operation Centre (NOC)
• At the upper international layer
• The Latin American Grid Initiative (LGI) which
  brings together all NGIs and supervises the
  various GridNGIs operation from its Grid Support
  Centre (GSC)
• CLARA it is the WAN network which integrates the
  Latin American NRENs and supervises the overall
  RedCLARA network activity by means of its Network
  Support Centre (NSC).

28
RC tasks
29
RC tasks(detailed description)

• Deploy and operate the local grid services such
  as Computing Elements and Storage Elements
• Provide 1st level support to users, application
  developers and Resource Centre administrators,
  usually through a trouble-shooting ticketing
  system
• Coordinate the deployment and update of the grid
  middleware used by its resource providers
• Monitor the status of the grid services run by
  the national resource providers to quickly detect
  problems, request of corrections and be sure that
  problems are fixed
• Deal with security incidents making sure that
  proper solutions are quickly applied
• Deploy an authentication service this includes
  one or more Registration Authorities of the
  National Certification Authority
• Operate an accounting repository to keep track of
  grid resource usage.

30
GOC tasks
31
GOC tasks(detailed description)

• Deploy and operate the core grid services needed
  to interconnect to the GSC. It includes Workload
  Management Systems, Grid Information Systems,
  Data Management Systems, etc.
• Provide 2nd level support to users, application
  developers and resource centre administrators in
  the region
• Setup, support the Operator on Duty (OOD) Teams
  responsible for the continuous monitoring and
  support of the Resource Centre fabric, for prompt
  error detection, diagnosis and resolution of
  operational problems
• Route and follow up trouble-shooting tickets to
  the proper support unit
• Coordinate the deployment and updating of the
  grid middleware
• Monitor the status of the grid services running
  in the country to quickly detect problems,
  request corrections and be sure that problems are
  fixed
• Coordinate the solution of security incidents,
  making sure that proper solutions are quickly
  applied
• Operate an accounting repository to keep track of
  grid resource usage
• Operate a configuration repository
• Operate the VO management systems for
  country-wide-scope Virtual Organisation
• Coordinate the resource allocation and brokering
  for national VOs
• Deploy and operate an Authentication Service at
  national level. This includes a Certification
  Authority and as many Registration Authorities as
  needed

32
GSC tasks
33
GSC tasks(detailed description 1/2)

• Coordinate the operation of all GOCs
• Deploy and operate the core grid services needed
  to interconnect all the GOCs and the single RCs
  in the infrastructure. It is usually done in an
  hierarchical way, profiting from the services
  already deployed and maintained by the GOCs,
  including Workload Management Systems, Grid
  Information Systems, Data Management Systems,
  etc.
• Provide 3rd level support to users, application
  developers and Resource Centre administrators in
  the infrastructure
• Route and follow up trouble tickets with the GOCs
• Maintain and Operate a central ticketing system
• Maintain and Operate a central Operations Portal
• Maintain a central Knowledge Database
• Maintain a Knowledge Sharing System
• Define best practices, operation procedures and
  operation requirements
• Coordinate the middleware deployment, release and
  support

34
GSC tasks(detailed description 2/2)

• Gather requirements for new services from users,
  resource centre administrators and grid operators
• Advertise new Infrastructure and Application
  services to users and applications developers
• Deploy and operate an Authentication Service.
  This includes a catch-all Certification
  Authority having as many Registration Authorities
  as needed in those countries which do not yet
  have a national Certification Authority the GSC
  should also provides consulting services to those
  countries in order to speed-up the creation of
  new Certification Authorities and their
  accreditation by TAGPMA, the relevant Policy
  Management Authority
• Operate the VO management for global VOs
• Coordinate the resource allocation and brokering
  for global VOs
• Coordinate the solution of security incidents,
  making sure that proper solutions are quickly
  applied
• Operate a global accounting repository to keep
  track of grid resource usage
• Operate a global configuration repository
• Coordinate interoperation efforts with other grid
  infrastructures
• Perform studies to plan the growth of the
  infrastructure

35
NSC tasks
Single point of access
36
NSC tasks(detailed description)

• Be a single point of contact for network related
  issues
• Trace and diagnose network problems
• The NSC could receive two types of tickets
• User ticket an e-Infrastructure user notifying
  about a network issue on his/her site or within
  some site that he/she is trying to access
• NREN ticket a NREN notifying the NSC about
  network issues. It is important to note the
  assumption that the NREN is not expected to
  notify (at least initially) the NSC only about
  network issues which involve the e Infrastructure
  sites, but actually about all the NREN issues,
  e.g., through a general mailing list assignment
• Support E2E communication between sites
• Provide quality indicators about the network
  infrastructure

37
Training dissemination tasks
38
Training dissemination tasks(detailed
description 1/2)

• The activities to be performed by the tutors
  operating at NGI and RC level include
• Participate as instructors in tutorials at local,
  national levels in cooperation with the LGI
• Update their knowledge as improvements and new
  services are added to the middleware(s)
• Create, update and translate training material
  maintained by the LGI
• The dissemination activities include
• Coordinate the dissemination actions at local and
  national levels in cooperation with the LGI
• Produce press releases, brochures, flyers,
  posters, multimedia material, and keep updated
  the contents (related to advancements plus news
  and reports) of websites in order to inform
  appropriately about the technology and events
  organized by the LGI as well as related
  communities
• Organise workshops and conferences at local and
  national levels in cooperation with the LGI

39
Training dissemination tasks(detailed
description 2/2)

• The activities at the LGI level can be summarized
  as follows
• Definition of training plans. At least 2 travels
  per country and per year should be foreseen in
  order to support and define common training
  strategies with all participating countries
• Designing and maintaining a master event calendar
  scheduling the most important LGI training and
  dissemination events like Grid schools,
  conferences and workshops
• Maintenance of a central training material
  repository training material should also be
  translated from English to Spanish and Portuguese
  by this task the maintenance of a training
  database will be also provided by this task
• Deliver Training courses for trainers
• Offer consultancy to Application Developers when
  needed by the Application support Team.

40
User support tasks
41
User support tasks(detailed description)

• The activities that must be performed by the
  Application Support teams located either at the
  GOCs at the GSC include
• Answer tickets issued by the central support
  ticket system
• Participate in periodic virtual meetings with
  application developers and forward their request
  if necessary to the middleware developers
• Contribute for the creation of FAQs and
  gridification guidelines
• Visit or accepting visits of application experts
  in events similar to the current EELA-2
  "Gridification Weeks
• Participate as a tutor in the training events
  organized by the LGI
• Make available a public central database
  containing the profile and the current status of
  all applications supported by the GridLGI
• Act as a liaison between the GridLGI and new
  application communities or related projects
• Evaluate the technical requirements and the
  impact of new projects/applications willing to be
  supported by the GridLGI

42
LGI Management / Governance
43
Possible structure of a Latin American NGI
A strong liaison with the NREN is strongly
suggested/recommended. Local conditions
permitting, the NGI should be incorporated into
the NREN.
44
Long-term sustainability status (1/2)

• Long-term sustainability of e-Infrastructures
  world wide is key for a continued support of
  scientific communities
• EELA-2 is definitely contributing to the
  establishment of JRUs and NGIs in Latin America,
  in close synergy with NRENs and CLARA, also in
  countries not initially involved in the project
• EELA-2 has defined a detailed model and
  architecture of the Latin American Grid
  Initiative
• Contacts with CLARA have formally been
  established and a Joint Committee has been formed
  at the Bogotá Conference (with Spanish
  participation) to analyse, agree on, and start
  implementing the proposed model

45
Long-term sustainability status (2/2)

• Clear support of the former CLARA President (C.
  Casasus)

46
Whats round the corner? GISELA?

• No EELA-3 foreseen then
• Grid Initiatives for e-Science virtual
  communities in Europe and Latin America (GISELA)
• Objective of the proposal Following about 4
  years of successful activities in EELA and
  EELA-2, the objective of the GISELA proposal is
  to guarantee the long-term sustainability of the
  EU-LA e-Infrastructure to ensure the continuity
  and the future enhancement of the European
  Latin American Virtual Research Communities.
  Building on the EELA-2 e-Infrastructure, its
  associated Services and the numerous User
  Communities, the project will focus on two
  inter-related goals
• Implement the NGI / LGI sustainability model, as
  specified in DSA1.3 (http//documents.eu-eela.org/
  record/1119/files/), in association with CLARA
  and collaborating with EGI.
• Provide the Virtual Research Communities with the
  suited e-Infrastructure and Application-related
  Services required to improve the effectiveness of
  their research.

47
GISELA summary proposal 1/3

• Call Identifier FP7-INFRASTRUCTURES-2010-2
• URL http//cordis.europa.eu/fp7/dc/index.cfm?fuse
  actionUserSite.CapacitiesDetailsCallPagecall_id
  263infopack
• Funding Scheme CP - CSA

48
GISELA summary proposal 2/3

• Work Packages and their objectives (short
  summary)
• NA1 Administrative and Technical Management of
  the Project
• EELA-2 type of Administrative and Technical
  management
• International Cooperation
• NA2 Dissemination and Outreach
• Organise Dissemination events focusing on
  Decision Makers interaction to get their support
  for the LGI / NGI implementation
• Promote outreach actions (e.g. Conference talks,
  seminars, exhibitions, Virtual Days,..) to widely
  spread the usefulness of e-Infrastructures for
  different research communities
• NA3 Virtual Communities Support
• Identify and support Virtual Research Communities
  (VRC) by fostering e-Science collaboration among
  multidisciplinary groups
• Create and maintain support tools to ease the
  utilisation of the e-Infrastructure
  (Applications portals and database,
  self-training material, Users and developers
  Guide, Help Desk support etc.)
• Organise training activities tailored to each
  VCR needs to transfer knowledge concerning the
  e-Infrastructure utilisation
• Validate the services provided by SA1, SA2 and
  JRA1 by performing tests with pilot applications

49
GISELA summary proposal 3/3

• SA1 NGI / LGI Infrastructure Services (as in
  DSA1.3)
• Deploy the catch-all Grid Operation Centre (GOC)
  as defined in DSA1.3 to serve as model for future
  ROCs
• Implement the Grid Support Centre (GSC) as
  defined in DSA1.3
• Support Virtual Research Environment (VRE) tools,
  systems and processes
• SA2 NGI / LGI Network Services (as in DSA1.3)
• Keep managing Network resources via the existing
  national Network Operation Centres (NOC)
• Implement the Network Support Centre (NSC) as
  defined in EELA-2 DSA1.3
• Proceed with the deployment of performing
  Network monitoring in collaboration with EGI
• JRA1 Infrastructure Application-oriented
  Services for Virtual Communities
• Increase the reach, the attractiveness, the
  usability and the effectiveness of the EU-LA
  e-Infrastructure for well-established and new
  communities. For this goal, collaborate with EGI
  to divulge the Services developed in the EELA-2
  context and devise those new Services required by
  the User Communities

50
Why is it possible?

• The EELAs
• Trained users, tutors and administrators in LA
• Disseminated the Grid culture
• Approached Decision Makers
• Proposed a model for long-term sustainability of
  Grid infrastructures in LA
• Is currently looking for funds in EU (EC
  CIEMAT) to help implementing the model during the
  next 2 years
• Conclusion
• It should work!!!!!!!!!!!!!!!!!!!!!

51

• Thank you for your attention
• marechal_at_if.ufrj.br