GridPP Public Service Summit - PowerPoint PPT Presentation

1 / 26
About This Presentation

GridPP Public Service Summit


GridSite - a certificate-based website management system, that allows members of ... Stress what is most critical to the future of your and/or your communities ... – PowerPoint PPT presentation

Number of Views:25
Avg rating:3.0/5.0
Slides: 27
Provided by: steve555


Transcript and Presenter's Notes

Title: GridPP Public Service Summit

GridPP delivering The UK Grid for Particle
Prof. David BrittonGridPP Project
leader University of Glasgow
RCUK e-Science Review 9th December 2009
Monday 23rd November, 2009
The LHC experiments saw first collisions data
was streamed off the detectors and distributed
to a world-wide network of computers.
and in preparation
--------- GridPP3 ---------
--------- GridPP2 ---------
--------- GridPP1 ---------
From Web to Grid
From Prototype to Production
From Production to Exploitation
What is GridPP?
GridPP is an internationally recognised
collaboration of particle physicists and computer
scientists from 19 UK universities, STFC and
CERN, who have built a Grid for particle physics.
GridPP delivers the UK part of the Worldwide LHC
Computing Grid (wLCG), a global project that the
UK, with funding via GridPP, played a pivotal
role in creating. The LHC will produce
unprecedented amounts of data, expected to rise
beyond 10PB/year.
National Impact
UK Tier-2s Leverage University funding, Local
  • GridPP has successfully developed and deployed a
    Grid across 19 sites in the UK with 20,000
    cores and 11 PB of storage.
  • RAL Tier-1 Oxfordshire.
  • ScotGrid Durham, Edinburgh, Glasgow.
  • NorthGrid Lancaster, Liverpool, Manchester, and
  • SouthGrid Birmingham, Bristol, Cambridge,
    Oxford, RAL PPD and Jet.
  • London Brunel, IC, QMUL, RHUL, and UCL.
  • This has changed the landscape of HEP computing
    in the UK, placing UK physicists at the forefront
    of research by allowing seamless access to local,
    national and international resources through the
    Grid paradigm.

New kit from Viglen at QMUL
UK Tier-2s Enabled development of new
University-based facilities
New machine room at Glasgow
Tier-2 15,000 CPUs, 3PB Disk
National Impact
  • GridPPs impact has been broader than just the
    LHC experiments with support for other
    experimental collaborations (such as BaBar, D0,
    Mice, T2K, ILC) and for theoretical physics
    collaborations (such as UKQCD, and PhenoGrid).
  • The national impact of GridPP goes beyond
    Particle Physics through the support of a wide
    range of Virtual Organisations from other
    disciplines in the UK through the European EGEE
  • (Plots show normalised CPU time per VO, Jan-Sep

All Regions Contribute
Supports more than LHC
Wide range of VOs
National Impact
European Framework-7 proposals
  • GridPP has complemented the development of the UK
    National Grid Service, by developing a
    larger-scale, but more specialised Grid. In so
    doing GridPP is a foundation stone for the future
    UK NGI, which will relate to the EU structures.
  • Many GridPP trained staff have moved to external
    positions, contributing to other areas of
    science, business, teaching, and software

International Impact
  • The UK, with funding via GridPP, played a pivotal
    role in initiating the Large Hadron Collider
    Computing Grid (LCG) project by funding 23 posts
    at CERN for three years.
  • Also via GridPP, the UK played a large role in
    the EDG and EGEE projects (a set of FP6 and FP7
    European projects) that created much of the Grid
    middleware together with an overlapping European
    Grid infrastructure that embraces other
    disciplines in addition to HEP.

International Impact
  • 56 Countries
  • 295 Sites
  • 180,000 CPUs

  • 21 Sites
  • 20,000 CPUs

International Impact
New kit from Viglen at QMUL
  • An international role for the RAL Tier-1 as one
    of 10 worldwide T1 centers.
  • Demanding service level required for receiving
    and processing data flowing from the LHC.
  • RAL has been central in developing Grid
    accounting for EGEE and wLCG.
  • RAL hosts the GOCDB, developed by GridPP staff,
    that collects and records the configuration of
    all EGEE sites.

International Impact
Delivering Resources The UK Tier-1 and Tier-2s
provide a large resource contribution to EGEE and
  • Delivering Expertise
  • GridPP members play leading roles
  • RAL started and chaired the European Group of
    Certification Authorities.
  • RAL leads and chairs the Joint Security Policy
    Group and Security Vulnerabilities Group.
  • GridPP members have Global responsibilities
    within the ATLAS and LHCb computing projects.
  • The UK chairs the wLCG Grid Deployment Board and
    previously the EGEE Project Board.
  • Members have worked on standardization in many
    areas in the international Open Grid Forum.

Wealth Creation
  • GridPP has supported
  • imense and iLexIR start-up companies at
    Cambridge (Camtology see below) with the
    GridPP/ATLAS/LHCb developed Ganga interface, and
  • Econophysica (mathamatical models for commodity
    trading) at QMUL, with access to resources.
  • Total Oil (geoscience research) from Aberdeen,
    with help establishing a Virtual Organisation to
    run their applications on the EGEE Grid.
  • GridPPs involvement in wLCG and EGEE has led
    directly to the formation of Constellation
    Technologies, a pioneer in Global Grid
    infrastructure software.

Wealth Creation
Quality of Life
The Grid allows rapid deployment of large-scale
resources to tackle topical problems. In 2006,
GridPP resources made the largest contribution to
a pan-european search for a cure for bird-flu.
Quality of Life
Search for a cure for Malaria In 2007, the
WISDOM docked over 41 million compounds in just
six weeks, the equivalent of 80 years work for a
single PC. The WISDOM team identified some 5000
interesting compounds, from which they found
three families of molecules that could be
effective against the malaria parasite. The UK
(GridPP) contributed almost half of the total
computing resources.
Impact on e-Science
GridPPs key impact on e-science is to
demonstrate the potential of a Grid for science
across multiple disciplines. GridPP has built
and runs a Grid in a complex, data-intensive
environment at a scale well above anything
previously done. A Grid infrastructure now
exists that can run tens of millions of jobs
Impact on e-Science
  • In June 2009 a world-wide full-scale test of the
    computing for LHC data-taking was performed
  • The UK provided over 2 million hours of CPU
    time in 2 weeks.
  • ATLAS data reprocessing recalled more than
    110TB of data from tape (7x higher than nominal
    rate for 2010 and the best performance
  • CMS data was imported to RAL at 100 MB/s and
    achieved an aggregated recall from tape of 180
  • The UK contributed 32 of the Global CPU time
    for the LHCb experiment.
  • The Glasgow Tier-2 ran more analysis jobs than
    any of the other 60 sites world-wide.
  • STEP09 was a notable success for the UK.
    Worldwide, the wLCG Grid demonstrated
    groundbreaking performance.

LHCb CPU contributions
Jobs at RAL
ATLAS reprocessing jobs.
Added Value
  • The GridPP project added value by
  • Creating and running a Grid that allows
    resources to be shared locally, nationally, and
  • By supporting the development of Ganga, the
    RealTime Monitor, GridSite, R-GMA and many other
    utilities to some degree.
  • Supporting up to 40 Virtual Organisations across
    many disciplines allowing the opportunistic use
    of available resources and an opportunity to
    develop experience with the Grid environment.
  • Enabling commercial interests to obtain
    experience with the Grid and gain the use of
    significant computational resources to develop
  • By driving technology, such as light-path
    networking and low-cost, high-performance
    computational resources.
  • By developing innovative management tools that
    were used by other projects.
  • By operating in a cutting-edge, complex
    environment that attracted funding for a study by
    the Pegasus project at the London School of

Panel Questions
  • Please provide any available data, or at least
    anecdotes, about how the e-science program to
    date has in fact enhanced collaboration in you
    area of research.
  • The LHC experiments require collaboration on a
    global scale to tackle the volume and complexity
    of the data from the LHC. The worldwide LHC
    Computing Grid, of which GridPP represents
    10-15, is an essential infrastructure, without
    which the LHC data could not be analysed. In
    essence, the e-infrastructure has realized the
    goal of global collaboration in High Energy
    Physics at a scale that could not be supported by
    the traditional computer centre.
  • Enhanced collaboration has been observed
  • within the regional groups of universities
    making up the distributed Tier-2s, driven by
    GridPP organisation and funding
  • between the Tier-2s and with the UK Tier-1,
    driven by the GridPP deployment team and support
    structure and by sharing the UK contribution to
    WLCG and EGEE.
  • between the UK and other countries both within
    the WLCG and EGEE projects. The coordination of
    GridPP has allowed the UK to present a credible,
    organized and common face to these international

Panel Questions
What evidence (hard data or anecdotes) of data
sharing has occurred in your e-science project?
  The STEP09 exercise in May 2009 exercised the
worldwide LHC Computing Grid at full scale and
across all areas. Data were shipped out from CERN
via the OPN optical network to the ten Tier-1
centers across the world. There, it was
reconstructed, archived, restored, and derived
data sets made available to the network of Tier-2
centers. At the latter, the data were analysed
and physics quantities extracted. This exercise
demonstrated that the infrastructure works at,
and in many areas, beyond, the levels required
for data taking in 2010 and demonstrates both the
divide-and-conquer philosophy of the Grid, and
the intrinsic sharing of physics data across the
Panel Questions
What evidence of sharing mechanisms or services
has occurred in your e-science project?
  Through e-science funding of GridPP, the UK
has been a major contributor to both wLCC and
EGEE. In the year to July 2009, GridPP provided
17 of EGEEs computer time and contributed
nearly 85 million CPU hours (KSI2K hours) to more
than 40 EGEE virtual organizations across a wide
range of disciplines. Of this, approaching 70
million CPU hours were provided to LHC VOs, 14
of WLCGs worldwide total.   The entire wLCG
infrastructure is a mechanism and service for
sharing data between members of a recognized
collaboration. Collaborations are recognized via
the Virtual Organisation Management System (VOMS)
that verifies the membership and rights of
individual. Higher-level tools are provided to
help locate and move data (the LHC file
catalogue, LFC, and the File Transfer Service,
FTS). GridPPs production grid includes a variety
of services that are shared by all UK institutes
and users, such as the BDII Information Service
Workload Management Services and Nagios
Panel Questions
What software have you built that has been used
outside your project?   GANGA User interface
and job submitter used extensively by HEP,
other disciplines and commercial users. Described
in later talks. APEL accounting used to
populate the GOCDB with information on all sites
in EGEE. GridSite - a certificate-based website
management system, that allows members of an
organisation to collaborate in maintaining web
pages etc. Used to create a dozen or so external
(to HEP) websites. The RealTimeMonitor high
profile tool produced by GridPP used throughout
the world to demonstrate the Grid in
action. R-GMA an information and monitoring
tool developed at RAL, which was been deployed in
LCG middleware and gLite. Contributions to many
Panel Questions
What disciplines have been involved in your
project? Describe their interaction. GridPP is
fundamentally a collaboration between Particle
Physicists and Computer Scientists. The PEGASUS
project based at LSE studied GridPP over a number
of years involved a lot of interaction
(observation interviews talks). GridPP
supports over 40 Virtual Organisations covering a
wide range of scientific disciplines.
Panel Questions
What major investments made by the e-science core
program and/or JISC (such as the National Grid
Service) have you used in your research? GridPP
is dependent on the academic network hosted by
JANET-UK and the dedicated OPN optical network to
CERN. GridPP developed the original UK
Certificate Authority and then passed this over
to the NGS/JISC to run.
Panel Questions
  • What has your project contributed back to the
    larger research community?
  • Leadership roles (in ATLAS, LHCb, wLCG, EGEE,
    international security)
  • Software (already described).
  • The UK CA just mentioned.
  • Trained staff who have moved on into the wider
  • Standardisation work in the international OGF
    context (eg information schema).
  • Resources for 40 VOs.
  • Driven technology pushed agenda for lightpath
    service in the UK (UKLIGHT project).

Panel Questions
What has been the major impact of the e-science
program to date on your research and allied
education program? The e-science programme has
enabled the UK to participate in the worldwide
e-infrastructure for LHC computing without the
e-Science program, UK physicists would not be in
a position to exploit the data expected shortly
from the LHC.
Panel Questions
What is your vision for e-science going into the
future? Stress what is most critical to the
future of your and/or your communities research
aspirations? See next slide.
GridPP has achieved everything that it set out to
do, but this is the start, not the end point The
Grid will have to serve the UK particle physics
community and others, for the next 15-20 years.
It will need to evolve to deal with increasing
demands (e.g. SuperLHC luminosities), cope with
new discoveries (e.g. Higgs), and respond to new
challenges (e.g. ILC). It will need to
incorporate new technology (e.g. many-core
processors, GPUs), adapt to new structures (e.g.
the UK NGI), and service a growing number of
communities without losing focus. What is
needed is the sort of forward-thinking vision of
Sir John Taylor in 2001 accompanied by a plan for
long-term support.
Write a Comment
User Comments (0)