GridPP Public Service Summit

1
GridPP delivering The UK Grid for Particle
Physics
Prof. David BrittonGridPP Project
leader University of Glasgow
RCUK e-Science Review 9th December 2009
2
Monday 23rd November, 2009
The LHC experiments saw first collisions data
was streamed off the detectors and distributed
to a world-wide network of computers.
and in preparation
--------- GridPP3 ---------
GridPP2
--------- GridPP2 ---------
--------- GridPP1 ---------
2001
2002
2003
2004
2005
2006
2007
2008
2009
From Web to Grid
From Prototype to Production
From Production to Exploitation
3
What is GridPP?
GridPP is an internationally recognised
collaboration of particle physicists and computer
scientists from 19 UK universities, STFC and
CERN, who have built a Grid for particle physics.
GridPP delivers the UK part of the Worldwide LHC
Computing Grid (wLCG), a global project that the
UK, with funding via GridPP, played a pivotal
role in creating. The LHC will produce
unprecedented amounts of data, expected to rise
beyond 10PB/year.
4
National Impact
UK Tier-2s Leverage University funding, Local
publicity

• GridPP has successfully developed and deployed a
  Grid across 19 sites in the UK with 20,000
  cores and 11 PB of storage.
• RAL Tier-1 Oxfordshire.
• ScotGrid Durham, Edinburgh, Glasgow.
• NorthGrid Lancaster, Liverpool, Manchester, and
  Sheffield.
• SouthGrid Birmingham, Bristol, Cambridge,
  Oxford, RAL PPD and Jet.
• London Brunel, IC, QMUL, RHUL, and UCL.
• This has changed the landscape of HEP computing
  in the UK, placing UK physicists at the forefront
  of research by allowing seamless access to local,
  national and international resources through the
  Grid paradigm.

New kit from Viglen at QMUL
UK Tier-2s Enabled development of new
University-based facilities
New machine room at Glasgow
Tier-2 15,000 CPUs, 3PB Disk
5
National Impact

• GridPPs impact has been broader than just the
  LHC experiments with support for other
  experimental collaborations (such as BaBar, D0,
  Mice, T2K, ILC) and for theoretical physics
  collaborations (such as UKQCD, and PhenoGrid).
• The national impact of GridPP goes beyond
  Particle Physics through the support of a wide
  range of Virtual Organisations from other
  disciplines in the UK through the European EGEE
  project.
• (Plots show normalised CPU time per VO, Jan-Sep
  2009).

All Regions Contribute
Supports more than LHC
Wide range of VOs
6
National Impact
European Framework-7 proposals
EGI.eu

• GridPP has complemented the development of the UK
  National Grid Service, by developing a
  larger-scale, but more specialised Grid. In so
  doing GridPP is a foundation stone for the future
  UK NGI, which will relate to the EU structures.
• Many GridPP trained staff have moved to external
  positions, contributing to other areas of
  science, business, teaching, and software
  development.

HEP SSC
EMI
CUE
UK NGI
GridPP
NGS
7
International Impact

• The UK, with funding via GridPP, played a pivotal
  role in initiating the Large Hadron Collider
  Computing Grid (LCG) project by funding 23 posts
  at CERN for three years.
• Also via GridPP, the UK played a large role in
  the EDG and EGEE projects (a set of FP6 and FP7
  European projects) that created much of the Grid
  middleware together with an overlapping European
  Grid infrastructure that embraces other
  disciplines in addition to HEP.

8
International Impact
Worldwide

• 56 Countries
• 295 Sites
• 180,000 CPUs

UK

• 21 Sites
• 20,000 CPUs

9
International Impact
New kit from Viglen at QMUL

• An international role for the RAL Tier-1 as one
  of 10 worldwide T1 centers.
• Demanding service level required for receiving
  and processing data flowing from the LHC.
• RAL has been central in developing Grid
  accounting for EGEE and wLCG.
• RAL hosts the GOCDB, developed by GridPP staff,
  that collects and records the configuration of
  all EGEE sites.

10
International Impact
Delivering Resources The UK Tier-1 and Tier-2s
provide a large resource contribution to EGEE and
wLCG.

• Delivering Expertise
• GridPP members play leading roles
• RAL started and chaired the European Group of
  Certification Authorities.
• RAL leads and chairs the Joint Security Policy
  Group and Security Vulnerabilities Group.
• GridPP members have Global responsibilities
  within the ATLAS and LHCb computing projects.
• The UK chairs the wLCG Grid Deployment Board and
  previously the EGEE Project Board.
• Members have worked on standardization in many
  areas in the international Open Grid Forum.

11
Wealth Creation

• GridPP has supported
• imense and iLexIR start-up companies at
  Cambridge (Camtology see below) with the
  GridPP/ATLAS/LHCb developed Ganga interface, and
  resources.
• Econophysica (mathamatical models for commodity
  trading) at QMUL, with access to resources.
• Total Oil (geoscience research) from Aberdeen,
  with help establishing a Virtual Organisation to
  run their applications on the EGEE Grid.
• GridPPs involvement in wLCG and EGEE has led
  directly to the formation of Constellation
  Technologies, a pioneer in Global Grid
  infrastructure software.

12
Wealth Creation
13
Quality of Life
The Grid allows rapid deployment of large-scale
resources to tackle topical problems. In 2006,
GridPP resources made the largest contribution to
a pan-european search for a cure for bird-flu.
14
Quality of Life
Search for a cure for Malaria In 2007, the
WISDOM docked over 41 million compounds in just
six weeks, the equivalent of 80 years work for a
single PC. The WISDOM team identified some 5000
interesting compounds, from which they found
three families of molecules that could be
effective against the malaria parasite. The UK
(GridPP) contributed almost half of the total
computing resources.
15
Impact on e-Science
GridPPs key impact on e-science is to
demonstrate the potential of a Grid for science
across multiple disciplines. GridPP has built
and runs a Grid in a complex, data-intensive
environment at a scale well above anything
previously done. A Grid infrastructure now
exists that can run tens of millions of jobs
annually.
16
Impact on e-Science

• In June 2009 a world-wide full-scale test of the
  computing for LHC data-taking was performed
  (STEP09)
• The UK provided over 2 million hours of CPU
  time in 2 weeks.
• ATLAS data reprocessing recalled more than
  110TB of data from tape (7x higher than nominal
  rate for 2010 and the best performance
  world-wide).
• CMS data was imported to RAL at 100 MB/s and
  achieved an aggregated recall from tape of 180
  MB/s.
• The UK contributed 32 of the Global CPU time
  for the LHCb experiment.
• The Glasgow Tier-2 ran more analysis jobs than
  any of the other 60 sites world-wide.
• STEP09 was a notable success for the UK.
  Worldwide, the wLCG Grid demonstrated
  groundbreaking performance.

LHCb CPU contributions
Jobs at RAL
ATLAS reprocessing jobs.
17
Added Value

• The GridPP project added value by
• Creating and running a Grid that allows
  resources to be shared locally, nationally, and
  internationally.
• By supporting the development of Ganga, the
  RealTime Monitor, GridSite, R-GMA and many other
  utilities to some degree.
• Supporting up to 40 Virtual Organisations across
  many disciplines allowing the opportunistic use
  of available resources and an opportunity to
  develop experience with the Grid environment.
• Enabling commercial interests to obtain
  experience with the Grid and gain the use of
  significant computational resources to develop
  products.
• By driving technology, such as light-path
  networking and low-cost, high-performance
  computational resources.
• By developing innovative management tools that
  were used by other projects.
• By operating in a cutting-edge, complex
  environment that attracted funding for a study by
  the Pegasus project at the London School of
  Economics.

18
Panel Questions

• Please provide any available data, or at least
  anecdotes, about how the e-science program to
  date has in fact enhanced collaboration in you
  area of research.
•  
• The LHC experiments require collaboration on a
  global scale to tackle the volume and complexity
  of the data from the LHC. The worldwide LHC
  Computing Grid, of which GridPP represents
  10-15, is an essential infrastructure, without
  which the LHC data could not be analysed. In
  essence, the e-infrastructure has realized the
  goal of global collaboration in High Energy
  Physics at a scale that could not be supported by
  the traditional computer centre.
• Enhanced collaboration has been observed
• within the regional groups of universities
  making up the distributed Tier-2s, driven by
  GridPP organisation and funding
• between the Tier-2s and with the UK Tier-1,
  driven by the GridPP deployment team and support
  structure and by sharing the UK contribution to
  WLCG and EGEE.
• between the UK and other countries both within
  the WLCG and EGEE projects. The coordination of
  GridPP has allowed the UK to present a credible,
  organized and common face to these international
  projects
•  

19
Panel Questions
What evidence (hard data or anecdotes) of data
sharing has occurred in your e-science project?
  The STEP09 exercise in May 2009 exercised the
worldwide LHC Computing Grid at full scale and
across all areas. Data were shipped out from CERN
via the OPN optical network to the ten Tier-1
centers across the world. There, it was
reconstructed, archived, restored, and derived
data sets made available to the network of Tier-2
centers. At the latter, the data were analysed
and physics quantities extracted. This exercise
demonstrated that the infrastructure works at,
and in many areas, beyond, the levels required
for data taking in 2010 and demonstrates both the
divide-and-conquer philosophy of the Grid, and
the intrinsic sharing of physics data across the
globe.  
20
Panel Questions
What evidence of sharing mechanisms or services
has occurred in your e-science project?
  Through e-science funding of GridPP, the UK
has been a major contributor to both wLCC and
EGEE. In the year to July 2009, GridPP provided
17 of EGEEs computer time and contributed
nearly 85 million CPU hours (KSI2K hours) to more
than 40 EGEE virtual organizations across a wide
range of disciplines. Of this, approaching 70
million CPU hours were provided to LHC VOs, 14
of WLCGs worldwide total.   The entire wLCG
infrastructure is a mechanism and service for
sharing data between members of a recognized
collaboration. Collaborations are recognized via
the Virtual Organisation Management System (VOMS)
that verifies the membership and rights of
individual. Higher-level tools are provided to
help locate and move data (the LHC file
catalogue, LFC, and the File Transfer Service,
FTS). GridPPs production grid includes a variety
of services that are shared by all UK institutes
and users, such as the BDII Information Service
Workload Management Services and Nagios
monitoring.  
21
Panel Questions
What software have you built that has been used
outside your project?   GANGA User interface
and job submitter used extensively by HEP,
other disciplines and commercial users. Described
in later talks. APEL accounting used to
populate the GOCDB with information on all sites
in EGEE. GridSite - a certificate-based website
management system, that allows members of an
organisation to collaborate in maintaining web
pages etc. Used to create a dozen or so external
(to HEP) websites. The RealTimeMonitor high
profile tool produced by GridPP used throughout
the world to demonstrate the Grid in
action. R-GMA an information and monitoring
tool developed at RAL, which was been deployed in
LCG middleware and gLite. Contributions to many
others. 
22
Panel Questions
What disciplines have been involved in your
project? Describe their interaction. GridPP is
fundamentally a collaboration between Particle
Physicists and Computer Scientists. The PEGASUS
project based at LSE studied GridPP over a number
of years involved a lot of interaction
(observation interviews talks). GridPP
supports over 40 Virtual Organisations covering a
wide range of scientific disciplines.
23
Panel Questions
What major investments made by the e-science core
program and/or JISC (such as the National Grid
Service) have you used in your research? GridPP
is dependent on the academic network hosted by
JANET-UK and the dedicated OPN optical network to
CERN. GridPP developed the original UK
Certificate Authority and then passed this over
to the NGS/JISC to run.
24
Panel Questions

• What has your project contributed back to the
  larger research community?
• Leadership roles (in ATLAS, LHCb, wLCG, EGEE,
  international security)
• Software (already described).
• The UK CA just mentioned.
• Trained staff who have moved on into the wider
  world.
• Standardisation work in the international OGF
  context (eg information schema).
• Resources for 40 VOs.
• Driven technology pushed agenda for lightpath
  service in the UK (UKLIGHT project).

25
Panel Questions
What has been the major impact of the e-science
program to date on your research and allied
education program? The e-science programme has
enabled the UK to participate in the worldwide
e-infrastructure for LHC computing without the
e-Science program, UK physicists would not be in
a position to exploit the data expected shortly
from the LHC.
26
Panel Questions
What is your vision for e-science going into the
future? Stress what is most critical to the
future of your and/or your communities research
aspirations? See next slide.
27
Conclusion
GridPP has achieved everything that it set out to
do, but this is the start, not the end point The
Grid will have to serve the UK particle physics
community and others, for the next 15-20 years.
It will need to evolve to deal with increasing
demands (e.g. SuperLHC luminosities), cope with
new discoveries (e.g. Higgs), and respond to new
challenges (e.g. ILC). It will need to
incorporate new technology (e.g. many-core
processors, GPUs), adapt to new structures (e.g.
the UK NGI), and service a growing number of
communities without losing focus. What is
needed is the sort of forward-thinking vision of
Sir John Taylor in 2001 accompanied by a plan for
long-term support.