title'open revolution execute

1
GridPP Collaboration Meeting 14 Feb 2002
Tony Doyle
2
GridPP Status Report

• Starting Point
• Project Management Structures
• Experiments Board (EB)
• CERN and the LHC Computing Grid (LCG)
• Technical Board (TB)
• Project Management Board

• High Level Deliverables
• Staff Effort and Recruitment
• DataGrid
• Inter Grids
• 2002 Challenges
• UK Testbed The Grid
• Summary
• Agenda

3
GridPP Management
Whos who?.. c.f. Steves web list of contacts
4
Collaboration Board

• Chair Steve Lloyd
• Birmingham Peter Watkins Liverpool Themis
  Bowcock
• Bristol Greg Heath Manchester Robin Marshall
• Brunel Peter Hobson Oxford Susan Cooper
• CERN Les Robertson Queen Mary Tony Carter
• Cambridge Janet Carter Royal Holloway John
  Strong
• Durham Mike Whalley RAL Ken Peach
• Edinburgh Steve Playfer Sheffield Neil Spooner
• Glasgow David Saxon Sussex David Wark
• IC Peter Dornan Swansea Chris Alton
• Lancaster Peter Ratoff UCL Peter Clarke
• Project Leader (in attendance) Tony Doyle
• e-Science Director (in attendance) Neil Geddes
• Members are 'Group Leaders' of all GridPP
  Institutes
• Chair elected by Board

5
Experiments Board

• ATLAS Roger Jones, Andy Parker
• CMS Dave Britton, Dave Newbold
• LHCb Nick Brook (Chair), Glenn Patrick
• ALICE Ian Bloodworth
• BaBar Roger Barlow (Deputy Chair)
• D0 Iain Bertram
• CDF Todd Huffman
• Others Dan Tovey
• Theory Alan Irving
• CERN Jamie Shiers
• TB Cross Members Pete Clarke, tbd
• Members chosen by Experiments
• Chair and Deputy elected by Board

6
Recruitment Timetable

• 8th October - EB input deadline
• Inputs
• 1. TECHNICAL CASE Complete
• 2. ASSESMENT OF HOW MUCH EFFORT IS REQUIRED
  Complete
• 3. WHO WILL CONTRIBUTE TO/LEAD THESE DEVELOPMENTS
  Complete
• 4. WHAT IS REMAINING TO BE DONE - I.E. A POSTS
  REQUIREMENT Complete
• 5. A JOB SPECIFICATION OF THE OUTSTANDING POST(S)
  INCLUDING DELIVERABLES AND MILESTONES
  Complete
• 19th October - TAG assessment document
• Next step - technical deployment issues
• 26th October - PMB definition of tasks
• Next step - deployment/resource issues
• first review of CERN posts
• 2nd November - Tender document
• 5-6th November - Collaboration Meeting
• 30th November- Deadline for submission by
  Institutes
• 7th December - PRSC mtg.
• 14th December- Oversight Committee mtg.

Ambitious timescales. Experiments Input? good
to very good
7
Experiments Board
Nick Brook

• New phase for EB moving from focussing on post
  creation to the need for exploitation and
  utilisation of GridPP resources
• Remit
• To ensure that the GridPP project is
  experimentally-driven
• Long term planning assessment of needs and
  reqts from GridPP
• Dissemination of experiences and information
  between UK collaborations
• To facilitate close working relationship between
  expts.
• Overall liaison between the expts GridPP

8
EB Meetings
Nick Brook

• Next meeting
• Late March, after LCG startup workshop
• Areas of discussion
• status of recruitment
• status experience of Grid work
• long term plans (changes to original submissions)
• reporting procedures to GridPP
• testbed experiences
• management "booking" of  UK Grid resources
• quarterly reporting to TB

9
CERN Status
Tony Cass

• The LCG Project is up and running
• PEB, SC2 and POB have all met.
• Launch Workshop, 11th-15th March
• Hardware for 2002 Prototype is available
• Scheduling ongoing.
• RTAGs launched for Persistency, Managing
  Software, Maths Libraries and Grid Requirements.
• Initial report from Persistency and Managing
  Software RTAGs at launch workshop.
• RTAG reports will determine assignments for
  GridPP staff in relevant areas.
• Fabric Management Activity Plan being prepared
• Similarly, this will determine assignments for
  GridPP staff working on Fabric Management.

10
CERN concerns
Tony Cass

• Ensuring compatibility between EU and US testbeds
• Understanding impact if first physics moves to
  2007
• On Production system
• On LCG Phase I
• Scheduling of the Prototype
• Understand needs and types of challenge - pushing
  the envelope or mass production for LHCC
  milestones
• Fitting these in resources available
• Tight schedule for new tape drives for ALICE
• And we need to support running experiments in a
  reduced budget
• Scheduling to allow test of Grid tools across
  multiple sites.

11
Technical Board
Pete Clarke

• Composition to date (reflects dynamic nature)
• Chair Pete Clarke
• Deputy Robin Middleton
• WP1 Dave Colling
• WP2 Gavin McCance
• WP3 Steve Fisher
• WP4 Lex Holt/Paul Anderson
• WP5 John Gordon
• WP6 Andrew McNab
• WP7 Richard Hughes-Jones
• WP8 Frank Harris
• Middleware Alex Martin
• Experiments Steve Burke
• Security Dave Kelsey
• EB Nick Brook

12
TB Meetings
Pete Clarke

• December
• adoption of DataGRID reporting mechanisms for
  WP1-7 (3-monthly reports)
• Jan 9th
• EDG Post quarterly submission review of
  delivered resources
• EDG Review of UK deliverable responsibilities
• Gantt chart testbed functionality
• Testbed rollout planning first discussion
• Application builders workshop planning
• adoption of TB-SUPPORT_at_JISCMAIL.AC.UK
• Feb 8th
• Detailed discussion of Testbed rollout planning
  gt SEE TOMORROW
• First discussion of non-LHC exp support
• GridPP DEVELOPMENT Grid

13
Resource Monitoring
Dave Britton
Next three-monthly reports in March (18th)..

• Input 3-monthly report
• A work-segment is a structure something like
• Quarter, Person, Institute, Experiment, Task,
  StaffMonths
• for example
• 2002Q1, P.Mealor, UCL, NONE, T7.2, 3.00
• Extract information such as
• 1) Total delivered effort as a function of time.
• 2) Effort as a function of task or workgroup.
• 3) Effort as a function of Institute or
  Experiment (as a function of time).
• 4) Histograms of any of these quanitites (e.g.
  effort vs workgroup or Institute or experiment).

• Technically - Input into Excel (visual basic
  macros)
• Need to integrate into project management tools
• Need to match to high level goals

14
TB Documentation
Pete Clarke

• EDG resource management document produced and
  circulated
• WP1-7 mechanisms now in place and being used
  usefully
• Applications posts monitoring mechanisms now
  being reviewed
• Testbed rollout planning and support document in
  preparation
• iterated within TB
• next step is circulation to EB and WPs affected
  (WP6) for comment
• will then be circulated to all for comment
• to be outlined by PC tomorrow
• Note TB-WWW pages being fixed now

15
TB Meetings
Pete Clarke

• Application Builders meeting 31st Jan _at_ RAL
• aim was how to use middleware talks where
  appropriate (e.g. WP1)
• in addition overview of what other middleware
  does
• All future meetings will be as requested by
  GridPP (i.e. the audience)
• Will probably only cover one theme
• Next planned is half day (or more) on use of Job
  submission system
• We will attempt to make the format more training
  oriented(perhaps use RAL or IC training rooms)
• Will circulate request for interest (6 weeks
  from now)
• Use other meetings, where appropriate e.g. RAL
  (SRB) and NeSC e-Science Institute (Globus)..
• But try to minimise..

16
(No Transcript)
17
Project Management Board

• Project Leader Tony Doyle Appointed by CB
• Deputy Project Leader John Gordon Appointed by
  Project Leader
• Chair of Collaboration Board Steve Lloyd Ex
  officio
• Chair of Technical Board Peter Clarke Appointed
  by Project Leader
• Chair of Experiments Board Nick Brook Ex officio
• Deputy Chair of EB Roger Barlow Ex officio
• Resource Manager Dave Britton Appointed by
  Project Leader
• DataGrid UK Project Leader Robin Middleton Ex
  officio
• CERN Liaison Tony Cass Appointed by CERN
• e-Science Director Neil Geddes Ex officio
• As proposal plus Resource Manager DataGrid UK
  Project Leader Part time Secretary at Glasgow
• Three year posts (except EB 12)
• Meets weekly via VC and monthly face to face

18
High Level Deliverables
To be re-defined to extend to The Grid i.e.
testbed Aspiration all that you expect from a
central service? Probably not.. Tomorrows
Discussion..
19
High Level Deliverables
Requires input from WP contacts
20
High Level Deliverables
Pre-defined at high level, but requires
further iteration
21
Projected Staff Effort SY

• Area GridPP _at_CERN CS
• WP1 Workload Management 0.5 IC 2.0 IC
• WP2 Data Management 1.5 Ggo 1.0 Oxf WP12
• WP3 Monitoring Services 5.0 RAL, QMW 1.0
  HW
• Security RAL 1.0 Oxf
• WP4 Fabric Management 1.5 Edin., Lpool
• WP5 Mass Storage 3.5 RAL,
  Lpool
• WP6 Integration Testbed 5.0 RAL/Mcr/IC/Brist
  ol
• WP7 Network Services 2.0 UCL/Mcr 1.0 UCL
• WP8 Applications 17.0
• ATLAS/LHCb (Gaudi/Athena) 6.5 Oxf, Cam,
  RHUL, Bham, RAL
• CMS 3.0 IC, Bristol, Brunel
• CDF/D0 (SAM) 4.0 IC, Ggo, Oxf, Lanc
• BaBar 2.5 IC, Mcr, Bristol
• UKQCD 1.0 Edin.
• Total 36.0 10.0 -gt20.0? 6.0

Resource Management
22
DataGrid Recruitment Status
Robin Middleton
Dec 01
Tools being developed
23
DataGrid-UK Status
Robin Middleton

• Recruitment
• EU funded posts filled in post (1 additional
  from 1/2/02)
• All but 2 of 15 unfunded posts active
• Reporting
• UK templates defined, with TB meetings before
  after quarterly submissions
• 81 of effort contracted delivered (but not
  uniform across WPs) for WP1-7 (WP8 hard to
  quantify)
• UK Progress (in 2001)
• Architectures, Testbed-1, network monitoring,
  certificates security, R-GMA, LCFG, MDS
  deployment, Spitfire

24
DataGrid-UK Effort
Robin Middleton
25
DataGrid-UK Status
Robin Middleton

• EU Review
• All UK deliverables passed EDG internal review
  and now submitted to EU
• Review at CERN on 1st March
• 3 external reviewers
• 40 deliverables (inc. quarterly reports)
• Summary presentations only
• Can expect detailed feedback of areas to address
• 4th Project Conference Paris 4th-8th March
• Plans for 2002 TB2
• Absorb initial feedback from EU review

26
DataGrid-UK News
Robin Middleton

• Licensing is still an issue
• ATF reconstituted with more Applications
  involvement
• New project web site at original address
• Based at CERN
• Uses EDMS for document handling
• Testbed (see elsewhere at meeting)
• Minor releases every 2 months
• V2 in September
• Transatlantic testbeds coming closer to reality
• Scope being clarified
• Interoperability a major issue ? architectures

27
InterGrid etc
Robin Middleton

• Strictly HENP-InterGrid
• HICB meeting on 17th Feb Sunday (prior to GGF)
• HIJTB meet monthly by phone
• Projects
• BaBar testbed
• GRIT Grid Reference International Testbed
• ATLAS CMS
• DataTAG started
• DataTAG WP2QoS, etc WP4m/w interoperability
• Collaborates with iVDGL
• CrossGrid, GridStart

28
GRIT
Ruth Pordes
29
The Grid is an evolving concept

• enabler for transient virtual organisations
• anatomy a software infrastructure that enables
  flexible, secure, co-ordinated resource sharing
  among dynamic collections of individuals,
  institutions and resources
• Foster, Kesselman Tuecke (2001)
• evolution of and integration with web services
• physiology everything is a Grid service i.e. a
  service that conforms to a set of conventions for
  management and exchanging messages
• Foster, Kesselman, Nick Tuecke (2002)
• http//www.globus.org/research/papers/ogsa.pdf
• Global Grid Forum define a standard Grid
  architecture
• big business and big science starting to work
  together
• GGF4 (Toronto) next week

30
GGF Architecture

• 1) scope the problem - what are we trying to
  produce
• 2) decide on the require functionality
• 3) modularize the functionality
• 4) decide on the relationships among the modules
• 5) identify the data that must flow among the
  modules
• 6) decide what state has to be maintained, and
  where
• 7) design the protocols for communicating among
  and with the modules
• 8) APIs
• The challenge in the Grid forum is
• - picked a large and amorphous problem (wide area
  distributed computing)
• - for which there is not agreement (yet) as to
  what all of the functionality should be
• - and even where we think that we understand the
  functionality requirements (e.g. data and
  resource management) there is not yet consensus
  on the modules
• ANSWERS??
• DataGrid is The Architecture we have adopted
  - look here first

31
e-Science and Grid

• We are asked to map onto this plane
• Recognisable words, focussing on middleware
• DataGridmiddleware
• (where SAM, SLAC-based developments)
• ExperimentsApplication Area
• MiddlewareApplication Area??
• Draft 2.0 status
• Response should be built up from technical
  definitions

32
2002 Challenges

• Complete rollout of TB-1 and plan future upgrades
• Reconvened ATF to work closely with applications
• Make TB-2 a success
• Deploy and exploit Tier-1/A
• Applications to make good use of testbeds
• Solve interoperability issues
• We are part of many larger collaborations/structur
  es/groupings - we need to collaborate/discuss
  engage here, and
• Focus on implementation in the UK this will tell
  us what works (and what doesnt) at any given
  point.

33
Status 13/2/02 Requires upgrades to
functionality (extract hosts from info.
services automatically) and maintainability (integ
rate into testbed workplan)
Technical Management See tomorrows discussion
34
Status 13/2/02 Expect that Experiments will
deliver production monitoring systems e.g.
AliEn for ALICE (integrate into testbed
workplan)
35
Guidelines and Policies

• Disk/CPU resources allocated to each group
• Grid is based on distributed resources - a
  group is an experiment
• An institute is typically involved in many
  experiments
• Institutes define priorities on computing
  resources via OPEN policy statements
• e.g. Edinburgh/Glasgow RAL
• ScotGrid - Priority 1 ATLAS, LHCb, Computing
  Science Data Mining BaBar (2/3-1/2-1/3) LHC
  (1/3-1/2-2/3) tomorrows
• - Priority 2 BaBar, CDF,
  UKQCD e-Science discussions
• BioInformatics
• All jobs submitted via Globus authentication -
  Certificates identified by user and experiment
• Need to implement priority
• Minimum amount of data to deliver at a time for a
  job?
• Where to store files?
• Which data access/storing activities have the
  highest priority?
• Sharing of the resources among groups?
• Users belong to multiple groups?
• How many jobs per group are allowed?
• What processing activities are allowed at each
  site?
• To which sites should data access and processing
  activities be sent?

Its all in the small print.
Implemented by site administrators, with
OPEN policies defined at each site based on
e.g. case to funding authority Whats new?
Ability to monitor/allocate unused resources
We will be judged by how well we work as a
Virtual Organisation
36
Summary

• Most effort so far directed towards
  DataGridmiddleware
• Significant UK contributions
• Startup difficulties largely overcome
• Recruitment, late filling of posts funded
  unfunded
• Lost ground being recovered..
• All WP with UK involvement active contributing
• Testbed-1 so far modest, predictable teething
  problems (many !)
• Next major challenges focus on INTEGRATION
• US experiments programme - planning for
  today/tomorrow..
• LCG programme - planning for 2007...
• Testbed (focus for tomorrows discussions) -
  requires input from system managers as well as
  software developers.
• Positioned reasonably well 6 months into
  GridPP..
• with many challenges ahead...

37
Closing thought 14/2/02
With many hidden words, meanings,
nuances communication is the answer to the
problem (apparently)
Grids are everywhere..
38
3rd Collab Meeting Agenda

• Feb 14th
• Opening Session (Chair - Andy Parker)
• 1100-1130 Welcome and Background - Steve Lloyd
• 1130-1200 GridPP Project Status - Tony Doyle
• 1200-1230 LHC Computing Grid Project Status -
  Tony Cass
• Middleware Status Reports and Development (Chair
  - Tony Doyle)
• 1330-1350 Information Services (FTREE and MDS)
  - Alex Martin
• 1350-1410 Information Services (RGMA) -
  Manfred Oevers
• 1410-1430 Mass Storage - Jens Jensen
• 1430-1450 Fabric management - Michael George
• 1450-1510 Grid Data Management - Will Bell
• 1510-1530 Networking - Paul Mealor
• Experiment Developments (Chair - John Gordon)
• 1600-1700 ATLAS, CMS, LHCb and theory - Nick
  Brook
• 1700-1730 WP8 status - Frank Harris
• 1730-1825 BaBar, CDF, D0, SAM and others -
  Roger Barlow
• 1825-1830 Logistics - Andy Parker
• 1830-1930 Project Management Board Meeting

• Feb 15th
• Cambridge Perspective and TestBed Introduction
  (Chair - Robin Middleton)
• 900-930 Cambridge e-Science Centre I - Andy
  Parker
• 930-1000 Cambridge e-Science Centre II - Mark
  Hayes
• 1000-1030 TestBed Technical Development - Pete
  Clarke
• Grid Deployment I (Chair - Pete Clarke)
• 1100-1130 Testbed Tools and Release in UK -
  Andrew McNab
• 1130-1140 Security issues - Linda Cornwall
• 1140-1155 Testbed status at RAL - Steve Traylen
• 1155-1210 Testbed status at M'cr - Andrew McNab
• 1210-1225 Testbed status at Bristol - Marc
  Kelly
• 1225-1240 Testbed status at IC - Dave Colling
• Grid Deployment II (Chair - Steve Lloyd)
• 1330-1340 Testbed status at Edinburgh/Glasgow -
  David Martin
• 1340-1350 Testbed status at Liveropool - Girish
  Patel
• 1350-1400 Testbed status at Oxford - Pete
  Gronbech
• 1400-1410 Testbed status at UCL - Paul Mealor
• 1410-1420 Testbed status at Cambridge - Mark
  Hayes
• 1420-1430 Testbed status at Brunel - Peter van
  Santen