DataTAG presentation (Bucarest) - PowerPoint PPT Presentation

View by Category
About This Presentation
Title:

DataTAG presentation (Bucarest)

Description:

The DataTAG Project CEOS workshop on Grid computing 6th May 2002, Frascati-Rome Mirco Mazzucato INFN-Padova Slides mainly from Olivier Martin – PowerPoint PPT presentation

Number of Views:21
Avg rating:3.0/5.0
Slides: 30
Provided by: Olivier150
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: DataTAG presentation (Bucarest)


1
The DataTAG Project
CEOS workshop on Grid computing 6th May 2002,
Frascati-Rome Mirco Mazzucato INFN-Padova Slides
mainly from Olivier Martin
2
Presentation outline
  • CERN networking
  • Grid networking requirements
  • DataTAG project
  • Partners
  • Goals
  • Positioning
  • Grid networking issues
  • Concluding remarks

3
Main Internet connections at CERN
Mission Oriented World Health Org.
IN2P3
Swiss National Research Network
General purpose AR and commodity Internet
connections (Europe/USA/World)
WHO
2.5Gbps
SWITCH
45Mbps
155Mbps
Europe
GEANT (1.25/2500Mbps)
1Gbps
USLIC 622Mbps
USA
1Gbps
CERN
CIXP
2.5Gbps
Commercial
DataTAG
Research
4
CERNs Distributed Internet Exchange Point
(CIXP)
  • Telecom Operators dark fibre providers
  • Cablecom, COLT, France Telecom, Global Crossing,
    GTS/EBONE, KPNQwest, LDCom(), Deutsche
    Telekom/Multilink, MCI/Worldcom, SIG,
    Sunrise/diAx, Swisscom (Switzerland), Swisscom
    (France), SWITCH (), Thermelec, VTX/Smartphone.
  • Internet Service Providers include
  • 3GMobile (), Infonet, ATT Global Network
    Services (formerly IBM), Cablecom, Callahan,
    Carrier1, Colt, DFI, Deckpoint, Deutsche Telekom,
    diAx (dplanet), Easynet, Ebone/GTS,
    Eunet/KPNQwest, France Telecom/OpenTransit,
    Global-One, InterNeXt, IS Internet Services
    (ISION), IS-Productions, Nexlink, Net Work
    Communications (NWC), PSI Networks (IProlink),
    MCI/Worldcom, Petrel, Renater, Sita/Equant(),
    Sunrise, Swisscom IP-Plus, SWITCH, GEANT, VTX,
    UUnet.

isp
isp
Telecom operators
c i x p
isp
isp
isp
isp
isp
isp
CERN firewall
Telecom operators
Cern Internal Network
5
Long term Data Grids networking requirements
  • A basic assumption of Data intensive Grids is
    that the underlying network is more or less
    invisible.
  • A prerequisite, therefore, is very fast links
    between Grid nodes
  • Is the hierarchical structure of European
    academic RE networks and the pan-European
    interconnection backbone GEANT a sustainable long
    term model, in order to adequately support Data
    intensive Grids such as the LHC Grid (Large
    Hadron Collider)?
  • Are lambda Grids, feasible affordable?
  • Interesting to note that the original LHC
    computing model which was itself hierarchical
    (Tier0, Tier1, etc) appears to be evolving
    towards a somewhat more flexible model.

6
Evolution of LHC bandwidth requirements
  • LHC Bandwidth Requirements (1999)
  • 622 Mbps between CERN and some (or all) LHC
    regional centers by 2005
  • There seems to be no other way to reach the LHC
    target than to significantly increase the budget
    for external networking by a factor of 3 to5,
    depending on when the bandwidth should be
    delivered.
  • LHC Bandwidth Requirements (2001)
  • 2.5 Gbps between CERN and some (or all) LHC
    regional centers by 2005
  • In any case, a great deal of optimism is needed
    in order to reach the LHC target!
  • LHC Bandwidth Requirements (2002)
  • 10 Gbps between CERN and some (or all) LHC
    regional centers by 2006
  • It is very likely that the first long haul 10Gbps
    circuits will already appear at CERN in
    2003/2004.

Evolution of circuit costs
7
What happened?
  • As a result of the EU wide deregulation of the
    Telecom that took place in 1998, there is an
    extraordinary situation today where circuit
    prices have gone much below the most optimistic
    forecasts!
  • An issue will this trend continue?
  • Whats the most efficient usage of network?
  • Especially for transatlantic connections

8
Evolution of transatlantic circuit costs
  • Since 1995, we have been tracking the prices of
    transatlantic circuits in order to assess the
    budget needed to meet the LHC bandwidth targets
  • The following scenarios have been considered
  • conservative (-20 per year)
  • very plausible (-29 per year, i.e. prices halved
    every two years)
  • Moores law (-37 per year, i.e. prices halved
    every 18 months)
  • optimistic (-50 per year)
  • N.B. Unlike raw circuits, where a price factor of
    2 to 2.5 for 4 times the capacity is usually the
    norm, commodity Internet pricing are essentially
    linear (e.g. 150 CHF/Mbps)

9

10

11

12

13

14

15
The DataTAG Project
http//www.datatag.org
16
Funding agencies
Cooperating Networks

17
EU partners

18
Associated US partners

19
The project
  • European partners INFN (IT), PPARC (UK),
    University of Amsterdam (NL) and CERN, as project
    coordinator.
  • INRIA (FR) will join in June/July2002.
  • ESA/ESRIN (IT) will provide Earth Observation
    demos together with NASA.
  • Budget 3.98 MEUR
  • Start date January, 1, 2002
  • Duration 2 years (aligned on DataGrid)
  • Funded manpower 15 persons/year

20
US Funding collaborations
  • US NSF support through the existing collaborative
    agreement with CERN (Eurolink award).
  • US DoE support through the CERN-USA line
    consortium.
  • Significant contributions to the DataTAG workplan
    have been made by Andy Adamson (University of
    Michigan), Jason Leigh (EVL_at_University of
    Illinois), Joel Mambretti (Northwestern
    University), Brian Tierney (LBNL).
  • Strong collaborations already in place with ANL,
    Caltech, FNAL, SLAC, University of Michigan, as
    well as Internet2 and ESnet.

21
In a nutshell
  • Two main focus
  • Grid related network research (WP2, WP3)
  • Interoperability between European and US Grids
    (WP4)
  • 2.5 Gbps transatlantic lambda between CERN
    (Geneva) and StarLight (Chicago) around July 2002
    (WP1).
  • Dedicated to research (no production traffic)
  • Fairly unique multi-vendor testbed with layer2
    and layer 3 capabilities
  • In principle open to other EU Grid projects as
    well as ESA for demonstrations

22
Multi-vendor testbed with layer3 as well as
layer2 capabilities
INFN (Bologna)
STARLIGHT (Chicago)
Abilene

CERN (Geneva)
GEANT
ESnet
1.25Gbps
Juniper
Juniper
2.5Gbps
Cisco 6509



M
M
Alcatel
Alcatel

Starlight
GBE
622Mbps
Cisco
Cisco
M Layer 2 Mux
23
Goals
  • End to end Gigabit Ethernet performance using
    innovative high performance transport protocols.
  • Assess experiment inter-domain QoS and
    bandwidth reservation techniques.
  • Interoperability between some major GRID projects
    in Europe and North America
  • DataGrid as reference. possibly other EU funded
    Grid projects
  • PPDG, GriPhyN, Teragrid, iVDGL (USA)

24
DataTAG project
NewYork
Abilene
STAR-LIGHT
ESNET
CERN
MREN
STAR-TAP
Major 2.5 Gbps circuits between Europe USA
25
Project positioning
  • Why yet another 2.5 Gbps transatlantic circuit?
  • Most existing or planned 2.5 Gbps transatlantic
    circuits are for production, which makes them
    basically not suitable for advanced networking
    experiments that require a great deal of
    operational flexibility in order to investigate
    new application driven network services, e.g.
  • deploying new equipment (routers, G-MPLS capable
    multiplexers),
  • activating new functionality (QoS, MPLS,
    distributed VLAN)
  • The only known exception to date is the Surfnet
    circuit between Amsterdam Chicago (Starlight)
  • Concerns
  • How far beyond Starlight can DataTAG extend?
  • How fast will US research network infrastructure
    match that of Europe!

26
The STAR LIGHT
  • Next generation STAR TAP with the following main
    distinguishing features
  • Neutral location (Northwestern University)
  • 1/10 Gigabit Ethernet based
  • Multiple local loop providers
  • Optical switches for advanced experiments
  • The STAR LIGHT will provide 2622 Mbps ATM
    connection to the STAR TAP
  • Started in July 2001
  • Also hosting other advanced networking projects
    in Chicago State of Illinois
  • N.B. Most European Internet Exchanges Points have
    already been deployed along the same principles.

27
Major Grid networking issues
  • QoS (Quality of Service)
  • still largely unresolved on a wide scale because
    of complexity of deployment
  • TCP/IP performance over high bandwidth, long
    distance networks
  • The loss of a single packet will affect a 10Gbps
    stream with 200ms RTT (round trip time) for 5
    hours. During that time the average throughput
    will be 7.5 Gbps.
  • On the 2.5Gbps DataTAG circuit with 100ms RTT,
    this translates to 38 minutes recovery time,
    during that time the average throughput will be
    1.875Gbps.
  • Line Error rates
  • A 2.5 Gbps circuit can absorb 0.2 Million
    packets/second
  • Bit error rates of 10E-9 means one packet loss
    every 250 mseconds
  • Bit error rates of 10E-11 means one packet loss
    every 25 seconds
  • End to end performance in the presence of
    firewalls
  • There is a lack of high performance firewalls,
    can we rely on products becoming available or
    should a new architecture be evolved?
  • Evolution of LAN infrastructure to 1Gbps then
    10Gbps
  • Uniform end to end performance

28
Single stream vs Multiple streams effect of a
single packet loss (e.g. link error, buffer
overflow)
Streams/Throughput 10 5 1 7.5 4.375 2 9.375
10
Avg. 7.5 Gbps
Throughput Gbps
7 5
Avg. 6.25 Gbps
Avg. 4.375 Gbps
5
Avg. 3.75 Gbps
2.5
T 2.37 hours! (RTT200msec, MSS1500B)
Time
T
T
T
T
29
Concluding remarks
  • The dream of abundant bandwith has now become a
    hopefully lasting reality!
  • Major transport protocol issues still need to be
    resolved.
  • Large scale deployment of bandwidth greedy
    applications still remain to be done,
  • Proof of concept has yet to be made.

30
Workplan (1)
  • WP1 Provisioning Operations (P. Moroni/CERN)
  • Will be done in cooperation with DANTE National
    Research Education Networks (NREN)
  • Two main issues
  • Procurement (largely done already for what
    concerns the circuit, equipment still to be
    decided).
  • Routing, how can the DataTAG partners access the
    DataTAG circuit across GEANT and their national
    network?
  • Funded participants CERN(1FTE), INFN (0.5FTE)
  • WP5 Information dissemination and exploitation
    (CERN)
  • Funded participants CERN(0.5FTE)
  • WP6 Project management (CERN)
  • Funded participants CERN(2FTE)

31
Workplan (2)
  • WP2 High Performance Networking (Robin
    Tasker/PPARC)
  • High performance Transport
  • tcp/ip performance over large bandwidthdelay
    networks
  • Alternative transport solutions using
  • Modified TCP/IP stack
  • UDP based transport conceptually similar to rate
    based TCP
  • End to end inter-domain QoS
  • Advance network resource reservation
  • Funded participants PPARC (2FTE), INFN (2FTE),
    UvA (1FTE), CERN(1FTE)

32
Workplan (3)
  • WP3 Bulk Data Transfer Application performance
    monitoring (Cees deLaat/UvA)
  • Performance validation
  • End to end user performance
  • Validation
  • Monitoring
  • Optimization
  • Application performance
  • Netlogger
  • Funded participants UvA (2FTE), CERN(0.6FTE)

33
WP4 Workplan (Antonia Ghiselli Cristina
Vistoli / INFN)
  • Main Subject
  • Interoperability between EU and US Grids services
    from DataGrid, GriPhyN, PPDG and in
    collaboration with iVDGL, for the HEP
    applications.
  • Objectives
  • Produce an assessment of interoperability
    solutions
  • Provide test environment to LHC Applications to
    extend existing use-cases to test
    interoperability of the grid components
  • Provide input to a common Grid LHC solution
  • Support EU-US Integrated grid deployment
  • Funded participants INFN (6FTE), PPARC (1FTE),
    UvA (1FTE)

34
WP4 Tasks
  • Assuming the same grid basic services
    (gram,gsi,gris)
  • between the differen grid projects, the main
    issues are
  • 4.1 resource discovery, coord. C.Vistoli
  • 4.2 authorization/VO managemnt, coord. R.Cecchini
  • 4.3 interoperability of collective services
    between EU-US grid domains, coord. F.Donno
  • 4.4 test applications, contact people from each
    application
  • Atlas / L.Perini, CMS / C.Grandi,
  • Alice / P.Cerello

35
WP4.1 - Resource Discovery Objectives
  • Enabling an interoperable system that allows for
    the discovery and access of the Grid services
    available at participant sites of all Grid
    domains, in particular between EU and US Grids.
  • Compatibility of the Resource Discovery System
    with the existent components/services of the
    available GRID systems.
  • Subproject on information schema established

36
Task 4.1 Time Plan
  • Reference agreement document on resource
    discovery schema
  • by 31st of May 2002
  • INTERGRID VO MDS test
  • by 31st of July 2002
  • Evaluation of the interoperability of multiple
    Resource Discovery Systems (FTree, MDS,
    etc) by 30th of September 2002
  • Network Element
  • by 31st of December 2002
  • Impact of the new Web Services Technology
  • by 31st of June 2003
  • Identify missing components.
  • by 31st of June 2003
  • Final deployment.
  • by 31st of December 2003

37
WP4.2 - Objectives
  • Identify Authentication, Authorization and
    Accounting (AAA) mechanisms allowing
    interoperability between grids
  • Compatibility of the AAA mechanisms with the
    existing components/services of the available
    GRID systems
  • Authorization/VO management subproject established

38
Task 4.2 Time Plan
  • Reference document
  • Issues
  • Minimum requirements for DataTAG CAs
  • Analysis of available authorization tools and
    policy languages and their suitability (in
    cooperation with the DataGrid Authorization WG)
  • Mapping of the policies of the VO domains
  • Information exchange protocol between the
    authorization systems
  • Feasibility study of an accounting system (in
    cooperation with the DataGrid WP1)
  • First draft 31 July 2002
  • Final version 30 April 2003
  • Deployment
  • First 30 September 2002
  • Final 31 December 2003

39
WP4.3 / WP4.4 - Objectives
  • Identify grid elements in EU and US grid
    projects, Identify common components in the
    testbeds used by the HEP experiments for
    semi-production activities in EU and US and
    classify them in an architectural framework.
  • Plan and Setup environment with common EU-US
    services.
  • Test common solutions in a EU-US domain in
    collaboration with iVDGL.

40
Task 4.3/4.4 Time Plan
The time plan will follow the schedule of each
experiment
  • Study of exp. Layout and Classification first
    result
  • by 31st of June 2002
  • First deployment (already started)
  • by 31st of September 2002
  • First report of integration and interoperability
    issues
  • by 30th of December 2002
  • First working version of a VO EU-US domain
  • by 31st of June 2003
  • Complete deployment.
  • by 31st of December 2003

41
DataTAG/WP4 framework and
relationships
Grid projects
DataGrid
PPDG
Griphyn
Globus
Condor
LCG
..
input
feedback
..
Grid Interoperability Activities
DataTAG/WP4
iVDGL
HICB/HIJTB
GGF
Integration
stardardization
Proposals
..
Applications
LHC experiments
CDF
Babar
ESA
42
Summary The
interoperability issues (1)
  • 1) Certificates. Solved
  • 2) GSI security. OK but users ask for improved
    error reporting for a production infrastructure 
  • 3) Authorization and VO management. A joint
    subproject has started for a common solution
  • 4) Information Schema. Work is progressing well
    and the group should be able to propose a common
    solution. 
  • 5) GIIS structure and hierarchies. Sub-project in
    the pipeline.
  • General issue of the different vision of the
    information system based on LDAP or on R-GMA
    still open 

43
Summary The
interoperability issues (2)
  • 6) Scheduling, use of JDL and experiment
    interface
  • Regular meetings are taking place between EDG
    WP1 and Condor/ PPDG people. Expected to produce
    a common recommendation.  
  • 7) Data management.
  • A good collaboration between EDG WP2 and
    Globus/PPDG teams. Should be able to make a
    common recommendation on the usage of GDMP,
    Replica Catalog and Replica Manager.  
  • 8) Packaging.
  • LCG should have common release, packaging and
    installation tools.
  • LCG application effort to define a common
    LHC solution started
  •  
About PowerShow.com