Harvey B Newman - PowerPoint PPT Presentation

Loading...

PPT – Harvey B Newman PowerPoint presentation | free to download - id: a3ade-ZTkxZ



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Harvey B Newman

Description:

Harvey B Newman. FAST Meeting, Caltech. July 1, 2002 ... 500 jobs per day. Monte Carlo. 5000 SI95sec/event. RAW Data. Reconstruction. Re-processing ... – PowerPoint PPT presentation

Number of Views:122
Avg rating:3.0/5.0
Slides: 40
Provided by: cms593
Category:
Tags: harvey | newman

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Harvey B Newman


1
HENP Grids and Networks Global Virtual
Organizations
  • Harvey B Newman
  • FAST Meeting, Caltech
  • July 1, 2002
  • http//l3www.cern.ch/newman/HENPGridsNets_FAST070
    202.ppt

2
Computing Challenges Petabyes, Petaflops,
Global VOs
  • Geographical dispersion of people and resources
  • Complexity the detector and the LHC environment
  • Scale Tens of Petabytes per year of data

5000 Physicists 250 Institutes 60
Countries
Major challenges associated with Communication
and collaboration at a distance Managing globally
distributed computing data resources
Cooperative software development and physics
analysis New Forms of Distributed Systems Data
Grids
3
Four LHC Experiments The
Petabyte to Exabyte Challenge
  • ATLAS, CMS, ALICE, LHCBHiggs New particles
    Quark-Gluon Plasma CP Violation

Data stored 40 Petabytes/Year and UP
CPU 0.30 Petaflops and UP 0.1
to 1 Exabyte (1 EB 1018
Bytes) (2007) (2012 ?) for the LHC
Experiments
4
LHC Higgs Decay into 4 muons (Tracker only)
1000X LEP Data Rate
109 events/sec, selectivity 1 in 1013 (1 person
in a thousand world populations)
5
LHC Data Grid Hierarchy
CERN/Outside Resource Ratio 12Tier0/(?
Tier1)/(? Tier2) 111
PByte/sec
100-400 MBytes/sec
Online System
Experiment
CERN 700k SI95 1 PB Disk Tape Robot
Tier 0 1
HPSS
2.5-10 Gbps
Tier 1
FNAL 200k SI95 600 TB
IN2P3 Center
INFN Center
RAL Center
2.5-10 Gbps
Tier 2
2.5-10 Gbps
Tier 3
Institute 0.25TIPS
Institute
Institute
Institute
Physicists work on analysis channels Each
institute has 10 physicists working on one or
more channels
0.110 Gbps
Physics data cache
Tier 4
Workstations
6
Emerging Data Grid User Communities
  • Grid Physics Network (GriPhyN)
  • ATLAS, CMS, LIGO, SDSS
  • Particle Physics Data Grid (PPDG)Intl Virtual
    Data Grid Lab (iVDGL)
  • NSF Network for Earthquake Engineering Simulation
    (NEES)
  • Integrated instrumentation, collaboration,
    simulation
  • Access Grid VRVS supporting group-based
    collaboration
  • And
  • Genomics, Proteomics, ...
  • The Earth System Grid and EOSDIS
  • Federating Brain Data
  • Computed MicroTomography
  • Virtual Observatories

7
HENP Related Data Grid Projects
  • Projects
  • PPDG I USA DOE 2M 1999-2001
  • GriPhyN USA NSF 11.9M 1.6M 2000-2005
  • EU DataGrid EU EC 10M 2001-2004
  • PPDG II (CP) USA DOE 9.5M 2001-2004
  • iVDGL USA NSF 13.7M 2M 2001-2006
  • DataTAG EU EC 4M 2002-2004
  • GridPP UK PPARC gt15M 2001-2004
  • LCG (Ph1) CERN MS 30 MCHF 2002-2004
  • Many Other Projects of interest to HENP
  • Initiatives in US, UK, Italy, France, NL,
    Germany, Japan,
  • Networking initiatives DataTAG, AMPATH,
    CALREN-XD
  • US Distributed Terascale Facility (53M, 12
    TeraFlops, 40 Gb/s network)

8
Daily, Weekly, Monthly and Yearly Statistics on
155 Mbps US-CERN Link
20 - 100 Mbps Used Routinely in 01 BaBar 600
Mbps Throughput in 02
BW Upgrades Quickly Followedby Upgraded
Production Use
9
Tier A
Two centers are trying to work as one -Data not
duplicated -Internationalization -transparent
access, etc
"Physicists have indeed foreseen to test the GRID
principles starting first from the Computing
Centres in Lyon and Stanford (California). A
first step towards the ubiquity of the GRID."
Pierre Le Hir Le Monde 12 april 2001
CERN-US Line Abilene
3/2002
Renater ESnet
D. Linglin LCG Wkshop
10
RNP Brazil (to 20 Mbps)
FIU Miami/So. America (to 80 Mbps)
11
Transatlantic Net WG (HN, L. Price)
Bandwidth Requirements

Installed BW. Maximum Link Occupancy 50
Assumed See http//gate.hep.anl.gov/lprice/TAN
12
Links Required to US Labs and Transatlantic

Maximum Link Occupancy 50 AssumedOC3155
Mbps OC12622 Mbps OC482.5 Gbps OC19210
GbpsNote New ESNet Upgrade Plan
13
MONARC CMS Analysis Process
  • Hierarchy of Processes (Experiment, Analysis
    Groups,Individuals)

RAW Data
3000 SI95sec/event 1 job year
3000 SI95sec/event 3 jobs per year
Reconstruction
Experiment- Wide Activity (109 events)
Re-processing 3 Times per year
New detector calibrations Or understanding
5000 SI95sec/event
25 SI95sec/event 20 jobs per month
Monte Carlo
Iterative selection Once per month
Trigger based and Physics based refinements
Selection
20 Groups Activity (109 ? 107 events)
10 SI95sec/event 500 jobs per day
Different Physics cuts MC comparison Once per
day
25 Individual per Group Activity (106 107
events)
Algorithms applied to data to get results
Analysis
14
Tier0-Tier1 Link Requirements Estimate for
Hoffmann Report 2001
  • 1) Tier1 ? Tier0 Data Flow for Analysis 0.5 - 1.0
    Gbps
  • 2) Tier2 ? Tier0 Data Flow for Analysis 0.2 - 0.5
    Gbps
  • 3) Interactive Collaborative Sessions (30 Peak)
    0.1 - 0.3 Gbps
  • 4) Remote Interactive Sessions (30 Flows Peak)
    0.1 - 0.2 Gbps
  • 5) Individual (Tier3 or Tier4) data transfers
    0.8 GbpsLimit to 10 Flows of 5
    Mbytes/sec each
  • TOTAL Per Tier0 - Tier1 Link 1.7 - 2.8 Gbps
  • NOTE
  • Adopted by the LHC Experiments given in the
    Steering Committee Report on LHC Computing 1.5
    - 3 Gbps per experiment
  • Corresponds to 10 Gbps Baseline BW Installed on
    US-CERN Link
  • Report also discussed the effects of higher
    bandwidths
  • For example all-optical 10 Gbps Ethernet WAN
    by 2002-3

15
Tier0-Tier1 BW RequirementsEstimate for
Hoffmann Report 2001
  • Does Not Include more recent ATLAS Data Estimates
  • 270 Hz at 1033 Instead of 100Hz
  • 400 Hz at 1034 Instead of 100Hz
  • 2 MB/Event Instead of 1 MB/Event ?
  • Does Not Allow Fast Download to Tier34 of
    Small Object Collections
  • Example Download 107 Events of AODs (104 Bytes)
    ? 100 Gbytes At 5 Mbytes/sec per person (above)
    thats 6 Hours !
  • This is a still a rough, bottoms-up, static, and
    hence Conservative Model.
  • A Dynamic distributed DB or Grid system with
    Caching, Co-scheduling, and Pre-Emptive data
    movement may well require greater bandwidth
  • Does Not Include Virtual Data
    operationsDerived Data Copies Data-description
    overheads
  • Further MONARC Model Studies are Needed

16
Maximum Throughput on Transatlantic Links (155
Mbps)
  • 8/10/01 105 Mbps reached with 30 Streams
    SLAC-IN2P3
  • 9/1/01 102 Mbps in One Stream CIT-CERN
  • 11/5/01 125 Mbps in One Stream (modified
    kernel) CIT-CERN
  • 1/09/02 190 Mbps for One stream shared on 2
    155 Mbps links
  • 3/11/02 120 Mbps Disk-to-Disk with One Stream
    on 155 Mbps link (Chicago-CERN)
  • 5/20/02 450 Mbps SLAC-Manchester on OC12 with
    100 Streams
  • 6/1/02 290 Mbps Chicago-CERN One Stream on
    OC12 (mod. Kernel)

Also see http//www-iepm.slac.stanford.edu/monitor
ing/bulk/ and the Internet2 E2E Initiative
http//www.internet2.edu/e2e
17
Some Recent EventsReported 6/1/02 to ICFA/SCIC
  • Progress in High Throughput 0.1 to 1 Gbps
  • Land Speed Record SURFNet Alaska (IPv6) (0.4
    Gbps)
  • SLAC Manchester (Les C. and Richard H-J) (0.4
    Gbps)
  • Tsunami (Indiana) (0.8 Gbps UDP)
  • Tokyo KEK (0.5 0.9 Gbps)
  • Progress in Pre-Production and Production
    Networking
  • 10 Mbytes/sec FNAL-CERN (Michael Ernst)
  • 15 Mbytes/sec disk-to-disk Chicago-CERN (Sylvain
    Ravot)
  • KPNQwest files for Chapter 11 Stops network
    yesterday.
  • Near Term Pricing of Competitor (DT) ok.
  • Unknown impact on prices and future planning in
    the medium and longer term

18
Baseline BW for the US-CERN Link HENP
Transatlantic WG (DOENSF)
Transoceanic NetworkingIntegrated with the
Abilene, TeraGrid, Regional Netsand Continental
NetworkInfrastructuresin US, Europe, Asia,
South America
Baseline evolution typicalof major HENPlinks
2001-2006
  • US-CERN Link 622 Mbps this month
  • DataTAG 2.5 Gbps Research Link in Summer 2002
  • 10 Gbps Research Link by Approx. Mid-2003

19
Total U.S. Internet Traffic
100 Pbps
Limit of same GDP as Voice
10 Pbps
1 Pbps
100Tbps
New Measurements
10Tbps
1Tbps
100Gbps
Projected at 3/Year
Voice Crossover August 2000
10Gbps
1Gbps
ARPA NSF Data to 96
100Mbps
4X/Year
10Mbps
2.8X/Year
1Mbps
100Kbps
10Kbps
1Kbps
100 bps
10 bps
U.S. Internet Traffic
Source Roberts et al., 2001
20
Internet Growth Rate Fluctuates Over Time
U.S. Internet Edge Traffic Growth Rate 6 Month
Lagging Measure
10/004/01 Growth Reported 3.6/year
4.50
10/004/01 Growth Reported 4.0/year
4.00
3.50
3.00
Growth Rate per Year
2.50
Average 3.0/year
2.00
1.50
1.00
0.50
0.00
Jan 00
Apr 00
Jul 00
Oct 00
Jan 01
Apr 01
Jul 01
Oct 01
Jan 02
Source Roberts et al., 2002
21
AMS-IX Internet Exchange Throughput
Accelerating Growth in Europe (NL)
Monthly Traffic2X Growth from 8/00 - 3/012X
Growth from 8/01 - 12/01
?
Hourly Traffic3/22/02
6.0 Gbps
4.0 Gbps
2.0 Gbps
22
ICFA SCIC Meeting March 9 at CERN Updates from
Members
  • Abilene Upgrade from 2.5 to 10 Gbps
  • Additional scheduled lambdas planned for targeted
    for targeted applications Pacific and
    National Light Rail
  • US-CERN
  • Upgrade On Track to 622 Mbps in July Setup and
    Testing Done in STARLIGHT
  • 2.5G Research Lambda by this Summer
    STARLIGHT-CERN
  • 2.5G Triangle between STARLIGHT (US), SURFNet
    (NL), CERN
  • SLAC IN2P3 (BaBar)
  • Getting 100 Mbps over 155 Mbps CERN-US Link
  • 50 Mbps Over RENATER 155 Mbps Link, Limited by
    ESnet
  • 600 Mbps Throughput is BaBar Target for this Year
  • FNAL
  • Expect ESnet Upgrade to 622 Mbps this Month
  • Plans for dark fiber to STARLIGHT underway, could
    be done in 4 Months Railway or Electric Co.
    provider

23
ICFA SCIC AR Backbone and International Link
Progress
  • GEANT Pan-European Backbone (http//www.dante.net/
    geant)
  • Now interconnects 31 countries
  • Includes many trunks at 2.5 and 10 Gbps
  • UK
  • 2.5 Gbps NY-London, with 622 Mbps to ESnet and
    Abilene
  • SuperSINET (Japan) 10 Gbps IP and 10 Gbps
    Wavelength
  • Upgrade to Two 0.6 Gbps Links, to Chicago and
    Seattle
  • Plan upgrade to 2 X 2.5 Gbps Connection to US
    West Coast by 2003
  • CAnet4 (Canada) Interconnect customer-owned
    dark fiber nets across Canada at 10 Gbps,
    starting July 2002
  • Lambda-Grids by 2004-5
  • GWIN (Germany) Connection to Abilene Upgraded
    to 2 X 2.5 Gbps early in 2002
  • Russia
  • Start 10 Mbps link to CERN and 90 Mbps to US Now

24
2.5?10 Gbps Backbone
210 Primary ParticipantsAll 50 States, D.C. and
Puerto Rico80 Partner Corporations and
Non-Profits22 State Research and Education Nets
15 GigaPoPs Support 70 of Members
Caltech Connection with GbE to New Backbone
25
National RE Network ExampleGermany DFN
TransAtlanticConnectivity Q1 2002
  • 2 X OC12 Now NY-Hamburg and NY-Frankfurt
  • ESNet peering at 34 Mbps
  • Upgrade to 2 X OC48 expected in Q1 2002
  • Direct Peering to Abilene and Canarie expected
  • UCAID will add another 2 OC48s Proposing a
    Global Terabit Research Network (GTRN)
  • FSU Connections via satelliteYerevan, Minsk,
    Almaty, Baikal
  • Speeds of 32 - 512 kbps
  • SILK Project (2002) NATO funding
  • Links to Caucasus and Central Asia (8
    Countries)
  • Currently 64-512 kbps
  • Propose VSAT for 10-50 X BW NATO State
    Funding

26
National Research Networks in Japan
  • SuperSINET
  • Started operation January 4, 2002
  • Support for 5 important areas
  • HEP, Genetics, Nano-Technology,Space/Astronomy,
    GRIDs
  • Provides 10 ?s
  • 10 Gbps IP connection
  • 7 Direct intersite GbE links
  • Some connections to 10 GbE in JFY2002
  • HEPnet-J
  • Will be re-constructed with MPLS-VPN in
    SuperSINET
  • Proposal Two TransPacific 2.5 Gbps
    Wavelengths, and Japan-CERN Grid Testbed by
    2003

NIFS
IP
Nagoya U
NIG
WDM path
IP router
Nagoya
Osaka
Osaka U
Tokyo
Kyoto U
NII Hitot.
ICR
Kyoto-U
U Tokyo
ISAS
Internet
IMS
NAO
U-Tokyo
27
DataTAG Project
NewYork
ABILENE
STARLIGHT
ESNET
GENEVA
Wave Triangle
CALREN
STAR-TAP
  • EU-Solicited Project. CERN, PPARC (UK), Amsterdam
    (NL), and INFN (IT)and US (DOE/NSF UIC, NWU
    and Caltech) partners
  • Main Aims
  • Ensure maximum interoperability between US and EU
    Grid Projects
  • Transatlantic Testbed for advanced network
    research
  • 2.5 Gbps Wavelength Triangle 7/02 (10 Gbps
    Triangle in 2003)

28
TeraGrid (www.teragrid.org)NCSA, ANL, SDSC,
Caltech
A Preview of the Grid Hierarchyand Networks of
the LHC Era
Abilene
Chicago
DTF Backplane 4 X 10 Gbps
Indianapolis
Urbana
Caltech
Starlight / NW Univ
UIC
San Diego
I-WIRE
Multiple Carrier Hubs
Ill Inst of Tech
ANL
OC-48 (2.5 Gb/s, Abilene)
Univ of Chicago
Multiple 10 GbE (Qwest)
Indianapolis (Abilene NOC)
Multiple 10 GbE (I-WIRE Dark Fiber)
NCSA/UIUC
Idea to extend the TeraGrid to CERN
Source Charlie Catlett, Argonne
29
CA ONI, CALREN-XD Pacific Light Rail Backbones
(Proposed)
Also LA-CaltechMetro Fiber National Light
Rail
30
Key Network Issues Challenges
  • Net Infrastructure Requirements for High
    Throughput
  • Packet Loss must be Zero (at and below 10-6)
  • I.e. No Commodity networks
  • Need to track down uncongested packet loss
  • No Local infrastructure bottlenecks
  • Multiple Gigabit Ethernet clear paths between
    selected host pairs are needed now
  • To 10 Gbps Ethernet paths by 2003 or 2004
  • TCP/IP stack configuration and tuning Absolutely
    Required
  • Large Windows Possibly Multiple Streams
  • New Concepts of Fair Use Must then be Developed
  • Careful Router, Server, Client, Interface
    configuration
  • Sufficient CPU, I/O and NIC throughput
    sufficient
  • End-to-end monitoring and tracking of performance
  • Close collaboration with local and regional
    network staffs
  • TCP Does Not Scale to the 1-10 Gbps Range

31
A Short List Revolutions in Information
Technology (2002-7)
  • Managed Global Data Grids (As Above)
  • Scalable Data-Intensive Metro and Long Haul
    Network Technologies
  • DWDM 10 Gbps then 40 Gbps per ? 1 to 10
    Terabits/sec per fiber
  • 10 Gigabit Ethernet (See www.10gea.org) 10GbE
    / 10 Gbps LAN/WAN integration
  • Metro Buildout and Optical Cross Connects
  • Dynamic Provisioning ? Dynamic Path Building
  • Lambda Grids
  • Defeating the Last Mile Problem(Wireless or
    Ethernet in the First Mile)
  • 3G and 4G Wireless Broadband (from ca.
    2003)and/or Fixed Wireless Hotspots
  • Fiber to the Home
  • Community-Owned Networks

32
A Short List Coming Revolutions in Information
Technology
  • Storage Virtualization
  • Grid-enabled Storage Resource Middleware (SRM)
  • iSCSI (Internet Small Computer Storage
    Interface)Integrated with 10 GbE ? Global File
    Systems
  • Internet Information Software Technologies
  • Global Information Broadcast Architecture
  • E.g the Multipoint Information Distribution
    Protocol (Tie.Liao_at_inria.fr)
  • Programmable Coordinated Agent Architectures
  • E.g. Mobile Agent Reactive Spaces (MARS) by
    Cabri et al., University of Modena
  • The Data Grid - Human Interface
  • Interactive monitoring and control of Grid
    resources
  • By authorized groups and individuals
  • By Autonomous Agents

33
HENP Major Links Bandwidth Roadmap (Scenario)
in Gbps
34
One Long Range Scenario (Ca. 2008-12)HENP As a
Driver of Optical NetworksPetascale Grids with
TB Transactions
  • Problem Extract Small Data Subsets of 1 to 100
    Terabytes from 1 to 1000 Petabyte Data Stores
  • Survivability of the HENP Global Grid System,
    with hundreds of such transactions per day
    (circa 2007)requires that each transaction be
    completed in a relatively short time.
  • Example Take 800 secs to complete the
    transaction. Then
  • Transaction Size (TB) Net Throughput
    (Gbps)
  • 1
    10
  • 10
    100
  • 100
    1000 (Capacity of

    Fiber Today)
  • Summary Providing Switching of 10 Gbps
    wavelengthswithin 3 years and Terabit
    Switching within 5-10 yearswould enable
    Petascale Grids with Terabyte transactions,as
    required to fully realize the discovery potential
    of major HENP programs, as well as other
    data-intensive fields.

35
Internet2 HENP WG
  • Mission To help ensure that the required
  • National and international network
    infrastructures(end-to-end)
  • Standardized tools and facilities for high
    performance and end-to-end monitoring and
    tracking, and
  • Collaborative systems
  • are developed and deployed in a timely manner,
    and used effectively to meet the needs of the
    US LHC and other major HENP Programs, as well as
    the at-large scientific community.
  • To carry out these developments in a way that is
    broadly applicable across many fields
  • Formed an Internet2 WG as a suitable framework
    Oct. 26 2001
  • Co-Chairs S. McKee (Michigan), H.
    Newman (Caltech) Secy J. Williams (Indiana)
  • Website http//www.internet2.edu/henp also see
    the Internet2End-to-end Initiative
    http//www.internet2.edu/e2e

36
True End to End Experience
  • User perception
  • Application
  • Operating system
  • Host IP stack
  • Host network card
  • Local Area Network
  • Campus backbone network
  • Campus link to regional network/GigaPoP
  • GigaPoP link to Internet2 national backbones
  • International connections

EYEBALL APPLICATION STACK JACK NETWORK . . . . .
. . . . . . .
37
HENP Scenario LimitationsTechnologies and Costs
  • Router Technology and Costs (Ports and
    Backplane)
  • Computer CPU, Disk and I/O Channel Speeds to
    Send and Receive Data
  • Link Costs Unless Dark Fiber (?)
  • MultiGigabit Transmission Protocols End-to-End
  • 100 GbE Ethernet (or something else) by 2006
    for LANs to match WAN speeds

38
Throughput quality improvementsBWTCP lt
MSS/(RTTsqrt(loss))
80 Improvement/Year ? Factor of 10 In 4 Years
China Improves But Far Behind
Eastern Europe Far Behind
See Macroscopic Behavior of the TCP
Congestion Avoidance Algorithm, Matthis, Semke,
Mahdavi, Ott, Computer Communication Review
27(3), 7/1997
39
11900 Hosts6620 Registered Users in 61
Countries 43 (7 I2) Reflectors Annual Growth 2
to 3X
40
Networks, Grids and HENP
  • Next generation 10 Gbps network backbones are
    almost here in the US, Europe and Japan
  • First stages arriving, starting now
  • Major transoceanic links at 2.5 - 10 Gbps in
    2002-3
  • Network improvements are especially needed in
    Southeast Europe, So. America and some other
    regions
  • Romania, Brazil India, Pakistan, China Africa
  • Removing regional, last mile bottlenecks and
    compromises in network quality are now All
    on the critical path
  • Getting high (reliable Grid) application
    performance across networks means!
  • End-to-end monitoring a coherent approach
  • Getting high performance (TCP) toolkits in
    users hands
  • Working in concert with AMPATH, Internet E2E, I2
    HENP WG, DataTAG the Grid projects and the GGF
About PowerShow.com