Network Developments and Network Monitoring in Internet2 - PowerPoint PPT Presentation

Loading...

PPT – Network Developments and Network Monitoring in Internet2 PowerPoint presentation | free to view - id: c57d9-ZDc1Z



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Network Developments and Network Monitoring in Internet2

Description:

Network Developments and Network Monitoring in Internet2 – PowerPoint PPT presentation

Number of Views:258
Avg rating:3.0/5.0
Slides: 77
Provided by: ebo6
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Network Developments and Network Monitoring in Internet2


1
Network Developments and Network Monitoring in
Internet2
  • Eric Boyd
  • Director of Performance Architecture
  • and Technologies
  • Internet2

2
Overview
  • Internet2 Network
  • Performance Middleware Supporting Network-Based
    Science
  • Internet2 Network Observatory

3
An Asset for the Community
Internet2 Network An Asset for the Community
Universities
Universities
Researchers
Researchers
Regional Networks
Regional Networks
K-12
K-12
Industry
Industry
International
International
4
Internet2 Network
  • Hybrid optical and IP network
  • Dynamic and static wavelength services
  • Fiber, equipment dedicated to Internet2 Level 3
    maintains network and service level
  • Platform supports production services and
    experimental projects

5
Internet2 Network - Layer 1

Internet2 Network Optical Switching Node Level3
Regen Site Internet2 Redundant Drop/Add
Site ESnet Drop/Add Site
6
Internet2 Network Deployment

Internet2 Network Optical Switching Node Level3
Regen Site Internet2 Redundant Drop/Add
Site ESnet Drop/Add Site
7
Internet2 Network Deployment

Internet2 Network Optical Switching Node Level3
Regen Site Internet2 Redundant Drop/Add
Site ESnet Drop/Add Site
8
Internet2 Network Deployment

Internet2 Network Optical Switching Node Level3
Regen Site Internet2 Redundant Drop/Add
Site ESnet Drop/Add Site
9
The New Internet2 Network
10
A New Wrinkle
  • Internet2 exploring a merger with National Lambda
    Rail (NLR)
  • Goal Consolidate national higher education and
    research networking organizations
  • Technical team is exploring what the merged
    technical infrastructure will look like

11
Overview
  • Internet2 Network
  • Performance Middleware Supporting Network-Based
    Science
  • Internet2 Network Observatory

12
Network-Based Science
  • Science is a global community
  • Networks links scientists
  • Collaborative research occurs across network
    boundaries
  • For the scientist, the value of the network is
    the achieved network performance
  • Scientists should not have to focus on the
    network good end-to-end performance should be a
    given

13
Large Hadron Collider
  • International Physics facility located in CERN
    Switzerland
  • Major US involvement
  • 2 major US data repositories (PetaBytes/year)
  • 17 US Institutions provide data analysis and
    storage
  • 68 Universities and National Laboratories with
    scientists looking at the data
  • Dedicated transatlantic networks connect US to
    CERN
  • Advanced network services required over existing
    campus, connector/regional, and national networks

14
Cyberinfrastructure
Cyberinfrastructure
Middleware
Performance
Policy
Security
End to End
15
Achieving Good End-to-End Performance
  • Internet2 consists of
  • Campuses
  • Corporations
  • Regional networks
  • Internet2 backbone network
  • Our members care about connecting with
  • Other members
  • Government labs networks
  • International partners
  • The Internet2 community cares about making all of
    this work

16
Identifying the Problem
Hey, this is not working right!
Others are getting in ok
Not our problem
Applications Developer
Applications Developer
The computer Is working OK
Looks fine
All the lights are green
How do you solve a problem along a path?
We dont see anything wrong
The network is lightly loaded
17
Status Quo
  • Performance is excellent across backbone networks
  • Performance is a problem end-to-end
  • Problems are concentrated towards the edge and in
    network transitions
  • We need to
  • Diagnose Understand limits of performance
  • Address Work with members and application
    communities to address those performance issues

18
Vision Performance Information is
  • Available
  • People can find it (Discovery)
  • Community of trust allows access across
    administrative domain boundaries (AA)
  • Ubiquitous
  • Widely deployed (Paths of interest covered)
  • Reliable (Consistently configured correctly)
  • Valuable
  • Actionable (Analysis suggests course of action)
  • Automatable (Applications act on data)

19
Goal No more mystery
  • Increase network awareness
  • Set user expectations accurately
  • Reduce diagnostic costs
  • Performance problems noticed early
  • Performance problems addressed efficiently
  • Network engineers can see act outside their
    turf
  • Transform application design
  • Incorporate network intuition into application
    behavior

20
Strategy Build Empower the Community
  • Decouple the Problem Space
  • Analysis and Visualization
  • Performance Data Sharing
  • Performance Data Generation
  • Grow the Footprint
  • Clean APIs and protocols between each layer
  • Widespread deployment of measurement
    infrastructure
  • Widespread deployment of common performance
    measurement tools

21
Tactics Leverage position
  • Internet2 is leveraged to help provide diagnostic
    information for US backbone portion of problem
  • Create some diagnostic tools (BWCTL, NDT,
    OWAMP)
  • Make network data as public as is reasonable
  • Work on efforts to more widely make performance
    data available (perfSONAR)
  • Contribute to base perfSONAR development
    (partnership with ESnet, Europe, and Brazil)
  • Contribute to standards for performance
    information sharing (Open Grid Forum Network
    Measurement Working Group)
  • Integrate our diagnostic tools as good
    example of perfSONAR services

22
From the scientists perspective
  • On behalf of the scientist, network engineer or
    application can easily/automatically
  • Discover additional monitoring resources
  • Authenticate locally
  • Authorized to use remote network resources to a
    limited extent
  • Acquire performance monitoring data from remote
    sites via standard protocol
  • Innovate where needed
  • Customize the analysis and visualization

23
Internet2 End-to-End Performance Initiative
(E2Epi)
  • Includes
  • Internet2 staff
  • Internet2 members
  • Federal partners
  • International partners
  • Building
  • Performance monitoring tools
  • Performance middleware frameworks
  • Performance improvement tools

24
Support for E2Epi
  • Funded out of network revenues
  • Partnerships
  • Leveraging GÉANT2, ESnet, and RNP resources
    through consortium leadership
  • Grants
  • NSF Apps - Targeted Assistance and
    Instrumentation for Internet2 Applications
  • NSF SGER - Leveraging Internet2 Facilities for
    the Network Research Community
  • NSF SGER2 - Network Measurement for International
    Connections
  • NSF BTG - Bridging the Gap End-to-End
    Networking for Landmark Applications
  • NLM Pilot - User Experience with the High
    Performance Internet Infrastructure Critical
    Incidents of Success and Failure
  • NLM NDT - Enhancing the Web 100-based Network
    Diagnostic Tool

25
Current Activities
  • Analysis/Diagnostic tools
  • Performance tools
  • Software distributions to enable partner network
    organizations to participate
  • Google Summer of Code
  • New network deployment of measurement
    infrastructure on new observatory

26
Software Distributions
  • NPToolkit (Network Performance Toolkit)
  • Will include much of the following eventually
  • NDT (avail now)
  • OWAMP (avail now)
  • BWCTL/Thrulay (May)
  • AMI (Fall?)
  • Regular testing and collection for OWAMP/BWCTL
  • perfSONAR-PS (Earliest - Fall)
  • SNMP collection/archive
  • AMI archive
  • TopoS and L2Status
  • perfSONAR UIs (Earliest - Winter)

27
Google Summer of Code
  • 5 Projects
  • NDT enhancements
  • Phoebus protocol enhancements
  • Chrolog (user-space timestamp)
  • OWAMP (Java Client)
  • perfSONAR/cacti interface

28
OWAMP (3.0c)
  • One-way latencies
  • Full support of RFC 4656
  • Deployment Status
  • Abilene all remaining nms4 hosts
  • New network newy and chic (nms-rlat)
  • Software available at
  • http//e2epi.internet2.edu/owamp/

29
BWCTL (1.2b)
  • Throughput Test Controller
  • Pending Software release
  • Additional throughput tools
  • Iperf/thrulay/nuttcp
  • More tolerant of questionable clocks
  • Deployment Status
  • Abilene open TCP testing
  • New network - awaiting new software release

30
What is perfSONAR?
  • Performance Middleware
  • perfSONAR is an international consortium in which
    Internet2 and GÉANT2 are founders and leading
    participants
  • perfSONAR is a set of protocol standards for
    interoperability between measurement and
    monitoring systems
  • perfSONAR is a set of open source web services
    that can be mixed-and-matched and extended to
    create a performance monitoring framework

31
perfSONAR Design Goals
  • Standards-based
  • Modular
  • Decentralized
  • Locally controlled
  • Open Source
  • Extensible
  • Applicable to multiple generations of network
    monitoring systems
  • Grows beyond our control
  • Customized for individual science disciplines

32
perfSONAR Integrates
  • Network measurement tools
  • Network measurement archives
  • Discovery
  • Authentication and authorization
  • Data manipulation
  • Resource protection
  • Topology

33
perfSONAR Credits
  • perfSONAR is a joint effort
  • ESnet
  • GÉANT2 JRA1
  • Internet2
  • RNP
  • ESnet includes
  • ESnet/LBL staff
  • Fermilab
  • Internet2 includes
  • University of Delaware
  • Georgia Tech
  • SLAC
  • Internet2 staff
  • GÉANT2 JRA1 includes
  • Arnes
  • Belnet
  • Carnet
  • Cesnet
  • CYNet
  • DANTE
  • DFN
  • FCCN
  • GRNet
  • GARR
  • ISTF
  • PSNC
  • Nordunet (Uninett)
  • Renater
  • RedIRIS
  • Surfnet
  • SWITCH

34
perfSONAR Adoption
  • RE Networks
  • Internet2
  • ESnet
  • GÉANT2
  • European NRENs
  • RNP
  • Application Communities
  • LHC
  • Roll-out to other application communities in late
    2007
  • Distributed Development
  • Individual projects (10 before first release)
    write components that integrate into the overall
    framework
  • Individual communities (5 before first release)
    write their own analysis and visualization
    software

35
perfSONAR-PS
  • perfSONAR (Perl Services)
  • Why?
  • Adoption of Java Services difficult
  • Many network administrators dont do Java, but
    are fluent in Perl)
  • Services more directly targeted at the data
    available from Internet2 observatory deployment.

36
perfSONAR Deployment Status
37
  • Demo

38
Overview
  • Internet2 Network
  • Performance Middleware Supporting Network-Based
    Science
  • Internet2 Network Observatory

39
History and Motivation
  • Original Abilene racks included measurement
    devices
  • Included a single (somewhat large) PC
  • Early OWAMP, Surveyor measurements
  • Optical splitters at some locations
  • Motivation was primarily operations, monitoring,
    and management - understanding the network and
    how well it operates
  • Data was collected and maintained whenever
    possible
  • Primarily a NOC function
  • Available to other network operators to
    understand the network
  • It became apparent that the datasets were
    valuable as a network research tool

40
The Abilene Upgrade Network
Rick Summerhill
41
Upgrade of the Abilene Observatory
  • An important decision was made during the Abilene
    upgrade process (Juniper T-640 routers and
    OC-192c)
  • Two racks, one of which was dedicated to
    measurement
  • Potential for research community to collocate
    equipment
  • Two components to the Observatory
  • Collocation - network research groups are able to
    collocate equipment in the Abilene router nodes
  • Measurement - data is collected by the NOC, the
    Ohio ITEC, and Internet2, and made available to
    the research community

42
An Abilene router node
Power (48VDC)
Power
Measurement Machines (nms)
Eth. Switch
Out-of-band
Space for Collocation!
(M-5)
Measurement(Observatory) Rack
T-640
43
Dedicated servers at each node
  • Houston Router Node - In this picture
  • Measurement machines
  • Collocated PlanetLab machines

44
Example Research Projects
  • Collocation projects
  • PlanetLab Nodes installed in all Abilene Router
    Nodes. See http//www.planet-lab.org
  • The Passive Measurement and Analysis Project
    (PMA) - The Router clamp. See http//pma.nlanr.ne
    t
  • Projects using collected datasets. See
    http//abilene.internet2.edu/observatory/research-
    projects.html
  • Modular Strategies for Internetwork Monitoring
  • Algorithms for Network Capacity Planning and
    Optimal Routing Based on Time-Varying Traffic
    Matrices
  • Spatio-Temporal Network Analysis
  • Assessing the Presence and Incidence of Alpha
    Flows in Backbone Networks

45
The New Internet2 Network
  • Expanded Layer 1, 2 and 3 Facilities
  • Includes SONET and Wave equipment
  • Includes Ethernet Services
  • Greater IP Services
  • Requires a new type of Observatory

46
The New Internet2 Network
47
The New Internet2 Observatory
  • Seek Input from the Community, both Engineers and
    Network Researchers
  • Current thinking is to support three types of
    services
  • Measurement (as before)
  • Collocation (as before)
  • Experimental Servers to support specific projects
    - for example, Phoebus (this is new)
  • Support different types of nodes
  • Optical Nodes
  • Router Nodes

48
The New York Node - First Installment
49
Existing Observatory Capabilities
  • One way latency, jitter, loss
  • IPv4 and IPv6 (owamp)
  • Regular TCP/UDP throughput tests 1 Gbps
  • IPv4 and IPv6 On-demand available (bwctl)
  • SNMP
  • Octets, packets, errors collected 1/min
  • Flow data
  • Addresses anonymized by 0-ing the low order 11
    bits
  • Routing updates
  • Both IGP and BGP - Measurement device
    participates in both
  • Router configuration
  • Visible Backbone Collect 1/hr from all routers
  • Dynamic updates
  • Syslog also alarm generation (nagios) polling
    via router proxy

50
Observatory Functions
Device Function Details
nms-rthr1 Measurement BWCTL on-demand 1 Gpbs router throughput, Thrulay
nms-rthr2 Measurement BWCTL on-demand 10 Gbps router throughput, Thrulay
nms-rexp Experimental NDT/NPAD
nms-rpsv Measurement Netflow collector
nms-rlat Measurement OWAMP with locally attached GPS timing
nms-rpho Experimental Phoebus 2 x 10GE to Multiservice Switch
nms-octr Management Controls Multiservice Switch
nms-oexp Experimental NetFPGA
nms-othr Measurement On-demand Multiservice Switch 10 Gbps throughput
51
Router Nodes
52
Router Nodes
53
Optical Nodes
54
Optical Nodes
55
Observatory Hardware
  • Dell 1950 and Dell 2950 servers
  • Dual Core 3.0 GHz Xeon processors
  • 2 GB memory
  • Dual RAID 146 GB disk
  • Integrated 1 GE copper interfaces
  • 10 GE interfaces
  • Hewlett-Packard 10GE switches
  • 9 servers at router sites, 3 at optical only sites

56
Observatory Databases Dat? Types
  • Data is collected locally and stored in
    distributed databases
  • Databases
  • Usage Data
  • Netflow Data
  • Routing Data
  • Latency Data
  • Throughput Data
  • Router Data
  • Syslog Data

57
Uses and Futures
  • Some uses of existing datasets and tools
  • Quality Control
  • Network Diagnosis
  • Network Characterization
  • Network Research
  • Consultation with researchers
  • Open questions

58
Observatory Deployment (July)
  • NDT/NPAD servers
  • OWAMP/BWCTL deployments
  • perfSONAR services (perfSONAR-PS)
  • LS (Discovery)
  • SNMP collection/archive
  • OWAMP/BWCTL archive
  • TopoS and L2 status (GN2 E2EMon compatible)

59
Recall Datasets
  • Usage Data
  • Netflow Data
  • Routing Data
  • Latency Data
  • Throughput Data
  • Router Data
  • Syslog Data
  • ND, NR
  • ND, NC, NR
  • NR
  • QC, ND, NR
  • QC, ND, NR
  • ND, NR
  • NR

And, of course, most used for operations
60
Quality Control e-VLBI
  • When starting to connect telescopes, needed to
    verify inter-site paths
  • Set up throughput testing among sites (using same
    Observatory tool bwctl)
  • Kashima, JP
  • Onsala, SE
  • Boston, MA (Haystack)
  • Collect and graph data distribute via web
  • Quick QC check before applications tests start

61
Quality Control e-VLBI Network
62
Quality Control eVLBI Result
  • Automated monitoring allowed view of network
    throughput variation over time
  • Highlights route changes, network outages
  • Automated monitoring also helps to highlight any
    throughput issues at end points
  • e.g. Network Interface Card failures, untuned TCP
    Stacks
  • Integrated monitoring provides overall view of
    network behavior at a glance

63
Network Diagnosis e-VLBI
  • Target at the time 50Mbps
  • Oops Onsala-Boston 1Mbps
  • Divide and Conquer
  • Verify Abilene backbone tests look good
  • Use Abilene test point in Washington DC
  • Eliminated European and trans-Atlantic pieces
  • Focus on problem found oversubscribed link

64
Quality Control IP Backbone
  • Machines with 1GE interfaces, 9000 MTU
  • Full mesh
  • IPv4 and IPv6
  • Expect gt 950 Mbps TCP
  • Keep list of Worst 10
  • If any path lt 900 Mbps for two successive testing
    intervals, throw alarm

65
(No Transcript)
66
Quality Control Peerings
  • Internet2 and ESnet have been watching the
    latency across peering points for a while.
  • Internet2 and DREN have been preparing to do some
    throughput and latency testing
  • During the course of this set up, found
    interesting routing and MTU size issues

67
Network Diagnosis End Hosts
  • NDT, NPAD servers
  • Quick check from a host that has a browser
  • Easily eliminate (or confirm) last mile problems
    (buffer sizing, duplex mismatch, )
  • NPAD can find switch limitations, provided the
    server is close enough

68
Network Diagnosis Generic
  • Generally looking for configuration loss
  • Dont forget security appliances
  • Is there connectivity reasonable latency? (ping
    -gt OWAMP)
  • Is routing reasonable (traceroute, proxy)
  • Is host reasonable (NDT NPAD)
  • Is path reasonable (BWCTL)

69
Network Characterization
  • Flow data collected with flow-tools package
  • All data not used for security alerts and
    analysis REN-ISAC is anonymized
  • Reports from anonymized data available (see
    truncated addresses)
  • Additionally, some Engineering reports

70
Network Research Projects
  • Major consumption
  • Flows
  • Routes
  • Configuration
  • Nick Feamster (while at MIT)
  • Dave Maltz (while at CMU)
  • Papers in SIGCOMM, INFOCOM
  • Hard to track folks that just pull data off of
    web sites

71
Lots of Work to be Done
  • Internet2 Observatory realization inside racks
    set for initial deployment, including new
    research projects (NetFPGA, Phoebus)
  • Software and links easily changed
  • Could add or change hardware depending on costs
  • Researcher tools, new datasets
  • Consensus on passive data

72
Not Just Research
  • Operations and Characterization of new services
  • Finding problems with stitched together VLANs
  • Collecting and exporting data from Dynamic
    Circuit Service...
  • Ciena performance counters
  • Control plane setup information
  • Circuit usage (not utilization, although that is
    also nice)
  • Similar for underlying Infinera equipment
  • And consider inter-domain issues

73
Sharing Observatory Data
  • We want to make Internet2 Network Observatory
    Data
  • Available
  • Access to existing active and passive measurement
    data
  • Ability to run new active measurement tests
  • Interoperable
  • Common schema and semantics, shared across other
    networks
  • Single format
  • XML-based discovery of whats available

74
Internet2 Deployment Status
  • Focus is on development of services for Internet2
    new network and integration with Indiana NOC
  • Submitting a proposal to NSF for additional
    funding
  • Target July 2007 as new Internet2 network goes
    operation
  • OWAMP MA
  • BWCTL MA/MP
  • IU-based Topology Service
  • Multi-LS
  • NOC Alarm Transformation Service

75
More Information
  • Eric Boyd
  • eboyd_at_internet2.edu
  • 734-352-7032
  • http//e2epi.internet2.edu/
  • http//bwctl.internet2.edu
  • http//ndt.internet2.edu/
  • http//owamp.internet2.edu/
  • http//vfer.internet2.edu/
  • http//www.perfsonar.net/
  • http//nwmg.internet2.edu

76
(No Transcript)
About PowerShow.com