Internet Performance Monitoring for the HENP Community

1
Internet Performance Monitoring for the HENP
Community

• Les Cottrell Warren Matthews SLAC
• www.slac.stanford.edu/grp/scs/net/talk/mon-pam-mar
  00/
• Presented at the Passive Active Measurement
  Workshop, University of Waikato, New Zealand
  April 3, 2000
• Partially funded by DOE/MICS Field Work Proposal
  on Internet End-to-end Performance Monitoring
  (IEPM), also supported by IUPAP

2
Overview

• Requirements
• PingER
• Validations
• Results
• Quality of Service
• IPv6 Monitoring
• Summary

3
HENP Requirements

• Large experiments with collaborators in over 50
  countries
• Hundreds or even 1000 people on experiment
• Data volumes of PetaBytes or even ExaBytes (1018)
• Distributed access
• Bulk transfer to regional centers
• Fast database queries
• Smooth interactive sessions
• ICFA created standing committee to review
  Inter-regional Connectivity
• Mainly use National Research Networks
• Set expectations, help troubleshoot, planning
  input

4
PingER

• Measurements from
• 30 monitors in 15 countries
• Over 500 remote hosts
• Over 70 countries
• Over 2100 monitor-remote site pairs
• Over 50 of HENP collaborator sites are
  explicitly monitored as remote sites by PingER
  project
• Atlas (37), BaBar (68), Belle (23), CDF (73),
  CMS (31), D0 (60), LEP (44), Zeus (35), PPDG
  (100), RHIC(64)
• Remainder covered by Beacons
• Currently 56, extending to 76

5
Beacons UK seen from ESnet
Direct peering between JANet and ESnet
Sites in UK track one another, so can represent
with single site 2 Beacons in UK
Indicates common source of congestion Increased
capacity by 155 times in 5 years
6
PingER Deployment Jan-00
7
Validations Ping vs. Surveyor
Scatter plot Ping RTT vs Surveyor RTT gives R2
0.92 www.slac.stanford.edu/comp/net/wan-mon/survey
or-vs-pinger.html
8
RIPE vs Surveyor 1/2
Little structure outliers dont match
Little short term correlation even for time
differences of 9
RIPE vs Surveyor 2/2
10
PingER vs AMP
Time series
Little obvious short term agreement (R2if compare ping vs. ping
Avg Ping distribution agrees with AMP Both show
95 of samples are 58-59 msec R2 0.95 for
min avg
11
Rate Limiting 1/2

• Have identified about 2 of sites probably
  limiting
• Using Sting (Stefan Savage) SynAck (SLAC) tools
  to identify loss(sting or synack probes) loss(ping)
• www.vincy.bg.ac.yu blocked 884 rounds of 10 ICMP
  packets each, out of 903
• islamabad-server2.comsats.net.pk
• blocked 554 out of 903
• leonis.nus.edu.sg
• blocked all non 56Byte packets
• All low loss with sting or synack

12
Rate Limiting 2/2

• Rate-limiting kicks in after the first few
  packets and hence later packets are more likely
  to be dropped

Calculate slope and histogram slope frequency for
all nodes, look at outliers (8)
Tail-drop behavior
Added as PingER metric, Still validating, some
sites consistent others vary from month to month
13
ResultsHow are the U.S. Nets doing?
In general performance is good (i.e. (vBNS/Abilene) is catching up with ESnet XIWT
(70 .com) 3-5 times worse than ESnet or I2
14
Europe seen from U.S.
15
Asia seen from U.S.
16
Latin America, Africa Australasia
17
Quality of Service How to improve

• More bandwidth
• Keep network load low (
• Costs (at least in the W) are coming down
  dramatically, but non-trivial to keep up
• Reserved/managed bandwidth generally on ATM via
  PVCs today
• Differentiated services

18
Effect of more managed bandwidth
19
RTT from ESnet to Groups of Sites
20
Loss seen from ESnet to groups of Sites
21
Bulk transfer - Performance Trends
Bandwidth TCP 22
Interactive apps - Jitter
IPDD(i) RTT(i) - RTT(i-1)
23
Interactive apps - Jitter
IPDD(i) RTT(i) - RTT(i-1)
24
SLAC-CERNJitter
25
Voice over IP Reachability
Within N. America, W. Europe loss, RTT and
jitter is acceptable for VoIP
26
Availability Outage Probabaility
Surveyor probes randomly 2/second Measure time
(Outage length) consecutive probes dont
get through
http//www-iepm.slac.stanford.edu/monitoring/surve
yor/outage.html
27
Error free seconds
Typical US phone company objectives are 99.999
What do we see for the Internet using Surveyor
measurements
http//www-iepm.slac.stanford.edu/monitoring/surve
yor/err-sec.html
28
IPv6 Monitoring

• Small amount of bandwidth carved off ESnet
  connection to provide native IPv6 service to SLAC

• Production IPv6 allocation
• 20014000808/48
• Addresses are in DNS

6REN
RTR-IPv6
29
Porting PingER to PingER6

• Recompiled Linux 2.2.5-15 (Red Hat 6.0) kernel
  with IPv6 support
• Downloaded installed inet-apps (including ping)
  from inner.net and patch for glibc-2.1 systems
• Wrote Perl module to provide IPv6 DNS lookup
• Got remote IPv6 sites to monitor
• 10 countries, 40 sites
• Currently one monitoring site at SLAC
• 6TAP to start soon
• China?

30
How does it look?
40
Nov/Dec 1999
35
The weekend
30
25
loss
20
15
10
5
0
22
24
26
28
30
2
4
6
8
10
Much of current 6BONE is congested
31
Summary

• Long term agreement between AMP, PingER,
  Surveyor, RIPE
• need persistent structure (e.g. congestion or
  route changes) for short term point by point
  agreement
• Rate limiting still a minor effect, but could
  become a problem, trying to get good signature
• International performance from US to sites
  outside W. Europe, JP, KR, SG, TW is generally
  poor to bad
• Managed bandwidth can be big help.
• ESnet Internet 2 doing well, even for VoIP,
  except reachability has a way to go
• PingER ported to IPv6, 6BONE congested

32
More Information

• This talk
• www.slac.stanford.edu/grp/scs/net/talk/mon-pam-mar
  00/
• IEPM/PingER home site
• www-iepm.slac.stanford.edu/
• Comparison of Surveyor RIPE PingER
• www.slac.stanford.edu/comp/net/wan-mon/surveyor-vs
  -ripe.html
• www.slac.stanford.edu/comp/net/wan-mon/surveyor-vs
  -pinger.html
• Detecting ICMP Rate Limiting
• www.slac.stanford.edu/grp/scs/net/talk/limiting-fe
  b00/
• IPv6 Monitoring
• www.slac.stanford.edu/grp/scs/net/talk/pinger6/