Traffic Dynamics at a Commercial Backbone POP

1
Traffic Dynamics at a Commercial Backbone POP

• Nina Taft
• Sprint ATL
• Co-authors Supratik Bhattacharyya, Jorjeta
  Jetcheva, Christophe Diot

2
Outline

• Part 1 what are the traffic demands between
  pairs of POPs?
• How stable is this demand?
• Part 2 what are the paths taken by those
  demands?
• Are link utilizations levels similar throughout
  the backbone?
• Part 3 is there a better way to spread the
  traffic across paths?
• Can we divert some traffic to lightly loaded
  paths?

3
The Sprint IPMon Project

• Passive monitoring
• Capture header (44 bytes) from every packet
• full TCP/IP headers, no http information
• Use GPS time stamping - allows accurate
  correlating of packets on different links
• Day long traces
• Simultaneously monitor multiple links and sites.
• Collect routing information along with packet
  traces.
• Traces archived for future use

4
IP Backbone POP-to-POP view
OC-192
OC-48
POP fanout one row of POP-to-POP traffic matrix
OC-12
5
POP-to-POP Traffic Matrix
For every ingress POP Identify total traffic
to each egress POP Further analyze this traffic
Measure traffic over different timescales Divide
traffic per destination prefix, protocol, etc.
6
The Mapping Problem
What is the egress POP for a packet entering a
given ingress POP?
7
Monitored links at a single POP
Publicpeer 2
Public peer 1

Core
Core
Core
ISP
web host
Date Aug 9, 2000
8
Traffic Fanout POP level granularity
9
Fanout web host links
10
Time-of-Day for POP level granularity
11
Day-Night Variation Webhost 1
reduction at night between 20-50 depending
upon access link
12
Summary so far ...

• Wide disparity in traffic demands among egress
  POPs
• POPs can be roughly categorized as small,
  medium, large and they maintain their rank
  during the day.
• Traffic is heterogeneous in space yet stable in
  time.
• 20-50 reduction at night

13
Outline

• Part 1 what are the traffic demands between
  pairs of POPs?
• How stable is this demand?
• Part 2 what are the paths taken by those
  demands?
• Are link utilizations levels similar throughout
  the backbone?
• Part 3 is there a better way to spread the
  traffic across paths?
• Can we divert some traffic to lightly loaded
  paths?

14
Paths used by traffic demands

• Our Observations (summary)
• routing policies concentrate traffic on a few
  paths between two POPs, all the traffic uses
  either the same route, or 1 or 2 routes
• the ISIS weights are changed very infrequently
  (once a month), so routing is fairly static
• there are many underutilized routes

15
Is backbone traffic balanced?
16
Part 3 Can we divert some traffic to lightly
loaded paths?

• Approach to improve load balancing by rerouting
  only a few flows
• scalable
• Which flows? Heavy hitters.
• How identify heavy hitters Consider
  destination prefix-based flows
• at fixed prefix lengths 8 and 16
• BGP table entries (variable prefix length)

17
Streams based on destination prefix
Traffic grouped by egress POPs
Stream all packets in a group with same /8
destination address prefix
Similar results for /16 and bgp table prefixes
Ingress Webhost Link
18
Stability of prefix-based streams
Stability of prefix rank
Ri(n) the rank of flow i at time slot n
Di,n,k Ri(n) - Ri(nk) each time slot
19
Conclusions

• We have used our data to build components of
  traffic matrices for traffic engineering
• Heterogeneous traffic fanout from POP
• Current routing practices lead to many
  underutilized links and paths
• thus, there is a lot of room for improved load
  balancing techniques.
• Load-balancing using flows selected via
  destination-prefixes is a simple and promising
  criterion

20
Ongoing Work

• Intra-domain Routing
• Choosing ISIS link weights
• Multi-path routing
• Flow Characterization at the network prefix level
• Inference techniques for building POP-to-POP
  traffic matices