Template

1
Peer Policy Policing with
NETFLOW
NANOG 25 June 9, 2002
2
Matthew Meyer
Traffic Engineering
NANOG 25 June 9, 2002
3
The Global Crossing Network

• 200 On Net Cities
• 27 On Net Countries
• Nearly 100,000 route miles
• 17 Metro Networks

4
Peer Policy Policing With Netflow

• Discovering and engaging the wayward packet flows
  that stumble onto your network
• Giving default free networking a fighting chance
• Get off my lawn
• Bottom line Just detecting a peer defaulting
  traffic us

5
Peer Policy Policing with Netflow
Defining the problem

• Telecom Internet-space companies going into
  Ch11
• Punctuated mass customer moves due to Ch7
  backbone liquidations
• Peering less flexible
• Some will resort to uncouth methods to mitigate
  the congestion and sidestep potential costs

6
Peer Policy Policing with Netflow
Defining the problem

• Fewer players, larger peerings
• Peering inherits more flux and less flexibility
  to deal with it
• Some more liberal peering channels may dry up or
  become heavily utilized

7
Peer Policy Policing with Netflow
Addressing the Problem

• Time to think like a bean counter
• Is peering being abused?
• Effect Lower capex due to longer upgrade cycles
• End goal Knowing that we run a tight ship and
  being alerted when uninvited traffic enters the
  network

8
Peer Policy Policing with Netflow
Measurement

• Not rocket science
• 1100 Netflow sampling
• Sampling points All traffic arriving on our
  border routers
• Currently set to do peer-as type flow export

9
Peer Policy Policing with Netflow
Measurement

• One centrally located collector
• Collector handling approximately 20 selected
  routers
• Collector iBGP peers with border routers
• Records route table changes every 5 minutes
• Dual Pentium III, 1G memory, multiple Ultra-160
  SCSI drives, directly connected to backbone

10
Peer Policy Policing with Netflow
Measurement

• DEFAULTING PEER REPORT
• Rec'd Peer Bytes
  percentage of total
• router interface destined for peer
  Bytes for interface
• br2.HUB1.gblx.net_so-2/1/3.0 0.011M
  0.006 lt-Peer A
• br2.HUB1.gblx.net_so-2/1/0.0 0.026M
  0.008 lt-Peer B
• br2.HUB1.gblx.net_so-3/1/0.0 0.087M
  0.008 lt-Peer C
• br2.HUB1.gblx.net_so-2/1/2.0 0.145M
  0.011 lt-Peer D
• br2.HUB1.gblx.net_at-2/2/0.0 0.167M
  0.024 lt-Peer E
• br2.HUB1.gblx.net_so-1/2/3.0 0.339M
  0.017 lt-Peer F
• br2.HUB1.gblx.net_so-3/1/2.0 2.464M
  0.246 lt-Peer G
• br2.HUB1.gblx.net_so-0/0/0.0 3319.615M
  56.722 lt-uplink
• br2.HUB1.gblx.net_so-1/0/0.0 3381.523M
  61.515 lt-uplink

11
Peer Policy Policing with Netflow
Measurement

• EXAMPLE OF FLOWDATA
• /Ixia/SeeFlow/bin/rseeas2as -S '20020603 0000'
  br2.w00t1.gblx.net
• Facets
• TimeInterval 06/04/2002 165049.217018 -
  06/04/2002 193152.879363 UTC
• RouterIpv4Addr 10.10.10.10
• InputIfIndex 67
• InputIfIpv4Addr 10.0.0.1
• InputIfName so-1/2/3.0
• RouterName br2.w00t1.gblx.net
• Src AS Dst AS Packets Pkts/sec
  Bytes Bits/sec
• ------- ------- -------------
  ------------- -------------
  -------------
• 1111 2222 654.061K 67.683
  321.386M 266.058K
• 1111 3333 177.794K 18.398
  130.125M 107.723K
• 99 44444 139.861K
  14.473 91.889M 76.070K
• 1111 3549 257.006K 26.595
  78.603M 65.071K
• 1111 5555 72.634K 7.516
  65.807M 54.478K
• 300 more lines clipped

12
Peer Policy Policing with Netflow
Manipulating the Data

• Extracted with Ixia tools
• 24 hour cumulative byte count per interface
  dest-as key pair
• Created a peer-as list
• Ignored incorrectly reported Netflow data
  according to routing policy

13
(No Transcript)
14
Peer Policy Policing with Netflow
Where to Look

• Our design is hierarchical
• Peers tend to be on dedicated peering routers
• Our peering in consistent and rich
• Collecting closer to the core would not catch
  this behavior universally

15
Peer Policy Policing with Netflow
Analysis

• BGP import policy gets in the way of trusting
  source AS
• Trace levels of false peer to peer traffic
  associated with most peering interfaces
• In initial beta, no peers have been found
  blatantly defaulting to us

16
Peer Policy Policing with Netflow
So Far So Good

• For the moment peer defaulting does not seem to
  be a problem
• We can move forward and easily complete a
  detection system
• Feeling more confident about possible tighter
  peering ahead

17
Peer Policy Policing with Netflow
Whats Next

• Change flow export style from peer-as to
  origin-as
• Putting the discovery on cron
• Long term
• Distribute collection
• Build some visualization
• Integrate with RRDtool

18
Peer Policy Policing with Netflow
Retrospect

• Good exercise in Netflow 101
• Sampling capability excellent
• Data quality excellent
• Restored confidence in Netflow reliability

19
SEAMLESS NETWORK.
GLOBAL REACH.
20
THANK YOU