CRIO: Scaling IP Routing with the Core RouterIntegrated Overlay

1
CRIOScaling IP Routing with the Core
Router-Integrated Overlay

• Xinyang (Joy) Zhang Paul Francis Jia Wang
  Kaoru Yoshida

2
Internet Architecture BoardRouting Workshop (Oct
2006)

• It was clear at the workshop, and probably
  clearly evident elsewhere, that if there is a
  highest ranked problem in the routing space
  then it would be that of scaling the routing
  system
• It appears that unbounded continued growth of the
  routing and forwarding system in the Internet
  appears to trigger off some real limitations
  relating to hardware design and switching centre
  infrastructure

---- Geoff Huston
3
Why is Scaling a Problem?

• A glimpse of current routing system
• Static table size
• Global IPv4 200K entries
• VPN 800K entries
• And more routes are coming IPV6,
  traffic-engineered, etc.
• Routing Dynamics
• BGP update churns
• Persistent instabilities
• Long convergence time (due to damping and MRAI
  timer)

• This talk is about the static
• characteristics of the scaling
• Validity of CRIO approach

• Looking into the future
• Can we support a routing table twice (or 10
  times) the size of today?
• Can we rely on the hardware advances to alleviate
  the scaling pressure?

4
CRIOs Approach to Scaling

• Tunneling
• Revisit old idea (by Deering)
• Decouples addressing from topology
• Virtual Prefix
• Novel approach
• Greatly shrink forwarding table

5
CRIO Tunneling an Illustration
Prefix TE Source
Mapping Adv. 24.1.1.0/24 TEPE2
Prefix TE Source
PE2 ---- BGP
24.1.1.0/24 ---- BGP/OSPF
24.1.1.0/24 PE2 Mapping
24.1.1.0/24 PE3 Mapping
Provider Networks
PE2
PE1
24.1.1.1
PE2
24.1.1.1
PE3
24.1.1.1
Routing Adv. 24.1.1.0/24 NHCE2
CE2
CE1
Customer Site C2 24.1.1.0/24
Customer Site C1
6
CRIO Tunneling Benefits

• Separate Mapping from Routing
• BGP only computes routes to TE prefixes
• On the order of one thousand entries
• Stable ISP provisioned prefixes
• Mappings are easy to distribute
• A mapping entry is the same no matter where it
  appears
• Support multi-homing without burdening the
  routing system

7
What about routers forwarding table?

• CRIO tunneling can not shrink forwarding
  information
• Forwarding table is expected to get larger
• Since CRIO supports for fine-grained multi-homing
• Benefits for having small forwarding tables
• Smaller memory requirement on routers line cards
• Faster transfer for forwarding table updates

8
CRIO Virtual Prefix an Illustration

• A virtual prefix is a super-prefix that spans a
  large portion of the address space
• Routers that advertise a given virtual prefix
  must hold the mappings for every prefix within
  the virtual prefix

Prefix TE Source
Routing Adv. 24.0.0.0/8
PE2 ---- BGP
24.1.1.0/24 PE2 Mapping
24.2.2.0/24 PE4 Mapping
PE3
24.1.1.1
PE2
PE2
24.1.1.1
24.1.1.0/24
24.1.1.1
Customer Site
PE1
CE2
Prefix TE Source
PE2 ---- BGP
PE3 ---- BGP
24.1.1.0/24 PE2 Mapping
24.0.0.0/8 ---- BGP
24.2.2.0/24 PE4 Mapping
9
CRIO Virtual Prefix Trade-off

• Virtual prefixes provide a tuning knob for the
  router
• trade-off forwarding table size for path length
• Per-prefix basis
• Its a good trade-off to make
• Few prefixes handle most traffic (power law)
• Routers could shed most of their prefixes with
  very little overall increase in traffic volume
• Save routers from handling large amount of
  mapping updates
• Virtual Prefix is particularly suitable for VPNs

10
CRIO Evaluation Static Analysis

• Evaluate the static performance of CRIO by
  simulation
• Quantify table size vs. path length tradeoff
• Simulated both Global Internet and VPN
• Based on actual Internet topology ISP traffic
  matrices
• Simulation tool C-BGP

11
CRIO Evaluation Data Collection

• Global Internet
• Topology
• POP-level from RocketFuel
• 23 Tier-1 ISP, 1219 POPs, 4159 inter-POP links
• Mappings
• Derived ltprefix, TEgt mappings from RocketFuel raw
  traces
• Internet Traffic Matrices
• Prefix-level, across all POP in our topology
• Use Netflow records from Tier-1 ISP backbone
• VPN
• Same data is collected for VPN from a large VPN
  provider and one of its national-sized customers

12
CRIO Evaluation Forwarding Table Content

• Direct Entries
• Virtual Prefix Entries
• Extra Path-Shrinking Entries

Routing Adv. 24.0.0.0/8
Prefix TE Source
PE2 ---- BGP
PE3
24.1.1.0/24 PE2 Mapping
24.1.1.0/24
Customer Site
PE1
CE2
Prefix TE Source
Prefix TE Source
PE2 ---- BGP
24.1.1.0/24 ---- BGP
PE3 ---- BGP
24.0.0.0/8 ---- BGP
24.1.1.0/24 PE2 Mapping
13
CRIO Evaluation Virtual Prefix Placement Policy

• Inter-ISP (Random)
• Intra-ISP
• Intra-ISP shortest customer path

Routing Adv. 24.0.0.0/8
Routing Adv. 24.0.0.0/8
Provider 2
Provider 1
24.1.1.0/24
PE1
Customer Site
CE2
14
CRIO Global Internet Simulation Results

• Path-length vs. Table-size Trade-off

• Virtual Prefix does increase the path length
• Average path length converges quickly as the
  path-shrinking entries increases
• Reduce FIB size by 3-5 times with very little
  path length penalty

Increase the percentage of shortest path
traffic by increasing of Path-Shrinking Entries

99 Traffic uses shortest path
15
CRIO VPN Simulation Results (One Customer)

• Hub-Spoke nature of VPN traffic exploits the
  tradeoff better
• Reduce table size by 10-20 times with very little
  path length penalty

Cumulative Distribution of PE Routers
PE Routers In Hub Sites
16
Conclusions

• CRIO is a new routing architecture, aimed to
  provide
• Scalable and stable core routing
• Reduce BGP RIB by two order of magnitude
• FIB size reduction
• Reduce FIB by one order of magnitude for global
  Internet, 10-20x for VPN

17
Future Work

• Design and implement the mapping distribution
  infrastructure
• Study the dynamics aspect of CRIO
• Study the security aspect of CRIO
• Explore the use of CRIO to provide traffic
  engineering for multi-homed site
• Address (??) new management challenges

18
Thank you!