MPLS-TE Doesn

1
MPLS-TE Doesnt ScaleAdrian FarrelOld Dog
Consultingadrian_at_olddog.co.uk
www.mpls2007.com
2
Is the Sky Falling?

• The only way to get your attention is to be
  alarmist
• MPLS-TE is perfectly functional in todays
  networks
• But
• MPLS-TE will not scale indefinitely
• The problem is the well-known full mesh or
  n-squared problem
• The number of LSPs scales as the square of the
  number of PEs

3
What Do We Want to Achieve?

• MPLS-TE is an important feature for many SPs
• Allow traffic to be groomed
• Optimize use of network resources
• Provide quality of service guaranties
• Carriers look to provide edge-to-edge tunnels
  across their core networks
• Differentiated Services
• VPNs
• VLANS and pseudowires
• Multimedia content distribution
• Normal IP traffic

4
What is the Scope of the Problem?

• Consider a service provider network with 1000 PEs
• This is not outrageously large
• Such a network may be broken into areas or ASes
• Consider a full mesh of PE-PE TE-LSPs
• Consider parallel tunnels for different services,
  QoS levels, and for protection
• May give rise to multiples of 999,000 LSPs in the
  core
• What is the capacity of a core LSR?
• What is the capacity of a management system?

5
What Are the Scaling Limits?

• Management
• NMS
• How many LSPs can the NMS process
• Management protocols
• Reporting on large numbers of LSPs may overload
  the management network
• LSR issues
• Memory capacity
• Per LSP data requirements
• CPU capacity largely an RSVP-TE protocol issue
• Degradation of LSP setup times
• Soft state addressed by Refresh Reduction
• MPLS forwarding plane
• Number of labels (Only 1048559 per interface)

6
The Snowflake Topology

• Example network for analysis
• Meshed core of P nodes
• Called P1 nodes
• Each Pi1 node connected to just one Pi node
• PE nodes connected to just one Pn node
• Well-defined connectivity and symmetry allows
  many important metrics to be computed
• Number of levels number of nodes per level may
  be varied
• We can vary the number of P1 nodes
• We can vary the ratio of Pi1 to Pi
• We can vary the value n
• We can vary the number of PE nodes per Pn node

PE
P2
P1
7
Analysing the Snowflake Topology

• Define
• Pn a node at the nth level (level 1 is core)
• Sn the number of nodes at the nth level
• Mn the multiplier at the nth level (how many Pn1
  nodes are connected to a Pn node)
• Ln number of LSPs seen by a Pn node
• Discover
• LPE 2(SPE - 1)
• L2 M2(2SPE - M2 - 1)
• L1 M1M2(2SPE M2(M1 1))
• Practical numbers
• S1 10, M1 10, and M2 20
• SPE 2000
• LPE 3998
• L2 79580
• L1 756000

8
The Ladder Topology

• Example network for analysis
• Core of P1 nodes looks like a ladder
• Similar to many nationalnetworks
• Symmetrical trees subtendedto core
• Each Pi1 node connected to just one Pi node
• Each PE node connected to just one P node
• Again
• Well-defined connectivity and symmetry allows
  many important metrics to be computed
• Number of levels number of nodes per level may
  be varied

9
Analysing the Ladder Topology

• Same definitions as for snowflake network
• E the number of subtended edge nodes (PEs) to
  each spar-node (E M1M2)
• Discover
• LPE 2(SPE - 1)
• L2 2M2(SPE - 1) - M2(M2 - 1)
• L1 EES1S1/2 EES1 3EE - EM2
• Practical numbers
• S 1 10, M1 10, and M2 20
• E 200
• SPE 2000
• LPE 3998
• L2 79580
• L1 2516000

10
Option 1 Solve a Different Problem!

• If a full mesh of PE-PE LSPs is too big, dont
  build it!
• This is the bottom line if we dont fix the
  problem
• The suggestion is to build a full mesh of
  Pn-to-Pn LSPs, and perform routing or
  routing-based MPLS between Pn and PE
• Scaling improves from O(10002)to O(1002)
• But we lose functionality
• Why did we want a PE-PE mesh?
• How do we handle private address spaces?
• What if the traffic is not routable?
• This may simply not be good enough to provide the
  function

11
Option 2 LSP Hierarchies

• Well-known, core MPLS function
• Label stacks
• Forwarding Adjacencies (RFC 4206)
• Configured or automatic grooming
• Possible to build a full or partialmesh of
  hierarchical tunnels
• For example connect all P2 nodes
• Each P2 node must encapsulate each PE-PE LSP in
  the correct tunnel
• Each P1 node only sees the P2-P2 tunnels

12
Scaling Properties of Hierarchies - Snowflake

• Note that PE-PE tunnels dont help
• P1-P1 tunnels are also no benefit (core is fully
  meshed)
• P2 nodes see all PE-PE LSPs and new tunnels
• L2 M2(2SPE - M2 - 1) 2(S2 - 1)
• Situation at P1 nodes is much better
• L1 M1(2S2 - M1 - 1)
• Numbers (S1 10, M1 10, and M2 20)
• Flat 2-Level Hierarchy
• SPE 2000 2000
• LPE 3998 3998
• L2 79580 79778
• L1 756000 1890
• Maybe insert another layer (P3 ) to increase the
  scaling?
• L3 remains high

13
Scaling Properties of Hierarchies - Ladder

• Note that PE-PE tunnels dont help
• But P1-P1 tunnels are good because core is not
  fully-meshed
• L1 S1S1/2 2S1 2EE(S1 - 1) - EM2 - 2
• Another level of hierarchy is also possible
• Add a mesh of P2-P2 tunnels
• L1 S1S1/2 2S1 2M1M1S1 - M1(M1 1) 2
• L2 2M2(S(PE) - 1) - M2(M2 - 1)
  2(S(1)M(1) - 1)
• Numbers (S 1 10, M1 10, and M2 20)
• Flat 2-Level 3-Level
• Hierarchy Hierarchy
• SPE 2000 2000 2000
• LPE 3998 3998 3998
• L2 79580 79580 79778
• L1 2516000 716060 1958

14
Issues and Drawbacks for Hierarchies

• Scaling is not good enough!
• Impact on layer adjacent to PEs is negligible
• Actually impact is slightly negative
• Management burden
• Plan and operate a secondary mesh
• Effectively the same burden as managing PEs or a
  layered network
• Possible to consider auto-mesh techniques
• Fast Reroute protection is a problem
• FRR struggles to protect tunnel end-points
• Not obvious how to arrange the hierarchy when the
  network is not symmetrical
• E.g., some PEs closer to the core

15
Option 3 Multipoint-to-Point LSPs

• LSPs merge automatically as they converge on the
  destination
• Reduces the number of LSPs toward the egress
• Other LSP properties (e.g.,bandwidth) must be
  cumulative
• TE is still possible, butde-merge is not
  considered
• Should count LSP state not number of LSPs
• New definition
• Xn the amount of LSP state held at each Pn node
• For flat and hierarchical networks
• Each LSP adds one state at ingress or egress
• Each LSP adds two states at each transit node

16
Scaling Properties of MP2P LSPs - Snowflake

• XPE 2(SPE - 1)
• X2 SPE(M2 1)
• X1 M1M2(S1 - 2) SPE(M1 1)
• Numbers (S1 10, M1 10, and M2 20)
• Flat 2-Level Hierarchy P2MP
• SPE 2000 2000 2000
• XPE 3998 3998 3998
• X2 159160 159358 42000
• X1 1512000 3780 23600

17
Scaling Properties of MP2P LSPs - Ladder

• XPE 2(SPE - 1)
• X2 (M2 1)S1E
• X1 (4 M1)S1E - M1E
• Numbers (S1 10, M1 10, and M2 20)
• Flat 2-Level 3-Level P2MP
• Hierarchy Hierarchy
• SPE 2000 2000 2000 2000
• XPE 3998 3998 3998 3998
• X2 159160 159160 159358 42000
• X1 5032000 1433998 3898 26000

18
Issues and Drawbacks for MP2P LSPs

• Clear scaling benefits
• Better than flat networks
• Only thing that improves the situation adjacent
  to PEs
• But
• Data plane support
• This will only ever be a packet/frame/cell
  technology
• Control plane support
• RSVP does have MP2P support
• RSVP-TE features not yet specified or implemented
• De-aggregation and disambiguation
• May be necessary to use label stack so that
  egress can detect sender of data
• OAM may be more complex and require source labels
• New management applications needed
• FRR still to be designed

19
Other Topics for Investigation

• Cost-effectiveness of the network
• Revenue only generated by PEs
• K S(PE)/(S(1)S(2) ... S(n))
• Many ways to improve scaling reduce
  cost-effectiveness
• Fast Reroute
• What are the implications of FRR to scaling?
• Can scaling contributions be designed that can be
  protected by FRR?
• Point-to-multipoint
• What are the scaling properties of P2MP MPLS-TE?
• Domain boundaries (in particular AS boundaries)
• Boundaries such as at area and AS borders cause
  constrictions
• How can we reduce the number of LSPs seen by ABRs
  and ASBRs?

20
Conclusions, Next Steps, and References

• MPLS-TE is not a scaling issue today
• But it wont scale arbitrarily
• We need to plan now for tomorrows scalability
• Hierarchical LSPs are not as good as expected
• MP2P LSPs may offer a better solution
• More research and implementation is needed
• draft-ietf-mpls-te-scaling-analysis-01.txt
• Seisho Yaukawa (NTT)
• Adrian Farrel (Old Dog Consulting)
• Olufemi Komolafe (Cisco Systems)

21

• Questions?
• adrian_at_olddog.co.uk