High Performance Routers

1
High Performance Routers
Slides originally by Nick McKeown Professor of
Electrical Engineering and Computer Science,
Stanford University nickm_at_stanford.edu www.stanfor
d.edu/nickm
Stanford High Performance Networking group
http//klamath.stanford.edu
2
Outline

• Background
• What is a router?
• Why do we need faster routers?
• Why are they hard to build?
• Architectures and techniques
• The evolution of router architecture.
• IP address lookup.
• Packet buffering.
• Switching.
• The Future

3
What is Routing Forwarding?
4
What is Routing?
5
What is Routing?
6
Points of Presence (POPs)
7
Where High Performance Routers are Used
(2.5 Gb/s)
R2
(2.5 Gb/s)
R1
R6
R5
R4
R3
R7
R9
R10
R8
R11
R12
R14
R13
R16
R15
(2.5 Gb/s)
(2.5 Gb/s)
8
What a Router Looks Like
Cisco GSR 12416
Juniper M160
19
19
Capacity 160Gb/sPower 4.2kW
Capacity 80Gb/sPower 2.6kW
6ft
3ft
2ft
2.5ft
9
Generic Router Architecture
Header Processing
Lookup IP Address
Update Header
Queue Packet
1M prefixes Off-chip DRAM
1M packets Off-chip DRAM
10
Generic Router Architecture
Buffer Manager
Buffer Memory
Buffer Manager
Buffer Memory
Buffer Manager
Buffer Memory
11
Why do we Need Faster Routers?

1. To prevent routers becoming the bottleneck in the
   Internet.
2. To increase POP capacity, and to reduce cost,
   size and power.

12
Why we Need Faster Routers 1 To prevent routers
from being the bottleneck
Packet processing Power
Link Speed
10000
1000
2x / 18 months
2x / 7 months
100
Fiber Capacity (Gbit/s)
10
1
1985
1990
1995
2000
0,1
TDM
DWDM
Source SPEC95Int David Miller, Stanford.
13
Why we Need Faster Routers 2 To reduce cost,
power complexity of POPs
POP with large routers
14
Why are Fast Routers Difficult to Make?

• Its hard to keep up with Moores Law
• The bottleneck is memory speed.
• Memory speed is not keeping up with Moores Law.

15
Why are Fast Routers Difficult to Make?Speed of
Commercial DRAM

• Its hard to keep up with Moores Law
• The bottleneck is memory speed.
• Memory speed is not keeping up with Moores Law.

16
Why are Fast Routers Difficult to Make?

• Its hard to keep up with Moores Law
• The bottleneck is memory speed.
• Memory speed is not keeping up with Moores Law.
• Moores Law is too slow
• Routers need to improve faster than Moores Law.

17
Router Performance Exceeds Moores Law

• Growth in capacity of commercial routers
• Capacity 1992 2Gb/s
• Capacity 1995 10Gb/s
• Capacity 1998 40Gb/s
• Capacity 2001 160Gb/s
• Capacity 2003 640Gb/s
• Average growth rate 2.2x / 18 months.

18
Outline

• Background
• What is a router?
• Why do we need faster routers?
• Why are they hard to build?
• Architectures and techniques
• The evolution of router architecture.
• IP address lookup.
• Packet buffering.
• Switching.
• The Future

19
First Generation Routers
Shared Backplane
Line Interface
20
Second Generation Routers
CPU
Buffer Memory
Route Table
Line Card
Line Card
Line Card
Buffer Memory
Buffer Memory
Buffer Memory
Fwding Cache
Fwding Cache
MAC
MAC
MAC
Typically lt5Gb/s aggregate capacity
21
Third Generation Routers
Switched Backplane
Line Card
CPU Card
Line Card
Local Buffer Memory
Local Buffer Memory
Line Interface
CPU
Routing Table
Memory
Fwding Table
MAC
MAC
Typically lt50Gb/s aggregate capacity
22
Fourth Generation Routers/SwitchesOptics inside
a router for the first time
Optical links
100s of metres
Switch Core
Linecards
0.3 - 10Tb/s routers in development
23
Outline

• Background
• What is a router?
• Why do we need faster routers?
• Why are they hard to build?
• Architectures and techniques
• The evolution of router architecture.
• IP address lookup.
• Packet buffering.
• Switching.
• The Future

24
Generic Router Architecture
Buffer Manager
Buffer Memory
Header Processing
Buffer Manager
Lookup IP Address
Update Header
Buffer Memory
Address Table
Buffer Manager
Buffer Memory
25
IP Address Lookup

• Why its thought to be hard
• Its not an exact match its a longest prefix
  match.
• The table is large about 120,000 entries today,
  and growing.
• The lookup must be fast about 30ns for a 10Gb/s
  line.

26
IP Lookups find Longest Prefixes
128.9.176.0/24
128.9.16.0/21
128.9.172.0/21
142.12.0.0/19
65.0.0.0/8
128.9.0.0/16
0
232-1
Routing lookup Find the longest matching prefix
(aka the most specific route) among all prefixes
that match the destination address.
27
IP Address Lookup

• Why its thought to be hard
• Its not an exact match its a longest prefix
  match.
• The table is large about 120,000 entries today,
  and growing.
• The lookup must be fast about 30ns for a 10Gb/s
  line.

28
Address Tables are Large
Source Geoff Huston, Oct 2001
29
IP Address Lookup

• Why its thought to be hard
• Its not an exact match its a longest prefix
  match.
• The table is large about 120,000 entries today,
  and growing.
• The lookup must be fast about 30ns for a 10Gb/s
  line.

30
Lookups Must be Fast
40B packets (Mpkt/s)
Line
Year
1.94
622Mb/s
1997
7.81
2.5Gb/s
1999
31.25
10Gb/s
2001
125
40Gb/s
2003
31
IP Address LookupBinary tries
Example Prefixes
0
1
a) 00001
b) 00010
c) 00011
d) 001
e) 0101
g
f
d
f) 011
g) 100
h
i
h) 1010
e
i) 1100
j) 11110000
a
b
c
j
32
Prefix Length Distribution
Source Geoff Huston, Oct 2001
99.5 prefixes are 24-bits or shorter
33
24-8 Direct Lookup Trie
00000000
11111111
24 bits
0
224-1
8 bits
0
28-1

• When pipelined, allows one lookup per memory
  access.
• Although inefficient use of memory, total memory
  cost lt 20.

34
IP Address LookupSummary

• Lookup limited by memory bandwidth.
• Lookup uses high-degree trie.
• State of the art 10Gb/s line rate.
• Scales to 40Gb/s line rate.
• By 2008, entire IPv4 address space will fit on
  one 20 DRAM!

35
Outline

• Background
• What is a router?
• Why do we need faster routers?
• Why are they hard to build?
• Architectures and techniques
• The evolution of router architecture.
• IP address lookup.
• Packet buffering.
• Switching.
• The Future

36
Generic Router Architecture
Queue Packet
Buffer Memory
Queue Packet
Buffer Memory
Queue Packet
Buffer Memory
37
Fast Packet Buffers
Example 40Gb/s packet buffer Size RTTBW
10Gb 40 byte packets
Write Rate, R
Read Rate, R
Buffer Manager
1 packet every 8 ns
1 packet every 8 ns
Buffer Memory
Use SRAM? fast enough random access time, but -
too low density to store 10Gb of data.
Use DRAM? high density means we can store data,
but - too slow (50ns random access time).
38
Packet Caches
Buffer Manager SRAM
1
3
4
2
Arriving
Departing
2
Packets
Packets
1
2
3
4
5
Q
1
2
3
4
5
6
bgtgt1 packets at a time
DRAM Buffer Memory
39
Packet BuffersSummary

• Packet buffers limited by memory bandwidth.
• Packet buffer caches use hybrid SRAMDRAM.
• State of the art 10Gb/s line rate.
• Scales to 40Gb/s line rate.

40
Outline

• Background
• What is a router?
• Why do we need faster routers?
• Why are they hard to build?
• Architectures and techniques
• The evolution of router architecture.
• IP address lookup.
• Packet buffering.
• Switching.
• The Future

41
Generic Router Architecture
1
1
Queue Packet
Buffer Memory
2
2
Queue Packet
Buffer Memory
N times line rate
N
N
Queue Packet
Buffer Memory
42
Generic Router Architecture
Queue Packet
Buffer Memory
Queue Packet
Buffer Memory
Scheduler
Queue Packet
Buffer Memory
43
Output Buffered Switches
The best that any queueing system can achieve.
44
Input Buffered SwitchesHead of Line Blocking
The best that any queueing system can achieve.
45
Head of Line Blocking
46
Virtual Output Queues
47
A Router with Virtual Output Queues
The best that any queueing system can achieve.
48
Maximum Weight Matching
S(n)
L11(n)
A11(n)
D1(n)
A1(n)
1
1
A1N(n)
AN1(n)
DN(n)
AN(n)
N
N
ANN(n)
LNN(n)
L11(n)
Maximum Weight Match
LN1(n)
Bipartite Match
Request Graph
49
The Evolution of Switching
Theory
Input Queueing (IQ)
58 Karol, 1987
Practice
Input Queueing (IQ)
Various heuristics, distributed algorithms, and
amounts of speedup
50
SwitchingSummary

• Routers use virtual output queues, and a
  centralized scheduler.
• State of the art 2.5Tb/s.
• Scales to 10Tb/s.
• 100 throughput will be standard soon.

51
Current Internet Router TechnologySummary

• There are three potential bottlenecks
• Address lookup,
• Packet buffering, and
• Switching.
• Techniques exist today for
• 10Tb/s Internet routers, with
• 40Gb/s linecards.
• But what comes next?

52
Outline

• Background
• What is a router?
• Why do we need faster routers?
• Why are they hard to build?
• Architectures and techniques
• The evolution of router architecture.
• IP address lookup.
• Packet buffering.
• Switching.
• The Future
• More parallelism.
• Eliminating schedulers.
• Introducing optics into routers.
• Natural evolution to circuit switching?

53
External Parallelism Multiple Parallel Routers
IP Router capacity 100s of Tb/s
What wed like
R
R
NxN
R
R
54
Multiple parallel routers Load Balancing
R
R
R
55
Intelligent Packet Load-balancingParallel Packet
Switching
Router
R/k
R/k
1
rate, R
rate, R
1
1
2
rate, R
rate, R
N
N
k
Bufferless
56
Parallel Packet Switching

• Advantages
• Single-stage of buffering
• No excess link capacity
• kh a power per subsystem i
• kh a memory bandwidth i
• kh a lookup rate i

57
Parallel Packet SwitchTheorem

• If S gt 2k/(k2) _at_ 2 then a parallel packet
  switch can precisely emulate a single big router.

58
Outline

• Background
• What is a router?
• Why do we need faster routers?
• Why are they hard to build?
• Architectures and techniques
• The evolution of router architecture.
• IP address lookup.
• Packet buffering.
• Switching.
• The Future
• More parallelism.
• Eliminating schedulers.
• Introducing optics into routers.
• Natural evolution to circuit switching?

59
Eliminating schedulersTwo-Stage Switch Chang et
al., 2001
Internal Inputs
External Inputs
External Outputs
First Round-Robin
Second Round-Robin
60
Outline

• Background
• What is a router?
• Why do we need faster routers?
• Why are they hard to build?
• Architectures and techniques
• The evolution of router architecture.
• IP address lookup.
• Packet buffering.
• Switching.
• The Future
• More parallelism.
• Eliminating schedulers.
• Introducing optics into routers.
• Natural evolution to circuit switching?

61
Do optics belong in routers?

• They are already there.
• Connecting linecards to switches.
• Optical processing doesnt belong on the
  linecard.
• You cant buffer light.
• Minimal processing capability.
• Optical switching can reduce power.

62
Optics in routers
Optical links
Switch Core
Linecards
63
Complex linecards
Typical IP Router Linecard

• Buffer
• State
• Memory

Lookup Tables
Switch Fabric
Buffer Mgmt Scheduling
Optics
Packet Processing
Framing Maintenance
Physical Layer
Buffer Mgmt Scheduling

• Buffer
• State
• Memory

Arbitration

• 10Gb/s linecard
• Number of gates 30M
• Amount of memory 2Gbits
• Cost gt20k
• Power 300W

64
Replacing the switch fabric with optics
electrical

• Candidate technologies
• MEMs.
• Fast tunable lasers passive optical couplers.
• Diffraction waveguides.
• Electroholographic materials.

65
The Stanford Phicticious Optical Router
Optical 2-stage Switch
Linecard 625
Linecard 1
160- 320Gb/s
160- 320Gb/s
40Gb/s

• Line termination
• IP packet processing
• Packet buffering

• Line termination
• IP packet processing
• Packet buffering

40Gb/s
160Gb/s
40Gb/s
40Gb/s
100 Tb/s IP Router, 625 linecards, each
operating at 160Gb/s.
66
Outline

• Background
• What is a router?
• Why do we need faster routers?
• Why are they hard to build?
• Architectures and techniques
• The evolution of router architecture.
• IP address lookup.
• Packet buffering.
• Switching.
• The Future
• More parallelism.
• Eliminating schedulers.
• Introducing optics into routers.
• Natural evolution to circuit switching?

67
Evolution to circuit switching

• Optics enables simple, low-power, very high
  capacity circuit switches.
• The Internet was packet switched for two reasons
• Expensive links statistical multiplexing.
• Resilience soft-state routing.
• Neither reason holds today.

68
Fast Links, Slow Routers
Processing Power
Link Speed (Fiber)
2x / 2 years
2x / 7 months
Source SPEC95Int Prof. Miller, Stanford Univ.
69
Fewer Instructions
Instructions per packet since 1996
70
Outline

• Background
• What is a router?
• Why do we need faster routers?
• Why are they hard to build?
• Architectures and techniques
• The evolution of router architecture.
• IP address lookup.
• Packet buffering.
• Switching.
• The Future
• More parallelism.
• Eliminating schedulers.
• Introducing optics into routers.
• Natural evolution to circuit switching?

71
References

• General
• J. S. Turner Design of a Broadcast packet
  switching network, IEEE Trans Comm, June 1988,
  pp. 734-743.
• C. Partridge et al. A Fifty Gigabit per second
  IP Router, IEEE Trans Networking, 1998.
• N. McKeown, M. Izzard, A. Mekkittikul, W.
  Ellersick, M. Horowitz, The Tiny Tera A Packet
  Switch Core, IEEE Micro Magazine, Jan-Feb 1997.
• Fast Packet Buffers
• Sundar Iyer, Ramana Rao, Nick McKeown Design of
  a fast packet buffer, IEEE HPSR 2001, Dallas.

72
References

• IP Lookups
• A. Brodnik, S. Carlsson, M. Degermark, S. Pink.
  Small Forwarding Tables for Fast Routing
  Lookups, Sigcomm 1997, pp 3-14.
• B. Lampson, V. Srinivasan, G. Varghese. IP
  lookups using multiway and multicolumn search,
  Infocom 1998, pp 1248-56, vol. 3.
• M. Waldvogel, G. Varghese, J. Turner, B.
  Plattner. Scalable high speed IP routing
  lookups, Sigcomm 1997, pp 25-36.
• P. Gupta, S. Lin, N. McKeown. Routing lookups in
  hardware at memory access speeds, Infocom 1998,
  pp 1241-1248, vol. 3.
• S. Nilsson, G. Karlsson. Fast address lookup for
  Internet routers, IFIP Intl Conf on Broadband
  Communications, Stuttgart, Germany, April 1-3,
  1998.
• V. Srinivasan, G.Varghese. Fast IP lookups using
  controlled prefix expansion, Sigmetrics, June
  1998.

73
References

• Switching
• N. McKeown, A. Mekkittikul, V. Anantharam, and J.
  Walrand. Achieving 100 Throughput in an
  Input-Queued Switch. IEEE Transactions on
  Communications, 47(8), Aug 1999.
• A. Mekkittikul and N. W. McKeown, "A practical
  algorithm to achieve 100 throughput in
  input-queued switches," in Proceedings of IEEE
  INFOCOM '98, March 1998.
• L. Tassiulas, Linear complexity algorithms for
  maximum throughput in radio networks and input
  queued switchs, in Proc. IEEE INFOCOM 98, San
  Francisco CA, April 1998.
• D. Shah, P. Giaccone and B. Prabhakar, An
  efficient randomized algorithm for input-queued
  switch scheduling, in Proc. Hot Interconnects
  2001.
• J. Dai and B. Prabhakar, "The throughput of data
  switches with and without speedup," in
  Proceedings of IEEE INFOCOM '00, Tel Aviv,
  Israel, March 2000, pp. 556 -- 564.
• C.-S. Chang, D.-S. Lee, Y.-S. Jou, Load balanced
  Birkhoff-von Neumann switches, Proceedings of
  IEEE HPSR 01, May 2001, Dallas, Texas.

74
References

• Future
• C.-S. Chang, D.-S. Lee, Y.-S. Jou, Load balanced
  Birkhoff-von Neumann switches, Proceedings of
  IEEE HPSR 01, May 2001, Dallas, Texas.
• Pablo Molinero-Fernndez, Nick McKeown "TCP
  Switching Exposing circuits to IP" Hot
  Interconnects IX, Stanford University, August
  2001
• S. Iyer, N. McKeown, "Making parallel packet
  switches practical," in Proc. IEEE INFOCOM 01,
  April 2001, Alaska.