Sizing Router Buffers How much packet buffers does a router need?

1
Sizing Router BuffersHow much packet buffers
does a router need?

• Smaller buffers have a major effect on router
  design
• An Example
• 10Gb/s linecard with 200,000 x 56kb/s flows
• Rule-of-thumb Buffer 2.5Gbits
• Requires external, slow DRAM
• Becomes Buffer 6Mbits
• Can use on-chip, fast SRAM
• 40Gb/s linecard with 40,000 x 1Mb/s flows
• Rule-of-thumb Buffer 10Gbits
• Becomes Buffer 50Mbits
• It is estimated that reduced buffers on a line
  cards could
• Reduce power consumption by 10-40
• Save 10-40 of board space
• Reduce cost by 10-30

• What determines router buffer size?
• TCP cant fully utilize a link without any
  buffers
• Only with maximum congestion window the link is
  full
• Window scales down just after it reaches maximum

• The current Rule of Thumb
• A router needs a buffer size
• 2T is the two-way propagation delay (or just
  250ms)
• C is capacity of bottleneck link
• Context
• Mandated in backbone and edge routers.
• Appears in RFPs and IETF architectural guidelines
• Usually referenced to Villamizar and Song High
  Performance TCP in ANSNET, CCR, 1994.
• Already known by inventors of TCP Van Jacobson,
  1988

• Our new rule for sizing buffers
• A router needs a buffer size
• 2T is the two-way propagation delay
• C is capacity of bottleneck link
• n is the number of long-lived flows through the
  router
• Why? What has changed?
• High-end routers serve many unsynchronized flows
• Statistical multiplexing reduces amount of
  buffering needed

• The main role of a router buffer is to
  compensatefor the difference in outstanding
  packets
• TCP Window sizes changes over time (the TCP saw
  tooth)
• Number of outstanding packets changes over time
• Buffer has to buffer these packets

• Do these results hold for a physical router?
• Yes. We verified our model by simulation and on a
  physical router
• Measurements on a Cisco GSR with an OC3 line card
  and 400 flows
• Simulations with ns2 over a broad range of
  parameters
• Below is a side by side comparison of Model,
  Simulation and Experiments on a physical router

• For many flows, TCP windows are multiplexed
• Buffer size is reduced

TCP Flows Router Buffer Router Buffer Router Buffer Link Utilization Link Utilization Link Utilization
TCP Flows Pkts RAM Model Sim Exp
100 0.5 x 1 x 2 x 3 x 64 129 258 387 1Mb 2Mb 4Mb 8Mb 96.9 99.9 100 100 94.7 99.3 99.9 99.8 94.9 98.1 99.8 99.7
400 0.5 x 1 x 2 x 3 x 32 64 128 192 512kb 1Mb 2Mb 4Mb 99.7 100 100 100 99.2 99.8 100 100 99.5 100 100 99.9

• What is the effect of too much or too little
  buffer?
• Too much buffer causes the buffer to never empty
  and increases latency
• Too little buffering causes the buffer to be
  empty over extended periods of time and the link
  to be underutilized

• Sum of congestion windows is Gaussian
• Reason The central limit theorem
• Width of Gaussian is dependent on square root of
  number of flows
• Buffer has to be wide enough to fit Gaussian

Right amount of buffering
Right amount of buffering

• Smaller buffers help short flows
• In mixes of long and short TCP flows, short flows
  benefit from smaller buffers
• RTT is decreased significantly (almost haved) by
  smaller buffers
• Loss increases, but effect of RTT decrease
  outweighs loss
• Example Completion time of short (14 packet)
  flows in a mix of long and short flows
• We increase the number of long flows and measure
  average completion time
• With smaller buffers completion time is
  substantially lower

• Buffer requirement decreases with 1/sqrt(n)
• Verified experimentally

Overbuffered Link
Underbuffered Link