CS 352 Queue management and Queuing theory

1
CS 352Queue management and Queuing theory

• Richard Martin
• Rutgers University

2
Queues and Traffic shaping

• Big Ideas
• Traffic shaping
• Modify traffic at entrance points in the network
• Can reason about states of the interior and link
  properties, loads. (E.g. Leaky, Token bucket)
• Modify traffic in the routers
• Enforce policies on flows
• Queuing theory
• Analytic analysis of properties of queues as
  functions of the inputs and service types

3
Congestion Control

• Too many packets in some part of the system
• Congestion

4
Simplified Network Model
Input Arrivals
System
Output Departures
Goal Move packets across the system from the
inputs to output System could be a single switch,
or entire network
5
Problems with Congestion

• Performance
• Low Throughput
• Long Latency
• High Variability (jitter)
• Lost data
• Policy enforcement
• User X pays more than user Y, X should get more
  bandwidth than Y.
• Fairness
• One user/flow getting much more throughput or
  better latency than the other flows

6
Congestion Control

• Causes of congestion
• Types of congestion control schemes
• Solving congestion

7
Causes of Congestion

• Many ultimate causes
• New applications, new users, bugs, faults,
  viruses, spam, stochastic (time based)
  variations, unknown randomness.
• Congestion can be long-lived or transient
• Timescale of the congestion is important
• Microseconds vs seconds vs hours vs days
• Different solutions to all the above!

8
Exhaustion ofBuffer Space (contd)
50 Mbps
Router
50 Mbps
50 Mbps
50 Mbps
Buffer
9
Types of Congestion Control Strategies

• Terminate existing resources
• Drop packets
• Drop circuits
• Limit entry into the system
• Packet level (layer 3)
• Leaky Bucket, token bucket, WFQ
• Flow/conversation level (layer 4)
• Resource reservation
• TCP backoff/reduce window
• Application level (layer 7)
• Limit types/kinds of applications

10
Leaky Bucket

• Across a single link, only allow packets across
  at a constant rate
• Packets may be generated in a bursty manner, but
  after they pass through the leaky bucket, they
  enter the network evenly spaced
• If all inputs enforce a leaky bucket, its easy to
  reason about the total resource demand on the
  rest of the system

11
Leaky Bucket Analogy
Packets from input
Leaky Bucket
Output
12
Leaky Bucket (contd)

• The leaky bucket is a traffic shaper It
  changes the characteristics of a packet stream
• Traffic shaping makes the network more manageable
  and predictable
• Usually the network tells the leaky bucket the
  rate at which it may send packets when a
  connection is established

13
Leaky Bucket Doesnt allow bursty transmissions

• In some cases, we may want to allow short bursts
  of packets to enter the network without smoothing
  them out
• For this purpose we use a token bucket, which is
  a modified leaky bucket

14
Token Bucket

• The bucket holds logical tokens instead of
  packets
• Tokens are generated and placed into the token
  bucket at a constant rate
• When a packet arrives at the token bucket, it is
  transmitted if there is a token available.
  Otherwise it is buffered until a token becomes
  available.
• The token bucket holds a fixed number of tokens,
  so when it becomes full, subsequently generated
  tokens are discarded
• Can still reason about total possible demand

15
Token Bucket
Packets from input
Token Generator (Generates a token once every T
seconds)
output
16
Token Bucket vs. Leaky Bucket
Case 1 Short burst arrivals
Arrival time at bucket
6
5
4
3
2
1
0
Departure time from a leaky bucket Leaky bucket
rate 1 packet / 2 time units Leaky bucket size
4 packets
6
5
4
3
2
1
0
Departure time from a token bucket Token bucket
rate 1 tokens / 2 time units Token bucket size
2 tokens
6
5
4
3
2
1
0
17
Token Bucket vs. Leaky Bucket
Case 2 Large burst arrivals
Arrival time at bucket
6
5
4
3
2
1
0
Departure time from a leaky bucket Leaky bucket
rate 1 packet / 2 time units Leaky bucket size
2 packets
6
5
4
3
2
1
0
Departure time from a token bucket Token bucket
rate 1 token / 2 time units Token bucket size
2 tokens
6
5
4
3
2
1
0
18
Multi-link congestion management

• Token bucket and leaky bucket manage traffic
  across a single link.
• But what if we do not trust the incoming traffic
  to behave?
• Must manage across multiple links
• Round Robin
• Fair Queuing

19
Multi-queue management

• If one source is sending too many packets, can we
  allow other sources to continue and just drop the
  bad source?
• First cut round-robin
• Service input queues in round-robin order
• What if one flow/link has all large packets,
  another all small packets?
• Smaller packets get more link bandwidth

20
Idealized flow model

• For N sources, we would like to give each host or
  input source 1/N of the link bandwidth
• Image we could squeeze factions of a bits on the
  link at once
• -gtfluid model
• E.g. fluid would interleave the bits on the
  link
• But we must work with packets
• Want to approximate fairness of the fluid flow
  model, but still have packets

21
Fluid model vs. Packet Model
22
Fair Queuing vs. Round Robin

• Advantages protection among flows
• Misbehaving flows will not affect the performance
  of well-behaving flows
• Misbehaving flow a flow that does not implement
  congestion control
• Disadvantages
• More complex must maintain a queue per flow per
  output instead of a single queue per output
• Biased toward large packets a flow receives
  service proportional to the number of packets

23
Virtual Time

• How to keep track of service delivered on each
  queue?
• Virtual Time is the number of rounds of queue
  service completed by a bit-by-bit Round Robin
  (RR) scheduler
• May not be an integer
• increases/decreases with of active queues

24
Approximate bit-bit RR

• Virtual time is incremented each time a bit is
  transmitted for all flows
• If we have 3 active flows, and transmit 3 bits,
  we increment virtual time by 1.
• If we had 4 flows, and transmit 2 bits, increment
  Vt by 0.5.
• At each packet arrival, compute time packet would
  have exited the router during virtual time.
• This is the packet finish number

25
Fair Queuing outline

• Compute virtual time packet exits system (finish
  time)
• Service in order of increasing Finish times I.e.,
  F(t)s
• Scheduler maintains 2 variables
• Current virtual time
• lowest per-queue finish number

26
Active Queues

• A queue is active if the largest finish number is
  greater than the current virtual time
• Notice the length of a RR round (set of queue
  services) in real time is proportional to the
  number of active connections
• Allows WFQ to take advantage of idle connections

27
Computing Finish Numbers

• Finish of an arriving packet is computed as the
  size of the packet in bits the greater of
• Finish of previous packet in the same queue
• Current virtual time

28
Finish Number

• Define
• - finish of packet k of flow i (in virtual
  time)
• - length of packet k of flow I (bits)
• - Real to virtual time function
• Then The finish of packet k of flow i is

29
Fair Queuing Summary

• On packet arrival
• Compute finish number of packet
• Re-compute virtual time
• Based on number of active queues at time of
  arrival
• On packet completion
• Select packet with lowest finish number to be
  output
• Recompute virtual time

30
A Fair Queuing Example

• 3 queues A, B, and C
• At real time 0, 3 packets
• Of size 1 on A, size 2 on B and C
• A packet of size 2 shows up at Real time 4 on
  queue A

31
FQ Example

• The finish s for queues A, B and C are set to
  1, 3 and 2
• Virtual time runs at 1/3 real time

32
FQ Example- cont.

• After 1 unit of service, each connection has
  received 10.33 0.33 units of service
• Packet from queue A departed at R(t)1
• Packet from queue C is transmitting (break tie
  randomly)

33
FQ Example- cont.

• Between T1,3 there are 2 connections virtual
  time V(t) function increases by 0.5 per unit of
  real time

34
FQ Example- cont.

• Between T3,4 only 1 connection virtual time
  increases by 1.0 per unit of real time

35
Final Schedule
36
Weight on the queues

• Define
• - finish time of packet k of flow i (in virtual
  time)
• - length of packet k of flow I (bits)
• - Real to virtual time function
• - Weight of flow i
• The finishing time of packet k of flow i is

Weight
37
Weighted Fair Queuing

• Increasing the weight gives more service to a
  given queue
• Lowers finish time
• Service is proportional to the weights
• E.g. Weights of 1, 1 and 2 means one queue gets
  50 of the service, other 2 queues get 25.

38
Introduction to Queuing Theory
CS 352 Richard Martin Rutgers University
39
Queuing theory

• View network as collections of queues
• FIFO data-structures
• Queuing theory provides probabilistic analysis of
  these queues
• Examples
• Average length
• Probability queue is at a certain length
• Probability a packet will be lost

40
Littles Law
System
Arrivals
Departures

• Littles Law Mean number tasks in system
  arrival rate x mean reponse time
• Observed before, Little was first to prove
• Applies to any system in equilibrium, as long as
  nothing in black box is creating or destroying
  tasks

41
Proving Littles Law
Arrivals
Packet
Departures
1 2 3 4 5 6 7 8
Time
J Shaded area 9 Same in all cases!
42
Definitions

• J Area from previous slide
• N Number of jobs (packets)
• T Total time
• l Average arrival rate
• N/T
• W Average time job is in the system
• J/N
• L Average number of jobs in the system
• J/T

43
Proof Method 1 Definition
in System (L)

1 2 3 4 5 6 7 8
Time (T)
44
Proof Method 2 Substitution
Tautology
45
Example using Littles law

• Observe 120 cars in front of the Lincoln Tunnel
• Observe 32 cars/minute depart over a period where
  no cars in the tunnel at the start or end (e.g.
  security checks)
• What is average waiting time before and in the
  tunnel?

46
Model Queuing System
Queuing System
Server System

• Strategy
• Use Littles law on both the complete system and
  its parts to reason about average time in the
  queue

47
Kendal Notation

• Six parameters in shorthand
• First three typically used, unless specified
• Arrival Distribution
• Probability of a new packet arrives in time t
• Service Distribution
• Probability distribution packet is serviced in
  time t
• Number of servers
• Total Capacity (infinite if not specified)
• Population Size (infinite)
• Service Discipline (FCFS/FIFO)

48
Distributions

• M Exponential
• D Deterministic (e.g. fixed constant)
• Ek Erlang with parameter k
• Hk Hyperexponential with param. k
• G General (anything)
• M/M/1 is the simplest realistic queue

49
Kendal Notation Examples

• M/M/1
• Exponential arrivals and service, 1 server,
  infinite capacity and population, FCFS (FIFO)
• M/M/m
• Same, but M servers
• G/G/3/20/1500/SPF
• General arrival and service distributions, 3
  servers, 17 queue slots (20-3), 1500 total jobs,
  Shortest Packet First

50
M/M/1 queue model
L

Lq


Wq
W
51
Analysis of M/M/1 queue

• Goal A closed form expression of the probability
  of the number of jobs in the queue (Pi) given
  only l and m

52
Solving queuing systems

• Given
• l Arrival rate of jobs (packets on input link)
• m Service rate of the server (output link)
• Solve
• L average number in queuing system
• Lq average number in the queue
• W average waiting time in whole system
• Wq average waiting time in the queue
• 4 unknowns need 4 equations

53
Solving queuing systems

• 4 unknowns L, Lq W, Wq
• Relationships using Littles law
• LlW
• LqlWq (steady-state argument)
• W Wq (1/m)
• If we know any 1, can find the others
• Finding L is hard or easy depending on the type
  of system. In general

54
Equilibrium conditions
l
l
l
l
n1
n
n-1
m
m
m
m
1
2
inflow outflow
1
2
3
stability
55
Solving for P0 and Pn
1
,
,
,
2
,
,
(geometric series)
3
5
4
56
Solving for L
57
Solving W, Wq and Lq
58
Response Time vs. Arrivals
59
Stable Region
linear region
60
Empirical Example
M/M/m system
61
Example

• Measurement of a network gateway
• mean arrival rate (l) 125 Packets/s
• mean response time per packet 2 ms
• Assuming exponential arrivals departures
• What is the service rate, m ?
• What is the gateways utilization?
• What is the probability of n packets in the
  gateway?
• mean number of packets in the gateway?
• The number of buffers so P(overflow) is lt10-6?

62
Example (cont)

• The service rate, m
• utilization
• P(n) packets in the gateway

63
Example (cont)

• Mean in gateway (L)
• to limit loss probability to less than 1 in a
  million

64
Properties of a Poisson processes

• Poisson process exponential distribution
  between arrivals/departures/service
• Key properties
• memoryless
• Past state does not help predict next arrival
• Closed under
• Addition
• Subtraction

65
Addition and Subtraction

• Merge
• two poisson streams with arrival rates l1 and l2
  
• new poisson stream l3l1l2
• Split
• If any given item has a probability P1 of
  leaving the stream with rate l1
• l2(1-P1)l1

66
Queuing Networks
l2
l1
0.3
0.7
l6
l3
l4
0.5
l5
0.5
l7