Loading...

PPT – CS 352 Queue management and Queuing theory PowerPoint presentation | free to view - id: 18195f-ODk4Z

The Adobe Flash plugin is needed to view this content

CS 352Queue management and Queuing theory

- Richard Martin
- Rutgers University

Queues and Traffic shaping

- Big Ideas
- Traffic shaping
- Modify traffic at entrance points in the network
- Can reason about states of the interior and link

properties, loads. (E.g. Leaky, Token bucket) - Modify traffic in the routers
- Enforce policies on flows
- Queuing theory
- Analytic analysis of properties of queues as

functions of the inputs and service types

Congestion Control

- Too many packets in some part of the system
- Congestion

Simplified Network Model

Input Arrivals

System

Output Departures

Goal Move packets across the system from the

inputs to output System could be a single switch,

or entire network

Problems with Congestion

- Performance
- Low Throughput
- Long Latency
- High Variability (jitter)
- Lost data
- Policy enforcement
- User X pays more than user Y, X should get more

bandwidth than Y. - Fairness
- One user/flow getting much more throughput or

better latency than the other flows

Congestion Control

- Causes of congestion
- Types of congestion control schemes
- Solving congestion

Causes of Congestion

- Many ultimate causes
- New applications, new users, bugs, faults,

viruses, spam, stochastic (time based)

variations, unknown randomness. - Congestion can be long-lived or transient
- Timescale of the congestion is important
- Microseconds vs seconds vs hours vs days
- Different solutions to all the above!

Exhaustion ofBuffer Space (contd)

50 Mbps

Router

50 Mbps

50 Mbps

50 Mbps

Buffer

Types of Congestion Control Strategies

- Terminate existing resources
- Drop packets
- Drop circuits
- Limit entry into the system
- Packet level (layer 3)
- Leaky Bucket, token bucket, WFQ
- Flow/conversation level (layer 4)
- Resource reservation
- TCP backoff/reduce window
- Application level (layer 7)
- Limit types/kinds of applications

Leaky Bucket

- Across a single link, only allow packets across

at a constant rate - Packets may be generated in a bursty manner, but

after they pass through the leaky bucket, they

enter the network evenly spaced - If all inputs enforce a leaky bucket, its easy to

reason about the total resource demand on the

rest of the system

Leaky Bucket Analogy

Packets from input

Leaky Bucket

Output

Leaky Bucket (contd)

- The leaky bucket is a traffic shaper It

changes the characteristics of a packet stream - Traffic shaping makes the network more manageable

and predictable - Usually the network tells the leaky bucket the

rate at which it may send packets when a

connection is established

Leaky Bucket Doesnt allow bursty transmissions

- In some cases, we may want to allow short bursts

of packets to enter the network without smoothing

them out - For this purpose we use a token bucket, which is

a modified leaky bucket

Token Bucket

- The bucket holds logical tokens instead of

packets - Tokens are generated and placed into the token

bucket at a constant rate - When a packet arrives at the token bucket, it is

transmitted if there is a token available.

Otherwise it is buffered until a token becomes

available. - The token bucket holds a fixed number of tokens,

so when it becomes full, subsequently generated

tokens are discarded - Can still reason about total possible demand

Token Bucket

Packets from input

Token Generator (Generates a token once every T

seconds)

output

Token Bucket vs. Leaky Bucket

Case 1 Short burst arrivals

Arrival time at bucket

6

5

4

3

2

1

0

Departure time from a leaky bucket Leaky bucket

rate 1 packet / 2 time units Leaky bucket size

4 packets

6

5

4

3

2

1

0

Departure time from a token bucket Token bucket

rate 1 tokens / 2 time units Token bucket size

2 tokens

6

5

4

3

2

1

0

Token Bucket vs. Leaky Bucket

Case 2 Large burst arrivals

Arrival time at bucket

6

5

4

3

2

1

0

Departure time from a leaky bucket Leaky bucket

rate 1 packet / 2 time units Leaky bucket size

2 packets

6

5

4

3

2

1

0

Departure time from a token bucket Token bucket

rate 1 token / 2 time units Token bucket size

2 tokens

6

5

4

3

2

1

0

Multi-link congestion management

- Token bucket and leaky bucket manage traffic

across a single link. - But what if we do not trust the incoming traffic

to behave? - Must manage across multiple links
- Round Robin
- Fair Queuing

Multi-queue management

- If one source is sending too many packets, can we

allow other sources to continue and just drop the

bad source? - First cut round-robin
- Service input queues in round-robin order
- What if one flow/link has all large packets,

another all small packets? - Smaller packets get more link bandwidth

Idealized flow model

- For N sources, we would like to give each host or

input source 1/N of the link bandwidth - Image we could squeeze factions of a bits on the

link at once - -gtfluid model
- E.g. fluid would interleave the bits on the

link - But we must work with packets
- Want to approximate fairness of the fluid flow

model, but still have packets

Fluid model vs. Packet Model

Fair Queuing vs. Round Robin

- Advantages protection among flows
- Misbehaving flows will not affect the performance

of well-behaving flows - Misbehaving flow a flow that does not implement

congestion control - Disadvantages
- More complex must maintain a queue per flow per

output instead of a single queue per output - Biased toward large packets a flow receives

service proportional to the number of packets

Virtual Time

- How to keep track of service delivered on each

queue? - Virtual Time is the number of rounds of queue

service completed by a bit-by-bit Round Robin

(RR) scheduler - May not be an integer
- increases/decreases with of active queues

Approximate bit-bit RR

- Virtual time is incremented each time a bit is

transmitted for all flows - If we have 3 active flows, and transmit 3 bits,

we increment virtual time by 1. - If we had 4 flows, and transmit 2 bits, increment

Vt by 0.5. - At each packet arrival, compute time packet would

have exited the router during virtual time. - This is the packet finish number

Fair Queuing outline

- Compute virtual time packet exits system (finish

time) - Service in order of increasing Finish times I.e.,

F(t)s - Scheduler maintains 2 variables
- Current virtual time
- lowest per-queue finish number

Active Queues

- A queue is active if the largest finish number is

greater than the current virtual time - Notice the length of a RR round (set of queue

services) in real time is proportional to the

number of active connections - Allows WFQ to take advantage of idle connections

Computing Finish Numbers

- Finish of an arriving packet is computed as the

size of the packet in bits the greater of - Finish of previous packet in the same queue
- Current virtual time

Finish Number

- Define
- - finish of packet k of flow i (in virtual

time) - - length of packet k of flow I (bits)
- - Real to virtual time function
- Then The finish of packet k of flow i is

Fair Queuing Summary

- On packet arrival
- Compute finish number of packet
- Re-compute virtual time
- Based on number of active queues at time of

arrival - On packet completion
- Select packet with lowest finish number to be

output - Recompute virtual time

A Fair Queuing Example

- 3 queues A, B, and C
- At real time 0, 3 packets
- Of size 1 on A, size 2 on B and C
- A packet of size 2 shows up at Real time 4 on

queue A

FQ Example

- The finish s for queues A, B and C are set to

1, 3 and 2 - Virtual time runs at 1/3 real time

FQ Example- cont.

- After 1 unit of service, each connection has

received 10.33 0.33 units of service - Packet from queue A departed at R(t)1
- Packet from queue C is transmitting (break tie

randomly)

FQ Example- cont.

- Between T1,3 there are 2 connections virtual

time V(t) function increases by 0.5 per unit of

real time

FQ Example- cont.

- Between T3,4 only 1 connection virtual time

increases by 1.0 per unit of real time

Final Schedule

Weight on the queues

- Define
- - finish time of packet k of flow i (in virtual

time) - - length of packet k of flow I (bits)
- - Real to virtual time function
- - Weight of flow i
- The finishing time of packet k of flow i is

Weight

Weighted Fair Queuing

- Increasing the weight gives more service to a

given queue - Lowers finish time
- Service is proportional to the weights
- E.g. Weights of 1, 1 and 2 means one queue gets

50 of the service, other 2 queues get 25.

Introduction to Queuing Theory

CS 352 Richard Martin Rutgers University

Queuing theory

- View network as collections of queues
- FIFO data-structures
- Queuing theory provides probabilistic analysis of

these queues - Examples
- Average length
- Probability queue is at a certain length
- Probability a packet will be lost

Littles Law

System

Arrivals

Departures

- Littles Law Mean number tasks in system

arrival rate x mean reponse time - Observed before, Little was first to prove
- Applies to any system in equilibrium, as long as

nothing in black box is creating or destroying

tasks

Proving Littles Law

Arrivals

Packet

Departures

1 2 3 4 5 6 7 8

Time

J Shaded area 9 Same in all cases!

Definitions

- J Area from previous slide
- N Number of jobs (packets)
- T Total time
- l Average arrival rate
- N/T
- W Average time job is in the system
- J/N
- L Average number of jobs in the system
- J/T

Proof Method 1 Definition

in System (L)

1 2 3 4 5 6 7 8

Time (T)

Proof Method 2 Substitution

Tautology

Example using Littles law

- Observe 120 cars in front of the Lincoln Tunnel
- Observe 32 cars/minute depart over a period where

no cars in the tunnel at the start or end (e.g.

security checks) - What is average waiting time before and in the

tunnel?

Model Queuing System

Queuing System

Server System

- Strategy
- Use Littles law on both the complete system and

its parts to reason about average time in the

queue

Kendal Notation

- Six parameters in shorthand
- First three typically used, unless specified
- Arrival Distribution
- Probability of a new packet arrives in time t
- Service Distribution
- Probability distribution packet is serviced in

time t - Number of servers
- Total Capacity (infinite if not specified)
- Population Size (infinite)
- Service Discipline (FCFS/FIFO)

Distributions

- M Exponential
- D Deterministic (e.g. fixed constant)
- Ek Erlang with parameter k
- Hk Hyperexponential with param. k
- G General (anything)
- M/M/1 is the simplest realistic queue

Kendal Notation Examples

- M/M/1
- Exponential arrivals and service, 1 server,

infinite capacity and population, FCFS (FIFO) - M/M/m
- Same, but M servers
- G/G/3/20/1500/SPF
- General arrival and service distributions, 3

servers, 17 queue slots (20-3), 1500 total jobs,

Shortest Packet First

M/M/1 queue model

L

Lq

Wq

W

Analysis of M/M/1 queue

- Goal A closed form expression of the probability

of the number of jobs in the queue (Pi) given

only l and m

Solving queuing systems

- Given
- l Arrival rate of jobs (packets on input link)
- m Service rate of the server (output link)
- Solve
- L average number in queuing system
- Lq average number in the queue
- W average waiting time in whole system
- Wq average waiting time in the queue
- 4 unknowns need 4 equations

Solving queuing systems

- 4 unknowns L, Lq W, Wq
- Relationships using Littles law
- LlW
- LqlWq (steady-state argument)
- W Wq (1/m)
- If we know any 1, can find the others
- Finding L is hard or easy depending on the type

of system. In general

Equilibrium conditions

l

l

l

l

n1

n

n-1

m

m

m

m

1

2

inflow outflow

1

2

3

stability

Solving for P0 and Pn

1

,

,

,

2

,

,

(geometric series)

3

5

4

Solving for L

Solving W, Wq and Lq

Response Time vs. Arrivals

Stable Region

linear region

Empirical Example

M/M/m system

Example

- Measurement of a network gateway
- mean arrival rate (l) 125 Packets/s
- mean response time per packet 2 ms
- Assuming exponential arrivals departures
- What is the service rate, m ?
- What is the gateways utilization?
- What is the probability of n packets in the

gateway? - mean number of packets in the gateway?
- The number of buffers so P(overflow) is lt10-6?

Example (cont)

- The service rate, m
- utilization
- P(n) packets in the gateway

Example (cont)

- Mean in gateway (L)
- to limit loss probability to less than 1 in a

million

Properties of a Poisson processes

- Poisson process exponential distribution

between arrivals/departures/service - Key properties
- memoryless
- Past state does not help predict next arrival
- Closed under
- Addition
- Subtraction

Addition and Subtraction

- Merge
- two poisson streams with arrival rates l1 and l2

- new poisson stream l3l1l2
- Split
- If any given item has a probability P1 of

leaving the stream with rate l1 - l2(1-P1)l1

Queuing Networks

l2

l1

0.3

0.7

l6

l3

l4

0.5

l5

0.5

l7