CPE 619Introduction to Queuing Theory

- Aleksandar Milenkovic
- The LaCASA Laboratory
- Electrical and Computer Engineering Department
- The University of Alabama in Huntsville
- http//www.ece.uah.edu/milenka
- http//www.ece.uah.edu/lacasa

Overview

- Queueing Notation
- Rules for All Queues
- Little's Law
- Types of Stochastic Processes

Queueing Models What Will You Learn?

- What are various types of queues?
- What is meant by an M/M/m/B/K queue?
- How to obtain response time, queue lengths, and

server utilizations? - How to represent a system using a network of

several queues? - How to analyze simple queueing networks?
- How to obtain bounds on the system performance

using queueing models? - How to obtain variance and other statistics on

system performance? - How to subdivide a large queueing network model

and solve it?

Basic Components of a Queue

2. Service time distribution

1. Arrival process

6. Service discipline

4. Waiting positions

5. Customer Population

3. Number of servers

Example students at a typical computer terminal

roomwith a number of terminals. If all terminals

are busy,the arriving students wait in a queue.

Kendall Notation A/S/m/B/K/SD

- A Arrival process
- S Service time distribution
- m Number of servers
- B Number of buffers (system capacity)
- K Population size, and
- SD Service discipline

Arrival Process

- Arrival times
- Interarrival times
- tj form a sequence of Independent and Identically

Distributed (IID) random variables - The most common arrival process Poisson arrivals
- Inter-arrival times are exponential IID ?

Poisson arrivals - Notation
- M Memoryless Poisson
- E Erlang
- H Hyper-exponential
- G General ? Results valid for all distributions

Service Time Distribution

- Time each student spends at the terminal
- Service times are IID
- Distribution M, E, H, or G
- Device Service center Queue
- Buffer Waiting positions

Service Disciplines

- First-Come-First-Served (FCFS)
- Last-Come-First-Served (LCFS)
- Last-Come-First-Served with Preempt and Resume

(LCFS-PR) - Round-Robin (RR) with a fixed quantum.
- Small Quantum ? Processor Sharing (PS)
- Infinite Server (IS) fixed delay
- Shortest Processing Time first (SPT)
- Shortest Remaining Processing Time first (SRPT)
- Shortest Expected Processing Time first (SEPT)
- Shortest Expected Remaining Processing Time first

(SERPT). - Biggest-In-First-Served (BIFS)
- Loudest-Voice-First-Served (LVFS)

Common Distributions

- M Exponential
- Ek Erlang with parameter k
- Hk Hyper-exponential with parameter k
- D Deterministic ? constant
- G General ? All
- Memoryless
- Expected time to the next arrival is always 1/l

regardless of the time since the last arrival - Remembering the past history does not help

Example M/M/3/20/1500/FCFS

- Time between successive arrivals is exponentially

distributed - Service times are exponentially distributed
- Three servers
- 20 Buffers 3 service 17 waiting
- After 20, all arriving jobs are lost
- Total of 1500 jobs that can be serviced
- Service discipline is first-come-first-served
- Defaults
- (Only the first 3 parameters are sufficient to

indicate the type) - Infinite buffer capacity
- Infinite population size
- FCFS service discipline
- G/G/1 G/G/1/1/1/FCFS

Group Arrivals/Service

- Bulk arrivals/service
- Mx x represents the group size
- Gx a bulk arrival or service process with

general inter-group times - Examples
- Mx/M/1 Single server queue with bulk Poisson

arrivals and exponential service times - M/Gx/m Poisson arrival process, bulk

service with general service time distribution,

and m servers

Key Variables

Key Variables (contd)

- t Inter-arrival time time between two

successive arrivals - l Mean arrival rate 1/EtMay be a function

of the state of the system, e.g., number of jobs

already in the system - s Service time per job
- m Mean service rate per server 1/Es
- Total service rate for m servers is mm
- n Number of jobs in the system. This is also

called queue length - Note Queue length includes jobs currently

receiving service as well as those waiting in the

queue

Key Variables (contd)

- nq Number of jobs waiting
- ns Number of jobs receiving service
- r Response time or the time in the system

time waiting time receiving service - w Waiting time Time between arrival and

beginning of service

Rules for All Queues

- Rules The following apply to G/G/m queues
- 1. Stability Condition l lt

mmFinite-population and the finite-buffer

systems are always stable - 2. Number in System versus Number in Queuen

nq nsNotice that n, nq, and ns are random

variables. EnEnqEnsIf the service rate

is independent of the number in the queue, - Cov(nq,ns) 0

Rules for All Queues (contd)

- 3. Number versus Time If jobs are not lost due

to insufficient buffers, Mean number of jobs in

the system Arrival rate ? Mean response time - 4. Similarly, Mean number of jobs in the queue

Arrival rate ? Mean waiting time - This is known as Little's law.
- 5. Time in System versus Time in Queue

r w sr, w, and s are random

variables - Er Ew Es

Rules for All Queues (contd)

- 6. If the service rate is independent
- of the number of jobs in the queue,

Cov(w,s)0

Little's Law

- Mean number in the system Arrival rate ? Mean

response time - This relationship applies to all systems or parts

of systems in which the number of jobs entering

the system is equal to those completing service - Named after Little (1961)
- Based on a black-box view of the system
- In systems in which some jobs are lost due to

finite buffers, the law can be applied to the

part of the system consisting of the waiting and

serving positions

Proof of Little's Law

4

NumberinSystem

3

2

1

1

2

3

4

5

6

7

8

Time

Time

- If T is large, arrivals departures N
- Arrival rate Total arrivals/Total time N/T
- Hatched areas total time spent inside the

system by all jobs J - Mean time in the system J/N
- Mean Number in the system J/T (N/T) ?(J/N)

Arrival rate ? Mean time in the system

Application of Little's Law

Arrivals

Departures

- Applying to just the waiting facility of a

service center - Mean number in the queue Arrival rate ? Mean

waiting time - Similarly, for those currently receiving the

service, we have - Mean number in service Arrival rate ? Mean

service time

Example 30.3

- A monitor on a disk server showed that the

average time to satisfy an I/O request was 100

milliseconds. The I/O rate was about 100

requests per second. What was the mean number of

requests at the disk server? - Using Little's law
- Mean number in the disk server
- Arrival rate ? Response time
- 100 (requests/second) ?(0.1 seconds)
- 10 requests

Stochastic Processes

- Process Function of time
- Stochastic Process Random variables, which are

functions of time - Example 1
- n(t) number of jobs at the CPU of a computer

system - Take several identical systems and observe n(t)
- The number n(t) is a random variable
- Can find the probability distribution functions

for n(t) at each possible value of t - Example 2
- w(t) waiting time in a queue

Types of Stochastic Processes

- Discrete or Continuous State Processes
- Markov Processes
- Birth-death Processes
- Poisson Processes

Discrete/Continuous State Processes

- Discrete Finite or Countable
- Number of jobs in a system n(t) 0, 1, 2, ....
- n(t) is a discrete state process
- The waiting time w(t) is a continuous state

process - Stochastic Chain discrete state stochastic

process

Markov Processes

- Future states are independent of the past and

depend only on the present - Named after A. A. Markov who defined and analyzed

them in 1907 - Markov Chain discrete state Markov process
- Markov ? It is not necessary to know how long the

process has been in the current state ? State

time has a memoryless (exponential) distribution - M/M/m queues can be modeled using Markov

processes - The time spent by a job in such a queue is a

Markov process and the number of jobs in the

queue is a Markov chain

Birth-Death Processes

- The discrete space Markov processes in which the

transitions are restricted to neighboring states - Process in state n can change only to state n1

or n-1. - Example the number of jobs in a queue with a

single server and individual arrivals (not bulk

arrivals)

Poisson Processes

- Interarrival time s IID and exponential ?

number of arrivals n over a given interval (t,

tx) has a Poisson distribution ? arrival

Poisson process or Poisson stream - Properties
- 1.Merging
- 2.Splitting If the probability of a job going to

ith substream is pi, each substream is also

Poisson with a mean rate of pi l

Poisson Processes (contd)

- 3.If the arrivals to a single server with

exponential service time are Poisson with mean

rate l, the departures are also Poisson with the

same rate l provided l lt m

Poisson Process (contd)

- 4. If the arrivals to a service facility with m

service centers are Poisson with a mean rate l,

the departures also constitute a Poisson stream

with the same rate l, provided l lt åi mi.

Here, the servers are assumed to have

exponentially distributed service times.

Relationship Among Stochastic Processes

Summary

- Kendall Notation A/S/m/B/k/SD, M/M/1
- Number in system, queue, waiting, serviceService

rate, arrival rate, response time, waiting time,

service time - Littles Law Mean number in system Arrival

rate X Mean time spent in the system - Processes Markov ? Memoryless, Birth-death ?

Adjacent states Poisson ? IID and exponential

inter-arrival

Analysis of A Single Queue

Overview

- Birth Death Processes
- M/M/1 Queue
- M/M/m Queue
- M/M/m/B Queue with Finite Buffers
- Results for other Queueing systems

Birth-Death Processes

- Jobs arrive one at a time (and not as a batch)
- State Number of jobs n in the system
- Arrival of a new job changes the state to n1 ?

birth - Departure of a job changes the system state to

n-1 ? death - State-transition diagram

Birth-Death Processes (contd)

- When the system is in state n, it has n jobs in

it. - The new arrivals take place at a rate ln
- The service rate is mn
- We assume that both the inter-arrival times and

service times are exponentially distributed

Theorem State Probability

- The steady-state probability pn of a birth-death

process being in state n is given by - Here, p0 is the probability of being in the zero

state

Proof

- Suppose the system is in state j at time t. There

are j jobs in the system. In the next time

interval of a very small duration Dt, the system

can move to state j-1 or j1 with the following

probabilities

Proof (contd)

- If there are no arrivals or departures, the

system will stay in state j and, thus - Dt small ? zero probability of two events (two

arrivals, two departure, or a arrival and a

departure) occurring during this interval - pj(t) probability of being in state j at time t

Proof (contd)

- The jth equation above can be written as follows
- Under steady state, pj(t) approaches a fixed

value pj, that is

Proof (contd)

- Substituting these in the jth equation, we

get - The solution to this set of equations is

Proof (contd)

- The sum of all probabilities must be equal to

one

M/M/1 Queue

- M/M/1 queue is the most commonly used type of

queue - Used to model single processor systems or to

model individual devices in a computer system - Assumes that the interarrival times and the

service times are exponentially distributed and

there is only one server - No buffer or population size limitations and the

service discipline is FCFS - Need to know only the mean arrival rate l and the

mean service rate m - State number of jobs in the system

Results for M/M/1 Queue

- Birth-death processes with
- Probability of n jobs in the system

Results for M/M/1 Queue (contd)

- The quantity l/m is called traffic intensity and

is usually denoted by symbol r. Thus - n is geometrically distributed. Utilization of

the server Probability of having one or more

jobs in the system

Results for M/M/1 Queue (contd)

- Mean number of jobs in the system
- Variance of the number of jobs in the system

Results for M/M/1 Queue (contd)

- Probability of n or more jobs in the system
- Mean response time (using the Little's law)Mean

number in the system Arrival rate Mean

response timeThat is

Results for M/M/1 Queue (contd)

- Cumulative distribution function of the response

time - The response time is exponentially distributed.

? q-percentile of the response time

Results for M/M/1 Queue (contd)

- Cumulative distribution function of the waiting

time - This is a truncated exponential distribution. Its

q-percentile is given by - The above formula applies only if q is greater

than 100(1-r). All lower percentiles are zero.

Results for M/M/1 Queue (contd)

- Mean number of jobs in the queue
- Idle ? there are no jobs in the system
- The time interval between two successive idle

intervals - All results for M/M/1 queues including some for

the busy period are summarized in Box 31.1 in the

book.

Example 31.2

- On a network gateway, measurements show that the

packets arrive at a mean rate of 125 packets per

second (pps) and the gateway takes about two

milliseconds to forward them. Using an M/M/1

model, analyze the gateway. What is the

probability of buffer overflow if the gateway

had only 13 buffers? How many buffers do we need

to keep packet loss below one packet per million?

- Arrival rate l 125 pps
- Service rate m 1/.002 500 pps
- Gateway Utilization r l/m 0.25
- Probability of n packets in the gateway

(1-r)rn 0.75(0.25)n)

Example 31.2 (contd)

- Mean Number of packets in the gateway (r/(1-r)

0.25/0.75 0.33) - Mean time spent in the gateway ((1/m)/(1-r)

(1/500)/(1-0.25) 2.66 milliseconds - Probability of buffer overflow
- To limit the probability of loss to less than

10-6 We need about ten buffers.

Example 31.2 (contd)

- The last two results about buffer overflow are

approximate. Strictly speaking, the gateway

should actually be modeled as a finite buffer

M/M/1/B queue. However, since the utilization is

low and the number of buffers is far above the

mean queue length, the results obtained are a

close approximation.

Summary

- Birth-death processes Compute probability of

having n jobs in the system - M/M/1 Queue Load-independent gt Arrivals and

service do not depend upon the number in the

system ln l, mnm - Traffic Intensity r l/ m
- Mean Number of Jobs in the system r/(1-r)
- Mean Response Time (1/m)/(1-r)

M/M/1 Queue

- Parameters
- Traffic intensity
- Stability Condition
- Traffic intensity r must be less than one
- Probability of zero jobs in the system p0 1- r
- Probability of n jobs in the system
- Mean number of jobs in the system

M/M/1 Queue (contd)

- Variance of number of jobs in the system
- Probability of k jobs in the queue

M/M/1 Queue (contd)

- Mean number of jobs in the queue
- Variance of number of jobs in the queue
- Cumulative distribution function of the response

time - Mean response time
- Variance of the response time
- q-percentile of the response time Er

ln100/(100-q) - 90-percentile of the response time 2.3Er

M/M/1 Queue (contd)

- Cumulative distribution function of waiting time
- Mean waiting time
- Variance of the waiting time
- q-percentile of the waiting time
- 90-percentile of the waiting time
- Probability of finding n or more jobs in the

system rn

M/M/1 Queue (contd)

- Probability of serving n jobs in one busy period

- Mean number of jobs served in one busy period

Kleinrock, p. 218, eq. 5.155, with

- Variance of number of jobs served in one busy

period - Mean busy period duration
- Variance of the busy period

M/M/m Queue