CPE 619 Introduction to Queuing Theory - PowerPoint PPT Presentation

View by Category
About This Presentation

CPE 619 Introduction to Queuing Theory


Last-Come-First-Served with Preempt and Resume (LCFS-PR) Round-Robin (RR) with a fixed quantum. ... Can find the probability distribution functions. for n(t) at ... – PowerPoint PPT presentation

Number of Views:112
Avg rating:3.0/5.0
Slides: 55
Provided by: Mil36
Learn more at: http://www.ece.uah.edu


Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: CPE 619 Introduction to Queuing Theory

CPE 619Introduction to Queuing Theory
  • Aleksandar Milenkovic
  • The LaCASA Laboratory
  • Electrical and Computer Engineering Department
  • The University of Alabama in Huntsville
  • http//www.ece.uah.edu/milenka
  • http//www.ece.uah.edu/lacasa

  • Queueing Notation
  • Rules for All Queues
  • Little's Law
  • Types of Stochastic Processes

Queueing Models What Will You Learn?
  • What are various types of queues?
  • What is meant by an M/M/m/B/K queue?
  • How to obtain response time, queue lengths, and
    server utilizations?
  • How to represent a system using a network of
    several queues?
  • How to analyze simple queueing networks?
  • How to obtain bounds on the system performance
    using queueing models?
  • How to obtain variance and other statistics on
    system performance?
  • How to subdivide a large queueing network model
    and solve it?

Basic Components of a Queue
2. Service time distribution
1. Arrival process
6. Service discipline
4. Waiting positions
5. Customer Population
3. Number of servers
Example students at a typical computer terminal
roomwith a number of terminals. If all terminals
are busy,the arriving students wait in a queue.
Kendall Notation A/S/m/B/K/SD
  • A Arrival process
  • S Service time distribution
  • m Number of servers
  • B Number of buffers (system capacity)
  • K Population size, and
  • SD Service discipline

Arrival Process
  • Arrival times
  • Interarrival times
  • tj form a sequence of Independent and Identically
    Distributed (IID) random variables
  • The most common arrival process Poisson arrivals
  • Inter-arrival times are exponential IID ?
    Poisson arrivals
  • Notation
  • M Memoryless Poisson
  • E Erlang
  • H Hyper-exponential
  • G General ? Results valid for all distributions

Service Time Distribution
  • Time each student spends at the terminal
  • Service times are IID
  • Distribution M, E, H, or G
  • Device Service center Queue
  • Buffer Waiting positions

Service Disciplines
  • First-Come-First-Served (FCFS)
  • Last-Come-First-Served (LCFS)
  • Last-Come-First-Served with Preempt and Resume
  • Round-Robin (RR) with a fixed quantum.
  • Small Quantum ? Processor Sharing (PS)
  • Infinite Server (IS) fixed delay
  • Shortest Processing Time first (SPT)
  • Shortest Remaining Processing Time first (SRPT)
  • Shortest Expected Processing Time first (SEPT)
  • Shortest Expected Remaining Processing Time first
  • Biggest-In-First-Served (BIFS)
  • Loudest-Voice-First-Served (LVFS)

Common Distributions
  • M Exponential
  • Ek Erlang with parameter k
  • Hk Hyper-exponential with parameter k
  • D Deterministic ? constant
  • G General ? All
  • Memoryless
  • Expected time to the next arrival is always 1/l
    regardless of the time since the last arrival
  • Remembering the past history does not help

Example M/M/3/20/1500/FCFS
  • Time between successive arrivals is exponentially
  • Service times are exponentially distributed
  • Three servers
  • 20 Buffers 3 service 17 waiting
  • After 20, all arriving jobs are lost
  • Total of 1500 jobs that can be serviced
  • Service discipline is first-come-first-served
  • Defaults
  • (Only the first 3 parameters are sufficient to
    indicate the type)
  • Infinite buffer capacity
  • Infinite population size
  • FCFS service discipline
  • G/G/1 G/G/1/1/1/FCFS

Group Arrivals/Service
  • Bulk arrivals/service
  • Mx x represents the group size
  • Gx a bulk arrival or service process with
    general inter-group times
  • Examples
  • Mx/M/1 Single server queue with bulk Poisson
    arrivals and exponential service times
  • M/Gx/m Poisson arrival process, bulk
    service with general service time distribution,
    and m servers

Key Variables
Key Variables (contd)
  • t Inter-arrival time time between two
    successive arrivals
  • l Mean arrival rate 1/EtMay be a function
    of the state of the system, e.g., number of jobs
    already in the system
  • s Service time per job
  • m Mean service rate per server 1/Es
  • Total service rate for m servers is mm
  • n Number of jobs in the system. This is also
    called queue length
  • Note Queue length includes jobs currently
    receiving service as well as those waiting in the

Key Variables (contd)
  • nq Number of jobs waiting
  • ns Number of jobs receiving service
  • r Response time or the time in the system
    time waiting time receiving service
  • w Waiting time Time between arrival and
    beginning of service

Rules for All Queues
  • Rules The following apply to G/G/m queues
  • 1. Stability Condition l lt
    mmFinite-population and the finite-buffer
    systems are always stable
  • 2. Number in System versus Number in Queuen
    nq nsNotice that n, nq, and ns are random
    variables. EnEnqEnsIf the service rate
    is independent of the number in the queue,
  • Cov(nq,ns) 0

Rules for All Queues (contd)
  • 3. Number versus Time If jobs are not lost due
    to insufficient buffers, Mean number of jobs in
    the system Arrival rate ? Mean response time
  • 4. Similarly, Mean number of jobs in the queue
    Arrival rate ? Mean waiting time
  • This is known as Little's law.
  • 5. Time in System versus Time in Queue
    r w sr, w, and s are random
  • Er Ew Es

Rules for All Queues (contd)
  • 6. If the service rate is independent
  • of the number of jobs in the queue,

Little's Law
  • Mean number in the system Arrival rate ? Mean
    response time
  • This relationship applies to all systems or parts
    of systems in which the number of jobs entering
    the system is equal to those completing service
  • Named after Little (1961)
  • Based on a black-box view of the system
  • In systems in which some jobs are lost due to
    finite buffers, the law can be applied to the
    part of the system consisting of the waiting and
    serving positions

Proof of Little's Law
  • If T is large, arrivals departures N
  • Arrival rate Total arrivals/Total time N/T
  • Hatched areas total time spent inside the
    system by all jobs J
  • Mean time in the system J/N
  • Mean Number in the system J/T (N/T) ?(J/N)
    Arrival rate ? Mean time in the system

Application of Little's Law
  • Applying to just the waiting facility of a
    service center
  • Mean number in the queue Arrival rate ? Mean
    waiting time
  • Similarly, for those currently receiving the
    service, we have
  • Mean number in service Arrival rate ? Mean
    service time

Example 30.3
  • A monitor on a disk server showed that the
    average time to satisfy an I/O request was 100
    milliseconds. The I/O rate was about 100
    requests per second. What was the mean number of
    requests at the disk server?
  • Using Little's law
  • Mean number in the disk server
  • Arrival rate ? Response time
  • 100 (requests/second) ?(0.1 seconds)
  • 10 requests

Stochastic Processes
  • Process Function of time
  • Stochastic Process Random variables, which are
    functions of time
  • Example 1
  • n(t) number of jobs at the CPU of a computer
  • Take several identical systems and observe n(t)
  • The number n(t) is a random variable
  • Can find the probability distribution functions
    for n(t) at each possible value of t
  • Example 2
  • w(t) waiting time in a queue

Types of Stochastic Processes
  • Discrete or Continuous State Processes
  • Markov Processes
  • Birth-death Processes
  • Poisson Processes

Discrete/Continuous State Processes
  • Discrete Finite or Countable
  • Number of jobs in a system n(t) 0, 1, 2, ....
  • n(t) is a discrete state process
  • The waiting time w(t) is a continuous state
  • Stochastic Chain discrete state stochastic

Markov Processes
  • Future states are independent of the past and
    depend only on the present
  • Named after A. A. Markov who defined and analyzed
    them in 1907
  • Markov Chain discrete state Markov process
  • Markov ? It is not necessary to know how long the
    process has been in the current state ? State
    time has a memoryless (exponential) distribution
  • M/M/m queues can be modeled using Markov
  • The time spent by a job in such a queue is a
    Markov process and the number of jobs in the
    queue is a Markov chain

Birth-Death Processes
  • The discrete space Markov processes in which the
    transitions are restricted to neighboring states
  • Process in state n can change only to state n1
    or n-1.
  • Example the number of jobs in a queue with a
    single server and individual arrivals (not bulk

Poisson Processes
  • Interarrival time s IID and exponential ?
    number of arrivals n over a given interval (t,
    tx) has a Poisson distribution ? arrival
    Poisson process or Poisson stream
  • Properties
  • 1.Merging
  • 2.Splitting If the probability of a job going to
    ith substream is pi, each substream is also
    Poisson with a mean rate of pi l

Poisson Processes (contd)
  • 3.If the arrivals to a single server with
    exponential service time are Poisson with mean
    rate l, the departures are also Poisson with the
    same rate l provided l lt m

Poisson Process (contd)
  • 4. If the arrivals to a service facility with m
    service centers are Poisson with a mean rate l,
    the departures also constitute a Poisson stream
    with the same rate l, provided l lt åi mi.
    Here, the servers are assumed to have
    exponentially distributed service times.

Relationship Among Stochastic Processes
  • Kendall Notation A/S/m/B/k/SD, M/M/1
  • Number in system, queue, waiting, serviceService
    rate, arrival rate, response time, waiting time,
    service time
  • Littles Law Mean number in system Arrival
    rate X Mean time spent in the system
  • Processes Markov ? Memoryless, Birth-death ?
    Adjacent states Poisson ? IID and exponential

Analysis of A Single Queue
  • Birth Death Processes
  • M/M/1 Queue
  • M/M/m Queue
  • M/M/m/B Queue with Finite Buffers
  • Results for other Queueing systems

Birth-Death Processes
  • Jobs arrive one at a time (and not as a batch)
  • State Number of jobs n in the system
  • Arrival of a new job changes the state to n1 ?
  • Departure of a job changes the system state to
    n-1 ? death
  • State-transition diagram

Birth-Death Processes (contd)
  • When the system is in state n, it has n jobs in
  • The new arrivals take place at a rate ln
  • The service rate is mn
  • We assume that both the inter-arrival times and
    service times are exponentially distributed

Theorem State Probability
  • The steady-state probability pn of a birth-death
    process being in state n is given by
  • Here, p0 is the probability of being in the zero

  • Suppose the system is in state j at time t. There
    are j jobs in the system. In the next time
    interval of a very small duration Dt, the system
    can move to state j-1 or j1 with the following

Proof (contd)
  • If there are no arrivals or departures, the
    system will stay in state j and, thus
  • Dt small ? zero probability of two events (two
    arrivals, two departure, or a arrival and a
    departure) occurring during this interval
  • pj(t) probability of being in state j at time t

Proof (contd)
  • The jth equation above can be written as follows
  • Under steady state, pj(t) approaches a fixed
    value pj, that is

Proof (contd)
  • Substituting these in the jth equation, we
  • The solution to this set of equations is

Proof (contd)
  • The sum of all probabilities must be equal to

M/M/1 Queue
  • M/M/1 queue is the most commonly used type of
  • Used to model single processor systems or to
    model individual devices in a computer system
  • Assumes that the interarrival times and the
    service times are exponentially distributed and
    there is only one server
  • No buffer or population size limitations and the
    service discipline is FCFS
  • Need to know only the mean arrival rate l and the
    mean service rate m
  • State number of jobs in the system

Results for M/M/1 Queue
  • Birth-death processes with
  • Probability of n jobs in the system

Results for M/M/1 Queue (contd)
  • The quantity l/m is called traffic intensity and
    is usually denoted by symbol r. Thus
  • n is geometrically distributed. Utilization of
    the server Probability of having one or more
    jobs in the system

Results for M/M/1 Queue (contd)
  • Mean number of jobs in the system
  • Variance of the number of jobs in the system

Results for M/M/1 Queue (contd)
  • Probability of n or more jobs in the system
  • Mean response time (using the Little's law)Mean
    number in the system Arrival rate Mean
    response timeThat is

Results for M/M/1 Queue (contd)
  • Cumulative distribution function of the response
  • The response time is exponentially distributed.
    ? q-percentile of the response time

Results for M/M/1 Queue (contd)
  • Cumulative distribution function of the waiting
  • This is a truncated exponential distribution. Its
    q-percentile is given by
  • The above formula applies only if q is greater
    than 100(1-r). All lower percentiles are zero.

Results for M/M/1 Queue (contd)
  • Mean number of jobs in the queue
  • Idle ? there are no jobs in the system
  • The time interval between two successive idle
  • All results for M/M/1 queues including some for
    the busy period are summarized in Box 31.1 in the

Example 31.2
  • On a network gateway, measurements show that the
    packets arrive at a mean rate of 125 packets per
    second (pps) and the gateway takes about two
    milliseconds to forward them. Using an M/M/1
    model, analyze the gateway. What is the
    probability of buffer overflow if the gateway
    had only 13 buffers? How many buffers do we need
    to keep packet loss below one packet per million?
  • Arrival rate l 125 pps
  • Service rate m 1/.002 500 pps
  • Gateway Utilization r l/m 0.25
  • Probability of n packets in the gateway
    (1-r)rn 0.75(0.25)n)

Example 31.2 (contd)
  • Mean Number of packets in the gateway (r/(1-r)
    0.25/0.75 0.33)
  • Mean time spent in the gateway ((1/m)/(1-r)
    (1/500)/(1-0.25) 2.66 milliseconds
  • Probability of buffer overflow
  • To limit the probability of loss to less than
    10-6 We need about ten buffers.

Example 31.2 (contd)
  • The last two results about buffer overflow are
    approximate. Strictly speaking, the gateway
    should actually be modeled as a finite buffer
    M/M/1/B queue. However, since the utilization is
    low and the number of buffers is far above the
    mean queue length, the results obtained are a
    close approximation.

  • Birth-death processes Compute probability of
    having n jobs in the system
  • M/M/1 Queue Load-independent gt Arrivals and
    service do not depend upon the number in the
    system ln l, mnm
  • Traffic Intensity r l/ m
  • Mean Number of Jobs in the system r/(1-r)
  • Mean Response Time (1/m)/(1-r)

M/M/1 Queue
  • Parameters
  • Traffic intensity
  • Stability Condition
  • Traffic intensity r must be less than one
  • Probability of zero jobs in the system p0 1- r
  • Probability of n jobs in the system
  • Mean number of jobs in the system

M/M/1 Queue (contd)
  • Variance of number of jobs in the system
  • Probability of k jobs in the queue

M/M/1 Queue (contd)
  • Mean number of jobs in the queue
  • Variance of number of jobs in the queue
  • Cumulative distribution function of the response
  • Mean response time
  • Variance of the response time
  • q-percentile of the response time Er
  • 90-percentile of the response time 2.3Er

M/M/1 Queue (contd)
  • Cumulative distribution function of waiting time
  • Mean waiting time
  • Variance of the waiting time
  • q-percentile of the waiting time
  • 90-percentile of the waiting time
  • Probability of finding n or more jobs in the
    system rn

M/M/1 Queue (contd)
  • Probability of serving n jobs in one busy period
  • Mean number of jobs served in one busy period
    Kleinrock, p. 218, eq. 5.155, with
  • Variance of number of jobs served in one busy
  • Mean busy period duration
  • Variance of the busy period

M/M/m Queue
About PowerShow.com