Continuous Time Markov Chains and Basic Queueing Theory

1
Continuous Time Markov Chainsand Basic Queueing
Theory

• EE384X Review 4
• Winter 2008

2
Review DTMC

• pij is the transition probability from i to j
  over one time slot
• The time spent in a state is geometrically
  distributed
• Result of the Markov (memoryless) property
• When there is a jump from state i, it goes to
  state j with probability

3
Continuous Time Version

• qij is the transition rate from state i to state
  j

4
CTMC

• Upon entering state i, a random timer
  TijExp(qij) is started for each potential
  transition i!j
• These timers are independent of each other
• Recall that Exponential distribution is
  memoryless
• When the first timer expires, the MC makes the
  corresponding transition
• Let Ti be the time spent in state i, and qiåj¹i
  qij, then Ti Exp(qi)
• When there is a transition, the probability of
  jumping to state j is qij /qi

5
Definitions

• X(t)t0 is a continuous time Markov chain
  ifPX(st)j X(u) us PX(st)j X(s)
• Similar to Discrete Time MCs, Continuous Time MCs
  have stationary distribution p
• Exists when Markov chain is positive recurrent
  and irreducible

6
Stationary Distribution

• Balance equations
• Transition rates in and out of state i are equal
• Define matrix transition rate Q (qij) with qii
  -qi , then p Q 0, where p is a row vector
• Together with åi p(i) 1, can solve for p

7
Queueing Theory Notation

• A/S/s/k
• A is the arrival process, e.g., Geometric,
  Poisson, Deterministic
• S is the service distribution, e.g., Geometric,
  Exponential, Deterministic
• s is the number of servers, e.g., 1, N, 1
• k is the buffer size (if k is absent, then k 1)
• E.g., Geom/M/1, M/M/1, M/D/1, M/M/1

8
M/M/1 Queue

• Arrivals are Poisson with rate l
• Inter-arrival times are exp(l)
• Services are exponential with rate m
• These are also transition rates for the Markov
  chain
• This looks very similar to Geom/Geom/1 queue, but
  different

9
Solving M/M/1 Queue

• We have pi l pi1 m
• Let r l/m, then pi pi-1 r p0 ri
• If r lt 1, the stationary distribution exists
  pi (1 - r) ri
• Average Queue size

10
M/M/1 Queue

• NQ is the queue size, excluding the one in
  service

11
M/M/1 Queue

• Customer arrival process is Poisson(l)
• All customers are served in parallel exp(m)
• Departure rate proportional to of customers

12
Solving M/M/1 Queue

• We have pi-1 l pi i m
• Let r l/m, then
• Thus

13
M/M/1 Queue

• The queue size distribution of the M/M/1 queue is
  Poisson(r)
• Therefore the average queue size is E(Q)r
• Whats the condition for the queue to be
  recurrent?