Flows and Networks Plan for today (lecture 2):

1
Flows and NetworksPlan for today (lecture 2)

• Questions?
• Birth-death process
• Example pure birth process
• Example pure death process
• Simple queue
• General birth-death process equilibrium
• Reversibility, stationarity
• Truncation
• Kolmogorovs criteria
• Summary / Next
• Exercises

2
Last time on Flows and NetworksHighlights
continuous time Markov chain

• stochastic process X(t) countable or finite
  state space SMarkov propertytransition
  rates
  independent tirreducible each
  state in S reachable from any other state in
  SAssume ergodic and regular
• global balance equations (equilibrium eqns)
• p is stationary distribution
• solution that can be normalised is
  equilibrium distributionif equilibrium
  distribution exists, then it is unique and is
  limiting distribution

3
Flows and NetworksPlan for today (lecture 2)

• Questions?
• Birth-death process
• Example pure birth process
• Example pure death process
• Simple queue
• General birth-death process equilibrium
• Reversibility, stationarity
• Truncation
• Kolmogorovs criteria
• Summary / Next
• Exercises

4
Birth-death process

• State space
• Markov chain, transition rates
• Bounded state spaceq(J,J1)0 then states
  space bounded above at Jq(I,I-1)0 then state
  space bounded below at I
• Kolmogorov forward equations
• Global balance equations

5
Flows and NetworksPlan for today (lecture 2)

• Questions?
• Birth-death process
• Example pure birth process
• Example pure death process
• Simple queue
• General birth-death process equilibrium
• Reversibility, stationarity
• Truncation
• Kolmogorovs criteria
• Summary / Next
• Exercises

6
Example pure birth process

• Exponential interarrival times, mean 1/?
• Arrival process is Poisson process
• Markov chain?
• Transition rates let t0ltt1ltlttnltt
• Kolmogorov forward equations for P(X(0)0)1

7
Flows and NetworksPlan for today (lecture 2)

• Questions?
• Birth-death process
• Example pure birth process
• Example pure death process
• Simple queue
• General birth-death process equilibrium
• Reversibility, stationarity
• Truncation
• Kolmogorovs criteria
• Summary / Next
• Exercises

8
Example pure death process

• Exponential holding times, mean 1/?
• P(X(0)N)1, S0,1,,N
• Markov chain?
• Transition rates let t0ltt1ltlttnltt
• Kolmogorov forward equations for P(X(0)N)1

9
Flows and NetworksPlan for today (lecture 2)

• Questions?
• Birth-death process
• Example pure birth process
• Example pure death process
• Simple queue
• General birth-death process equilibrium
• Reversibility, stationarity
• Truncation
• Kolmogorovs criteria
• Summary / Next
• Exercises

10
Simple queue

• Poisson arrival proces rate ?, single server
  exponential service times, mean 1/?
• Assume initially emptyP(X(0)0)1,
  S0,1,2,,
• Markov chain?
• Transition rates

11
Simple queue

• Poisson arrival proces rate ?, single server
  exponential service times, mean 1/?
• Kolmogorov forward equations, jgt0

12
Simple queue (ctd)

• ? ? ?
• j
  j1? ?
• Equilibrium distribution ?lt?
• Stationary measure summable ? eq. distrib.
• Proof Insert into global balance
• Detailed balance!

13
Flows and NetworksPlan for today (lecture 2)

• Questions?
• Birth-death process
• Example pure birth process
• Example pure death process
• Simple queue
• General birth-death process equilibrium
• Reversibility, stationarity
• Truncation
• Kolmogorovs criteria
• Summary / Next
• Exercises

14
Birth-death process

• State space
• Markov chain, transition rates
• Definition Detailed balance equations
• Theorem A distribution that satisfied detailed
  balance is a stationary distribution
• Theorem Assume that then is the
  equilibrium distrubution of the birth-death
  prcess X.

15
Flows and NetworksPlan for today (lecture 2)

• Questions?
• Birth-death process
• Example pure birth process
• Example pure death process
• Simple queue
• General birth-death process equilibrium
• Reversibility, stationarity
• Truncation
• Kolmogorovs criteria
• Summary / Next
• Exercises

16
Reversibility stationarity

• Stationary process A stochastic process is
  stationary if for all t1,,tn,??
• Theorem If the initial distribution is a
  stationary distribution, then the process is
  stationary
• Reversible process A stochastic process is
  reversible if for all t1,,tn,??
• NOTE labelling of states only gives suggestion
  of one dimensional state space this is not
  required

17
Reversibility stationarity

• Lemma A reversible process is stationary.
• Theorem A stationary Markov chain is reversible
  if and only if there exists a collection of
  positive numbers p(j), j?S, summing to unity that
  satisfy the detailed balance equationsWhen
  there exists such a collection p(j), j?S, it is
  the equilibrium distribution
• Proof

18
Flows and NetworksPlan for today (lecture 2)

• Questions?
• Birth-death process
• Example pure birth process
• Example pure death process
• Simple queue
• General birth-death process equilibrium
• Reversibility, stationarity
• Truncation
• Kolmogorovs criteria
• Summary / Next
• Exercises

19
Truncation of reversible processes
10
Lemma 1.9 / Corollary 1.10 If the transition
rates of a reversible Markov process with state
space S and equilibrium distribution
are altered by changing q(j,k) to
cq(j,k) for where cgt0
then the resulting Markov process is
reversible in equilibrium and has equilibrium
distribution
where B is the normalizing constant. If
c0 then the reversible Markov process is
truncated to A and the resulting Markov
process is reversible with equilibrium
distribution
20
Time reversed process

• X(t) reversible Markov process ? X(-t) also, but
• Lemma 1.11 tijdshomogeneity not inherited for
  non-stationary process
• Theorem 1.12 If X(t) is a stationary Markov
  process with transition rates q(j,k), and
  equilibrium distribution p(j), j?S, then the
  reversed processX(?-t) is a stationary Markov
  process with transition ratesand the same
  equilibrium distribution
• Theorem 1.13 Kellys lemmaLet X(t) be a
  stationary Markov processwith transition rates
  q(j,k). If we can find a collection of numbers
  q(j,k) such that q(j)q(j), j?S, and a
  collection of positive numbers ?(j), j?S, summing
  to unity, such thatthen q(j,k) are the
  transition rates of the time-reversed process,
  and ?(j), j?S, is the equilibrium distribution of
  both processes.

21
Flows and NetworksPlan for today (lecture 2)

• Questions?
• Birth-death process
• Example pure birth process
• Example pure death process
• Simple queue
• General birth-death process equilibrium
• Reversibility, stationarity
• Truncation
• Kolmogorovs criteria
• Summary / Next
• Exercises

22
Kolmogorovs criteria

• Theorem 1.8A stationary Markov chain is
  reversible ifffor each finite sequence of
  states Notice that

23
Flows and NetworksPlan for today (lecture 2)

• Questions?
• Birth-death process
• Example pure birth process
• Example pure death process
• Simple queue
• General birth-death process equilibrium
• Reversibility, stationarity
• Truncation
• Kolmogorovs criteria
• Summary / Next
• Exercises

24
Summary / next

• Birth-death process
• Simple queue
• Reversibility, stationarity
• Truncation
• Kolmogorovs criteria
• Nextinput / output simple queuePoisson
  procesPASTAOutput simple queueTandem netwerk

25
Exercises

• RSN 1.3.2, 1.3.3, 1.3.5, 1.5.1, 1.5.2, 1.5.5,
  1.6.2, 1.6.3, 1.6.4