EE255/CPS226 Stochastic Processes

1
EE255/CPS226Stochastic Processes

• Dept. of Electrical Computer engineering
• Duke University
• Email bbm_at_ee.duke.edu, kst_at_ee.duke.edu

2
What is a stochastic process?

• Stochastic Process is a family of rvs X(t)t e
  T (T is an index set it may be discrete or
  continuous)
• Values assumed by X(t) are called states.
• State space (I) set of all possible states
• Example cosmic radio noise at antenna a1, a2,
  .., ak.

t1
3
Stochastic Process Characterization

• Sample space S set of antennas.
• Sample the output of all antennas at time t1 ( ?
  rv), i.e. we can define rv X(t1).
• In general, we can define
• At a fixed time tt1, we can define Xt1(s)
  X(t1,s) (rv X(t1)). Similarly, we can define,
  X(t2), .., X(tk).
• X(t1) can be characterized by its distribution
  function,
• We can also a joint variable, characterized by
  its CDF as,
• Discrete and continuous cases
• States X(t) (i.e. time t) may be
  discrete/continuous
• State space I (i.e. sample space S) may be
  discrete/continuous

4
Classification of Stochastic Processes

• Four classes of stochastic processes
• Discrete-state process ? chain (e.g., DJIA index
  at any time)
• discrete-time process ? stochastic sequence Xn
  n ? T (e.g., probing a system every 10 ms.)

5
Example a Queuing System

• Inter arrival times Y1, Y2, (mutually
  independent) (FY)
• Service times S1, S2, (mutually
  independent) (FS)
• Notation for a queuing system Fy /FY /m
• Possible arrival/service time distributions
  types are
• M Memory-less (i.e., EXP)
• D Deterministic
• G General distribution
• Ek k-stage Erlang etc.
• M/M/1 ? Memory-less arrival/departure
  processes with 1-service station

6
Discrete/Continuous Stochastic Processes

• Nk Number of jobs waiting in the system at the
  time of kth jobs departure ? Stochastic process
  Nkk1,2,
• Discrete time, discrete space

Nk
Discrete
k
Discrete
7
Continuous Time, Discrete Space

• X(t) Number of jobs in the system at time t.
  X(t)t ? T forms a continuous-time,
  discrete-state stochastic process, with,

X(t)
Discrete
Continuous
8
Discrete Time, Continuous Space

• Wk wait time for the kth job. Then Wk k ? T
  forms a Discrete-time, Continuous-state
  stochastic process, where,

Wk
Continuous
k
Discrete
9
Continuous Time, Continuous Space

• Y(t) total service time for all jobs in the
  system at time t. Y(t) forms a continuous-time,
  continuous-state stochastic process, Where,

Y(t)
t
10
Further Classification

• Similarly, we can define nth order distribution
• Difficult to compute nth order distribution.

(1st order distribution)
(2nd order distribution)

11
Further Classification (contd.)

• Can the nth order distribution computations be
  simplified?
• Yes. Under some simplifying assumptions
• Stationary (strict)
• F(xt) F(xtt) ? all moments are
  time-invariant
• Independence
• As consequence of independence, we can define
  Renewal Process
• Discrete time independent process Xnn1,2,
  (X1, X2, .. are iid, non-negative rvs), e.g.,
  repair/replacement after a failure. Markov
  process removes independence restriction.
• Markov Process
• Stochastic proc. X(t) t ? T is Markov if for
  any t0 lt t1lt lt tnlt t, the conditional
  distribution

12
Markov Process

• Mostly, we will deal with discrete state Markov
  process i.e., Markov chains
• In some situations, a Markov process may also
  exhibit invariance wrt to the time origin, i.e.
  time-homogeneity
• time-homogeneity does not imply stationarity.
  This also means that while conditional pdf may be
  stationary, the joint pdf may not be so.
• Homogeneous Markov process ? process is
  completely summarized by its current state
  (independent of how it reached this particular
  state).
• Let, Y time spent in a given state

13
Markov Process-Sojourn time

• Y is also called the sojourn time
• This result says that for a homogeneous discrete
  time Markov chain, sojourn time in a state
  follows EXP( ) distribution.
• Semi-Markov process is one in which the sojourn
  time in state may not be EPX( ) distributed.

14
Renewal Counting Process

• Renewal counting process of renewals (repairs,
  replacements, arrivals) in time t a continuous
  time process
• If time interval between two renewals follows EXP
  distribution, then ? Poisson Process

15
Stationarity Properties

• Strict sense Stationarity
• Stationary in the mean ? EX(t) EX
• In general, if
• Then, a process is said to be wide-sense
  stationary
• Strict-sense stationarity ? wide-sense
  stationarity

16
Bernoulli Process

• A set of Bernoulli sequences, Yii1,2,3,.., Yi
  1 or 0
• Yi forms a Bernoulli Process. Often Yis are
  independent.
• EYi p EYi2 p VarYi p(1-p)
• Define another stochastic process ,
  Snn1,2,3,.., where Sn Y1 Y2 Yn
  (i.e. Sn sequence of partial sums)
• Sn Sn-1 Yn (recursive form)
• PSn k Sn-1 k PYn 0 (1-p) and,
• PSn k Sn-1 k-1 PYn 1 p
• Sn n1,2,3,.., forms a Binomial process
• PSn k

17
Binomial Process Properties

• Viewing successes in a Bernoulli process as
  arrivals, then,
• define discrete rv T1 trials up to including
  1st success (arrival)
• T1 First order inter-arrival time and v has a
  Geometric distribution
• PT1 i p(1-p)i-1, i1,2, ET1 1/p
  VarT1 (1-p)/p2
• Geometric Distribution ? memory-less property.
• Cond. pmf PT1 i no success in the previous m
  trials p
• Since we treat arrival as success in Sn,
  occupancy time in state Sn is memory-less
• Generalization to rth order inter-arrival time
  Tr trial trials up to including rth arrival.
• Distribution for Tr r-fold convolution of T1s
  distribution.
• Non-homogeneous Bernouli process.

18
Poisson Process

• A continuous time, discrete state process.
• N(t) no. of events occurring in time (0, t.
  Events may be,
• of packets arriving at a router port
• of incoming telephone calls at a switch
• of jobs arriving at file/computer server
• Number of failed components in time interval
• Events occurs successively and that intervals
  between these successive events are iid rvs, each
  following EXP( )
• ? average arrival rate (1/ ? average time
  between arrivals)
• ? average failure rate (1/ ? average time
  between failures)

19
Poisson Process (contd.)

• N(t) forms a Poisson process provided
• N(0) 0
• Events within non-overlapping intervals are
  independent
• In a very small interval h, only one event may
  occur (prob. p(h))
• Letting, pn(t) PN(t)n,
• Hence, for a Poisson process, interval arrival
  times follow EXP( ) (memory-less) distribution.
  Such a Poisson process is non-stationary.
• Mean Var ?t What about EN(t)/t, as t ?
  infinity?

20
Merged Multiple Poisson Process Streams

• Consider the system,
• Proof Using z-transform. Letting, a ?t,

21
Decomposing a Poisson Process Stream

• Decompose a Poisson process into multiple streams
• N arrivals decomposed into n1, n2, .., nk N
  n1n2, ..,nk
• Cond. pmf
• Since,
• The uncond. pmf

22
Renewal Counting Process

• Poisson process ? EXP( ) distributed
  inter-arrival times.
• What if the EXP( ) assumption is removed ?
  renewal proc.
• Renewal proc. Xii1,2, (Xis are iid
  non-EXP rvs)
• Xi time gap between the occurrence of ith and
  (i1)st event
• Sk X1 X2 .. Xk ? time to occurrence of
  the kth event.
• N(t)- Renewal counting process is a
  discrete-state, continuous-time stochastic. N(t)
  denotes no. of renewals in the interval (0, t.

23
Renewal Counting Processes (contd.)
Sn t

• For N(t), what is P(N(t) n)?
• nth renewal takes place at time t (account for
  the equality)
• If the nth renewal occurs at time tn lt t, then
  one or more renewals occur in the interval (tn lt
  t.

tn
More arrivals possible
24
Renewal Counting Process Expectation

• Let, m(t) EN(t). Then, m(t) mean no. of
  arrivals in time (0,t. m(t) is called the
  renewal function.

25
Renewal Density Function

• Renewal density function
• For example, if the renewal interval X is EXP(?
  x), then
• d(t) ? , t gt 0 and m(t) ? t , t gt 0.
• PN(t)n
• Fn(t) will turn out to be

e? t (? t)n/n! i.e Poisson process pmf
n-stage Erlang
26
Availability Analysis

• Availability is defined is the ability of a
  system to provide the desired service.
• If no repairs/replacements, Availability
  Reliability.
• If repairs are possible, then above def. is
  pessimistic.
• MTBF EDiTi1 ETiDiEXiMTTFMTTR

MTBF
T1 D1 T2 D2
T3 D3 T4 D4 .
27
Availability Analysis (contd.)

• Two mutually exclusive situations
• System does not fail before time t ? A(t) R(t)
• System fails, but the repair is completed before
  time t
• Therefore, A(t) sum of these two probabilities

renewal
Repair is completed with in this interval
t
x
28
Availability Expression

• dA(x) Incremental availability
• dA(x) Prob(that after renewal, life time is gt
  (t-x) that the renewal occurs in the interval
  (x,xdx)

Repair is completed with in this interval
x
t
xdx
0
Renewed life time gt (t-x)
29
Availability Expression (contd.)

• A(t) can also be expressed in the Laplace domain.
• Since, R(t) 1-W(t) or LR(s) 1/s LW(s) 1/s
  Lw(s)/s
• What happens when t becomes very large?
• However,

30
Availability, MTTF and MTTR

• Steady state availability A is
• for small values of s,

31
Availability Example

• Assuming EXP( ) density fn for g(t) and w(t)