Loading...

PPT – Lecture 3: Markov processes, master equation PowerPoint presentation | free to download - id: 74c5cd-ZmQ3Y

The Adobe Flash plugin is needed to view this content

Lecture 3 Markov processes, master equation

- Outline
- Preliminaries and definitions
- Chapman-Kolmogorov equation
- Wiener process
- Markov chains
- eigenvectors and eigenvalues
- detailed balance
- Monte Carlo
- master equation

Stochastic processes

Random function x(t)

Stochastic processes

Random function x(t) Defined by a distribution

functional Px, or by all its moments

Stochastic processes

Random function x(t) Defined by a distribution

functional Px, or by all its moments

Stochastic processes

Random function x(t) Defined by a distribution

functional Px, or by all its moments or by

its characteristic functional

Stochastic processes

Random function x(t) Defined by a distribution

functional Px, or by all its moments or by

its characteristic functional

Stochastic processes (2)

Cumulant generating functional

Stochastic processes (2)

Cumulant generating functional

Stochastic processes (2)

Cumulant generating functional where

Stochastic processes (2)

Cumulant generating functional where

correlation function

Stochastic processes (2)

Cumulant generating functional where etc.

correlation function

Stochastic processes (3)

Gaussian process

Stochastic processes (3)

Gaussian process

Stochastic processes (3)

Gaussian process (no higher-order

cumulants)

Stochastic processes (3)

Gaussian process (no higher-order

cumulants) Conditional probabilities

Stochastic processes (3)

Gaussian process (no higher-order

cumulants) Conditional probabilities

Stochastic processes (3)

Gaussian process (no higher-order

cumulants) Conditional probabilities

probability of x(t1) x(tk), given x(tk1)

x(tm)

Wiener-Khinchin theorem

Fourier analyze x(t)

Wiener-Khinchin theorem

Fourier analyze x(t) Power spectrum

Wiener-Khinchin theorem

Fourier analyze x(t) Power spectrum

Wiener-Khinchin theorem

Fourier analyze x(t) Power spectrum

Wiener-Khinchin theorem

Fourier analyze x(t) Power spectrum

Wiener-Khinchin theorem

Fourier analyze x(t) Power spectrum

Wiener-Khinchin theorem

Fourier analyze x(t) Power spectrum

Power spectrum is Fourier transform of the

correlation function

Markov processes

No information about the future from past values

earlier than the latest available

Markov processes

No information about the future from past values

earlier than the latest available

Markov processes

No information about the future from past values

earlier than the latest available

Can get general distribution by iterating

Q

Markov processes

No information about the future from past values

earlier than the latest available

Can get general distribution by iterating

Q

Markov processes

No information about the future from past values

earlier than the latest available

Can get general distribution by iterating

Q where P(x(t0)) is the initial

distribution.

Markov processes

No information about the future from past values

earlier than the latest available

Can get general distribution by iterating

Q where P(x(t0)) is the initial

distribution. Integrate this over x(tn-1),

x(t1) to get

Markov processes

No information about the future from past values

earlier than the latest available

Can get general distribution by iterating

Q where P(x(t0)) is the initial

distribution. Integrate this over x(tn-1),

x(t1) to get

Markov processes

No information about the future from past values

earlier than the latest available

Can get general distribution by iterating

Q where P(x(t0)) is the initial

distribution. Integrate this over x(tn-1),

x(t1) to get The case n 2 is the

Chapman-Kolmogorov equation

Chapman-Kolmogorov equation

Chapman-Kolmogorov equation

(for any t)

Chapman-Kolmogorov equation

(for any t)

- Examples
- Wiener process (Brownian motion/random walk)

Chapman-Kolmogorov equation

(for any t)

- Examples
- Wiener process (Brownian motion/random walk)

Chapman-Kolmogorov equation

(for any t)

- Examples
- Wiener process (Brownian motion/random walk)
- (cumulative) Poisson process

Markov chains

Both t and x discrete, assuming stationarity

Markov chains

Both t and x discrete, assuming stationarity

Markov chains

Both t and x discrete, assuming stationarity

(because

they are probabilities)

Markov chains

Both t and x discrete, assuming stationarity

(because

they are probabilities) Equation of motion

Markov chains

Both t and x discrete, assuming stationarity

(because

they are probabilities) Equation of motion

Formal solution

Markov chains (2) properties of T

T has a left eigenvector

Markov chains (2) properties of T

T has a left eigenvector

(because )

Markov chains (2) properties of T

T has a left eigenvector

(because ) Its eigenvalue is

1.

Markov chains (2) properties of T

T has a left eigenvector

(because ) Its eigenvalue is

1. The corresponding right eigenvector is

Markov chains (2) properties of T

T has a left eigenvector

(because ) Its eigenvalue is

1. The corresponding right eigenvector

is (the stationary state, because the

eigenvalue is 1 )

Markov chains (2) properties of T

T has a left eigenvector

(because ) Its eigenvalue is

1. The corresponding right eigenvector

is (the stationary state, because the

eigenvalue is 1 ) For all other

right eigenvectors with components

Markov chains (2) properties of T

T has a left eigenvector

(because ) Its eigenvalue is

1. The corresponding right eigenvector

is (the stationary state, because the

eigenvalue is 1 ) For all other

right eigenvectors with components

Markov chains (2) properties of T

T has a left eigenvector

(because ) Its eigenvalue is

1. The corresponding right eigenvector

is (the stationary state, because the

eigenvalue is 1 ) For all other

right eigenvectors with components (because

they must be orthogonal to

)

Markov chains (2) properties of T

T has a left eigenvector

(because ) Its eigenvalue is

1. The corresponding right eigenvector

is (the stationary state, because the

eigenvalue is 1 ) For all other

right eigenvectors with components (because

they must be orthogonal to

) All other eigenvalues are lt 1.

Detailed balance

If there is a stationary distribution P0 with

components and

Detailed balance

If there is a stationary distribution P0 with

components and

Detailed balance

If there is a stationary distribution P0 with

components and

Detailed balance

If there is a stationary distribution P0 with

components and can prove (if ergodicity)

convergence to P0 from any initial state

Detailed balance

If there is a stationary distribution P0 with

components and can prove (if ergodicity)

convergence to P0 from any initial state

Can reach any state from any other and no cycles

Detailed balance

If there is a stationary distribution P0 with

components and can prove (if ergodicity)

convergence to P0 from any initial state Define

,

Can reach any state from any other and no cycles

Detailed balance

If there is a stationary distribution P0 with

components and can prove (if ergodicity)

convergence to P0 from any initial state Define

, make a similarity

transformation

Can reach any state from any other and no cycles

Detailed balance

If there is a stationary distribution P0 with

components and can prove (if ergodicity)

convergence to P0 from any initial state Define

, make a similarity

transformation

Can reach any state from any other and no cycles

Detailed balance

If there is a stationary distribution P0 with

components and can prove (if ergodicity)

convergence to P0 from any initial state Define

, make a similarity

transformation R is symmetric, has complete set

of eigenvectors , components (Eigenvalues ?j

same as those of T.)

Can reach any state from any other and no cycles

Detailed balance (2)

Detailed balance (2)

Detailed balance (2)

Detailed balance (2)

Right eigenvectors of T

Detailed balance (2)

Right eigenvectors of T Now look at

evolution

Detailed balance (2)

Right eigenvectors of T Now look at

evolution

Detailed balance (2)

Right eigenvectors of T Now look at

evolution

Detailed balance (2)

Right eigenvectors of T Now look at

evolution

Detailed balance (2)

Right eigenvectors of T Now look at

evolution (since )

Detailed balance (2)

Right eigenvectors of T Now look at

evolution (since )

Monte Carlo

an example of detailed balance

Monte Carlo

an example of detailed balance Ising model

Binary spins Si(t) 1

Monte Carlo

an example of detailed balance Ising model

Binary spins Si(t) 1 Dynamics at every time

step,

Monte Carlo

an example of detailed balance Ising model

Binary spins Si(t) 1 Dynamics at every time

step, (1) choose a spin (i) at random

Monte Carlo

an example of detailed balance Ising model

Binary spins Si(t) 1 Dynamics at every time

step, (1) choose a spin (i) at random (2)

compute field of neighbors hi(t) SjJijSj(t)

Monte Carlo

an example of detailed balance Ising model

Binary spins Si(t) 1 Dynamics at every time

step, (1) choose a spin (i) at random (2)

compute field of neighbors hi(t) SjJijSj(t)

Jij Jji

Monte Carlo

an example of detailed balance Ising model

Binary spins Si(t) 1 Dynamics at every time

step, (1) choose a spin (i) at random (2)

compute field of neighbors hi(t) SjJijSj(t)

Jij Jji (3) Si(t ?t) 1 with

probability

Monte Carlo

an example of detailed balance Ising model

Binary spins Si(t) 1 Dynamics at every time

step, (1) choose a spin (i) at random (2)

compute field of neighbors hi(t) SjJijSj(t)

Jij Jji (3) Si(t ?t) 1 with

probability

Monte Carlo

an example of detailed balance Ising model

Binary spins Si(t) 1 Dynamics at every time

step, (1) choose a spin (i) at random (2)

compute field of neighbors hi(t) SjJijSj(t)

Jij Jji (3) Si(t ?t) 1 with

probability

Monte Carlo

an example of detailed balance Ising model

Binary spins Si(t) 1 Dynamics at every time

step, (1) choose a spin (i) at random (2)

compute field of neighbors hi(t) SjJijSj(t)

Jij Jji (3) Si(t ?t) 1 with

probability (equilibration of Si, given

current values of other Ss)

Monte Carlo (2)

In language of Markov chains, states (n) are

Monte Carlo (2)

In language of Markov chains, states (n)

are Single-spin flips transitions only between

neighboring points on hypercube

Monte Carlo (2)

In language of Markov chains, states (n)

are Single-spin flips transitions only between

neighboring points on hypercube

Monte Carlo (2)

In language of Markov chains, states (n)

are Single-spin flips transitions only between

neighboring points on hypercube T matrix

elements

Monte Carlo (2)

In language of Markov chains, states (n)

are Single-spin flips transitions only between

neighboring points on hypercube T matrix

elements all other Tmn 0.

Monte Carlo (2)

In language of Markov chains, states (n)

are Single-spin flips transitions only between

neighboring points on hypercube T matrix

elements all other Tmn 0. Note

Monte Carlo (2)

In language of Markov chains, states (n)

are Single-spin flips transitions only between

neighboring points on hypercube T matrix

elements all other Tmn 0. Note

Monte Carlo (3)

T satisfies detailed balance

Monte Carlo (3)

T satisfies detailed balance where p0 is the

Gibbs distribution

Monte Carlo (3)

T satisfies detailed balance where p0 is the

Gibbs distribution After many Monte Carlo

steps, converge to p0

Monte Carlo (3)

T satisfies detailed balance where p0 is the

Gibbs distribution After many Monte Carlo

steps, converge to p0 Ss sample Gibbs

distribution

Monte Carlo (3) Metropolis version

The foregoing was for heat-bath MC. Another

possibility is the Metropolis algorithm

Monte Carlo (3) Metropolis version

The foregoing was for heat-bath MC. Another

possibility is the Metropolis algorithm If hiSi

lt 0, Si(t?t) -Si(t),

Monte Carlo (3) Metropolis version

The foregoing was for heat-bath MC. Another

possibility is the Metropolis algorithm If hiSi

lt 0, Si(t?t) -Si(t), If hiSi gt 0,

Si(t?t) -Si(t) with probability

exp(-hiSi)

Monte Carlo (3) Metropolis version

The foregoing was for heat-bath MC. Another

possibility is the Metropolis algorithm If hiSi

lt 0, Si(t?t) -Si(t), If hiSi gt 0,

Si(t?t) -Si(t) with probability

exp(-hiSi) Thus,

Monte Carlo (3) Metropolis version

The foregoing was for heat-bath MC. Another

possibility is the Metropolis algorithm If hiSi

lt 0, Si(t?t) -Si(t), If hiSi gt 0,

Si(t?t) -Si(t) with probability

exp(-hiSi) Thus,

Monte Carlo (3) Metropolis version

The foregoing was for heat-bath MC. Another

possibility is the Metropolis algorithm If hiSi

lt 0, Si(t?t) -Si(t), If hiSi gt 0,

Si(t?t) -Si(t) with probability

exp(-hiSi) Thus, In either case,

Monte Carlo (3) Metropolis version

The foregoing was for heat-bath MC. Another

possibility is the Metropolis algorithm If hiSi

lt 0, Si(t?t) -Si(t), If hiSi gt 0,

Si(t?t) -Si(t) with probability

exp(-hiSi) Thus, In either

case, i.e., detailed balance with Gibbs

Continuous-time limit master equation

For Markov chain

Continuous-time limit master equation

For Markov chain

Continuous-time limit master equation

For Markov chain Differential

equation

Continuous-time limit master equation

For Markov chain Differential equation In

components

Continuous-time limit master equation

For Markov chain Differential equation In

components (using normalization of columns of

T)

Continuous-time limit master equation

For Markov chain Differential equation In

components (using normalization of columns of

T) (expect , m ? n)

Continuous-time limit master equation

For Markov chain Differential equation In

components (using normalization of columns of

T) (expect , m ? n)

transition rate matrix