# The Markov Chain Monte Carlo Method - PowerPoint PPT Presentation

PPT – The Markov Chain Monte Carlo Method PowerPoint presentation | free to download - id: 3ec305-ZWIyY

The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
Title:

## The Markov Chain Monte Carlo Method

Description:

### The Markov Chain Monte Carlo Method Isabelle Stanton May 8, 2008 Theory Lunch Monte Carlo vs Las Vegas Las Vegas Algorithms are randomized and always give the correct ... – PowerPoint PPT presentation

Number of Views:345
Avg rating:3.0/5.0
Slides: 27
Provided by: Isabelle45
Category:
Tags:
Transcript and Presenter's Notes

Title: The Markov Chain Monte Carlo Method

1
The Markov Chain Monte Carlo Method
• Isabelle Stanton
• May 8, 2008 Theory Lunch

2
Monte Carlo vs Las Vegas
• Las Vegas Algorithms are randomized and always
give the correct results but gamble with
computation time
• Quicksort
• Monte Carlo algorithms have fixed running time
but may be wrong
• Simulated Annealing
• Estimating volume

3
Markov Chains
• a memoryless stochastic process, eg, flipping a
coin

1/6
1/6
3
2
4
1
1/6
1/6
6
5
1/6
1/6
4
Other Examples of Markov Chains
• Shuffling cards
• Flipping a coin
• PageRank Model
• Particle systems focus of MCMC work

5
General Idea
• Model the system using a Markov Chain
• Use a Monte Carlo Algorithm to perform some

6
Applications
• Approximate Counting - of solutions to 3-SAT or
Knapsack
• Statistical Physics when do phase transitions
occur?
• Combinatorial optimization simulated annealing
type of algorithms
• We'll focus on counting

7
Monte Carlo Counting
• How do you estimate the volume
• of a complex solid?
• Render with environment maps
• efficiently?
• Estimate an integral numerically?

8
(Picnic) Knapsack
weighs 5
weighs 4
weighs 10
What is a solution?
How many solutions are there?
weighs 2
weighs 4
Holds 20
9
Counting Knapsack Solutions
• Item weights a (a0,...an)
• Knapsack size a real number b
• Estimate the number of 0,1 vectors, x, that
satisfy ax b
• Let N denote the number of solutions

10
Na?ve Solution
• Randomly generate x
• Calculate ax
• If ax b return 2n
• else return 0
• This will return N in expectation
• 0(2n-N) N2n / 2n

11
Is this fast?
• Counterexample
• a (1, ... 1) and b n/3
• Any solution has less than n/3 1's
• There are (n choose n/3)2n/3 solutions

12
no
• Pr(sample x, x n/3) lt (n choose n/3)2-2n/3
• In expectation, need to generate 2n/3 x's before
we get a single solution!
• Any polynomial number of trials will grossly
underestimate N

13
Knapsack with MCMC
• Let Mknap be a markov chain withstate space O(b)
x ax b
• This will allow us to sample a solution

14
Various Mknap
a(0,.5,.5) b 1.5
a(0,1,1) b 1.5
111
011
101
110
110
101
011
110
101
001
010
100
001
010
100
001
010
100
000
000
000
15
Mknap Transitions
• Transitions
• With probability 1/2, x transitions to x
• Otherwise, select an i u.a.r.
• from 0 to n-1 and flip
• the ith bit of x.
• If x' is a
• solution,
• transition there.

111
0.5
0.5
011
101
110
110
101
110
101
1/6
1/6
1/6
1/6
0.5
001
010
100
001
010
100
001
010
100
0.5
0.5
1/6
1/6
1/6
000
000
000
0.5
a(0,1,1) b 1.5
16
Connected?
• Is Mknap connected?
• Yes. To get from x to x' go through 0.

17
Ergodicity
• What is the stationary distribution of Knapsack?
• Sample each solution with prob 1/N
• A MC is ergodic if the probability distribution
over the states converges to the stationary
distribution of the system, regardless of the
starting configuration
• Is Mknap ergodic? Yes.

18
Algorithm Idea
• Start at 0 and simulate Mknap for enough steps
that the distribution over the states is close to
uniform
• Why does uniformity matter?
• Does this fix the problem yet?

19
The trick
• Assume that a0 a1 ... an (0,1,2,,n-1,n)
• Let b0 0 and bi minb, Siaj
• O(bi-1) O(bi) - why?
• O(bi) (n1)O(bi-1) - why?

Change any element of O(bi) to one of O(bi-1) by
switching the rightmost 1 to a 0
20
How does that help?
• O(b) O(bn) O(bn)/O(bn-1) x
O(bn-1)/O(bn-2) x ... x
O(b1)/O(b0) x O(b0)
• We can estimate each of these ratios by doing a
walk on O(bi) and computing the fraction of
samples in O(bi-1)?
• Good estimate since
• O(bi-1) O(bi) (n1)O(bi-1)

21
Analysis
• Ignoring bias, the expectation of each trial is
O(bi-1)/O(bi)
• We perform t 17e-2n2 steps
• Focus on Var(X)/E(X)2 in analyzing efficiency
for MCMC methods

22
Analysis
• If Z is the product of the trials,
EZ ? O(bi-1)/O(bi)
• Magic Statistics Steps
• Var(Z)/(EZ)2 e2/16
• By Chebyshev's
• Pr(1-e/2)O(b) Z (1e/2)O(b) 3/4

23
Analysis
• We used nt 17e-2n3 steps
• This is a FPRAS (Fully Polynomial Randomized
Approximation Scheme)?
• Except... what assumption did I make?

24
Mixing Time
• Assumption We are close to the uniform
distribution in 17e-2n2 steps
• This is known as the mixing time
• It is unknown if this distribution mixes in
polynomial time

25
Mixing Time
• What does mix in polynomial time?
• Dice 1 transition
• Shuffling cards 7 shuffles
• ferromagnetic Ising model at high temperature
O(nlog n)?
• What doesn't?
• ferromagnetic Ising model at low temperature
starts to form magnets

26
Wes Weimer Memorial Conclusion Slide
• The markov chain monte carlo
• method models the problem
• as a Markov Chain and then
• uses random walks
• Mixing time is important
• P problems are hard
• Wes likes trespassing