Computing Distributions using Random Walks on Graphs

1
Computing Distributions using Random Walks on
Graphs

• Guy Kindler
• DIMACS

Dan Romik Weizmann Institute of Science
2
Computing distributions

• Knuth, Yao 76
• Given a source of random bits,
• output a sample with given distribution D.

3
Formalization

• Example Compute D(0,1/2), (1,1/4),
  (2,1/4)

Hright Tleft
0
Interpretation Computing a distribution using a
random walk on a binary tree.
1
1
2
1
4
Infinite trees are sometimes needed

• D(0,2/3), (1,1/3)

• Requirement Output is reached with probability
  1
• Knuth, Yao 76 Output can be reached in
  expected time Ent(D)O(1)

Tight!
5
Some other models

• Romik 99 Generate dist. B from dist. A in
  optimal time.
• von Neumann 51 Generate unbiased coins from
  biased ones (when bias is unknown).
• Keane OBrien 94 Generate f(p)-biased coins
  from p-biased ones.
• Peres Nacu 03 Generate f(p)-biased in good
  time.
• Mossel Peres 03 Generate f(p)-biased coins
  from p-biased, using a finite graph.

6
Finite state generators
KnuthYao

• Output

1
1
0

7
Finite state generators

• Interpretation binary representation
• Generating a random variable on 0,1 using a
  random walk on a graph
• Definition A distribution function is
  computable, if it is the output distribution of
  some f.s.g.
• Question KnuthYao which distributions are
  computable?

1
1
0
smooth/analytic
0
0
Output
1
1
0

8
History of the problem

• KnuthYao 76 Computable analytic density
  functions must be polynomials with rational
  coefficients
• Yao 84 The roots of such functions must be
  rational
• this work
• All functions with above properties can be
  computed
• Allowing smooth functions does not add computable
  functions.

9
Well discuss

• Theorem Let D be a distribution with density
  function f. If
• f is a non-negative polynomial
• with rational coefficients
• and no irrational roots in 0,1,
• then D is computable.

10
Generating some distributions

• All order statistics of independent uniform
  variables

• uniform distribution

• All distributions with density of the form
• f(x)c xm(1-x)n

• Generating max(X,Y)
• run two f.s.gs in parallel
• output the maximum

KnuthYao 76
11
More distributions

• KnuthYao 76
• uniform on a,b, for a,b rational.

All distributions with density of the
form f(x)c (x-a)m(b-x)n1a,b(x)

• Generating max(X,Y)
• run two f.s.gs in parallel
• output the maximum

12
All distributions
All distributions with density of the
form f(x)c (x-a)m(b-x)n1a,b(x)
Question what is the set of rational mixtures of
such functions ?

• Proof
• Geometric in nature
• Non-constructive

Q.E.D. !
Answer all polynomials with rational
coefficients, and no irrational roots in 0,1 !

• easy if f1,..,fk are
• computable, then so isa1f1akfk
• (for ai rational)

13
Conclusions

• We solved the computability problem in the f.s.g.
  model, for smooth functions.
• We have no good bounds on complexity (size of
  graph) in this model.

Open problems

• Solve for other computational models (stack
  automaton? Yao84)
• Solve the general computablity question (no
  smoothness restriction)
• Solve the complexity question

14
The End