Optimization in mean field random models

1
Optimization in mean field random models

• Johan Wästlund
• Linköping University
• Sweden

2
Statistical Mechanics

• Each particle has a spin
• Energy Hamiltonian depends on spins of
  interacting particles
• Ising model Spins 1, H interacting pairs of
  opposite spin

3
Statistical Mechanics

• Spin configuration ? has energy H(?)
• Gibbs measure depends on temperature T

• T?8 random state
• T?0 ground state, i.e. minimizing H(?)

4
Statistical Mechanics

• Thermodynamic limit N ?8
• Average energy? (suitably normalized)

5
Disordered Systems

• Spin glasses
• AuFe random alloy
• Fe atoms interact

6
Disordered Systems

• Random interactions between Fe atoms
• Sherrington-Kirkpatrick model

7
Disordered Systems

• Quenched random variables gi,j
• S-K is a mean field model No correlation
  betweeen quenched variables
• NP hard to find ground state given gi,j

8
Computer Science

• Test / evaluate heuristics for NP-hard problems
• Average case analysis
• Random problem instances

9
Combinatorial Optimization

• Minimum Matching / Assignment
• Minimum Spanning Tree
• Traveling Salesman
• Shortest Path
• Points with given distances, minimize total
  length of configuration

10
Statistical Physics / Computer Science

• Spin configuration
• Hamiltonian
• Ground state energy
• Temperature
• Gibbs measure
• Thermodynamic limit

• Feasible solution
• Cost of solution
• Cost of minimal solution
• Artificial parameter T
• Gibbs measure
• N?8

11
Mean field models

• Replica-cavity method has given good results for
  mean field models
• Parisi solution of S-K model
• The same methods can be applied to combinatorial
  optimization problems in mean field models

12
Mean field models of distance

• N points
• Abstract geometry
• Inter-point distances given by i. i. d. random
  variables
• Exponential distribution easiest to analyze
  (pseudodimension 1)

13
Matching

• Set of edges giving a pairing of all points

14
Spanning tree

• Network connecting all points

15
Traveling salesman

• Tour visiting all points

16
Mean field limits

• No normalization needed! (pseudodimension 1)
• Matching ?2/120.822 (Mézard Parisi 1985,
  rigorous proof by Aldous 2000)
• Spanning tree ?(3) 11/81/27 1.202 (Frieze
  1985)
• Traveling salesman 2.0415 (Krauth-Mézard-Parisi
  1989), now established rigorously!

17
Cavity results

• Non-rigorous method
• Aldous derived equivalent equations with the
  Poisson-Weighted Infinite Tree (PWIT)

18
Cavity results

• Non-rigorous quantity X cost of minimal
  solution cost of minimal solution with the root
  removed
• Define X1, X2, X3, similarly on sub-trees
• Leads to the equation

• Xi distributed like X, ?i are times of events in
  rate 1 Poisson process

19
Cavity results

• Analytically, this is equivalent to

where
20
Cavity results

• Explicit solution
• Ground state energy

21
Cavity results

• Note that the integral is equal to the area under
  the curve when f(u) is plotted against f(-u)
• In this case, f satisfies the equation

22
Cavity results
23
K-L matching
24
K-L matching

• Similarly, the K-L matching problem leads to the
  equations

• ? has rate K and ? has rate L
• minK stands for Kth smallest

25
K-L matching

• Shown by Parisi (2006) that this system has an
  essentially unique solution
• The ground state energy is given by
• where x and y satisfy an explicit equation
• For KL2, this equation is
• Unfortunately the cavity method is not rigorous

26
The exponential bipartite assignment problem
n
27
The exponential bipartite assignment problem

• Exact formula conjectured by Parisi (1998)
• Suggests proof by induction
• Researchers in discrete math, combinatorics and
  graph theory became interested
• Generalizations

28
Generalizations

• by Coppersmith Sorkin to incomplete matchings
• Remarkable paper by M. Buck, C. Chan D. Robbins
  (2000)
• Introduces weighted vertices
• Extremely close to proving Parisis conjecture!

29
Incomplete matchings
n
m
30
Weighted assignment problems

• Weights ?1,,?m, ?1,, ?n on vertices
• Edge cost exponential of rate ?i?j
• Conjectured formula for the expected cost of
  minimum assignment
• Formula for the probability that a vertex
  participates in solution (trivial for less
  general setting!)

31
The Buck-Chan-Robbins urn process

• Balls are drawn with probabilities proportional
  to weight

32
Proofs of the conjectures

• Two independent proofs of the Parisi and
  Coppersmith-Sorkin conjectures were announced on
  March 17, 2003 (Nair, Prabhakar, Sharma and
  Linusson, Wästlund)

33
Annealing

• Powerful idea Let T?0, forcing the system to
  converge to its ground state
• Replica-cavity approach
• Simulated annealing meta-algorithm (optimization
  by random local moves)

34
In the mean field model
Underlying rate 1 variables Yi

• ri plays the same role as T
• Local temperature
• Associate weight to vertices rather than edges

35
Cavity/annealing method

• Relax by introducing an extra vertex
• Let the weight of the extra vertex go to zero
• Example Assignment problem with
• ?1?m1, ?1?n1, and ?m1 ?
• p P(extra vertex participates)
• p/n P(edge (m1,n) participates)

36
Annealing

• p/n P(edge (m1,n) participates)
• When ??0, this is
• Hence
• By Buck-Chan-Robbins urn theorem,

37
Annealing

• Hence
• Inductively this establishes the
  Coppersmith-Sorkin formula

38
Results with annealing

• Much simpler proofs of Parisi, Coppersmith-Sorkin,
  Buck-Chan-Robbins formulas
• Exact results for higher moments
• Exact results and limits for optimization
  problems on the complete graph

39
The 2-dimensional urn process

• 2-dimensional time until k balls have been drawn

40
Limit shape as n?8

• Matching
• TSP/2-factor

41
Mean field TSP

• If the edge costs are i.i.d and satisfy
  P(lltt)/t?1 as t?0 (pseudodimension 1), then as n
  ?8,
• A. Frieze proved that whp a 2-factor can be
  patched to a tour at small cost

42
Further exact formulas
43
LP-relaxation of matching in the complete graph Kn
44
Future work

• Explain why the cavity method gives the same
  equation as the limit shape in the urn process
• Establish more detailed cavity predictions
• Use proof method of Nair-Prabhakar-Sharma in more
  general settings

45
Thank you!