NearlyLinear Time Algorithms: Graph Partitioning, Graph Sparsification, and Solving Linear Systems

1
Nearly-Linear Time Algorithms Graph
Partitioning, Graph Sparsification, and Solving
Linear Systems

• Daniel Spielman
• MIT
• Shang-Hua Teng
• Boston University

2
Laplacian of Weighted Graph

• G (V,E,w) w positive weights
• Adjacency matrix A
• Laplacian

3
Graph Approximation Sparsification
d-sparse at most dn edges à g-approximates A
if
That is,
4
Algorithm for Sparsification

• Partition
• Decompose graph into components with high
  isoperimetric numbers
• Sample and rescale
• Apply sampling to sparsify each component
• Properly rescale the edge weights

5
Isoperimetric Number of a Graph
6
Algorithm for Sparsification

• Partition
• Decompose graph into components with high
  isoperimetric numbers
• Sample and rescale
• Apply sampling to sparsify each component
• Properly rescale the edge weights

7
Building Sparsifiers
For unweighted , if
Random sample to degree and rescale
w.h.p
2-approximate
We approximate this partition quickly!
8
Nearly Linear-Time Sparsification

• Theorem
• For all e gt 0, in random O(m logO(1) m) time
• an (1e )-approximation
• logO(1)(m/e) / e2 sparse

9
Related Work I Benczur-Karger
à g-cut-approximates A if
10
Related Work I Benczur-Karger
à g-cut-approximates A if
good cut approximation, but bad for
edges for i-j mod n lt k
11
Related Work I Benczur-Karger
à g-cut-approximates A if
0
n/2
good cut approximation, but bad for
edges for i-j mod n lt k
12
Related Work II SamplingApproximate in Limited
Subspace

• Achlioptas-McSherry
• Frieze-Kannan-Vempala
• Approximate for x
• in the span of a few singular
• vectors of largest singular values.

13
Partitioning Algorithms
Multicommodity Flow too slow Spectral one cut
quickly, but might be small
cut Multilevel (Chaco/Metis) cant analyze,
miss small
sparse cuts New Algorithm Crude
partition in time
14
Nibble Partition with Random Walk

• Approximate the distribution of a random walk
• Cost Proportional to the cut obtained
• Based on an analysis of Lovász and Simonovits

15
Fast Partitioning With Nibble
If exists s.t. then find s.t.
and In time
16
Ultra-Sparsification

• Theorem
• (n-1) t no(1) edges
• (m/t)-approximate
• in random O(m logO(1) m) time

17
Building Ultra-Sparsifier
number clusters
18
Building Ultra-Sparsifier
Tree for each cluster
19
Building Ultra-Sparsifier
Tree for each cluster One edge between connected
clusters
20
Building Ultra-Sparsifier
Tree for each cluster One edge between connected
clusters Sparsify inter-cluster graph
21
Progress in Ultra-Sparsification
Edges Vaidya
Boman-Hendrickson ST03
This paper
Approximation
22
Symmetric Diagonally Dominant Systems
Sym Diagonally Dominant matrix, non-zeros
By recursive preconditioned iterative solver
23
Future work
Extension to other families of linear systems
Implementation and application in
practice Analysis of what we really
implement Other applications of sparsification
24
(No Transcript)
25
Symmetric Diagonally Dominant Systems
compute in time
non-zeros
26
History of Combinatorial Preconditioning
1.75 1.5 1.5 1.31 1

• Vaidya 1990
• Boman-Hendrickson 2001 O(m1.5 log (1/e))
• Maggs-Miller-Parekh-Ravi-Wu2002
• O(m1.5 log
  (1/e))
• Spielman-Teng 2003 O(m1.31 log (1/e))
• This paper Nearly Linear-Time

27
Low-Stretch Spanning Trees
28
The slides after this one

• Are my workspace.
• So they are not part of the talk

29
Preliminary Experiments

• Theoretically, our new algorithm almost
  asymptotically optimizes the potential of the
  combinatorial preconditioners introduced by
  Vaidya.
• Reasonably good performance of linear systems
  defined by 2D meshes.

30
Linear-Time Algorithms for Partitioning and
Sparsification Now and Then

• Almost asymptotically optimize the potential of
  the combinatorial preconditioners introduced by
  Vaidya
• Nearly linear-time algorithms for other
  optimization problems using our linear-time
  partitioning and sparsification algorithms.

31
Outline
Sparse Linear Solver
Preconditioner
Conjugate Gradient
Almost Linear Time
Ultra-sparsification
Sparsification
AKPW
Partitioning
Sampling
Random Walk
32
Related Work Sampling

• Achlioptas-McSherry
• Frieze-Kannan-Vempala
• Füredi-Komlós

33
Weighted Graph and Weighted Laplacian

• G (V,E,w) w positive weights
• Adjacency matrix A
• Laplacian L(A) D A.
• Symmetric, positive semi-definite
• 0 is the smallest eigenvalue with
• (1, 1, 1)T is eigenvector

34
Connection with Partitioning

• Ours
  Benczur-Karger
• sparsity of cut size of cut

E(U, U) min(vol(U),vol(U))
E(U, U)