CS 290H 31 October and 2 November Support graph preconditioners

1
CS 290H 31 October and 2 NovemberSupport graph
preconditioners

• Final projects
• Read and present two related papers on a topic
  not covered in class
• Or, experiment with preconditioning in an
  application of your choice
• Either way, talk to me by November 9 about your
  project
• Laplacian and generalized Laplacian matrices
• Maximum-weight spanning tree preconditioners
• Algebraic analysis of congestion/dilation and
  condition number
• Augmenting the MST to improve congestion
  dilation
• Extensions
• Reverse support MILU analysis
• Positive matrix elements diagonally dominant
  SPSD and matroid analysis
• Hierarchical partitioning Spielman/Teng
• Remarks on factor width

2
Support Graph Preconditioning

• CFIM Complete factorization of incomplete
  matrix
• Define a preconditioner B for matrix A
• Explicitly compute the factorization B LU
• Choose nonzero structure of B to make factoring
  cheap (using combinatorial tools from direct
  methods)
• Prove bounds on condition number using both
  algebraic and combinatorial tools

• New analytic tools, some new preconditioners
• Can use existing direct-methods software
• - Current theory and techniques limited

3
Definitions

• The Laplacian matrix of an n-vertex undirected
  graph G is the n-by-n symmetric matrix A with
• aij -1 if i ? j and (i, j) is an edge of
  G
• aij 0 if i ? j and (i, j) is not an
  edge of G
• aii the number of edges incident on vertex i
• Theorem The Laplacian matrix of G is symmetric,
  singular, and positive semidefinite. The
  multiplicity of 0 as an eigenvalue is equal to
  the number of connected components of G.
• A generalized Laplacian matrix (more accurately,
  a symmetric weakly diagonally dominant M-matrix)
  is an n-by-n symmetric matrix A with
• aij 0 if i ? j
• aii S aij where the sum is over j ? i

4
Edge-vertex factorization of generalized
Laplacians

• A generalized Laplacian matrix A can be factored
  as A UUT where U has
• a row for each vertex
• a column for each edge, with two nonzeros of
  opposite sign
• a column for each excess-weight vertex, with one
  nonzero

5
Spanning Tree Preconditioner Vaidya
G(A)
G(B)

• A is generalized Laplacian (symmetric positive
  semidefinite with negative off-diagonal nzs)
• B is the gen Laplacian of a maximum-weight
  spanning tree for A (with diagonal modified to
  preserve row sums)
• Form B costs O(n log n) or less time (graph
  algorithms for MST)
• Factorize B RTR costs O(n) space and O(n)
  time (sparse Cholesky)
• Apply B-1 costs O(n) time per iteration

6
Combinatorial analysis cost of preconditioning
G(A)
G(B)

• A is generalized Laplacian (symmetric positive
  semidefinite with negative off-diagonal nzs)
• B is the gen Laplacian of a maximum-weight
  spanning tree for A (with diagonal modified to
  preserve row sums)
• Form B costs O(n log n) time or less (graph
  algorithms for MST)
• Factorize B RTR costs O(n) space and O(n)
  time (sparse Cholesky)
• Apply B-1 costs O(n) time per iteration
  (two triangular solves)

7
Combinatorial analysis Graphs and sparse
Cholesky
Fill new nonzeros in factor
Symmetric Gaussian elimination for j 1 to n
add edges between js higher-numbered
neighbors
G(A)chordal
G(A)
8
Numerical analysis quality of preconditioner
G(A)
G(B)

• support each edge of A by a path in B
• dilation(A edge) length of supporting path in B
  
• congestion(B edge) of supported A edges
• p max congestion, q max dilation
• condition number ?(B-1A) bounded by pq (at most
  O(n2))

9
Numerical analysis Support numbers

• Intuition from networks of electrical resistors
  
• graph circuit edge resistor weight
  1/resistance conductance
• How much must you amplify B to provide as much
  conductance as A?
• How big does t need to be for tB A to be
  positive semidefinite?
• What is the largest eigenvalue of B-1A ?
• The support of B for A is
• s(A, B) min t xT(tB A)x ?
  0 for all x and all t ? t
• If A and B are SPD then s(A, B) max? Ax
  ?Bx ?max(A, B)
• Theorem If A and B are SPD then ?(B-1A)
  s(A, B) s(B, A)

10
Old analysis, splitting into paths and edges

• Split A A1 A2 Ak and B B1 B2
  Bk
• such that Ai and Bi are positive semidefinite
• Typically they correspond to pieces of the graphs
  of A and B (edge, path, small subgraph)
• Theorem s(A, B) ? maxi s(Ai , Bi)
• Lemma s(edge, path) ? (worst weight ratio)
  (path length)
• In the MST case
• Ai is an edge and Bi is a path, to give s(A, B) ?
  pq
• Bi is an edge and Ai is the same edge, to give
  s(B, A) ? 1

11
New analysis Algebraic Embedding Lemma
vvBoman/Hendrickson

• Lemma If VWU, then s(UUT, VVT) ?
  W22
• (with equality for some
  choice of W)
• Proof
• take t ? W22 ?max(WWT)
  max y?0 yTWWTy / yTy
• then yT (tI - WWT) y ? 0 for all y
• letting y VTx gives xT (tVVT - UUT) x
  ? 0 for all x
• recall s(A, B) mint xT(tB A)x ? 0 for
  all x, all t ? t
• thus s(UUT, VVT) ? W22

12
s(A, B) ? W22 ? W? x W1
(max row sum) x (max col sum) ? (max
congestion) x (max dilation)
13
Spanning Tree Preconditioner Vaidya
G(A)
G(B)

• can improve congestion and dilation by adding a
  few strategically chosen edges to B
• cost of factorsolve is O(n1.75), or O(n1.2) if A
  is planar
• in experiments by Chen Toledo, often better
  than drop-tolerance MIC for 2D problems, but not
  for 3D.

14
Complexity of direct methods
Time and space to solve any problem on any
well-shaped finite element mesh
15
Support-graph analysis of modified incomplete
Cholesky

• B has positive (dotted) edges that cancel fill
• B has same row sums as A
• Strategy Use the negative edges of B to support
  both the negative edges of A and the positive
  edges of B.

16
Supporting positive edges of B

• Every dotted (positive) edge in B is supported by
  two paths in B
• Each solid edge of B supports one or two dotted
  edges
• Tune fractions to support each dotted edge
  exactly
• 1/(2?n 2) of each solid edge is left over to
  support an edge of A

17
Analysis of MIC Summary

• Each edge of A is supported by the leftover
  1/(2?n 2) fraction of the same edge of B.
• Therefore s(A, B) ? 2?n 2
• Easy to show s(B, A) ? 1
• For this 2D model problem, condition number is
  O(n1/2)
• Similar argument in 3D gives condition number
  O(n1/3) or O(n2/3) (depending on boundary
  conditions)

18
Open problems I

• Other subgraph constructions for better bounds on
  W22 ?
• For example Boman,
• W22 ? WF2 sum(wij2) sum of
  (weighted) dilations,
• and Alon, Karp, Peleg, West show there exists a
  spanning tree with average weighted dilation
  exp(O((log n loglog n)1/2)) o(n? )
• this gives condition number O(n1?) and solution
  time O(n1.5?),
• compared to Vaidya O(n1.75) with augmented
  spanning tree
• Spielman, Teng recursive partitioning
  construction gives solution time O(n1?) for all
  generalized Laplacians! (Uses yet another
  matrix norm inequality.)
• Is there a construction that minimizes W22
  directly?

19
Open problems II

• Make spanning tree methods more effective in 3D?
• Vaidya gives O(n1.75) in general, O(n1.2) in 2D
• Issue 2D uses bounded excluded minors, not just
  separators
• Support graph methods for more general matrices?
• All SPD matrices? (Boman, Chen, Hendrickson,
  Toledo different matroid for all diagonally
  dominant SPD matrices)
• Finite element problems? (Boman
  Element-by-element preconditioner for bilinear
  quadrilateral elements)
• Matrices of bounded factor width?

20
Hierarchy of matrix classes (all real)

• General nonsymmetric
• Diagonalizable
• Normal
• Symmetric indefinite
• Symmetric positive (semi)definite Factor width
  n
• Factor width k
• . . .
• Factor width 4
• Factor width 3
• Diagonally dominant SPSD Factor width 2
• Generalized Laplacian Symm diag dominant
  M-matrix
• Graph Laplacian