CS 290H 26 October Sparse approximate inverses, support graphs

1
CS 290H 26 OctoberSparse approximate inverses,
support graphs

• Homework 2 due Mon 7 Nov.
• Sparse approximate inverse preconditioners
• Introduction to support theory

2
Preconditioned conjugate gradient iteration
x0 0, r0 b, d0 B-1 r0, y0
B-1 r0 for k 1, 2, 3, . . . ak
(yTk-1rk-1) / (dTk-1Adk-1) step length xk
xk-1 ak dk-1 approx
solution rk rk-1 ak Adk-1
residual yk B-1 rk
preconditioning
solve ßk (yTk rk) / (yTk-1rk-1)
improvement dk yk ßk dk-1
search direction

• Several vector inner products per iteration (easy
  to parallelize)
• One matrix-vector multiplication per iteration
  (medium to parallelize)
• One solve with preconditioner per iteration (hard
  to parallelize)

3
Sparse approximate inverses

• Compute B-1 ? A explicitly
• Minimize A B-1 I F (in parallel, by
  columns)
• Variants factored form of B-1, more fill, . .
• Good very parallel, seldom breaks down
• Bad effectiveness varies widely

4
Support Graph Preconditioning

• CFIM Complete factorization of incomplete
  matrix
• Define a preconditioner B for matrix A
• Explicitly compute the factorization B LU
• Choose nonzero structure of B to make factoring
  cheap (using combinatorial tools from direct
  methods)
• Prove bounds on condition number using both
  algebraic and combinatorial tools

• New analytic tools, some new preconditioners
• Can use existing direct-methods software
• - Current theory and techniques limited

5
Some classes of matrices

• Diagonally dominant symmetric positive definite
• Diagonally dominant M-matrix
• Laplacian
• Generalized Laplacian

6
Combinatorial half Graphs and sparse Cholesky
Fill new nonzeros in factor
Symmetric Gaussian elimination for j 1 to n
add edges between js higher-numbered
neighbors
G(A)chordal
G(A)
7
Spanning Tree Preconditioner Vaidya
G(A)
G(B)

• A is symmetric positive definite with negative
  off-diagonal nzs
• B is a maximum-weight spanning tree for A (with
  diagonal modified to preserve row sums)
• factor B in O(n) space and O(n) time
• applying the preconditioner costs O(n) time per
  iteration

8
Numerical half Support numbers

• Intuition from resistive networks How much
  must you amplify B to provide the conductivity of
  A?
• The support of B for A is s(A, B) mint
  xT(tB A)x ? 0 for all x, all t ? t
• In the SPD case, s(A, B) max? Ax ?Bx
  ?max(A, B)
• Theorem If A, B are SPD then ?(B-1A) s(A, B)
  s(B, A)

9
Splitting Lemma

• Split A A1 A2 Ak and B B1 B2
  Bk
• Ai and Bi are positive semidefinite
• Typically they correspond to pieces of the graphs
  of A and B (edge, path, small subgraph)
• Lemma s(A, B) ? maxi s(Ai , Bi)

10
Spanning Tree Preconditioner Vaidya
G(A)
G(B)

• A is symmetric positive definite with negative
  off-diagonal nzs
• B is a maximum-weight spanning tree for A (with
  diagonal modified to preserve row sums)
• factor B in O(n) space and O(n) time
• applying the preconditioner costs O(n) time per
  iteration

11
Spanning Tree Preconditioner Vaidya
G(A)
G(B)

• support each edge of A by a path in B
• dilation(A edge) length of supporting path in B
  
• congestion(B edge) of supported A edges
• p max congestion, q max dilation
• condition number ?(B-1A) bounded by pq (at most
  O(n2))

12
Spanning Tree Preconditioner Vaidya
G(A)
G(B)

• can improve congestion and dilation by adding a
  few strategically chosen edges to B
• cost of factorsolve is O(n1.75), or O(n1.2) if A
  is planar
• in experiments by Chen Toledo, often better
  than drop-tolerance MIC for 2D problems, but not
  for 3D.

13
Splitting and Congestion/Dilation Lemmas

• Split A A1 A2 Ak and B B1 B2
  Bk
• Ai and Bi are positive semidefinite
• Typically they correspond to pieces of the graphs
  of A and B (edge, path, small subgraph)
• Lemma s(A, B) ? maxi s(Ai , Bi)
• Lemma s(edge, path) ? (worst weight ratio)
  (path length)
• In the MST case
• Ai is an edge and Bi is a path, to give s(A, B) ?
  pq
• Bi is an edge and Ai is the same edge, to give
  s(B, A) ? 1

14
Algebraic framework

• The support of B for A is s(A, B) mint
  xT(tB A)x ? 0 for all x, all t ? t
• In the SPD case, s(A, B) max? Ax ?Bx
  ?max(A, B)
• If A, B are SPD then ?(B-1A) s(A, B) s(B, A)
  
• Boman/Hendrickson If VWU, then
  s(UUT, VVT) ? W22

15
Algebraic framework
Boman/Hendrickson

• Lemma If VWU, then s(UUT, VVT) ?
  W22
• Proof
• take t ? W22 ?max(WWT)
  max y?0 yTWWTy / yTy
• then yT (tI - WWT) y ? 0 for all y
• letting y VTx gives xT (tVVT - UUT) x
  ? 0 for all x
• recall s(A, B) mint xT(tB A)x ? 0 for
  all x, all t ? t
• thus s(UUT, VVT) ? W22

16
s(A, B) ? W22 ? W? x W1
(max row sum) x (max col sum) ? (max
congestion) x (max dilation)