On implicit-factorization block preconditioners

1
On implicit-factorization block preconditioners

• Sue Dollar1,2, Nick Gould3,
• Wil Schilders2,4 and Andy Wathen1

1 Oxford University Computing Laboratory,
Oxford, UK 2 CASA, Technische Universiteit
Eindhoven, The Netherlands 3 Rutherford Appleton
Laboratory, Chilton, UK 4 Philips Research
Laboratory, Eindhoven, The Netherlands
2
Summary of the talk

• Introduction
• Direct versus Iterative Methods
• Preconditioned Conjugate Gradient Method
• Constraint Preconditioners
• Implicit Constraint Preconditioners
• Numerical Examples
• Future work and conclusions

3
Introduction

• Interested in solving structured linear systems
  of equations
• full rank
• positive semi-definite

4
Example 1 Equality constrained minimization

• Interior-point sub-problem
• T (small) barrier weights
• First-order optimality

5
Factory
6
Example 2 Inequality constrained minimization

• Interior-point sub-problem
• C (small) barrier weights
• First-order optimality

7
Direct vs. Iterative Methods

• Direct methods
• Gaussian elimination with a pivoting strategy
• Bunch-Parlett factorization
• Iterative methods
• Krylov subspace methods
• MINRES GMRES find solution of (1) within nm
  iterations
• CG may fail because of the indefiniteness of
  system
• Often advantageous to use a preconditioner P

8
PCG

• Possible to use the preconditioned
  conjugate-gradient method

(2)
(Gould, Hribar, Nocedal)
9

• Projected PCG
• Can perform iteration in the original (x, z)
  space so long as preconditioner chosen carefully
• Solve
• Iterate until convergence
• Solve

10
Constraint Preconditioners

• Case C0

• The matrix P-1H has
• an eigenvalue at 1 with multiplicity 2m
• n-m eigenvalues which are defined by the
  generalized eigenvalue problem

(Keller, Gould, Wathen)
11

• Case rank(C)l

• The matrix P-1H has
• at least 2m-l eigenvalues at 1
• n-m eigenvalues which are defined by the
  generalized eigenvalue problem
• m eigenvalues defined by the generalized
  eigenvalue problem
• where wxy xz, subject to Y2BY1xy?0.

12
CVXQP1_S m50, n100, Gdiag(A)
13
CVXQP1_S m50, n100, Gdiag(A)
14
Explicit vs. implicit constraint preconditioner

• Explicit constraint preconditioners choose G,
  and then factorize
• Implicit constraint preconditioners find
  easily-invertible factors R and S so that
• always holds
• Pick parts of R and S to match ( G) to parts
  of A

15

• Easily invertible B1

• Require that both R and S are easily block
  invertible
• - some sub-blocks should be zero

16
Example 1 C0

• can recover any G
  (Schilders)

17
Example 2
Example 3
18
Numerical Examples
CUTEr test set Case C0
Explicit GI Explicit GI Explicit GI Implicit G22I Implicit G22I Implicit G22I
name n m fact. iter. total fact. iter. total ratio
CVXQP1 10000 5000 1.73 2 2.98 0.066 27 1.47 0.49
CVXQP2 10000 2500 0.045 2 0.15 0.032 9 0.28 1.80
CVXQP3 10000 7500 3.11 2 6.09 0.085 16 1.35 0.22
POWELL20 10000 5000 0.12 1 0.19 0.040 314 6.52 33.0
PRIMALC1 239 9 0.0047 2 0.0092 0.0023 1 0.0046 0.50
PRIMALC2 238 7 0.0041 1 0.0071 0.0023 1 0.0042 0.58
QPNBOEI1 726 351 0.028 59 0.44 0.0044 13 0.060 0.13
QPNBOEI2 305 166 0.0071 55 0.11 0.0029 17 0.031 0.29
UBH1 9009 6000 0.11 9 0.50 0.049 1 0.16 0.33
26/40 problems solved faster (20 if take into
account perm time)
19
Case CI
Explicit GI Explicit GI Explicit GI Implicit G22I Implicit G22I Implicit G22I
name n m fact. iter. total fact. iter. total ratio
CVXQP1 10000 5000 1.98 2 3.41 0.062 14 1.02 0.30
CVXQP2 10000 2500 0.048 2 0.16 0.030 11 0.33 2.10
CVXQP3 10000 7500 6.73 2 10.5 0.083 8 1.09 0.10
POWELL20 10000 5000 0.081 2 0.17 0.040 2 0.13 0.77
PRIMALC1 239 9 0.0045 1 0.0080 0.0024 1 0.0044 0.55
PRIMALC2 238 7 0.0037 1 0.0069 0.0021 1 0.0038 0.56
QPNBOEI1 726 351 0.029 26 0.27 0.0040 17 0.077 0.28
QPNBOEI2 305 166 0.0080 6 0.031 0.0027 14 0.032 1.03
UBH1 9009 6000 0.12 2 0.26 0.043 2 0.22 0.83
29/40 problems solved faster (25 if take into
account perm time)
20
Conclusions

• New method for constructing preconditioners for
  CG methods for a variety of important problems
• Preconditioners
• implicitly respect crucial structure
• easy to apply
• flexible and capable of improving eigenvalue
  clusters
• Extend the class of problems for which CG is
  applicable
• Still under developmentbut will be available as
  part of GALAHAD
• Open questions
• How to pick basis B1 efficiently and so as to
  improve eigenvalue clusters
• How to approximate blocks of H in G
• What about Mr Greedy?

21
(No Transcript)
22
Conclusions

• New method for constructing preconditioners for
  CG methods for a variety of important problems
• Preconditioners
• implicitly respect crucial structure
• easy to apply
• flexible and capable of improving eigenvalue
  clusters
• Extend the class of problems for which CG is
  applicable
• Still under developmentbut will be available as
  part of GALAHAD
• Open questions
• How to pick basis B1 efficiently and so as to
  improve eigenvalue clusters
• How to approximate blocks of H in G