1

1
Dynamical Systems for Extreme Eigenspace
Computations

• Maziar Nikpour
• UCL Belgium

2
Co-workers

• Iven M. Y. Mareels
• Jonathan H. Manton
• University of Melbourne, Australia.
• Vadym Adamyan
• Odessa State University, Ukraine.
• Uwe Helmke
• University of Wurzberg, Germany.

3
Problem

• For Hermitian matrices (A, B), with B gt 0
• find the non-trivial solutions (l, x) of

with the smallest or largest generalised
eigenvalues l.
n size of matrices (A,B) k no. of desired
generalised eigenvalue/eigenvector pairs.
4
Outline

• Introduction
• Motivation
• Brief history of literature
• Penalty function approach
• Gradient flow
• Convergence
• Discrete-time Algorithms
• Applications
• Conclusions

5
Motivation

• Signal Processing
• Telecommunications
• Control
• Many others

6
Brief History of Problem

• Numerical Linear Algebra Literature
• Methods for general A and B
• QZ algorithm, Moler and Stewart 1973.
• (what MATLAB does when you type eig)
• Methods for large and sparse A, B.
• Trace minimisation method, Sameh Wisiniewski,
  1981.

• Engineering Literature
• Methods largely for computing largest/smallest
  generalised evs adaptively
• Mathew and Reddy 1998 (inflation approach,
  special case of approach in this work).
• Strobach, 2000 (tracking algorithms).

7
Brief History of Problem

• Dynamical systems literature
• Brockett flow
• Oja

• Above approaches cannot be adapted to the
  Generalised Eigenvalue problem without
  manipulating A and/or B.
• Recent paper by Manton et al. presents an
  approach that can

8
Penalty Function Approach

• The minimisation of the following cost can lead
  to algorithms for computing extreme generalised
  evs.

9
Dynamical Systems for Numerical Computations

• Gradient descent like flows on a cost function.

Discretisation of flows.
Efficient numerical algorithms.
10
Examples

• Power flow

• Oja subspace flow

• Brockett flow

11
Contributions

• Gradient flow on f(A, B)
• Discretisation of Gradient Flow
• Steepest Descent
• Conjugate Gradient
• Stochastic minor/principal component tracking
  algorithms

• The case B I, and Z real has already been
  treated.
• (see Manton et al. 2003).
• Extending the domain to the complex matrices
  complicates the analysis substantially
• Allowing B to be any p.d. matrix expands the
  range of applications

12
Gradient Flow

• Main Result For almost all initial conditions,
  solutions of

converge to a single point in the stable
invariant set of the flow.
13
Gradient Flow

• The stable invariant set is

14
Critical Points of f(A, B)

• Hessian of f(A, B) is degenerate at critical
  points,
• N.B.

• Proposition

15
Stability analysis of critical points

• Linear stability analysis will not suffice.
• Use center manifold theorem at each c.p.

• Proposition

Why?
Nullspace of hessian of cost func. Tangent
space of critical subman.
16
Stability analysis of critical points

• Reduction principle of dynamical systems

17
Stability analysis of critical points

• Main result follows.
• Proposition level sets are compact gt flow
  converges to one of the critical components.
• Center manifold thm. reduction principle gt
  converge to a single point on a critical
  component.
• Converges to stable invariant set for an open
  dense set of initial conditions.

18
Remarks

• Conditions used in proof gt f(A, B) is a
  Morse-Bott function gt solutions converge to a
  single point instead of a set (see Helmke
  Moore, 1994).
• Also f(A, B) is a real analytic function (Cn x k
  considered as a real vector space) gt convergence
  to a single point (Lojasiewicz, 1984).

19
Further Remarks

• Generalised eigenvectors not unique but
• convergence to particular g.evs can be achieved
  by the following flow in reduced dimensions

where truncX denotes X with imaginary
components of diagonal set to 0.
Flow converges to an element of critical
component with real diagonal elements.
20
Systems of Flows

• Consider the system of cost functions

21
Systems of Flows

• System of partial gradient descent flows allows
  the possibility to add or take away components
  without affecting the computation of others

• Proposition Z(t) converges to smallest
  generalised eigenvalues for a generic initial
  condition.

22
Discrete-time algorithms

• Since flow evolves on a Euclidean space
  discretisation is not complicated

• Steepest descent

• Conjugate gradient

23
Discrete-time algorithms

• Can solve the Hermitian definite GEVP without any
  factorisation or manipulation of A or B.
• Only matrix small matrix multiplications are
  required.
• Suitable for cases where A and B are large and
  sparse.
• Conjugate gradient algorithm superlinear
  convergence but no increase in order of
  computational complexity.
• Complexity O(n2k).
• Exact line search can be performed.

24
(No Transcript)
25
Discrete-time algorithms

• Tracking algorithm

- signal plus noise model

• O(nk2) complexity when Rnn I.

26
(No Transcript)
27
Conclusion

• Proposing and deriving convergence theory of a
  gradient flow for solving GEVP.
• Modular system of flows.
• Discretisation CG and SD algorithms.
• Application to Minor component tracking.

28
Questions