Computing illConditioned Eigenvalues

1
Computing ill-Conditioned Eigenvalues and
Polynomial Roots
Zhonggang Zeng
Northeastern Illinois University
International Conference on Matrix Theory and its
Applications -- Shanghai
2
Can you solve (x-1.0 )100 0
Can you solve x100-100 x99 4950 x98 - 161700
x973921225x96 - ... - 100 x 1 0
3
Eigenvalues of
1 0 1 1 1 1
... ... 1 1 0
1 1
A X
X-1
4
The Wilkinson polynomial p(x)
(x-1)(x-2)...(x-20) x20 - 210 x19
20615 x18 ...
Wilkinson wrote in 1984 Speaking for
myself I regard it as the most traumatic
experience in my career as a numerical analyst.
5
Myths on multiple eigenvalues/roots
- multiple evalues/roots are ill-conditioned, or
even intractable
- extension of machine precision is necessary to
calculate multiple roots
- there is an attainable precision for multiple
eigenvalues/roots
machine precision
attainable precision ---------------------------
--
multiplicity
Example for a 100-fold eigenvalue, to get 5
digits right
500 digits in machine precision
5 digits precision ---------------------------
--------------
100 in multiplicity
6
The forward error 5
-- Ouch! Whos responsible?
The backward error 5 x 10-10
-- method is good!
Conclusion the problem is bad
7
If the answer is highly sensitive to
perturbations, you have probably asked the wrong
question. Maxims about numerical mathematics,
computers, science and life, L. N. Trefethen.
SIAM News
A Customer B Numerical analyst
Who is asking a wrong question?
A The polynomial or matrix B The computing
objective
What is the wrong question?
8
Kahans pejorative manifolds
All n-polynomials having certain multiplicity
structure form a pejorative manifold
xn a1 xn-1...an-1 x an ltgt (a1 , ...,
an-1 , an )
Example ( x-t )2 x2 (-2t) x t2
Pejorative manifold a1 -2t a2 t2
9
Pejorative manifolds of 3-polynomials
( x - s )( x - t )2 x3 (-s-2t) x2 (2stt2)
x (-st2)
a1 -s-2t a2 2stt2 a3 -st2
Pejorative manifold of multiplicity structure
1,2
( x - s )3 x3 (-3s) x2 (3s2) x (-s3)
a1 -3s a2 3s2 a3 -s3
Pejorative manifold of multiplicity structure 3

10
Pejorative manifolds of 3-polynomials
The wings a1 -s-2t a2 2stt2 a3 -st2
The edge a1 -3s a2 3s2 a3 -s3
General form of pejorative manifolds u G(z)
11
W. Kahan, Conserving confluence curbs
ill-condition, 1972
1. Ill-condition occurs when a polynomial/matrix
is near a pejorative manifold.
2. A small drift of the problem on that
pejorative manifold does not cause large forward
error to the multiple roots, except
3. If a multiple root/eigenvalue is sensitive to
small perturbation on the pejorative manifold,
then the polynomial/matrix is near a pejorative
submanifold of higher multiplicity.
Ill-condition is caused by solving
polynomial equations on a wrong manifold
12
Pejorative manifolds of 3-polynomials
The wings a1 -s-2t a2 2stt2 a3 -st2
The edge a1 -3s a2 3s2 a3 -s3
13
Given a polynomial p(x) xn a1
xn-1...an-1 x an
Find ( z1, ..., zn ) such that p(x) ( x - z1
)( x - z2 ) ... ( x - zn )
/ / / / / / / / / / / / / / / / /
/ / / / / / / / / / / / / / / / /
/ /
The wrong question
because you are asking for simple roots!
Find distinct z1, ..., zm such that p(x) (
x - z1 ) s1( x - z2 )s2 ... ( x - zm )sm
s1... sm n, m lt n
The right question
do it on the pejorative manifold!
14
For ill-conditioned polynomial p(x) xn
a1 xn-1...an-1 x an a (a1 ,
..., an-1 , an )
The objective find uG(z) that is nearest to
p(x)a
15
Let ( x - z1 ) s1( x - z2 )s2 ... ( x - zm
)sm xn g1 ( z1, ..., zm ) xn-1...gn-1 (
z1, ..., zm ) x gn ( z1, ..., zm )
Then, p(x) ( x - z1 ) s1( x - z2 )s2 ...
( x - zm )sm ltgt
g1 ( z1, ..., zm ) a1 g2( z1, ..., zm ) a2 ...
... ... gn ( z1, ..., zm ) an
(mltn)
n
m
I.e. An over determined polynomial system G(z)
a
16
The polynomial a
Project to tangent plane u1
G(z0)J(z0)(z1- z0)

tangent plane P0 u G(z0)J(z0)(z- z0)
pejorative root uG(z )
initial iterate u0G(z0)
new iterate u1G(z1)
Pejorative manifold u G( z )
Solve G( z ) a for
nonlinear least squares solution zz
Solve G(z0)J(z0)( z - z0 ) a
for linear least squares solution z z1
G(z0)J(z0)( z - z0 ) a
J(z0)( z - z0 ) - G(z0) - a
z1 z0 - J(z0) G(z0) - a
17
zi1zi - J(zi ) G(zi )-a , i0,1,2 ...
Theorem If z(z1, ..., zm) with z1, ..., zm
distinct, then the Jacobian J(z) of G(z) is of
full rank.
Theorem Let uG(z) be nearest to p(x)a,
if 1. z(z1, ..., zm) with z1, ..., zm
distinct 2. z0 is sufficiently close to
z 3. a is sufficiently close to u then the
iteration converges with a linear rate. Further
assume that a u , then the convergence
is quadratic.
18
The pejorative condition number
v G(z)
u G(y)
u - v 2 backward error y - z 2
forward error
u - v G(y) - G(z) J(z) (y - z) h.o.t.
u - v 2 J(z) (y - z) 2 gt s y - z
2
y - z 2 lt (1/s) u - v 2
1/s is the pejorative condition number where s
is the smallest singular value of J(z) .
19
Example (x-0.9)18(x-1.0)10(x-1.1)16 0
Step z1 z2 z3 ------------------------------
-------------------------------------- 0 .92
.95 1.12 1 .87 1.05 1.10 2 .92
.95 1.11 3 .88 1.01 1.10 4 .90
.97 1.12 5 .901 .992 1.101 6 .89993
.9998 1.1002 7 .9000003 .999998 1.1000007 8
.899999999997 .999999999991 1.100000000009 9 .9
00000000000006 .99999999999997 1.10000000000001
forward error 6 x 10-15 backward error 8 x
10-16 Pejorative condition 58
Even clustered multiple roots are pejoratively
well conditioned
20
Example (x-.3-.6i)100 (x-.1-.7i) 200 (x -
.7-.5i) 300 (x-.3-.4i) 400 0
Scary enough? Round coefficients to 6 digits.
Z1 z2 z3 z4 .289 .601i
.100 .702i .702 .498i .301
.399i .309 .602i .097 .698i
.698 .499i .299 .401i .293
.596i .101 .7003i .7002 .5005i
.3007 .4003i .300005 .600006i .099998
.6999992i .69999992.4999993i .2999992
.3999992i .3000002.60000005i .09999995.69999998
i .69999997.49999998i .29999997.400000002i
Roots are correct up to 7 digits!
Pejorative condition 0.58
21
Example The Wilkinson polynomial p(x)
(x-1)(x-2)...(x-20) x20 - 210 x19 20615
x18 ...
There are 605 manifolds in total. It is near
some manifolds, but which ones?
Multiplicity backward error condition
Estimated structure number
error --------------------------------------------
---------------------------- 1,1,1,1,1,1,1,1,1...
,1 .000000000000003 550195997640164
1.6 1,1,1,1,2,2,2,4,2,2,2 .000000003
29269411 .09 1,1,1,2,3,4,5,3 .0000001
33563 .003 1,1,2,3,4,6,3
.000001 6546
.007 1,1,2,5,7,4 .000005
812 .004 1,2,5,7,5 .00004
198 .008 1,3,8,8 .0002
25 .005 2,8,10 .003
6 .02 5,15 .04
1 .04 20 .9
.2 .2
What are the roots of the Wilkinson polynomial?
Choose your poison!
22
The right question for ill-conditioned
eigenproblem
Given a matrix A
Find a structured Schur form S and a matrix U
such that
AU - US 0 UU - I 0
Over-determined!!!
3 1 3
l l

A
m
2
S
m m
2 1 2
Minimize AU - US F2 UU - I F2

---- nonlinear least squares problem
23
Example A E, where E A 1.0e-7
Step l m ----------------------------------
-------- 0 4.0 1.1 1 2.99 2.01
2 3.0006 1.9998 3 3.000001 1.9999997
4 3.00000001 1.99999992
3 3

UT(AE)U
O(10 -7)
2
2 2
The pejorative condition number 22.8
24
Conclusion
1. Ill-condition is cause by a wrong identity
2. Multiple eigenvalues/roots are pejoratively
well conditioned, thereby tractable.
3. Extension of machine precision is NOT
needed, a change in computing concept is.
4. To calculate ill-conditioned
eigenvalues/roots, one has to figure out the
pejorative structure (how?)