THE MATRIX EIGENVALUE PROBLEM

Find scalars and vectors x =

0 for which
Ax = x
The form of the matrix aects the way in which we
solve this problem, and we also have variety as to what
is to be found.

A symmetric and real (or Hermitian and com-

plex). This is the most common case. In some
cases we want only the eigenvalues (and perhaps
only some of them); and in other cases, we also
want the eigenvectors. There are special classes
of such A, e.g. banded, positive denite, sparse,
and others.

A non-symmetric, but with a diagonal Jordan

canonical form. This means there is a nonsin-
gular matrix P for which
P 1AP = D = diag[1, ..., n ]
 Then AP = P D and the columns of P are the
eigenvectors of A.
As we see later, these matrix eigenvalue problems
may be ill-conditioned. There are special sub-
classes of problems, as with the symmetric case.
Note that when A is real, the complex eigenvalues
(if any) must occur in conjugate pairs.

A non-symmetric and the Jordan canonical form

is not diagonal. These are very dicult problems,
especially when calculating the eigenvectors.

GENERAL APPROACH. Begin by nding the eigen-

values. Then nd the eigenvectors, if they are needed.

Finding the eigenvalues. Proceed in two steps.

(1) Reduce A to a simpler form T , usually using or-
thogonal similarity transformations.
(2) Apply some method to nding the eigenvalues of
T.
 GERSCHGORINS CIRCLE THEOREM

Where are the eigenvalues of A located? We know

that for any matrix norm ,

max || A
(A)
How can this be improved? Let

n  
 
ri = ai,j  , i = 1, ..., n
j=1
j
=i
   
 
Zi = z C | z ai,i ri , i = 1, ..., n
The set Zi is a circle with center ai,i and radius ri.
Then the eigenvalues of A are located in the union of
the circles Zi:
n

(A) Zi
i=1
Moreover, break this union into disjoint components,
say C1, ..., Cm. Then each such component contains
exactly as many eigenvalues as circles Zi.
PROOF. Let Ax = x, x
= 0. Let k be an index for
which

x = |xk | > 0

From Ax = x,
n


ai,j xj = xi, i = 1, ..., n
j=1
Solve equation k for xk :
  n


ak,k xk = ak,j xj
j=1
j
=k
Taking absolute values,
 
n   
n  
      
 ak,k  |xk | ak,j  xj  ak,j  |xk |
j=1 j=1
j
=k j
=k
Cancel |xk | to get
 
n  
   
 ak,k  ak,j  = rk
j=1
j
=k
 EXAMPLE

Recall the matrix

c 1 0 0 0

1 c 1 0 0

A= 0 1 c 1 0


0 0 1 c 1

0 0 0 1 c
which is used as an example at the end of Chapter 7.
In this case,

r1 = r5 = 1; ri = 2, i = 2, 3, 4
The centers of the circles are all ai,i = c. Then the
union of the circles Zi is the circle

{z | |z c| 2}
The matrix A is real and symmetric, and thus all
eigenvalues are real. Thus the eigenvalues must
be located in the interval

c2c+2
 BAUER-FIKE THEOREM

Assume A has a diagonal Jordan canonical form, mean-

ing there is a nonsingular matrix P for which

P 1AP = D = diag[1, ..., n]

Assume we are using a matrix norm for which

D = max |i |
1in
Then consider the eigenvalues of the perturbed ma-
trix A + E . For such , we have
 
 1 
min | i| P  P  E
1in
PROOF. Write
(A + E ) x = x, x=

0
(I A) x = Ex
Assume
= 1, ..., n, as otherwise the theorem is
easily true. Substitute A = P DP 1,
 
I P DP 1 x = Ex
    
(I D) P 1x = P 1EP 1
P x
  
1
P 1x = (I D) 1
P EP P x1

Take norms of both sides,

     
 1   1  1   1 
P x (I D)  P EP  P x
 
 1 
Cancel P x,
  
 1  1 
1 (I D)  P EP 
Also note that by our assumption on the matrix norm,
  1 1
 1 
(I D )  = max =
i | i | mini | i|
Then
   
 1   1 
min | i| P EP  P  P  E
i
This completes the proof.
Consider the case in which A is symmetric and real.
Then the matrix P can be chosen to be orthogonal,
and P 1 = P T . If we use the matrix norm 2
induced by the Euclidean vector norm 2, then from
Problem 13 of Chapter 7, P 2 = 1. Thus for this
particular matrix norm,
    
 1  T
min | i | P EP  E2 = sqrt r E E
i 2
Thus small changes in the matrix lead to small changes
in the eigenvalues.

We can also use the inequality

 
 1 
min | i| P  P  E
i
to dene a condition number for the eigenvalue prob-
lem. For it, we would use
 
 1 
cond (A) = inf P  P 
P 1 AP =D
Then

min | i | cond (A) E

i
This says the changes in the eigenvalues are small.
But there may still be a large relative change.

From the book, consider the 33 Hilbert matrix and

its version rounded to four decimal digits.
1 1

1 2 3
1
H3 =
2
1
3
1
4
1 1 1
3 4 5

1.000 .5000 .3333

H 3 = .5000 .3333 .2500
.3333 .2500 .2000
In this case, A = H3 and A + E = H 3, or E =
H 3 H3. Using the matrix norm 2, the Bauer-
Fike result says that for each eigenvalue of H 3,

min | i | E2 = 3.3 105

i
In fact the true eigenvalues of H3 are

1 = 1.408319, 2 = .1223271, 3 = .002687340

and the true eigenvalues of H 3 are
 = 1.408294,
 = .1223415,
 = .002664489

1 2 3

For the errors,


1 1 = 2.49 105

2 5
2 = 1.44 10

3 2.29 105
3 =
which is in agreement with

min | i | E2 = 3.3 105

i
For the relative errors,
   
Rel 1 = 1.77 105,
 Rel 2 = 1.18 104


 
 3
Rel 3 = 8.5 10
 EXAMPLE

   
101 90 100.999 90.001
A= , A+E =
110 98 110 98
For A, the eigenvalues are 1, 2. For A + E , the eigen-
values are
.
= 1.298, 1.701
This is a very signicant change in eigenvalues for a
very small change in the matrix. It is illustrative of
what can happen with the non-symmetric eigenvalue
problem.
 WIELANDT-HOFFMAN THEOREM

Let A and E be real and symmetric, and let A =

A + E . Let the eigenvalues of A be

1 2 n
and let those of A be



1 2 n

Then
1 1
n 

2 2 n


n  2 2
  
j j F (E )  E i,j 
j=1 i=1 j=1
 EXAMPLE - NONSYMMETRIC

Consider the n n matrix

1 1 0 0 0

0 1 1 0 0
... ..

A=
.. ...




1 1

0 0 1
Its characteristic polynomial is

f () = (1 )n
Its only eigenvalue is = 1; and there is only a one-
dimensional family of eigenvectors, all multiples of

x = [1, 0, , 0]T
Now perturb the matrix to

1 1 0 0 0

0 1 1 0 0
... ..

A() =
.. ...




0 1 1

 0 0 1
Its characteristic polynomial is

f() = (1 )n (1)n 
Its roots, and the eigenvalues of A(), are

k () = 1 + k 1/n, k = 1, ..., n

with {k } the nth roots of unity,

k = e2ki/n, k = 1, ..., n
Thus

|k k ()| = ||1/n
For n = 10 and  = 1010, |k k ()| = 0.1.
 STABILITY FOR
NONSYMMETRIC MATRICES

Assume the matrix A has a diagonal Jordan canonical

form:

P 1AP = D = diag[1, ..., n]

Let P = [u1, ..., un]. Then AP = P D implies

Aui = i ui, i = 1, ..., n

and the vectors {u1, ..., un} form a basis of Cn .

To see some of the nonuniqueness in the choice of P ,

let F be an arbitrary nonsingular diagonal matrix,

F = diag[f1, ..., fn]

Then
F 1P 1AP F = F 1DF
(P F )1 A (P F ) = D
The matrix P F is another nonsingular matrix; and
since F is diagonal,
P F = [u1, ..., un] F = [f1u1, ..., fnun]
The vectors fiui are again eigenvectors of A. There-
fore, we assume that P has been so chosen that the
vectors ui all have Euclidean length 1:
ui ui = 1, i = 1, ..., n
Note that because the eigenvalues can be complex,
we must now work in Cn ; and we also allow A to be
complex.

Form the complex conjugate transpose of P 1AP =

D:
P A (P )1 = D = diag[1, ..., n]
Write
(P )1 = [w1, ..., wn]
Then as before with A, we have
A wi = i wi , i = 1, ..., n
wiA = i wi
The vectors wi are sometimes called left eigenvectors
of A. Taking the transpose of
(P )1 = [w1, ..., wn]

w1
..
P 1 =
wn

Write out P 1P = I to get

w 1 0
.1 . . . ..
. [u1, ..., un] = ..
wn 0 1

1, i = j
wiuj =
0, i =

j
Normalize the eigenvectors {wi} by
wi
vi = , i = 1, ..., n
wi2
giving eigenvectors of A of length 1.

Dene
1
si = viui = , i = 1, ..., n
wi2
We can write
 
v1 vn
(P )1 = , ,
s1 sn
and also

A vi = i vi , vi2 = 1, i = 1, ..., n

With these tools, we can now do a stability analysis for

isolated eigenvalues of A. In particular, assume the
eigenvalue k is a simple eigenvalue of A. Consider
what happens to it with a perturbation of the matrix
A, namely

A() = A + B, >0

Let 1(), ..., n () denote the perturbed eigenvalues
for A(). We want to estimate k () k .
Using the matrix P ,
P 1A()P = P 1 (A + B ) P = D + C
with

v1

s1
1 ..
C = P BP = B [u1, ..., un]

vn
sn
1
ci,j = vi Buj , 1 i, j n
si
We want to prove that
  
k () = k + vk Buk + O 2
sk
The argument for this is given on page 598 of the
text, which I omit here.

Using the vector and matrix 2-norms,

  
|k () k | B2 + O 2
sk
since u2 = v2 = 1. Thus the size of sk is of
crucial importance in determining the stability of k .
 EXAMPLE

Consider
   
101 90 1 0
A= , P 1AP =
110 98 0 2
9 10

P = sqrt(181) sqrt(221)
10 11
sqrt(181) sqrt(221)
 
11 sqrt (181) 10 sqrt (181)
P 1 =
10 sqrt (221) 9 sqrt (221)
This denes the vectors {u1, u2} and {w1, w2}, and
thus
1 1 .
s1 = v1u1 = = = .005
w12 sqrt (221 181)

  
|1() 1| B2 + O 2
s1  
.
= 200 B2 + O 2
 ORTHOGONAL TRANSFORMATIONS

Suppose we transform A using an orthogonal similarity

transformation,
= U AU,
A U U = I
What are the vectors u
 i, v
i, w
 i for this new matrix?
Tranform

P 1AP = D = diag[1, ..., n]

to

P 1U (U AU ) U P = D

(U P )1 A (U P ) = D
This says that the columns {u
 i} are obtained from

U P = U [u1, ..., un] = [U u1, ..., U un]

 i = U ui ,
u i = 1, ..., n
Similarly,

vi = U vi, i = 1, ..., n

Now consider the numbers si which measure the sen-
sitivity of the eigenvalues (provided they are simple).
For the new matrix, call these numbers si. Then
si = viu
i
= (U vi) (U ui)
= viU U ui
= si
Thus an orthogonal similarity transformation of A does
not change these numbers {si}, and thus the condi-
tioning of the eigenvalue problem is not changed. This
is a major reason for using orthogonal transformations.