Eigenvalues and Eigenvectors

Shahid Hussain
May 19, 2009
Outline
Introduction
Determinant
Introduction to Eigensystems
Similarity
Introduction and Preliminaries
Permutation
A permutation p = (p
1
, p
2
, . . . , p
n
) of numbers (1, 2, . . . , n is
simply any rearrangement. There are total n! dierent
permutations for (1, 2, . . . , n).
For example, the set
{(1, 2, 3) (1, 3, 2) (2, 1, 3) (2, 3, 1) (3, 1, 2) (3, 2, 1)}
contains the six distinct permutations of (1, 2, 3).
Cont.
Parity
The parity of a permutation is unique i.e., if a permutation p
can be restored to natural order by an even (odd) number of
interchanges, then every other sequence of interchanges that
restores p to natural order must also by even (odd). Accordingly,
the sign of a permutation p is dened to be the number
(p) =
_

_
+1
if p can be restored to natural order by an
even number of interchanges,
1
if p can restored to natural order by an
odd number of interchanges.
For example, if p = (1, 4, 3, 2), then (p) = 1, and if
p = (4, 3, 2, 1), then (p) = +1. The sign of natural order of
p = (1, 2, 3, 4) is naturally (p) = +1.
Determinant
Determinant
For an n n matrix A = [a
ij
], the determinant of A is dened to
be the scalar
det(A) =

p
(p)a
1p
1
a
2p
2
a
np
n
,
where the sum is taken over the n! permutations
p = (p
1
, p
2
, . . . , p
n
) of (1, 2, . . . , n). Observe that each term
a
1p
1
a
2p
2
a
np
n
contains exactly one entry from each row and
each column of A. The determinant of A can be denoted by
det(A) or |A|, whichever is more convenient.
Cont.
For example, when A is 2 2 there are 2! = 2 permutations of
(1, 2) namely, {(1, 2) (2, 1)}, so det(A) contains the two terms:
(1, 2)a
11
a
22
and (2, 1)a
12
a
21
.
Since (1, 2) = +1 and (2, 1) = 1, we obtain the familiar
formula

a
11
a
12
a
21
a
22

= a
11
a
22
a
12
a
21
.
Example
Use the denition to compute det(A), where A =
_
_
1 2 3
4 5 6
7 8 9
_
_
.
Solution. There are six permutations 3! = 6 of (1, 2, 3) shown
together with the terms in the expansion of det(A) in following
table.
p = (p
1
, p
2
, p
3
) (p) a
1p
1
a
2p
2
a
3p
3
(1, 2, 3) + 1 5 9 = 45
(1, 3, 2) 1 6 8 = 48
(2, 1, 3) 2 4 9 = 72
(2, 3, 1) + 2 6 7 = 84
(3, 1, 2) + 3 4 8 = 96
(3, 2, 1) 3 5 7 = 105
Therefore,
det(A) =

p
(p)a
1p
1
a
2p
2
a
3p
3
= 454872+84+96105 = 0.
Example
Use the denition to compute det(A), where A =
_
_
1 2 3
4 5 6
7 8 9
_
_
.
Solution. There are six permutations 3! = 6 of (1, 2, 3) shown
together with the terms in the expansion of det(A) in following
table.
p = (p
1
, p
2
, p
3
) (p) a
1p
1
a
2p
2
a
3p
3
(1, 2, 3) + 1 5 9 = 45
(1, 3, 2) 1 6 8 = 48
(2, 1, 3) 2 4 9 = 72
(2, 3, 1) + 2 6 7 = 84
(3, 1, 2) + 3 4 8 = 96
(3, 2, 1) 3 5 7 = 105
Therefore,
det(A) =

p
(p)a
1p
1
a
2p
2
a
3p
3
= 454872+84+96105 = 0.
Nilpotent Matrices
Nilpotent Matrices

N
nn
is said to be nilpotent whenever N
k
= 0 for some
positive integer k.

k = index (N) is the smallest positive integer such that

N
k
= 0. (Some authors refer to as the index of nilpotency.)

Furthermore, if A is nilpotent then trace (A) = 0. (It has

something to do with the spectrum of A! To be covered
later.)
Cont.
Verify that N =
_
_
0 1 0
0 0 1
0 0 0
_
_
is a nilpotent matrix, and determine
its index.
Solution. Computer the powers
N
2
=
_
_
0 0 1
0 0 0
0 0 0
_
_
and N
3
=
_
_
0 0 0
0 0 0
0 0 0
_
_
,
reveals that N is indeed nilpotent, and it shows that
index (N) = 3 because N
3
= 0, but N
2
= 0.
Cont.
Verify that N =
_
_
0 1 0
0 0 1
0 0 0
_
_
is a nilpotent matrix, and determine
its index.
Solution. Computer the powers
N
2
=
_
_
0 0 1
0 0 0
0 0 0
_
_
and N
3
=
_
_
0 0 0
0 0 0
0 0 0
_
_
,
reveals that N is indeed nilpotent, and it shows that
index (N) = 3 because N
3
= 0, but N
2
= 0.
Elementary properties
Consider the problem of solving the system of two rst-order linear
dierential equations du
1
/dt = 7u
1
4u
2
and
du
2
/dt = 5u
1
2u
2
. In matrix notation, this systems is
_
u

1
u

2
_
=
_
7 4
5 2
_ _
u
1
u
2
_
or, equivalently, u

= Au.
Solution of single equation u

= u have the form u = e

t
, we
are motivated to seek solutions of the above system that also have
the form
u
1
=
1
e
t
and u
2
=
2
e
t
.
Cont.
Dierentiating these two expressions and substituting the results
yields

1
e
t
= 7
1
e
t
4
2
e
t

2
e
t
= 5
2
e
t
2
2
e
t
We get:

1
= 7
1
4
2

2
= 5
1
2
2
We get:
_
7 4
2 2
_ _

2
_
=
_

2
_
Eigenvalues and Eigenvectors
Eigenvalues and Eigenvectors
For an n n matrix A, scalars and vectors x
n1
= 0 satisfying
Ax = x are called eigenvalues and eigenvectors of A, respectively,
and any such pair (, x) is called eigenpair for A. The set of
distinct eigenvalues, denoted by (A), is called the spectrum of A.

(A) AI is singular det(AI) = 0.

{x = 0 | x N(AI)} is the set of all eigenvectors

associated with . From now on, N(AI) is called an
eigenspace for A.

Nonzero row vectors y

such that y

(AI) = 0 are called

left-hand eigenvectors for A.
Geometry
Geometrically, Ax = x says that under transformation by A,
eigenvectors experience only changes in magnitude or sign the
orientation of Ax in R
n
is the same as that of x. The eigenvalue
is simply the amount of stretch or shrink to which the
eigenvector x is subjected when transformed by A. See following
gure.
Computing the Eigenvalues
Let A =
_
7 4
5 2
_
. The eigenvalues are the scalars for which
det(AI) = 0. We get:
p() = det(AI) =

7 4
5 2

=
2
5+6 = (2)(3).
We call p() the characteristic polynomial for A.
Cont.
The eigenvectors associated with = 2 and = 3 are simply the
nonzero vectors in the eigenspaces N(A2I) and N(A3I),
respectively. We solve two homogeneous systems, (A2I)x = 0
and (A3I)x = 0.
For = 2,
A2I =
_
5 4
5 4
_

_
1 4/5
0 0
_
=
x
1
= (4/5)x
2
x
2
is free
We get
N(A2I) =
_
x |x =
_
4/5
1
__
.
For = 3,
A3I =
_
3 4
5 5
_

_
1 1
0 0
_
=
x
1
= x
2
x
2
is free
We get
N(A3I) =
_
x |x =
_
1
1
__
.
Characteristic Polynomial

The characteristic polynomial of A

nn
is p() = det(AI).
The degree of p() is n and the leading term in p() is
(1)
n

n
.

The characteristic equation for A is p() = 0.

The eigenvalues of A are the solutions of the characteristic

equation or, equivalently the roots of characteristic
polynomial.

Altogether, A has n eigenvalues, but some may be complex

numbers (even if the entries of A are real numbers), and some
eigenvalues may be repeated.

If A contains only real numbers, then its complex eigenvalues

must occur in conjugate pairs i.e., if (A), then
(A).
Some Problems
1. Determine the eigenvalues and eigenvectors for the following
matrices.
A =
_
10 7
14 11
_
. B =
_
_
2 16 8
4 14 8
8 32 18
_
_
. C =
_
_
3 0 0
0 3 0
0 0 3
_
_
.
2. Prove that 0 (A) if and only if A is a singular matrix.
3. Explain that the matrix A =
_

_
4 1 1 1
1 4 1 1
1 1 4 1
1 1 1 4
_

_
does not have a
zero eigenvalue, and hence why A is nonsingular.
4. Prove that the eigenvalues of A

A and AA

are real and

nonnegative for every A C
mn
.
Introduction

Consider the obliquely ellipse in following gure in the

xy-coordinate system whose equation is
13x
2
+ 10xy + 13y
2
= 72.

If we rotate the xy-coordinate system counterclockwise

through an angle of /4 radians into a uv-coordinate system,
the cross-product term is eliminated,
and the equation of the ellipse simplies to become:
u
2
9
+
v
2
4
= 1.
Similarity
Similarity

Two n n matrices A and B are said to be similar whenever

there exists a nonsingular matrix P such that P
1
AP = B.
The product P
1
AP is called a similarity transformation on
A.

A Fundamental Problem. Given a square matrix A, reduce it

to the simplest possible form by means of a similarity
transformation.
Diagonal Matrices
Is every square matrix similar to a diagonal matrix?
Unfortunately, NO. For example, consider A =
_
0 1
0 1
_
, and
observe that A
2
= 0. If there exists a nonsingular matrix P such
that P
1
AP = D, where D is diagonal, then
D
2
= P
1
APP
1
AP = P
1
A
2
P = 0 = D = 0 = A = 0,
which is false. Thus A, as well as any other nonzero nilpotent
matrix, is not similar to a diagonal matrix.
Diagonal Matrices
Is every square matrix similar to a diagonal matrix?
Unfortunately, NO. For example, consider A =
_
0 1
0 1
_
, and
observe that A
2
= 0. If there exists a nonsingular matrix P such
that P
1
AP = D, where D is diagonal, then
D
2
= P
1
APP
1
AP = P
1
A
2
P = 0 = D = 0 = A = 0,
which is false. Thus A, as well as any other nonzero nilpotent
matrix, is not similar to a diagonal matrix.
Cont.
So, if not all square matrices can be diagonalized by a singularity
transformation, what are the characteristics of those that can? An
answer is easily derived by examining the equation
P
1
A
nn
P = D =
_

1
0 0
0
2
0
.
.
.
.
.
.
.
.
.
.
.
.
0 0
n
_

_
Cont.
which implies
A[P
1
| | P
n
] = [P
1
| | P
n
]
_

1
0 0
0
2
0
.
.
.
.
.
.
.
.
.
.
.
.
0 0
n
_

_
or
equivalently, [AP
1
| | AP
n
] = [
1
P
1
| |
n
P
n
].
Consequently, AP
j
=
j
P
j
for each j, so each (
j
, P
j
) is an
eigenpair for A. In simple words, P
1
AP = D implies that P
must be a matrix whose columns constitute n linearly independent
eigenvectors, and D is a diagonal matrix whose diagonal entries
are the corresponding eigenvalues.
Diagonalizability
Diagonalizability

A square matrix A is said to be diagonalizable whenever A is

similar to a diagonal matrix.

A complete set of eigenvectors for A

nn
is any set of n
linearly independent eigenvectors for A. Not all matrices have
complete sets of eigenvectors. Matrices that fail to possess
complete sets of eigenvectors are sometimes called decient or
defective matrices.

A
nn
is diagonalizable if and only if A possesses a complete
set of eigenvectors. Moreover,
P
1
AP = diag (
1
,
2
, . . . ,
n
) if and only if the columns of
P constitute a complete set of eigenvectors and the
j
s are
the associated eigenvalues, i.e., each (
j
, P
j
) is an eigenpair
for A.
Cont.
If possible, diagonalize the following matrix with a similarity
transformation:
A =
_
_
1 4 4
8 11 8
8 8 5
_
_
.
Solution. Determine whether or not A has a complete set of three
linearly independent eigenvectors. The characteristic equation,
perhaps computed is:

3
+ 5
2
+ 3 9 = ( 1)( + 3)
2
= 0.
= 1 is a simple eigenvalue, = 3 is repeated twice (we say its
algebraic multiplicity is 2). Following are the eigenspaces:
N(A1I) = span
_
_
_
_
_
1
2
2
_
_
_
_
_
and N(A+3I) = span
_
_
_
_
_
1
1
0
_
_
,
_
_
1
0
1
_
_
_
_
_
,
Cont.
If possible, diagonalize the following matrix with a similarity
transformation:
A =
_
_
1 4 4
8 11 8
8 8 5
_
_
.
Solution. Determine whether or not A has a complete set of three
linearly independent eigenvectors. The characteristic equation,
perhaps computed is:

3
+ 5
2
+ 3 9 = ( 1)( + 3)
2
= 0.
= 1 is a simple eigenvalue, = 3 is repeated twice (we say its
algebraic multiplicity is 2). Following are the eigenspaces:
N(A1I) = span
_
_
_
_
_
1
2
2
_
_
_
_
_
and N(A+3I) = span
_
_
_
_
_
1
1
0
_
_
,
_
_
1
0
1
_
_
_
_
_
,
Cont.
When combined these three eigenvectors constitute a linearly
independent set. Consequently, A must be diagonalizable. To
explicitly exhibit the similarity transformation that diagonalize A,
set
P =
_
_
1 1 1
2 1 0
2 0 1
_
_
, and verify P
1
AP =
_
_
1 0 0
0 3 0
0 0 3
_
_
= D.
Cont.

Not all square matrices are diagonalizable!

Can every square matrix be triangularized by similarity?

Fortunately, the answer is yes!

Before we proceed we need to following observation. (This is

not strange!)
Cont.

Not all square matrices are diagonalizable!

Can every square matrix be triangularized by similarity?

Fortunately, the answer is yes!

Before we proceed we need to following observation. (This is

not strange!)
Cont.

Not all square matrices are diagonalizable!

Can every square matrix be triangularized by similarity?

Fortunately, the answer is yes!

Before we proceed we need to following observation. (This is

not strange!)
Cont.

Not all square matrices are diagonalizable!

Can every square matrix be triangularized by similarity?

Fortunately, the answer is yes!

Before we proceed we need to following observation. (This is

not strange!)
Similarity Preserves Eigenvalues
Similarity Preserves Eigenvalues
Row reductions dont preserve eigenvalues (try a simple example).
However, similar matrices have the same characteristic polynomial,
so they have the same eigenvalues with the same multiplicities.
Caution! Similar matrices need not have the same eigenvectors.
Proof.
Can you?
Similarity Preserves Eigenvalues
Similarity Preserves Eigenvalues
Row reductions dont preserve eigenvalues (try a simple example).
However, similar matrices have the same characteristic polynomial,
so they have the same eigenvalues with the same multiplicities.
Caution! Similar matrices need not have the same eigenvectors.
Proof.
Can you?
Cont.

This means that the eigenvalues of a matrix representation of

a linear operator L are invariant under a change of basis.

Equivalently, we may say that the eigenvalues are intrinsic to

L in the sense that they are independent of any coordinate
representation.
Cont.

This means that the eigenvalues of a matrix representation of

a linear operator L are invariant under a change of basis.

Equivalently, we may say that the eigenvalues are intrinsic to

L in the sense that they are independent of any coordinate
representation.
Schurs Triangularization Theorem
Schurs Triangularization Theorem
Every square matrix is unitarily similar to an upper-triangular
matrix. That is, for each A
nn
there exists a unitary matrix U (not
unique) and an upper-triangular matrix T (not unique) such that
U

AU = T, and the diagonal entries of T are eigenvalues of A.

Multiplicities
Multiplicities
For (A) = {
1
,
2
, . . . ,
s
}, we adopt the following
denitions.

The algebraic multiplicity of is the number of times it is

repeated as a root of the characteristic polynomial. In other
words, alg mult
A
(
i
) = a
i
, if and only if
(x
i
)
a
1
(x
s
)
a
s
= 0 is the characteristic equation for
A.

When alg mult

A
() = 1, is called a simple eigenvalue.

The geometric multiplicity of is dim N(AI). In other

words, geo mult
A
() is the maximal number of linearly
independent eigenvectors associated with .

Eigenvalues such that alg mult

A
() = geo mult
A
() are
called semisimple eigenvalues of A. If follows that a simple
eigenvalue is always semisimple, but not conversely.
Cont.
Consider the following nilpotent matrix A =
_
0 1
0 0
_
has only one
distinct eigenvalue, = 0 that is repeated twice, so
alg mult
A
(0) = 2. But
dim N(A0I) = dimN(A) = 1 = geo mult
A
(0) = 1.
In other words, there is only one linearly independent eigenvector
associated with = 0 even tough = 0 is repeated twice as an
eigenvalue.
Cont.
Multiplicity Inequality
For every A C
nn
, and for each (A),
geo mult
A
() alg mult
A
()
Independent Eigenvectors
Independent Eigenvectors
Let {
1
,
2
, . . . ,
k
} be a set of distinct eigenvalues for A.

If {(
1
, x
1
), (
2
, x
2
), . . . , (
k
, x
k
)} is a set of eigenpairs for A,
then S = {x
1
, x
2
, . . . , x
k
} is a linearly independent set.

If B
i
is a basis for N(A
i
I), then B = B
1
B
2
B
k
,
is a linearly independent set.
Cont.
Diagonalizability and Multiplicities
A matrix A
nn
is diagonalizable if and only if
geo mult
A
() = alg mult
A
()
for each (A), i.e., if and only if every eigenvalue is
semisimple.
Cont.
Determine which of the following matrix(matrices) is(are)
diagonalizable:
A =
_
_
1 1 2
8 11 8
10 11 7
_
_
, B =
_
_
1 4 4
8 11 8
8 8 5
_
_
.
Cont.
Distinct Eigenvalues
If no eigenvalue of A is repeated, then Ais diagonalizable.
Caution! The converse is not true.
Spectral Theorem for Diagonalizable Matrices
Theorem
A matrix A
nn
with spectrum A = {
1
,
2
, . . . ,
k
} is
diagonalizable if and only if there exist matrices {G
1
, G
2
, . . . , G
k
}
such that
A =
1
G
1
+
2
G
2
+ +
k
G
k
,
where the G
i
s have the following properties.

G
i
is the projector onto N(A
i
I) along R(A
i
I).

G
i
G
j
= 0 whenever i = j.

G
1
+G
2
+ G
k
= I
The above equation is known as the spectral decomposition of A,
and the G
i
s are called the spectral projectors associated with A.
Cont.
Simple Eigenvalues and Projectors
If x and y

are respective right-hand and left-hand eigenvectors

associated with a simple eigenvalue (A), then
G = xy

/y

x
is the projected onto N(AI) along R(AI). In the context
of the spectral theorem, this means that G is the spectral projector
associated with .
Problem
Determine the spectral projector for A =
_
_
1 4 4
8 11 8
8 8 5
_
_
.
Solution. We already know that A is diagonalizable and there are
two distinct eigenvalues
1
= 1 and
2
= 3, there are two
spectral projector,
G
1
= the projector onto N(A1I) along R(A1I).
G
2
= the projector onto N(A+ 3I) along R(A+ 3I).
There are several dierent ways to nd these projectors.
1. Compute bases for the necessary nullspaces and ranges,
2. Compute G
i
= X
i
Y
T
i
. The required computations are
essentially the same as those needed above. Since much of
the work has already been done in previous example, lets
complete the arithmetic. We have
P =

1 1 1
2 1 0
2 0 1

= [X
1
|X
2
] , P
1
=

1 1 1
2 3 2
2 2 1

Y
T
1
Y
T
2

,
Problem
Determine the spectral projector for A =
_
_
1 4 4
8 11 8
8 8 5
_
_
.
Solution. We already know that A is diagonalizable and there are
two distinct eigenvalues
1
= 1 and
2
= 3, there are two
spectral projector,
G
1
= the projector onto N(A1I) along R(A1I).
G
2
= the projector onto N(A+ 3I) along R(A+ 3I).
There are several dierent ways to nd these projectors.
1. Compute bases for the necessary nullspaces and ranges,
2. Compute G
i
= X
i
Y
T
i
. The required computations are
essentially the same as those needed above. Since much of
the work has already been done in previous example, lets
complete the arithmetic. We have
P =

1 1 1
2 1 0
2 0 1

= [X
1
|X
2
] , P
1
=

1 1 1
2 3 2
2 2 1

Y
T
1
Y
T
2

,
Cont.
so
G
1
= X
1
Y
T
1
=
_
_
1 1 1
2 2 2
2 2 2
_
_
, G
2
= X
2
Y
T
2
=
_
_
0 1 1
2 3 2
2 2 1
_
_
.
3. Since
1
= 1 is a simple eigenvalue, we can compute G
1
as
G
1
=
xy
T
y
T
x
=

_
_
1
2
2
_
_

_
1 1 1

=
_
_
1 1 1
2 2 2
2 2 2
_
_
.
Other spectral projects is G
2
= I G
1
.
Cont.
4. An even easier solutions is obtained from the spectral theorem
by writing
AI = (1G
1
3G
2
) (G
1
+G
2
) = 4G
2
,
A+ 3I = (1G
1
3G
2
) + 3(G
1
+G
2
) = 4G
1
,
so that
G
1
=
A+ 3I
4
and G
2
=
(AI)
4
.
5. In face, the above is really a special case of a completely
general formula giving each G
i
as a function Aand
i
as
G
i
=
k

j=1
j=i
(A
j
I)
k

j=1
j=i
(
i

j
)
.
Summary
Summary of Diagonalizability
For an n n matrix A with spectrum (A) = {
1
,
2
, . . . ,
k
},
the following statements are equivalent.

A is similar to a diagonal matrix, i.e., P

1
AP = D.

A has a complete linearly independent set of eigenvectors.

Every
i
is semisimple, i.e., geo mult
A
(
i
) = alg mult
A
(
i
).

A =
1
G
1
+
2
G
2
+ +
k
G
k
, where

G
i
is the projector onto N(A
i
I) along R(A
i
I),

G
i
G
j
= 0 whenever i = j,

G
1
+G
2
+ +G
k
= I,

G
i
=
k

j=1
j=i
(A
j
I)
_
k

j=1
j=i
(
i

j
),

If
i
is a simple eigenvalue associated with right-hand and
left-hand eigenvectors x and y

, respectively, then
G
i
= xy

/y

x.