Topic 7: Eigenvalues and Eigenvectors

[AR Chapt 5]

7.1 Definition of eigenvalues and eigenvectors

7.2 Finding eigenvalues
7.3 Finding eigenvectors
7.4 The Cayley-Hamilton theorem
7.5 Diagonalization

287

7.1 Definition of eigenvalues and eigenvectors

The topic of eigenvalues and eigenvectors is fundamental to many
applications of linear algebra. These include quantum mechanics in
physics, image compression and reconstruction in computing and
engineering, the analysis of high dimensional data in statistics and
algorithms for searching the web.
The key idea is to look for non-zero vectors 0 6= v V that are
stretched by a linear transformation T : V V .
In other words
T (v) = v
for some scalar .
Then Span{v} is a subspace of V of dimension 1 that is mapped into
itself by T .
Note that is a scale factor which stretches the subspace and possibly
changes its sense (if is negative).
288

Definition
Let T : V V be a linear transformation.
A scalar is an eigenvalue of T if there is a non-zero vector v V such
that
T (v) = v
()
The vector v is called an eigenvector of T (with eigenvalue ).
If is an eigenvalue of T , then the set of all v satisfying () is a
subspace of V (exercise!) and is called the eigenspace of .
We will restrict attention to the case that V is finite dimensional. Then
T can be represented by a matrix [T ], which we know depends on the
choice of bases.

289

In fact the idea of eigenvalues and eigenvectors can be applied directly

to square matrices.

Definition
Let A be an n n matrix and let be a scalar. Then a non-zero n 1
column matrix v with the property that
Av = v
is called an eigenvector, while is called the eigenvalue.
To develop some geometric intuition, it is handy to think of A as being
the standard matrix of a linear transformation.

290

Example


1 4
Consider the matrix
as the standard matrix of a linear
1 1
transformation. What is the effect of the transformation on the vectors
(2, 1) and (2, 1)?

291

7.2 Finding eigenvalues

If I denotes the n n identity matrix, the defining equation
Av = v
for eigenvalues and eigenvectors can be rewritten
(A I )v = 0
The values of for which this equation has non-zero solutions are
precisely the eigenvalues.

Theorem
The homogeneous linear system (A I )v = 0 has a non-zero solution
if and only if det(A I ) = 0. Consequently, the eigenvalues of A are
the values of for which
det(A I ) = 0
292

Notation
The equation det(A I ) = 0 is referred to as the characteristic
equation. From our study of determinants, we know that det(I A) is
a polynomial of degree n in . It is called the characteristic polynomial.
Example


1 4
Find the eigenvalues of
.
1 1

293

Example


0 1
Find the eigenvalues of
.
1 0

If we use the standard basis, how can the corresponding linear

transformation be described geometrically? How does this tell you that
the matrix does not have any invariant subspaces?

294

Example

1
0
Find the eigenvalues of the matrix A =
0
0

0
1
0
0

0 0
0 0
.
1 1
1 1

295

7.3 Finding eigenvectors

To find the eigenvectors of a matrix

For each eigenvalue , solve the homogeneous linear system

(A I )v = 0.
Use row reduction as usual.

Note that rank(A I ) < n, so you always obtain at least one row
of zeros.

296

Example


1 4
For each of the eigenvalues 3, 1 of the matrix
, find a
1 1
corresponding eigenvector.

297

Example
For each of the eigenvalues
8 (the eigenvalue 1 is repeated
= 1,
3 2 4
twice) of the matrix A = 2 0 2, find a basis for the corresponding
4 2 3
eigenspace.

298

7.4 The Cayley-Hamilton theorem

We note (without proof) the following

Theorem (Cayley-Hamilton Theorem)

Every square matrix satisfies its characteristic equation, i.e. if
p() = det(A I ) = a0 + a1 + . . . + an n
is the characteristic polynomial of an n n matrix A, then
p(A) = a0 I + a1 A + . . . + an An = 0
is the zero matrix.
We will see some useful consequences of this theorem on the next slides.

299

Theorem
An n n matrix A is invertible if and only if 0 is not an eigenvalue of A.
Proof: If A is invertible the null space of A is {0}. Hence, there are no
non-zero vectors v Rn such that Av = 0v.
If 0 is not an eigenvalue of A, then a0 on the previous slide is not zero,
so a0 I + a1 A + . . . + an An = 0 can be manipulated as follows:
a1 A + . . . + an An = a0 I

an
a1
A . . . An = I since a0 6= 0
a0
a0
a1
an
A( I . . . An1 ) = I .
a0
a0

Hence A is invertible and

A1 =

an
a1
I . . . An1 .
a0
a0
300

The Cayley-Hamilton Theorem and the theorem on the previous slide

means (among other things) that

Every (positive) power of A can be expressed as a linear

combination of I , A,. . . , An1 .
If A is invertible, then A1 can be expressed as a linear
combination of I , A,. . . , An1 .

301

Examples


3 2
Let A =
1 4
1. Calculate the characteristic equation of A.

2. Verify that A satisfies its characteristic equation.

3. Express A1 as a linear combination of A and I and hence

calculate it.

4. Express A3 as a linear combination of A and I and hence calculate

it.
302

7.5 Diagonalization

[AR 5.2]

We now take up the problem of studying the question of when the

eigenvectors of an n n matrix A form a basis for Rn . Such bases are
extremely important in the applications of eigenvalues and eigenvectors
mentioned earlier.

Definition
A square matrix A is said to be diagonalizable if there is an invertible
matrix P such that P 1 AP is a diagonal matrix. The matrix P is said
to diagonalize A.

303

To test if A is diagonalizable, the following theorem can be used.

Theorem
An n n matrix A is diagonalizable if and only if there is a basis for Rn
all of whose elements are eigenvectors of A.
Idea of proof:
If we can form a basis in which all the basis vectors are eigenvectors of
T , then the new matrix for T will be diagonal.

304

A simple case in which A is diagonalizable is when A has n distinct

eigenvalues. This follows from the theorem above and the following

Lemma
Eigenvectors corresponding to distinct eigenvalues are linearly
independent.
Idea of proof:

305

Example
Decide whether or not the following matrices are diagonalizable.






2 0
1 2
2 2
.
, C=
, B=
A=
0 2
0 1
0 1

306

How to diagonalize a matrix

Suppose A is diagonalizable. How can we find an invertible matrix P,
and a diagonal matrix D with D = P 1 AP ?

Theorem
Let A be a diagonalizable n n matrix. Thus there exists a basis
{v1 , . . . , vn } for Rn whose
elements are eigenvectors of A.

Let P = [v1 ] [vn ] and let i be the eigenvalue of the eigenvector vi .
Then
P 1 AP = diag [1 , 2 , . . . , n ]
So, if we have found a basis consisting of eigenvectors, then we can
write down matrices P and D without further calculation.

307

Example

1
1
2

Check that 1 , 2 , 0 are eigenvectors of the matrix

2
0
1

3 2 4
A = 2 0 2
4 2 3

and read off the corresponding eigenvalues.

Write down an invertible matrix P such that D = P 1 AP is diagonal,

and write down the diagonal matrix D.
Check your answer by evaluating both sides of the equation PD = AP.

308

309

Orthogonal matrices
Because a matrix often represents a physical system it can be important
that the change-of-basis transformation does not affect shape.
This will happen when the change-of-basis matrix is orthogonal:

Definition
An n n matrix P is orthogonal if the columns of P form an
orthonormal basis of Rn using the dot product as inner product.
Examples

1/ 2 1/ 6
0
2/ 6

1/ 2
1/ 6



1 1
1
but
and
0 1
1



1/3
cos sin
are orthogonal
1/3 and
sin cos
1/ 3

1
are not.
1

310

Orthogonal matrices have some good properties:

If P is orthogonal n n and u, v Rn then

P 1 = P T

kPuk = kuk

hPu, Pvi = hu, vi where hu, vi = u v = dot product in Rn .

We can summarise this by saying that orthogonal matrices preserve

lengths and angles.
Geometrically, orthogonal matrices represent rotations and rotations
composed with reflections.
The proof of the first property above also shows that a square matrix P
is orthogonal if and only if P T P = I is the identity matrix.

311

Real symmetric matrices

Definition
A matrix A is symmetric if AT = A
Examples


3 1
1 2
and 1 1
2 3
4
5



3
2 1
but
and 1
3 2
4

4
5
9
1
1
6

are symmetric,

4
5 are not
9

Symmetric matrices arise (for example) in:

quadratic functions

inner products

maxima and minima of function of more than 1 variable

312

Theorem
Let A be an n n real symmetric matrix. Then:

1. all roots of the characteristic polynomial of A are real;

2. eigenvectors from distinct eigenvalues are orthogonal;
3. A is diagonalizable;
(In fact, there is an orthonormal basis of eigenvectors.)
4. we can write A = QDQ 1 where D is diagonal and Q is an
orthogonal matrix.

Note that, in this case the diagonalization formula can be written

A = QDQ T .
Idea of proof of 2:

313

General method to orthogonally diagonalize real, symmetric A

1. Find eigenvalues and n linearly independent eigenvectors for A.
2. Find an orthonormal basis for each eigenspace. (Use
Gram-Schmidt if there is a repeated eigenvalue.)
3. Let Q be the matrix with the orthonormal basis vectors in its
columns. Then
Q T AQ = D and Q 1 = Q T ,
where D = diag (1 , 2 , . . . , n ) and i is the eigenvalue
corresponding to the i th column of Q.

314

Example


2 1
Find matrices D and Q as above for A =
1 2

315

Powers of a matrix
In applications, one often comes across the need to apply a
transformation many times. In the case that the transformation can be
represented by diagonalizable matrix A, it is easy to compute Ak and
thus the action of the k-th application of the transformation.
The first point to appreciate is that computing powers of a diagonal
matrix D is easy.
Example
With D = diag[1, 3, 2], write down D 2 , D 3 and D k .

316

In general, we have

Lemma
Suppose A is diagonalizable, so that D = P 1 AP is diagonal. Then for
all integers k > 1
Ak = PD k P 1 .

317

Example
For the matrix of slide 293 we can write




1 4
3 0
1
A = PDP
with A =
D=
1 1
0 1


2 2
P=
1 1

Thus
Ak = PD k P 1

where

P 1 =

Explicitly,
Ak =

318

Example Population Movement between Victoria and Queensland

Assume 2% of Victorians move to Queensland each year, 1% of
Queenslanders move to Victoria each year and everybody else stays put.
Let xi be the Victorian population (in millions) after i years and yi be
the Queensland
  population (in millions) after i years.
x
Let pi = i
yi
1. Find a matrix A such that pn+1 = Apn .
2. Find p1 , p2 and pn in terms of p0 and A.
3. Check that A has eigenvalues 0.97, 1.00 with corresponding
eigenvectors (1, 1) and (1, 2).
4. Find matrices P and D such that A = PDP 1 .
5. Use P and D to estimate the population pn in terms of p0 .
6. What happens in the long run (as n ) to the populations of
Victoria and Queensland?
319

320

321

Conic Sections [AR 7.3]

Consider equations in x and y of the form
ax 2 + bxy + cy 2 + dx + ey + f = 0
where a, b, c, d, e, f R are constants.
The graphs of such equations are called conic sections or conics.
Example
Plot the curve given by the equation:
9x 2 4xy + 6y 2 10x 20y 5 = 0
y

We will see how the shape of

this graph can be calculated
using diagonalization.

-0.5

0.5

1.5

2.5

x
322

We shall assume that d and e are zero.

See [AR 7.3] for a discussion of how to reduce to this case.
If the equation is simple enough, we can identify the conic by inspection.
Standard central conics:

ellipse

y2
x2
+
=1
2 2

hyperbola

x2
y2
2 =1
2

The dashed lines are y = x

323

The equation in matrix form

Consider the curve defined by the equation
ax 2 + bxy + cy 2 = k
The equation can be written in matrix form as
 


 a 1b x
2
x y 1
=k
y
c
2b
That is,
xT Ax = k
 
x
where x =
, and A is a real symmetric matrix.
y
We can diagonalize in order to simplify the equation so that we can
identify the curve.
Lets demonstrate with an example.
324

Identify and sketch the conic defined by x 2 + 4xy + y 2 = 1



1 2
T
This can be written as x Ax = 1 where A =
2 1




3 0
1 1
Diagonalizing gives A = QDQ T with D =
, Q = 12
0 1
1 1
 
x
Let x = be the co-ordinates of (x, y ) relative to the orthonormal
y
basis of eigenvectors: B = {( 12 , 12 ), ( 12 , 12 )}
Then x = Qx , (Q is precisely the transition matrix PS,B )
and the equation of the conic can be rewritten:
xT Ax = 1 (Qx )T QDQ T Qx = 1

(x )T Q T QDQ T Qx = 1
(x )T Dx = 1

3(x )2 (y )2 = 1
So the curve is a hyperbola
325

The x -axis and the y -axis are called the principal axes of the conic
The directions of the principal axes are given by the eigenvectors.
In this example the directions of the principal axes are:
1 1
( , )
2
2

and

1 1
( , ).
2 2
y
y

2
2

-2

-1

-4

-2

(y )2 = 1

-2

-1

3(x )2

-2

-4

x2

+ 4xy + y 2 = 1
326

Summary

We can represent the equation of a conic (centered at the origin)

by a matrix equation x T Ax = k with A symmetric.

The eigenvectors of A will be parallel to the principal axes of the

conic.

So if Q represents the change of basis matrix then Q will be

orthogonal and Q T AQ = D will be diagonal.

If x = Qx then x represents the coordinates with respect to the

new basis and the equation of the conic with respect to this basis
is xT Dx = k or 1 (x )2 + 2 (y )2 = k.

The conic can now be identified and sketched.

327

Example (a quadric surface)

The equation
x 2 + 2y 2 + 2z 2 + 4xy + 4xz 2yz = 1
represents a quadric surface in R3 .
In matrix form it can be represented as xT Ax = 1 with

x
1 2
2
x = y
2 1
and
A= 2
z
2 1 2

The eigenvalues of A are 3, 3, 3. So the equation of the surface with

respect to an orthonormal basis of eigenvectors is
3X 2 + 3Y 2 3Z 2 = 1

328

The surface is a hyperboloid of one sheet; see the sketch below.

(You are not expected to identify quadric surfaces in three dimensions.)

329