1

Math 1b Practical March 02, 2011

Real symmetric matrices
A matrix A is symmetric when A

= A. In the following A

will denote the conjugate-

transpose of A, the matrix whose entries are the complex conjugates of the entries of A

.
A (complex) matrix is called Hermitian when A

= A. There are results about

Hermitian matrices that are similar to those about real symmetric matrices proved here.
See e.g. Section 2 of Chapter 6 of LADW.
Theorem 1. The eigenvalues of a real symmetric matrix are real.
Proof: Let A be real symmetric and suppose Ax = x where and x are possibly complex,
and x = 0. We take the conjugate tranpose to get
x

= (x)

= (Ax)

= x

= x

A,
the latter equality because A is symmetric and real. Then
x

x = (x

A)x = x

(Ax) = x

x.
Now x

x is nonzero (it is real too; it is what we called ||x||

2
for complex x). So we cancel
to nd = , which means that is real.
Remark. We have seen examples where eigenvectors have complex numbers as coordi-
nates. But if is a real eigenvalue of a real matrix A, then we can always nd a real
eigenvector corresponding to . A reason is that the row operations we can use to nd a
basis for the null space of A I produce real vectors.
Exercise. All eigenvalues of a Hermitian matrix are real.
Theorem 2. If x and y are eigenvectors of a real symmetric matrix A corresponding to
distinct eigenvalues and , then x, y = 0.
Proof: Think of both x and y as column vectors, as usual. Then x, y = x

y. We have
Ax = x, Ay = y. Then x

= x

A = x

, and
x

y = (x

A)y = x

(Ay) = x

y.
Since = , we must have x

y = 0.
Theorem 3. A real symmetric matrix is diagonalizable. In fact, R
n
has an orthonormal
basis of eigenvectors for any real symmetric matrix S. If these are taken as the columns
of an (orthogonal) matrix E, then E

SE is a diagonal matrix with the eigenvalues of S

on the diagonal.
Proof: If the columns of a matrix E are an orthonormal basis for R
n
(i.e. if E is an
orthogonal matrix), then E

E = I; that is, E
1
= E

.
2
We prove that there is an orthogonal matrix E such that E

SE is diagonal by in-
duction on the size of the matrix S. That will prove the theorem because the columns of
E will be eigenvectors for S. Our assertion is trivial if S is 1 1. Assume it holds for
symmetric (n 1) (n 1) matrices, and let S be a symmetric n n matrix.
Let be a zero of the characteristic polynomial of S. So there exists an eigenvector
e
1
of S corresponding to , which we can assume has unit length and which we can extend
to an orthonormal basis e
1
, a
2
, . . . , a
n
; and we take these as the columns of a matrix A.
Let S
0
= A

SA = A
1
SA. Then SA = AS
0
. The rst column of SA is Se
1
= e
1
, so
that is the rst column of AS
0
which means that the rst column of S
0
is [, 0, 0, . . . , 0]

.
This is true if S is symmetric or not. But since S is symmetric, so is S
0
= A

SA, and
then the rst row of S
0
is [, 0, 0, . . . , 0]. That is,
A

SA = S
0
=

0
0
.
.
.
0
0 0 . . . 0
S
1
where S
1
is an (n 1) (n 1) symmetric matrix.
By the induction hypothesis, there is an orthogonal matrix E
1
so that E

1
S
1
E
1
= D
1
,
where D
1
is diagonal. Then
1
0
0
.
.
.
0
0 0 . . . 0
E
1

0
0
.
.
.
0
0 0 . . . 0
S
1
1
0
0
.
.
.
0
0 0 . . . 0
E
1
=

0
0
.
.
.
0
0 0 . . . 0
D
1
,
or E

0
S
0
E
0
= D
0
where E
0
and D
0
are the bordered matrices obtained from E
1
and D
1
as indicated above. Finally,
D
0
= E

0
S
0
E
0
= E

0
A

SAE
0
= (AE
0
)

S(AE
0
).
Both A and E
0
are orthogonal matrices, so their product AE
0
is also (why?), and we have
shown what we wanted to show.
If you know a basis of eigenvectors for a symmetric matrix S, you can get an orthonor-
mal basis easily enough. First, we get an orthogonal basis. Eigenvectors corresponding
to distinct eigenvalues are already orthogonal. And if we have a set of several eigenvec-
tors e
1
, . . . , e
k
that span the eigenspace U

corresponding to an eigenvector , we can use

the Gram-Schmidt Process on them to nd an orthogonal basis e

1
, . . . , e

k
for that same
eigenspace. If we do this for each eigenvalue, we get our orthogonal basis. Then normalize.
Example. Let
S =

3 2 4
2 0 8
4 8 12

.
3
If we ask Mathematica Eigensystem[S], it responds with
{{-4,-4,17},{{-4,0,1},{-2,1,0},{1,2,4}}
which means that the eigenvalues are 4 of multiplicity two, and 17. (We could have
done that by hand.) The eigenvector (1, 2, 4) corresponding to 17 is indeed orthogonal to
the eigenvectors (4, 0, 1) and (2, 1, 0) corresponding to 4, but the latter two arent
orthogonal. So replace (4, 0, 1) with (4/5, 8/5, 1) = (4, 0, 1) (8/5)(2, 1, 0), or
with (4, 8, 5), and we have an orthogonal basis of eigenvectors. Take
E =

2/

5 4/

105 1/

21
1/

5 8/

105 2/

21
0 5/

105 4/

21

,
and we have an orthogonal matrix E such that E

SE is a diagonal matrix with 4, 4, 17

on the diagonal.
It is worth noting that the converse of Theorem 3 holds. Say that a matrix A has
an orthonormal basis of eigenvectors, which are the columns of a matrix E, so that E is
an orthogonal matrix and E

AE = D where D is a diagonal matrix. Then taking the

transpose of this equation,
E

E = E

(E

= D

= D = E

AE.
We can cancel the E

on the left and the E on the right to get A

= A. That is, A is
symmetric.
Since it is easy to see that there exists an orthogonal basis of eigenvectors for projection
and reection matrices, it follows that these matrices are symmetric matrices!
A real symmetric matrix S is said to be positive denite when
x

Sx > 0
for all nonzero column vectors x. We say S is positive semidenite when
x

Sx 0
for all column vectors x. Similarly, we say S is negative denite when x

Sx < 0 for every

nonzero x. You may guess what negative semidenite means, but I, personally, have never
heard anyone use that term.
Theorem 4. A symmetric matrix A is positive denite if and only if all of its eigenvalues
are positive. A symmetric matrix A is positive semidenite if and only if all its eigenvalues
are nonnegative.
Proof: Let e
1
, e
2
, . . . , e
n
be an orthonormal basis of eigenvectors (columns) for A. Given
a column vector x, write
x =
1
e
1
+
2
e
2
+. . . +
n
e
n
.
4
Then, as usual,
Ax =
1

1
e
1
+
2

2
e
2
+. . . +
n

n
e
n
where
i
corresponds to e
i
, and then, since e

i
e
j
is 0 for i = j and 1 if i = j,
x

Ax =
2
1

1
+
2
2

2
+. . . +
2
n

n
. ()
So the theorem should be clear now. If we assume all
i
s are positive, then the
expression above is positive (not all
i
s are 0, of course). And if all
i
s are nonnegative,
then the quantity in () is nonnegative. If some
i
0, then () is nonpositive when
x = e
i
. If some
i
< 0, then () is negative when x = e
i
.
Quadractic forms, change of variables, and congruence
Symmetric matrices can be associated with quadratic forms. A quadratic form in
(commuting) variables x
1
, x
2
, . . . , x
n
in a polynomial in x
1
, x
2
, . . . , x
n
in which every term
has degree exactly 2. An example of a quadratic form in four variables is
f = 3x
2
1
2x
1
x
3
+5x
1
x
4
4x
2
2
x
3
x
4
+x
2
4
.
The symmetric matrix associated to f is
S =

3 0 1 5/2
0 4 0 0
1 0 0 1/2
5/2 0 1/2 1

.
We have f = xSx

.
In general, a symmetric matrix S = (a
ij
) gives us the quadratic form f = xSx

,
where the coecient of x
i
x
j
is a
ij
+ a
ji
for i = j (remember that we have a
ij
= a
ji
in a
symmetric matrix). So the quadratic form associated with

1 1
1 1

is x
2
1
+2x
1
x
2
+x
2
2
.
If we have an expression in variables x
1
, x
2
, . . . , x
n
, it can be helpful in many situations
to make a (linear) change of variables. We introduce new variables y
1
, y
2
, . . . , y
n
by the
rule y = Ax where A is a nonsingular matrix. We insist that A be nonsingular because
we want to be able to obtain the values of the x
i
s from the y
i
s also. For example, the
quadratic form x
2
1
+6x
1
x
2
+7x
2
2
has a somewhat simpler form (the word form is being used
in two dierent ways in this sentence, one mathematical and one as in English) y
2
1
2y
2
2
when we introduce y
1
= x
1
+ 3x
2
and y
2
= x
2
.
To be continued....