Linear Algebra Lecture Notes

1
Matrices and Determinants

Text references: Lecture Notes Series: Engineering Mathematics Volume 1, 2
nd

Edition, Pearson 2006.

Matrices and Some Special Matrices

A linear system of equations

3 3 33 2 32 1 31
2 3 23 2 22 1 21
1 3 13 2 12 1 11
b x a x a x a
b x a x a x a
b x a x a x a
= + +
= + +
= + +

(
(
(
3 33 32 31
2 23 22 21
1 13 12 11
b a a a
b a a a
b a a a

A matrix is a rectangular array of numbers (or function) enclosed in brackets
(or parentheses).

Let m, n to be positive integers,
(
(
(
(
=
mn m m
n
n
a a a
a a a
a a a
A
L
M O M M
L
L
2 1
2 22 21
1 12 11

where
ij
a (i = 1, , n; j = 1, , m)
are the real numbers which are the
entries or element of A.

A also can be written as [ ]
ij
a A = .
The entries
nn
a a a a , , , ,
33 22 11
L

are the main diagonal (or principal
diagonal).

A matrix containing one row is called a row matrix.
A matrix containing one column is called a column matrix.

If n m = , we call A as a n n (or m m ) square matrix.

If we omit some rows or columns (or both) from A, we call it as sub-matrix.

can be written as

2
Example :
1.
(
(
(
9 6 3
8 5 2
7 4 1
is a 3 3 matrix (which is a square matrix).

2.
(
(
(
(
1 0 6 8
3 1 9 5
7 3 3 2
6 8 2 1
is a square matrix (or 4 4 matrix).
The entries 1, 3, 1, 1 are along the main diagonal.

3.
(
(
(
c
b
a
is a 1 3 matrix (or column matrix).

4. [ ] 3 2 1 is a 3 1 matrix (or row matrix).

5.
(
f e d
c b a
contains how many 2 2 sub-matrices? 1 2 sub-matrices?
2 1 sub-matrices?
S

A square matrix in which 0 =
ij
a for all j i is called diagonal matrix.
(
(
(
(
=
nn
a
a
a
A
L
M O M M
L
L
0 0
0 0
0 0
22
11
R a
ii
, n i , , 3 , 2 , 1 L = .

A square matrix in which 1 =
ii
a and 0 =
ij
a for all j i is called identity
matrix. Usually, we write the n n identity matrix as
n
I .

3
(
(
(
(
=
1 0 0
0 1 0
0 0 1
L
M O M M
L
L
n
I .

A square matrix in which all the entries above the main diagonal are zero is
called lower triangular matrix.

A square matrix in which all the entries below the main diagonal are zero is
called upper triangular matrix.

A matrix, either upper triangular or lower triangular matrix is called
triangular matrix.

Example :
1.
(
(
(
(
7 9 1 4
0 3 2 6
0 0 5 2
0 0 0 1
,
(
(
(
6 6 9
0 0 3
0 0 1
,
(
0 1
0 1
are the lower triangular
matrix.

2.
(
(
(
(
7 0 0 0
8 3 0 0
1 4 0 0
5 0 5 1
,
(
(
(

6 0 0
5 8 0
2 9 1
,
(
1 0
3 1
are the upper triangular
matrix.

3.
(
(
(
(
2 0 0 0
0 9 0 0
0 0 5 0
0 0 0 4
,
(
(
(
8 0 0
0 6 0
0 0 4
,
(
9 0
0 7
are the diagonal matrix.

4
Example :
What kinds of matrices are both upper triangular and lower triangular?
If A is both upper and lower triangular, then every entry off the main
diagonal must be zero. Hence A is a diagonal matrix.

Matrix Operations
1. Addition and subtraction: If [ ]
ij
a A = , [ ]
ij
b B = and both are the n m
matrices, then
[ ]
ij ij
b a B A + = + and [ ]
ij ij
b a B A =
for all m i , 2, , 1 L = n j , 2, , 1 L = .

Example :(Addition and Subtraction)
Given that
(
=
5 1 5
3 4 1
A ,
(

=
6 1 5
3 7 0
B and
(
=
2 6
9 1
C .
Find B A+ , B A and C A+ .

(

= +
11 2 0
6 11 1
B A ,
(
=
1 0 10
0 3 1
B A ,

The sum of C A+ is not defined, since the matrices have different
sizes.

5
2. Scalar multiplication: If [ ]
ij
a A = is a n m matrix and C is any real
number, then [ ]
ij
a C CA = for all m i , 2, , 1 L = n j , 2, , 1 L = .
Example : (scalar multiplication)
Compute
(
(
(
5 9
2 4
4 1
3 and
(
0 2
4 1
4 .

(
(
(
=
(
(
(
15 27
6 12
12 3
5 9
2 4
4 1
3 and
(
=
(
0 8
16 4
0 2
4 1
4 .

3. Matrix multiplication: Let [ ]
ij
a A = be a n m matrix and [ ]
ij
b B = be a
p n matrix. Then
(
(
(
=
(
(
(
(
(
(
(
(
(
(
(
(
=
mp m m
ij
p
np n n
ip i i
p
mn m m
in i i
n
c c c
c
c c c
b b b
b b b
b b b
a a a
a a a
a a a
AB
L
M M
L
L
M M M
L
M M M
L
L
M M M
L
M M M
L
2 1
1 12 11
2 1
2 1
1 12 11
2 1
2 1
1 12 11

where
=
= + + + =
n
k
kj ik nj in j i j i ij
b a b a b a b a c
1
2 2 1 1
L .

6

Example : (matrix multiplication)
Find the product AB for
(
=
1 2
3 1
A and
(
=
6 2 3
4 0 2
B .
Since A is 2 2 and B is 3 2 , the product AB is defined as a 3 2
matrix. To obtain the entries in the first row of AB, multiply the first
row [ ] 3 1 of A by the column
(
3
2
,
(
2
0
and
(
6
4
of B, respectively.

( )( ) ( )( ) ( )( ) ( )( ) ( )( ) ( )( )
(
+ + +
=
(
? ? ?
6 3 4 1 2 3 0 1 3 3 2 1
6 2 3
4 0 2
1 2
3 1

(

=
(
+ +
=
? ? ?
14 6 11
? ? ?
18 4 6 0 9 2

Similarly, to obtain the entries in the second row of AB.
Thus,
(
=
14 2 1
14 6 11
AB .

4. Some properties of Matrix Operations
Let A, B and C be the n m matrices, D and E be the p n matrices, F
be a q p matrix and
1
e ,
2
e are any real number.
(a) A B B A + = + ,
(b) ( ) ( ) C B A C B A + + = + + ,
(c) ( ) B e A e B A e
1 1 1
+ = + ,
(d) ( ) A e A e A e e
2 1 2 1
+ = + ,
(e) ( ) ( ) A e e A e e
2 1 2 1
= ,
(f) ( ) AE AD E D A + = + ,
(g) ( ) CE BE E C B + = + ,
(h) ( ) ( )F AD DF A = ,
(i) ( ) ( ) ( ) D e A D A e AD e
1 1 1
= = .

7

Example :
Find x, y, z and w, if
(
+ +
+
(
=
(
1 1
2 1
2
w z y x
w y w y
z x
.

First write each side as a single matrix:
(
+
+ + + +
=
(
1 1
2 1
2 2
2 2
w y
w z y x
w y
z x

Set corresponding entries equal to each other to obtain the system of
four equations,
1 2
2 2
1 2
1 2
=
+ + =
+ =
+ + =
w w
w z z
y y
y x x
or
1 3
2
1
1
=
+ =
=
+ =
w
w z
y
y x

The solution is:
3
1
3
5
1 2 = = = = w z y x , , , .

8
Example :
Find two matrices A and B such that AB and BA are defined and have
the same size, but BA AB .
Let
(

=
0 2
1 1
A and
(
=
1 1
1 2
B , then

(
=
(

=
2 4
0 1
1 1
1 2
0 2
1 1
AB

(

=
(
=
1 3
2 4
0 2
1 1
1 1
1 2
BA

Matrix multiplication does not obey the commutative law.

Example :
Can we have 0 = AB , with 0 A and 0 B ?
Let
(
=
2 2
1 1
A ,
(

=
2 3
2 3
B
(
=
(
=
0 0
0 0
2 3
2 3
2 2
1 1
AB .

5. Transpose of matrix (Transposition)
If [ ]
ij
a A = is a n m matrix, then the transposition of A is [ ]
ji
T
a A =
(
T
A becomes a m n matrix).

Some properties for the Transposition
Let A and B be the n m matrices, C be the p n matrices, and
1
e is
any real number.
(a) ( )
T T T
B A B A + = + ,
(b) ( )
T T T
B A B A = ,
(c) ( )
T T
A e A e
1 1
= ,
(d) ( ) A A
T
T
= ,
(e) ( )
T T T
A C AC = .

9

Example :
Find the transposes of the following vectors:
(
(
(
=
9
6
1
u , [ ] 6 9 4 2 = v ,
(
=
9
4
w , [ ] 1 6 = z

[ ] 9 6 1 =
T
u ,
(
(
(
(
=
6
9
4
2
T
v , [ ] 9 4 =
T
w ,
(
=
1
6
T
z

Example :
Given
(
=
6 5 4
3 2 1
A , find
T
A and ( )
T
T
A .

(
(
(
=
6 3
5 2
4 1
T
A , ( )
(
=
6 5 4
3 2 1 T
T
A .
Observe that ( ) A A
T
T
= .

Example :
Given
(
(
(
=
4 1
3 3
2 1
A , find
T
AA and A A
T
.
(
(
(
=
17 9 7
9 18 9
7 9 5
T
AA ,
(
=
29 7
7 11
A A
T

Observe that A A AA
T T
.

10
Example :
Given
(

=
3 2
1 1
A and
(
=
5 3
1 2
B , find ( )
T
AB ,
T T
B A and
T T
A B .

( )
(
=
17 4
13 1
T
AB ,
(
=
12 1
13 4
T T
B A ,
(
=
17 4
13 1
T T
A B .
Observe that ( )
T T T
B A AB ,
T T T T
A B B A and ( )
T T T
A B AB = .

Is it true in general that ( )
T T T
A B AB = ? (OPTIONAL)
If [ ]
ij
a A = and [ ]
kj
b B = , the ij-entry of AB is
mj im j i j i
b a b a b a + + + L
2 2 1 1
(1)
Thus (1) is the ji-entry [reverse order] of ( )
T
AB .
On the other hand, column j of B becomes row j of
T
B , and row I of A
becomes column I of
T
A . Consequently, the ji-entry of
T T
A B is
[ ]
im mj i j i ij
im
i
i
mj j ij
a b a b a b
a
a
a
b b b + + + =
(
(
(
(
L
M
L
2 2 1
2
1
2

Thus, ( )
T T T
A B AB = , since corresponding entries are equal.

11
6. Let [ ]
ij
a A = be a n m matrix. Then
(a) A is said to be a symmetric matrix if A A
T
= .
(b) A is said to be a skew-symmetric matrix if A A
T
= .

Example :
Given
(
=
2 4
4 1
A ,
(
=
4 1
1 6
B ,
(
=
0 1
1 0
C and
(

=
0 3
3 0
D .

Which of the above matrices are (i) symmetric, (ii) skew-symmetric?
A is symmetric because
(
=
2 4
4 1
T
A which is equal to A.
B is also symmetric because
(
=
4 1
1 6
T
B which is equal to B.
C is skew-symmetric because
(

=
0 1
1 0
T
C which is equal to C .
D is also skew-symmetric because
(
=
0 3
3 0
T
D which is equal to
D .

Theorem
(a) If A, B are symmetric matrices and k is any constant, then kA,
B A+ , B A , are symmetric matrices.

(b) If A, B are skew-symmetric matrices and k is any constant, then
kA, B A+ , B A , are skew-symmetric matrices.
(c) If A is any n n matrix then
-
T
A A+ and
T
AA are symmetric matrices,
-
T
A A is a skew-symmetric matrix.

12

Example :
For the matrices of example 14, find kA, B A+ and B A . Are they
symmetric?
(
=
k k
k k
kA
2 4
4
,
(
= +
6 3
3 7
B A and
(
=
2 5
5 5
B A . All of them
are symmetric matrices.

Example :
For the matrices of example 14, find kC, D C + and D C . Are they
skew-symmetric?
(
=
0
0
k
k
kC ,
(

= +
0 2
2 0
D C ,
(
=
0 4
4 0
D C . All of them
are skew-symmetric matrices.

Example :
Given
(

=
7 5
6 1
A , find
T
A A+ ,
T
AA and
T
A A .
(
=
7 6
5 1
T
A .
(
= +
14 1
1 2
T
A A ,
(
=
74 47
47 37
T
AA and
(
=
0 11
11 0
T
A A .
Observe that
T
A A+ and
T
AA are symmetric matrices,
T
A A is a
skew-symmetric matrix.

Example :
Show that the diagonal elements of a skew-symmetric matrix must be
zero.
If [ ]
ij
a A = is skew-symmetric then
ii ii
a a = . Hence each 0 =
ii
a .

13

Determinant
Let A be a n n matrix, the determinant of A is denoted by ( ) A det or A .
To evaluate determinant of A,
(1) If 2 = n , then ( )
21 12 22 11
22 21
12 11
a a a a
a a
a a
A det = = .
(2) If 3 n , then
( )
nn n n
n
n
a a a
a a a
a a a
A det
L
M M M
L
L
2 1
2 22 21
1 12 11
=

( ) ( ) ( )
in in
n i
i i
i
i i
i
M a M a M a
+ + +
+ + + = 1 1 1
2 2
2
1 1
1
L
( )
=
+
=
n
j
ij ij
j i
M a
1
1
where
ij
M is a determinant of ( ) ( ) 1 1 n n matrix obtained from A
by omitting row i and column j.
ij
M is introduced as minor for matrix A.
( )
ij
j i
M
+
1 is introduced as cofactor for matrix A.

Example :
Find ( ) 5 det , ( ) 91 det and ( ) 8 + s det .
The determinant is the scalar itself; hence, ( ) 5 5 = det , ( ) 91 91 = det
and ( ) 8 8 + = + s s det .

Example :
Given
(
=
2 1
8 5
A ,
(
=
d b
c a
B . Find ( ) A det and ( ) B det .
( )( ) ( )( ) 2 8 10 1 8 2 5
2 1
8 5
= = = ,
bc ad
d b
c a
= .

14

Example :
Determine the values of s for which 0
3
2
=
s
s s
.

0 3 2
3
2
2
= = s s
s
s s
, or ( ) 0 3 2 = s s . Hence, 0 = s or
2
3
= s .

Example :
Given
(
(
(
=
3 3 3
2 2 2
1 1 1
c b a
c b a
c b a
A , find ( ) A det .

( ) ( ) ( ) ( )
3 2 3 2 1 3 2 3 2 1 3 2 3 2 1
3 3 3
2 2 2
1 1 1
a b b a c a c c a b b c c b a
c b a
c b a
c b a
A det + = =

3 2 1 3 2 1 3 2 1 3 2 1 3 2 1 3 2 1
a b c c a b b c a b a c a c b c b a + + =

Example :
Given
(
(
(
=
4 3 1
2 5 0
1 1 2
A . Find ( ) A det .
( ) ( )( )( ) ( )( )( ) ( )( )( ) ( )( )( )
( )( )( ) ( )( )( ) 0 1 4 2 2 3
1 5 1 0 3 1 1 2 1 4 5 2
4 3 1
2 5 0
1 1 2

+ + = = A det

45 0 12 5 0 2 40 = + + =

15

Some properties of determinants
(1) If matrix B is obtained from matrix A by interchanging two rows (or
two columns), then ( ) ( ) A det B det = .
(2) If two rows (or two columns) in matrix A are identical, then
( ) 0 = A det .
(3) If matrix B is obtained from matrix A by multiplying one complete
row (or column) by a constant then ( ) ( ) A det B det = . Hence,
( ) ( ) A det A det
n
= .
(4) If matrix A has a zeros row (or column) then ( ) 0 = A det .
(5) If matrix B is obtained from matrix A by adding to one row(column) a
multiple of another row(column) then ( ) ( ) A det B det = .
(6) If A and B are square matrix of same size, then
( ) ( ) ( ) B det A det AB det = .
(7) If A is a square matrix, then ( ) ( ) A det A det
T
= .

Example :
Given
(
=
4 2
3 1
A and
(
=
3 1
4 2
( ) 2 6 4 = = A det and ( ) 2 4 6 = = B det .
Observe that ( ) ( ) B det A det = , where matrix B is obtained from matrix A by
interchanging two rows.

Example :
Given
(
(
(
=
1 1 1
4 3 2
1 1 1
A . Find ( ) A det .
( ) ( )( )( ) ( )( )( ) ( )( )( ) ( )( )( ) ( )( )( ) ( )( )( ) 2 1 1 1 4 1 1 3 1 2 1 1 1 4 1 1 3 1 + + = A det
0 2 4 3 2 4 3 = + + + =
Observe that row 1 and row 3 in matrix A are identical.

16

Example :
Given
(
=
10 3
2 1
A ,
(
=
10 3
20 10
B and
(
=
50 15
10 5
C . Find ( ) A det , ( ) B det
and ( ) C det .
( ) 4 6 10 = = A det , ( ) 40 60 100 = = B det and ( ) 100 150 250 = = C det .
Observe that ( ) ( ) A det B det 10 = where matrix B is obtained from matrix A
by multiplying the first row by 10.
Observe that ( ) ( ) A det C det
2
5 = .

Example :
Given
(
=
10 3
2 1
A and
(
=
10 3
12 4
( ) 4 6 10 = = A det and ( ) 4 36 40 = = B det .
Observe that ( ) ( ) B det A det = where matrix B is obtained from matrix A : the
1
st
row of B is summation of row 1 and row 2 of matrix A.

Example :
Given
(
=
1 2
6 1
A and
(
=
1 3
5 2
B . Find ( ) A det , ( ) B det and ( ) AB det .
(

=
(
=
11 1
1 20
1 3
5 2
1 2
6 1
AB ,
( ) ( )( ) ( )( ) 13 2 6 1 1 = = A det , ( ) ( )( ) ( )( ) 17 15 2 3 5 1 2 = = = B det and
( ) ( )( ) ( )( ) 221 1 1 11 20 = = AB det .
Observe that ( ) ( ) ( ) B det A det AB det = .

17

Example :
Given
(
=
2 6
1 4
A . Find ( ) A det and ( )
T
A det .
(
=
2 1
6 4
T
A ,
( ) ( )( ) ( )( ) 14 6 8 6 1 2 4 = = = A det and
( ) ( )( ) ( )( ) 14 2 6 2 4 = =
T
A det .
Observe that ( ) ( )
T
A det A det = .

18
Inverse Matrix
Invertible Matrices:

AB = BA = I

A square matrix, A
Identity matrix, I
If there exists a matrix B, I BA AB = = A is invertible.
B is inverse of A: B A =
1
.

1
= A B A B =
1

Invertible matrix: nonsingular matrix ( ( ) 0 det A )
Not invertible: singular matrix ( ( ) 0 det = A )

19

Properties of inverse matrix:
(1) AB = BA = I
AC = CA = I
C B =
(CANNOT have TWO different inverses for a square matrix A.)
(2) A and B are invertible AB is invertible and
( )
1 1 1
= A B AB .
(3) If A is an invertible matirx, then
(i) A
-1
is invertible and (A
-1
)
-1
= A.
(ii) A
T
T
)
-1
= (A
-1
)
T

(iii) A
n
n
)
-1
= (A
-1
)
n
for n = 0, 1,2,
Example:
Show that
|
|
\
|
=
0 1
1 1
B is the inverse of
|
|
\
|
=
1 1
1 0
A .
Solution:

|
|
\
|
=
|
|
\
|
|
|
\
|
=
1 0
0 1
0 1
1 1
1 1
1 0
AB

|
|
\
|
=
|
|
\
|
|
|
\
|
=
1 0
0 1
1 1
1 0
0 1
1 1
BA
Hence AB = BA = I, so B is indeed the inverse of A.

Example:
Suppose A is an invertible matrix and I A =
3
. Find
1
A .

Solution:
AAA = I
A(AA)=I
AA A =
1

20
(iv) For any nonzero scalar k, the matrix kA is invertible and
( ) . A
k
kA
1 1
1

=

Example:
Can you find the inverse of
|
|
\
|
6 4 2
5 3 1
?

NO! why??

Example :
Which of the following matrices are singular?
(
=
1 1
0 0
A ,
(
=
7 3
4 1
B ,
(
(
(
=
2 9 2
1 5 1
1 3 1
C ,
(
(
(
=
4 3 1
2 5 0
1 1 2
D .

Matrices A and C are singular.

The inverse of a nonsingular n n matrix [ ]
ij
a A = is given by
(1) If
(
=
d c
b a
A then
(
a c
b d
bc ad
A
1
1
.
(2) If 3 n ,
( )
( )
T
ij
A
A det
A
1
1
=
where
ij
A is the cofactor of A
( )
|
|
\
|
=
+
ij
j i
ij
M A 1 .

21

Example :
Find the inverse for the matrices B and D in example 31.
(
(
(
=
(
5
1
5
3
5
4
5
7
1 3
4 7
12 7
1
1
B ,
( )
[ ]
T
ij
A
D det
D
1
1
=
where
ij
A is the cofactor of A.

( ) 45 = D det (from example 24)

( ) 26
4 3
2 5
1
1 1
11
=
=
+
A , ( ) 2
4 1
2 0
1
2 1
12
=
=
+
A , ( ) 5
3 1
5 0
1
3 1
13
= =
+
A

( ) 1
4 3
1 1
1
1 2
21
= =
+
A , ( ) 7
4 1
1 2
1
2 2
22
= =
+
A , ( ) 5
3 1
1 2
1
3 2
23
= =
+
A

( ) 7
2 5
1 1
1
1 3
31
=
=
+
A , ( ) 4
2 0
1 2
1
2 3
32
=
=
+
A , ( ) 10
5 0
1 2
1
3 3
33
= =
+
A

(
(
(

=

10 5 5
4 7 2
7 1 26
45
1
1
D .

Theorem
(1) A square matrix A is invertible if and only if ( ) 0 A det .
(2) If A and B are invertible matrices of same size, then
(a) AB is invertible.
(b) ( )
1 1 1
= A B AB .
(3) If A is an invertible matrix, then
(a)
1
A is invertible and ( ) A A =
1
1
.
(b)
n
A is invertible and ( ) ( )
n
n
A A
1
1
= for any L 2, 1, , 0 = n .

22
(c)
T
A is invertible and ( ) ( )
T
T
A A
1
1
= .
(d) For any { } 0 \ R , the matrix A is invertible and
( )
1 1 1
= A A .

Remark
If A is invertible, then ( ) ( )
1 1
= A det A det

Example :
Determine whether
(
=
2 1
5 3
A is an inverse of
(
=
3 1
5 2
B .
(
=
(
=
1 0
0 1
3 1
5 2
2 1
5 3
AB . Thus, A is an inverse of B.

Example :
Let A and B be invertible matrices of the same size. Show that the product
AB is also invertible and ( )
1 1 1
= A B AB .
( )( ) ( ) I AA AIA A BB A A B AB = = = =
1 1 1 1 1 1
,
( )( ) ( ) I B B IB B B A A B AB A B = = = =
1 1 1 1 1 1
.

Example :
Given
(
=
2 1
0 1
A and
(
=
1 1
1 2
B . Find ( )
1
AB ,
1 1
B A and
1 1
A B .
(
=
3 4
1 2
AB ,
(
2 1
1 1
1
B ,
(
1 1
0 2
2
1
1
A .

( )
(
2 4
1 3
2
1
1
AB ,
(
=

3 2
2 2
2
1
1 1
B A ,
(
=

2 4
1 3
2
1
1 1
A B .
Observe that ( )
1 1 1
= A B AB but ( )
1 1 1
= B A AB .

23

Example :
Show that if A is an invertible matrix, then
T
A is also invertible and
( ) ( )
T
T
A A
1
1
= .
( ) ( ) I I A A A A
T
T T
T
= = =
1 1

( ) ( ) I I AA A A
T
T
T
T
= = =
1 1
.

Example :
If A is invertible, show that kA is invertible when 0 k , with inverse
1 1
A k .
Since 0 k ,
k
k
1
1
=
exists. Then ( )( ) ( )( ) . I I AA kk A k kA = = =

1
1 1 1 1

Hence,
1 1
A k is the inverse of kA.

Example :
Given
(
=
2 1
1 2
A . Determine whether
1
1

= A A .

( )( ) ( )( ) 3 1 1 2 2 = = A ,
3
1
3
1
1
= =

A .
(
(
(
3
2
3
1
3
1
3
2
1
A ,
3
1
3
1
3
1
3
2
3
2
1
=
|
\
|
|
\
|
\
|
|
\
|
=
A .

Thus,
1
1

= A A .

24
System of linear Equations
A linear system of m linear equations in n unknowns (
1
x ,
2
x ,
3
x ,
4
x ,
n
x , L ):

1 1 2 12 1 11
b x a x a x a
n n
= + + + L
M
L

2 2 2 22 1 21
b x a x a x a
n n
= + + +
( )
m n mn m m
b x a x a x a = + + + L
2 2 1 1

If 0
2 1
= = = =
m
b b b L the system is homogeneous. Note that
0
3 2 1
= = = = =
n
x x x x L is always a solution of a homogeneous system.
This is known as trivial solution.
If not all
m
b b b , , ,
2 1
L equal to zero then the system is non-homogeneous.
We can write the system ( ) in the form
B X = A
where
(
(
(
(
(
(
=
mn mj m
in ij i
n j
a a a
a a a
a a a
A
L L
M O M M
L L
M M O M
L L
1
1
1 1 11
;
(
(
(
(
(
(
=
n
x
x
x
M
M
2
1
X ;
(
(
(
(
(
(
=
m
b
b
b
M
M
2
1
B
A is called coefficient matrix.

[ ]
(
(
(
(
(
(
=
m mn mj m
i in ij i
n j
b a a a
b a a a
b a a a
A
L L
M M O M M
M M M O M
L L
1
1
1 1 1 11
B is called augmented matrix.

25
Solving a system of linear equations by using Inverse of a matrix
If B X = A with 0 A then
1
A is exist and the unique solution of X can
be found by using
1
A with the following procedures:

Step 1:
Write down the system of linear equations in the form of B X = A . Then
find
1
A .

Step 2:
Multiply both sides of the system of linear equation B X = A with
1
A i.e
B X
1 1
= A A A
B X
1
= A I
B X
1
= A
Step 3:
Find B
1
A to get the solution of X.

Example :
Solve the following system of linear equations by using Inverse of a matrix.
3 4 3
1 2 5
2 2
3 2 1
3 2
3 2 1
= + +
=
= + +
x x x
x x
x x x

Step 1:
The system of linear equations can be written as B X = A .
i.e
(
(
(
=
(
(
(
(
(
(
3
1
2
4 3 1
2 5 0
1 1 2
3
2
1
x
x
x

Let
(
(
(
=
4 3 1
2 5 0
1 1 2
A ,
1
A can be found as shown in example 32.

(
(
(
10 5 5
4 7 2
7 1 26
45
1
1
A .

26
Step 2:
Multiply both sides of the system of linear equation B X = A with
1
A i.e
(
(
(
(
(
(

=
(
(
(
(
(
(
(
(
(

3
1
2
10 5 5
4 7 2
7 1 26
45
1
4 3 1
2 5 0
1 1 2
10 5 5
4 7 2
7 1 26
45
1
3
2
1
x
x
x

(
(
(
(
(
(

=
(
(
(
(
(
(
3
1
2
10 5 5
4 7 2
7 1 26
45
1
1 0 0
0 1 0
0 0 1
3
2
1
x
x
x

(
(
(
(
(
(

=
(
(
(
3
1
2
10 5 5
4 7 2
7 1 26
45
1
3
2
1
x
x
x

Step 3:
(
(
(
(
(
(
=
(
(
(
3
1
3
1
3
2
3
2
1
x
x
x
.

27

Elementary Row Operations

Let A be an m x n matrix. The following operations are called elementary
row operations:

(1): Interchange the ith row and the jth row: R
i
R
j
.

(2): Multiply the ith row by a nonzero scalar k: R
i
kR
i
, k 0.

(3): Replace the ith row by k times the jth row plus the ith row:
R
i
R
i
+

kR
j
.

Example:
Performs the operations R
1
R
2,
R
3
-7R
3
and R
2
R
2
-3R
1
to the
identity matrix
|
|
|
\
|
=
1 0 0
0 1 0
0 0 1
3
I .

Solution:
|
|
|
\
|
1 0 0
0 1 0
0 0 1

|
|
|
\
|
1 0 0
0 0 1
0 1 0

|
|
|
\
|
1 0 0
0 1 0
0 0 1

|
|
|
\
|
7 0 0
0 1 0
0 0 1

|
|
|
\
|
1 0 0
0 1 0
0 0 1

|
|
|
\
|
1 0 0
0 1 3
0 0 1

R
1
R
2

R
3
-7R
3

R
2
R
2
-3R
1

Changes in row 1
and row 2
Changes in row 3.
Changes in row 2.

28

Elementary Matrix

Identity matrix

E
ij
: Interchange the ith row and the jth row in Identity matrix
I: R
i
R
j
.

E
i
(k): Multiply the ith row by a nonzero scalar k in Identity
matrix I: R
i
kR
i
, k 0.

E
ij
(k): Replace the ith row by k times the jth row plus the ith row
in Identity matrix I: R
i
R
i
+

kR
j
.

|
|
|
\
|
1 0 0
0 1 0
0 0 1

|
|
|
\
|
1 0 0
0 1 0
5 0 1

Questions:
Which of the following are elementary matrices?
|
|
|
|
|
\
|
0 0 1 0
0 1 0 0
1 0 0 0
0 0 0 1
,
|
|
|
\
|
2 0 0
1 1 0
0 1 1
,
|
|
\
|
1 0
0 5
.

Single elementary
row operation
Elementary
Matrix
( ) 5
13
E or
3 1 1
5R R R +

29
Theorem
If the elementary matrix E results from performing a certain row
operation on I
m
and if A is an m x n matrix, then the product EA is
the matrix that results when the same row operation is performed
on A.

Example:
(a) Find an elementary matrix, E by using E
31
(3). and
(b) Perform the same row operation as given in (a) to
|
|
|
\
|
=
0 4 4 1
6 3 1 2
3 2 0 1
A . Compare the answer with EA.

Solution:

(a) E I
R R R
=
|
|
|
\
|

|
|
|
\
|
=
+
1 0 3
0 1 0
0 0 1
1 0 0
0 1 0
0 0 1
1 3 3
3

(b) Same row operation is performed on A:
|
|
|
\
|

|
|
|
\
|
=
+
9 10 4 4
6 3 1 2
3 2 0 1
0 4 4 1
6 3 1 2
3 2 0 1
1 3 3
3R R R
A

E multiple by A:
|
|
|
\
|
=
|
|
|
\
|
|
|
|
\
|
=
9 10 4 4
6 3 1 2
3 2 0 1
0 4 4 1
6 3 1 2
3 2 0 1
1 0 3
0 1 0
0 0 1
EA

A single elementary row operation
is performed on I.
Elementary
Matrix
Same result

30
Row equivalent

A is said to be row equivalent to a matrix B:
- If B can be obtained from A by performing a finite sequence
of elementary row operations successively to A.
- If and only if there exist elementary matrices E
1
, E
2
, . . . , E
s

such that E
s
. . . E
2
E
1
A = B.

Example:

1 2 2

5 1 1
1 2 1
R R R
A

=
|
|
\
|

6 1 0
1 2 1
1
A E =
|
|
\
|

2 2
R R

6 1 0
1 2 1
1 2
) A E ( E =
|
|
\
|

2
2 1 1
R - R R

B ) A E E ( E = =
|
|
\
|

6 1 0
11 0 1
1 2 3

1 2 2
1 0
0 1
R R R
I

=
|
|
\
|

1
1 1
0 1
E =
|
|
\
|

2 2
R R
I

2
1 0
0 1
E =
|
|
\
|

2 1 1
2R - R R
I

3
1 0
2 1
E =
|
|
\
|

A is row
equivalent
to B.

31
Theorem
Every elementary matrix is invertible, and the inverse is also an
elementary matrix. More precisely:
E
-1
is the elementary matrix obtained from I by the inverse of the row
operation that produced E from I.
(a) E
ij

-1
= E
ij.
: R
i
R
j
.
(b) E
i
(k)
-1
= E
i
(
k
1
):R
i
k
1
R
i
, k 0
(c) E
ij
(k)
-1
= E
ij
(-k), i j, k 0: R
i
R
i
-

kR
j
.

Example:

|
|
|
\
|
1 0 0
0 1 0
0 0 1

|
|
|
\
|
1 0 0
0 1 0
0 0 1

|
|
|
\
|
1 0 0
0 1 0
0 0 1

|
|
|
\
|
1 0 0
0 1 0
0 0 1

|
|
|
\
|
0 1 0
1 0 0
0 0 1

|
|
|
\
|
0 1 0
1 0 0
0 0 1

|
|
|
\
|
1 0 0
0 3 0
0 0 1

|
|
|
\
|
1 0 0
0 0
0 0 1
3
1

|
|
|
\
|
=
|
|
|
\
|

0 1 0
1 0 0
0 0 1
0 1 0
1 0 0
0 0 1
1

|
|
|
\
|
=
|
|
|
\
|

1 0 0
0 0
0 0 1
1 0 0
0 3 0
0 0 1
3
1
1

E
23
1
23
E = E
23
E
2
(3) E
2
(3)
-1
=E
2
(1/3)

32
Matrix Inverse by Elementary Operations

Theorem: If an nxn matrix can be reduced to the identity matrix, then it is
invertible.

Theorem: If A is an nxn invertible matrix, then A can be reduced to the
identity matrix I
n
by elementary row operations.

It follows that E
s
. . . E
1
A = I where E
1
, . . . , E
s
are the row elementary matrices
corresponding to the row operations and A
-1
= E
s
. . . E
1.

How to find the inverse of an invertible matrix?
1. By using Elementary Row Operations.
2. By using Classical Adjoint.

By using Elementary Row Operations:

( ) I A ( ) B I

B A =
1

( ) I A : augmented matrix
Perform elementary
operations

33

Example:
The matrix
|
|
|
\
|
=
1 0 0
4 2 0
0 1 1
A is nonsingular. Find its inverse by using
elementary operations.

Solution:
( )
|
|
|
\
|

|
|
|
\
|
=

1 0 0
4 1 0
0 0 1

1 0 0
0 2 0
0 1 1
1 0 0
0 1 0
0 0 1

1 0 0
4 2 0
0 1 1
3 2 2
4
3
R R R
I / A

|
|
|
|
\
|

|
|
|
\
|

1 0 0
2 0
2 1

1 0 0
0 1 0
0 0 1
1 0 0
2 0
0 0 1

1 0 0
0 1 0
0 1 1
2
1
2
1
2
1
2
2 1 1 2 2
R R R / R R

|
|
|
|
\
|
=

1 0 0
2 0
2 1
2
1
2
1
1
A

34

By using Classical Adjoint

|
|
|
|
|
\
|
=
nn n n
n
n
a a a
a a a
a a a
A
K
M M M M
L
L
2 1
2 22 21
1 12 11

adj A =
|
|
|
|
|
\
|
nn n n
n
n
A A A
A A A
A A A
L
M L M M
L
L
2 1
2 22 12
1 21 11
, A
ij
=(-1)
i+j
M
ij
, where M
ij
is the minor of |A|,
that is, a determinant obtained by deleting the i-th row and j-th column of |A|.

adjA
A det
A
1
1
=

cofactor

35
Example:
The matrix
|
|
|
\
|
=
1 0 0
4 2 0
0 1 1
A is nonsingular. Find its inverse by using classical
adjoint.

Solution:
1. Find the determinant of A, det A = 2.
2. Find the cofactors
ij
A .
2
1 0
4 2
11
= = A , 1
1 0
0 1
21
= = A , 4
4 2
0 1
31
= = A
0
1 0
4 0
12
= = A , 1
1 0
0 1
22
= = A , 4
4 0
0 1
32
= = A
0
0 0
2 0
13
= = A , 0
0 0
1 1
23
= = A , 2
2 0
1 1
33
= = A

3.
|
|
|
\
|
=
2 0 0
4 1 0
4 1 2
A adj
4. adjA
A det
A
1
1
=

|
|
|
\
|
=
2 0 0
4 1 0
4 1 2
2
1

|
|
|
|
\
|
=
1 0 0
2 0
2 1
2
1
2
1

36

Remarks
To solve a system of n equations in n unknowns by Cramers Rule, it is
necessary to evaluate 1 + n determinants of n n matrices. For system with
more than three equations, Gaussian elimination is far more efficient, since
it is only necessary to reduce one ( ) 1 + n n augmented matrix.

Gaussian Elimination
Gaussian Elimination is a method of solving system of linear equations.

Elementary Operations
The basic method for solving a system of linear equations is to replace the
given system by a new system that has the same solution set but which is
easier to solve.
Elementary Operations for Linear Equations
(1) Multiply an equation by a nonzero number.
(2) Interchange two equations.
(3) Add a multiple of one equation to another.

Since the rows of the augmented matrix correspond to the equations in the
associated system, these operations correspond to the following elementary
row operations which are performed on the augmented matrix.
Elementary Row Operations
(1) Multiply a row through by nonzero constant:
i i
R R , { } 0 \ R .
(2) Interchange two rows:
j i
R R .
(3) Addition of a constant multiple of one row to another row:
j i i
R R R + .

Reduced Row-Echelon form
A matrix in reduced row-echelon form must contain following properties.
(1) If a row does not consist entirely of zeros, then the first nonzero
number in row is a 1, called a leading 1 or pivot.
(2) Any rows consisting entirely of zeros are grouped together at the
bottom of the matrix.
(3) If any two successive rows that do not consist entirely of zeros, the
leading 1 in the lower row occurs farther to the right than the leading
1 in the higher row.
(4) Each column that contains a leading 1 has zeros everywhere else.

37
A matrix having properties (1), (2) and (3)(but not necessarily 4) is said to
be in row-echelon form.

Example :
(
(
(
1 0 0
0 1 0
0 0 1
in reduced row-echelon form.

(
(
(
0 0 0
0 1 0
0 1 1
in row-echelon form.

Example :
Which of the following matrices are said to be in (i) row-echelon form (r.e.f)
(ii) reduced row-echelon form (r.r.e.f)?
(
(
(
=
1 0 0
5 1 0
8 2 1
A
(
(
(
(
=
1 0 0 0
0 0 0 0
0 0 1 0
0 0 0 1
B
n
I C =
(
(
(
=
1 0 0
0 0 1
0 1 1
D
(
(
(
=
1 0 0
0 0 1
0 1 0
E
A - r.e.f.
C r.e.f / r.r.e.f
B , D and E - neither.

Procedure to reduce a matrix to reduced row-echelon form/ row-echelon
form: (Study this together with example 43 below)
(1) Locate the leftmost column that does not consist entirely of zeros.
(2) Interchange the top row with another row, if necessary, to bring a
nonzero entry to the top of the column found in (1).
(3) If the entry that is now at the top of the column found in (1) is a, then
multiply the first row by
a
1
in order to get a leading 1.
(4) Add suitable multiples of the top row to the rows below so that all
entries below the leading 1 become zeros.
(5) Ignore the rows already containing leading 1. Locate the leftmost
column that does not consist entirely of zeros. Repeat (2), (3), (4) and
(5) until the entire matrix in reduced row-echelon form/ row-echelon.

38
The procedure for reducing a matrix to reduced row-echelon form is called
Gauss-Jordan elimination.
If we use the procedure to reduce a matrix up to a row-echelon form is called
Gaussian elimination.

Example :
Reduce this matrix into reduced row-echelon form.
(
(
(
=
1 1 0 1
2 0 2 2
2 0 1 0
A

(1)
(
(
(
1 1 0 1
2 0 2 2
2 0 1 0

(2)
2 1
R R (interchange row 1 and row 2)
(
(
(
1 1 0 1
2 0 1 0
2 0 2 2

(3)
1 1
2
1
R R
(
(
(
1 1 0 1
2 0 1 0
1 0 1 1

(4)
1 3 3
R R R
(
(
(
0 1 1 0
2 0 1 0
1 0 1 1

Leftmost nonzero column

39
(5)
(
(
(
0 1 1 0
2 0 1 0
1 0 1 1

Repeat step (2), (3), (4) and (5).

(
(
(

2 1 0 0
2 0 1 0
1 0 0 1
is the reduced row-echelon form for matrix A.

Example :
Reduce the following matrix into row-echelon form and reduced row-
echelon form.
(
(
(
=
3 3 3
1 1 2
1 2 1
A

(
(
(
3 3 3
1 1 2
1 2 1

(
(
(

0 3 0
1 3 0
1 2 1

(
(
(
0 3 0
3
1
1 0
1 2 1

(
(
(
1 0 0
3
1
1 0
1 2 1

(
(
(
(
(
(
1 0 0
3
1
1 0
3
1
0 1

Leftmost nonzero column
1 2 2
2R R R
1 3 3
3R R R
2 2
3
1
R R
2 3 3
3R R R +
2 1 1
2R R R
3 1 1
3
1
R R R
3 2 2
3
1
R R R
Row-echelon form

40

(
(
(
1 0 0
0 1 0
0 0 1

How to use Gauss Elimination/Gauss-Jordan Elimination to solve a
system of linear equations? (Study the following example)

Reduced row-echelon form

41

Example :
Solve the following system of linear equation by using Gauss-Jordan
Elimination method.
3 3 3 3
1 2
2 2
= + +
= + +
= + +
z y x
z y x
z y x

Above system is in standard form.
The augmented matrix is
(
(
(
=
3 3 3 3
1 1 1 2
2 1 2 1
A .
Reduce the matrix A into Reduced row-echelon form.

(
(
(
3 3 3 3
1 1 1 2
2 1 2 1

(
(
(

3 0 3 0
3 1 3 0
2 1 2 1

(
(
(
3 0 3 0
1
3
1
1 0
2 1 2 1

(
(
(
0 1 0 0
1
3
1
1 0
2 1 2 1

(
(
(
(
(
(
0 1 0 0
1
3
1
1 0
0
3
1
0 1

(
(
(
0 1 0 0
1 0 1 0
0 0 0 1

0 , 1 , 0 = = = z y x
.

1 2 2
2R R R
1 3 3
3R R R
2 2
3
1
R R
2 3 3
3R R R +
2 1 1
2R R R
3 1 1
3
1
R R R
3 2 2
3
1
R R R
Row-echelon form

42
Existence and properties of solution of Linear System
After we apply the Gauss elimination / GaussJordon elimination to the
augmented matrix [ ] B A .
1. If there are zero rows in the row echelon form and there is a leading 1 in
the B column, the system is inconsistent ( no solution).
For example,
(
(
(
1 0 0 0
3 0 1 0
2 0 0 1

2. If there are one or more zero rows in the augmented matrix, then the
system is having infinitely many solutions.
For example,
(
(
(
0 0 0 0
3 0 1 0
2 0 0 1

(
(
(
0 0 0 0
0 0 0 0
2 0 0 1

3. If there is NO zero row in the augmented matrix, then the system is
having precisely one solution.
For example,
(
(
(
1 1 0 0
3 0 1 0
2 0 0 1

The equation is
inconsistent
Infinitely many
solutions
Infinitely many
solutions
Precisely one
solution

43
Example : (Gauss elimination if no solution exists)

Solve the system
6 4 2 6
0 2
3 2 3
3 2 1
3 2 1
3 2 1
= + +
= + +
= + +
x x x
x x x
x x x

( )
1 3 3
1 2 2
2
3
2
6
0
3
4 2 6
1 1 2
1 2 3
R R R
R R R
+
|
\
|
+
|
|
|
\
|

( )
2 3 3
6 0
2
3
2 2 0
3
1
3
1
0
1 2 3
R R R +
|
|
|
|
\
|

|
|
|
|
\
|

12
2
3
0 0 0
3
1
3
1
0
1 2 3

3 2 3
3 2 1
= + + x x x
2
3
1
3
1
3 2
= + x x
12 0 =

This shows that the system has no solution.

44

Example : (Gauss elimination if infinitely many solutions exist)

Solve the linear system of three equations in four unknowns

1 2 4 2 3 0 3 0 2 1
7 2 4 5 5 1 5 1 6 0
0 8 0 5 0 2 0 2 0 3
4 3 2 1
4 3 2 1
4 3 2 1
. x . x . x . x .
. x . x . x . x .
. x . x . x . x .
= +
= + +
= + +

1 3 3
1 2 2
4 0
2 0
1 2
7 2
0 8
4 2 3 0 3 0 2 1
4 5 5 1 5 1 6 0
0 5 0 2 0 2 0 3
R ) . ( R R
R ) . ( R R
.
.
.
. . . .
. . . .
. . . .
+
+
|
|
|
\
|

2 3 3
1 1
1 1
0 8
4 4 1 1 1 1 0
4 4 1 1 1 1 0
0 5 0 2 0 2 0 3
R R R .
.
.
. . .
. . .
. . . .
+
|
|
|
\
|

|
|
|
\
|
0
1 1
0 8
0 0 0 0
4 4 1 1 1 1 0
0 5 0 2 0 2 0 3
.
.
. . .
. . . .

Back substitution:
4 3 2
4 1 x x x + =

4 1
2 x x =

Therefore we have infinitely many solutions. If we choose a value of
3
x and
a value of
4
x then the corresponding values of
1
x and
2
x are uniquely
determined.

45

Example : (Gauss elimination if a unique solution exists)

Solve the following system of linear equation by using the Gauss elimination
method
6 3 2
3 2 1
= + + x x x
14 2 3 2
3 2 1
= + x x x
2 3
3 2 1
= + x x x

The system can be written as
B X = A
i.e

(
(
(
=
(
(
(
(
(
(
2
14
6
1 1 3
2 3 2
3 2 1
3
2
1
x
x
x

|
|
|
\
|

2
14
6
1 1 3
2 3 2
3 2 1

|
|
|
\
|

20
2
6
10 5 0
4 7 0
3 2 1

|
|
|
\
|

4
2
6
2 1 0
4 7 0
3 2 1

|
|
|
\
|
4
2
6
4 7 0
2 1 0
3 2 1

|
|
|
\
|
30
4
6
10 0 0
2 1 0
3 2 1

|
|
|
\
|
3
4
6
1 0 0
2 1 0
3 2 1

~ row echelon form

( )
1 2 2
2 R R R +
( )
1 3 3
3 R R R +
3 3
5
1
R R
3 2
R R
2 3 3
7R R R +
3 3
10
1
R R

46

=
= +
= + +
= +
= +
= + +
3
4 2
6 3 2
2 3
14 2 3 2
6 3 2
3
3 2
3 2 1
3 2 1
3 2 1
3 2 1
x
x x
x x x
x x x
x x x
x x x

Back substitution,
3
3
= x
2 2 4
3 2
= = x x
1 2 3 6
2 3 1
= = x x x

The solution for X ( ) ( ) 3 2 1
3 2 1
, , x , x , x = =

47
Gauss-Jordan elimination also can be use to find the inverse of a matrix?
The algorithm is as follows.

[ ]
n
I A [ ] B I
n
then B A =
1
.

Example :
Find the inverse of
(
(
(
=
2 2 0
7 2 0
2 1 1
A .

[ ]
n
I A

(
(
(
1 1 0
0 1 0
0 0 1

5 0 0
7 2 0
2 1 1

(
(
(

2 0 2 0 0
0 5 0 0
0 0 1

1 0 0
5 3 1 0
2 1 1
. .
. .

A finite series of elementary row
operations
3 3
2 2
1 1
2 0
5 0
R . R
R . R
R R

3 2 2
3 1 1
5 3
2
R . R R
R R R

+

2 3 3
R R R

48
(
(
(

2 0 2 0 0
7 0 2 0 0
4 0 4 0 1

1 0 0
0 1 0
0 1 1
. .
. .
. .

(
(
(
2 0 2 0 0
7 0 2 0 0
3 0 2 0 1

. .
. .
. .
I
n

Hence
(
(
(
2 0 2 0 0
7 0 2 0 0
3 0 2 0 1
1
. .
. .
. .
A .

Exercise
What is the different within Gauss Elimination and Gauss-Jordan
Elimination?
Find the answer from lecture notes.

2 1 1
R R R +

49
Linear Dependence and Independence, Rank of a matrix
Let
n
R be the collection of all ordered n-tuples row vectors, that is
( ) { } n i R x x x x R
i n
n
, 2, , 1 , , , ,
2 1
L L = = .

Definition
Let
n
k
R A A A , , ,
2 1
L . Any row vector in
n
R of the form
k k
A A A + + + L
2 2 1 1

where
i
, k i , , 3 , 2 , 1 L = are scalar in R, is called a linear combination of
vectors
k
A A A , , ,
2 1
L . A row vector A is a linear combination of
k
A A A , , ,
2 1
L if and only if there exists scalar R
k
, , ,
2 1
L such that
k k
A A A A + + + = L
2 2 1 1
.

Definition
The vectors
n
k
R A A A , , ,
2 1
L are said to be linearly independent if
0
2 2 1 1
= + + +
k k
A A A L implies 0
2 1
= = = =
k
L .

Definition
The vectors
n
k
R A A A , , ,
2 1
L are said to be linearly dependent if there
exists scalars R
k
, , ,
2 1
L , not all equal to zero, such that
0
2 2 1 1
= + + +
k k
A A A L .

Theorem
(1) A finite set of vectors that contains the zero vectors is linearly
dependent.
(2) Two vectors are linearly dependent if and only if either vector is a
scalar multiple of the other.

Example :
Show that any nonzero vector v is, by itself, linearly independent.
Suppose 0 = kv , but 0 v . Then 0 = k . Hence v is linearly independent.

50
Example :
Determine whether u and v are linearly dependent.
(a) ( ) 5 , 3 = u , ( ) 10 , 6 = v (b) ( ) 4 , 3 = u , ( ) 10 , 6 = v

Two vectors u and v are dependent if and only if one is a multiple of the
other.
(a) Yes; u v 2 = (b) No; neither is a multiple of the other.

Example :
Determine whether the vectors ( ) 1 , 2 , 1 , ( ) 0 2, , 2 and ( ) 3 , 4 , 5 are linearly
dependent.
Method 1:
Set a linear combination of the vectors equal to the zero vector using
unknown scalars
1
,
2
and
3
:
( ) ( ) ( ) ( ) 0 0, , 0 3 , 4 , 5 0 2, , 2 1 , 2 , 1
3 2 1
= + +
Then, ( ) ( ) 0 0, 0, 3 , 4 2 2 , 5 2
3 1 3 2 1 3 2 1
= + + + +

Set corresponding components equal to each other to obtain the equivalent
homogeneous system, and reduce to echelon form:
0 3
0 4 2 2
0 5 2
3 1
3 2 1
3 2 1
= +
= +
= + +
or
0 2 2
0 6 6
0 5 2
3 2
3 2
3 2 1
=
= +
= + +
or

0
0 5 2
3 2
3 2 1
= +
= + +

The system, in echelon form, has only two nonzero equations in the three
unknowns; hence the system has a nonzero solution. Thus the original
vectors are linearly dependent.

51
Method 2:
Form the matrix whose rows are the given vectors, and reduce to echelon
form using the elementary row operations:
(
(
(
3 4 5
0 2 2
1 2 1
to
(
(
(
2 6 0
2 6 0
1 2 1
to
(
(
(
0 0 0
3
1
1 0
1 2 1

Since the echelon matrix has a zero row, the vectors are dependent.

Example :
Determine whether ( ) 3 - 2, 1, , ( ) 2 3, - 1, and ( ) 5 1, - 2, are linearly
independent.

Form the matrix whose rows are the given vectors, and reduce to echelon
form using the elementary row operations:
(
(
(
5 1 2
2 3 1
3 2 1
to
(
(
(
11 5 0
5 5 0
3 2 1
to to
(
(
(
1 0 0
1 1 0
3 2 1

Since the echelon matrix has no zero rows, the vectors are independent.

Definition
The maximum number of linearly independent row vectors of a matrix
[ ]
ij
a A = is called the rank of A and is denoted by ( ) A rank or ( ) A .

Theorem
(1) If A is any matrix, then ( ) ( ) A rank A rank
T
= .
(2) If A is an n m matrix in row echelon form, then ( ) A rank is the
number of nonzero rows of A.
(3) Row equivalent matrices have the same rank.

52

Example :
Given
(
(
(
=
5 1 2
2 3 1
3 2 1
A . Find ( ) A rank .
From example 54, number of nonzero rows of row equivalent matrix for A is
3. Thus ( ) 3 = A rank .

53
Theorem (Fundamental Theorem of Linear Systems)
A linear system of m equation in n unknowns (
1
x ,
2
x ,
3
x ,
4
x ,
n
x , L ):

1 1 2 12 1 11
b x a x a x a
n n
= + + + L
M
L

2 2 2 22 1 21
b x a x a x a
n n
= + + +
( )
m n mn m m
b x a x a x a = + + + L
2 2 1 1

has solution if and only if the coefficient matrix A and the augmented matrix
~
A have the same rank; that is, ( ) |
\
|
=
~
A rank A rank . Moreover,
(1) If ( ) n A rank A rank = |
\
|
=
~
, then the system ( ) has precisely one
solution.
(2) If ( ) n A rank A rank < |
\
|
=
~
, then the system ( ) has infinitely many
solutions.
(3) If ( )
|
\
|
<
~
A rank A rank , then the system (*) has no solution.

Theorem
Let A be an n n matrix. Then the following are equivalent:
(1) A is invertible,
(2) ( ) n A rank = ,
(3) B X = A has exactly one solution for every 1 n matrix B,
(4) 0 = X A has only the trivial solution.

54

Solving a system of linear equations by using Determinant (or Cramers
Rule)
Cramers Rule
Cramers Rule is useful in solving a system of linear equation of n equations
in n unknowns.

Theorem (Cramers Rule)
If
~ ~
b x = A is a system of n linear equations in n unknowns such that
( ) 0 A det , then the system has a unique solution. This solution is
( )
( ) A det
A det
x
i
i
= , n i , 3, 2, , 1 L =
where
i
A , n i , 3, 2, , 1 L = is the matrix obtained by replacing the entries in
the i
th
column of A by the entries in the matrix
(
(
(
(
=
n
b
b
b
M
2
1
~
b .

55

Example :
Solve by determinant (or Cramers Rule):
5 2 = y x
0 3 2 1 = + + x y .
First arrange the system in standard form:
1 2 3
5 2
= +
=
y x
y x

The coefficient matrix is
(

=
2 3
1 2
A .
The determinant of coefficient matrix is ( ) 7
2 3
1 2
=
= A det .
Since ( ) 0 A det , the system has a unique solution,
( ) 9
2 1
1 5
1
=
= A det , ( ) 17
1 3
5 2
2
=
= A det .

Thus the unique solution of the system is
( )
( ) 7
9
1
= =
A det
A det
x ,
( )
( ) 7
17
2
= =
A det
A det
y .

56

Example :
Solve by determinant (or Cramers Rule):
y x z
y z x
z x y
2 1 3
8 3
2 3
=
= +
= +

First arrange the system in standard form:
1 3 2
8 3
0 3 2
= + +
= + +
= +
z y x
z y x
z y x

The coefficient matrix is
(
(
(
=
3 2 1
1 1 3
1 3 2
A .
The determinant of coefficient matrix is ( ) 35 = A det .
Since ( ) 0 A det , the system has a unique solution,
( ) 84
3 2 1
1 1 8
1 3 0
1
=
= A det , ( ) 35
3 1 1
1 8 3
1 0 2
2
=
= A det ,
( ) 63
1 2 1
8 1 3
0 3 2
3
=
= A det .

Thus the unique solution of the system is

( )
( ) 35
84
1
= =
A det
A det
x ,
( )
( )
1
2
= =
A det
A det
y ,
( )
( ) 35
63
3
= =
A det
A det
z .

57

Further Matrices

Partitioning
Any matrix A may be partitioned into a number of smaller matrices
called blocks by vertical lines that extend from bottom to top, and
horizontal lines that extend from left to right.

Example:
|
|
|
\
|
=
|
|
|
|
|
\
|

=
|
|
|
|
|
\
|

=
32 31
22 21
12 11
1
0
7
3

6 4
4 3
2 1
0 1
1 6 4
0 4 3
7 2 1
3 0 1
A A
A A
A A
A
where the blocks are
( ) ( ) ( ) ( ) 1 6 4
0
7
4 3
2 1
3 0 1
32 31 22 21 12 11
= =
|
|
\
|
=
|
|
\
|
= = = A , A , A , A , A , A .

Clearly the partition is NOT unique. We could also have
|
|
|
|
|
\
|
=
1
0
7
3
6
4
2
0
4
3
1
1
A =
|
|
\
|
23 22 21
13 12 11
A A A
A A A

58

Some properties: Example:

If A and B are partitioned as
|
|
|
\
|
=
mn m m
n
A A A
A A A
A
L
M M M
L
2 1
1 12 11
and

|
|
|
\
|
=
pq p p
q
B B B
B B B
B
L
M M M
L
2 1
1 12 11
then

(i)
|
|
|
\
|

=
mn m m
n
A A A
A A A
A
L
M M M
L
2 1
1 12 11

(ii) If A and B are matrices of the same size,
i.e., m = p and n = q and each A
ij
block is
of the same form as the corresponding B
ij

block, then
|
|
|
\
|
+ + +
+ + +
= +
mn mn m m m m
n n
B A B A B A
B A B A B A
B A
L
M M M
L
2 2 1 1
1 1 12 12 11 11

(iii) If n = p and we denote AB = C, then
, B A C
n
k
kj ik ij
=
=
1

provided that the number of columns in
each A
ik
is the same as the number of rows
in the corresponding B
kj
.

If A and B are partitioned as
|
|
\
|
=
|
|
|
\
|
=
22 21
12 11
3 2 3
1 1 2
4 0 1
A A
A A
A and

|
|
\
|
=
|
|
|
\
|
=
22 21
12 11
1 0 3
2 2 1
1 0 0
B B
B B
B then
(i)
|
|
|
\
|
=
|
|
\
|
=
15 10 15
5 5 10
20 0 5
5 5
5 5
5
22 21
12 11
A A
A A
A

(ii)
|
|
\
|
+ +
+ +
= +
22 22 21 21
12 12 11 11
B A B A
B A B A
B A

|
|
|
\
|
=
4 2 6
3 3 3
5 0 1

(iii)
|
|
\
|
|
|
\
|
=
22 21
12 11
22 21
12 11
B B
B B
A A
A A
AB
|
|
\
|
+ +
+ +
=
22 22 12 21 21 22 11 21
22 12 12 11 21 12 11 11
B A B A B A B A
B A B A B A B A

( ) 0 3
1
4
2 1
0 0
1 2
0 1
21 12 11 11
|
|
\
|
+
|
|
\
|
|
|
\
|
= + B A B A

|
|
\
|
=
2 4
0 12

( )
|
|
\
|
=
|
|
\
|
+
|
|
\
|
|
|
\
|
= +
5
5
1
1
4
2
1
1 2
0 1
22 12 12 11
B A B A
( ) ( )( ) ( ) 4 11 0 3 3
2 1
0 0
2 3
21 22 11 21
= +
|
|
\
|
= + B A B A
( ) ( )( ) ( ) 10 1 3
2
1
2 3
22 22 12 21
= +
|
|
\
|
= + B A B A
|
|
|
\
|
=
10 4 11
5 2 4
5 0 12
AB

59
Suppose that an n x n matrix A can be partitioned into the block
diagonal form
|
|
|
|
|
\
|
m
A
A
A
0 0
0
0
0 0
2
1
L
O M
M
L

where A
1
, . . . , A
m
are all square matrices. (not necessarily all of
the same size)
Then
(i) A is invertible if and only if A
1
, . . . , A
m
are invertible.
(ii) assuming that A
1
, . . . , A
m
are invertible, then
|
|
|
|
|
\
|
=
1
1
2
1
1
1
0 0
0
0
0 0
m
A
A
A
A
L
O M
M
L

Example:
Find entry(1,2) in A
2
, where A =
|
|
|
\
|

2 1 1
0 1 1
0 1 1
.

Solution:

By partitioning: By original way:
|
|
\
|
=
|
|
|
\
|

22 12
12 11
2 1 1
0 1 1
0 1 1
A A
A A

|
|
|
\
|
=
4 4 4
0 2 2
0 2 2
2
A

|
|
\
|
=
|
|
\
|
|
|
\
|
=
2 2
2 2
1 1
1 1
1 1
1 1
11 11
A A 2
12
= a
2
12
= a .

60
Example:
Find entry(1,2) in A
2
, where A =
|
|
|
\
|

2 1 1
1 1 1
1 1 1
.

Solution:
By partitioning: By original way:

|
|
\
|
=
|
|
|
\
|

22 12
12 11
2 1 1
1 1 1
1 1 1
A A
A A

|
|
|
\
|

=
4 0 0
2 1 1
2 3 3
2
A

|
|
\
|
=
|
|
\
|
|
|
\
|
=
2 2
2 2
1 1
1 1
1 1
1 1
11 11
A A 3
12
= a

2
12
= a . Is this TRUE??

61
Eigenvalue and Eigenvector

Let A be an n n matrix. A number is said to be an eigenvalue of
A if there exists a nonzero solution vector X of the linear system
AX = X.
The solution vector X is said to be an eigenvector corresponding to
the eigenvalue .

From the definition, = 0 can be an eigenvalue but the zero
vector is NOT an eigenvector for any matrix.

Example:
Is
|
|
\
|
1
1
an eigenvector of the matrix
|
|
\
|
=
3 4
2 1
A ?

Solution:
|
|
\
|
=
|
|
\
|
|
|
\
|
1
1
1
1
1
3 4
2 1

Hence,
|
|
\
|
1
1
is an eigenvector of A associated to the eigenvalue 1 .

A X = X

62
How to find eigenvalues of an n n matrix A?
Solve characteristic equation of A to get eigenvalues.

What is characteristic equation of A?
We rewrite x Ax = as Ix Ax = , or equivalently,
( ) 0 = x A I .

( ) 0 = A I det : Characteristic equation of A.

When ( ) A I det is expanded, it is a polynomial in called
characteristic polynomial of A. This characteristic polynomial
has the form:
n
+ c
n-1
n-1
+. . . + c
1
+ c
0
.

Example:
Find the eigenvalues of
(
(
(
=
6 11 6
1 0 0
0 1 0
A .

Solution:
Characteristic polynomial:
( ) 6 11 6
6 11 6
1 0
0 1
2 3
+ =
(
(
(
= det A I det .

Characteristic equation:
( ) 0 = A I det
0 6 11 6
2 3
= +
( )( )( ) 0 3 2 1 =
3 2 1 , , =
The eigenvalues of A are 1, 2 and 3.

Nonzero ( ) 0 = A I det

63
Eigenvalues of triangular matrices

Theorem:
If A is an n x n triangular matrix (upper triangular, lower
triangular, or diagonal), then the eigenvalues of A are the entries
on the main diagonal of A.

Example:
Find the eigenvalues of
|
|
|
\
|
=
5 2 7
0 2 9
0 0 1
A (lower triangular matrix).

Solution:
By inspection, the eigenvalues are 1, 2 and 5.

How to find eigenvectors of an n n matrix A?
The eigenvectors of A corresponding to an eigenvalue are the
nonzero vectors x that satisfy x Ax = .

Study the following 3 examples:
Example: (All the eigenvalues are different)

Example: (Some of the eigenvalues are the same)

Example: (All the eigenvalues are the same)

64
Example: (All the eigenvalues are different)
Determine the eigenvalues and eigenvectors for the matrix A.
|
|
|
\
|
=
1 1 0
1 2 1
2 1 1
A .

Solution:
The characteristic equation, |A - I| = 0
0
1 1 0
1 2 1
2 1 1
=

(1 )[(2 )(1 + ) 1] + [(1 ) + 2] = 0
(1 )[
2
- 3] + (1 ) = 0
(1 )[
2
2] = 0
(1 )( + 1)( 2) = 0
Hence, the eigenvalues are
1
= 2,
2
= 1 and
3
= 1.
For =
1
= 2, we have
A - I =
|
|
|
\
|

=
3 1 0
1 0 1
2 1 1
2I A
Let e
1
be the eigenvector of

= 2. Then

(A 2I)e
1
= 0

0
3 1 0
1 0 1
2 1 1
3
2
1
=
|
|
|
\
|
|
|
|
\
|

x
x
x

that is,
x
1
+ x
2
2x
3
= 0
x
1
+ 0x
2
+ x
3
= 0
0x
1
+ x
2
3x
3
= 0
x
1
= x
3

x
2
= 3x
3

65
Thus the eigenvector e
1
corresponding to the eigenvalue

= 2 is
|
|
|
\
|
=
1
3
1
1
e ,
where is an arbitrary nonzero scalar.

For =
2
= 1, we have
(A (1)I)e
2
= 0
0
2 1 0
1 1 1
2 1 0
3
2
1
=
|
|
|
\
|
|
|
|
\
|
x
x
x

that is,
x
2
2x
3
= 0
x
1
+ x
2
+ x
3
= 0
x
2
2x
3
= 0
x
2
= 2x
3

x
1
= x
2
+ x
3
= 3 x
3

2

= 1 is
|
|
|
\
|
=
1
2
3
2
e
where is an arbitrary nonzero scalar.

For =
3
= 1, we obtain
(A (1)I)e
3
= 0
0
0 1 0
1 3 1
2 1 2
3
2
1
=
|
|
|
\
|
|
|
|
\
|
x
x
x

that is,
2x
1
+ x
2
2x
3
= 0
x
1
+ 3x
2
+ x
3
= 0
x
2
= 0
x
2
= 0 , x
1
= x
3

3

= 1 is
|
|
|
\
|
=
1
0
1
3
e .

66
Example: (Some of the eigenvalues are the same)
Determine the eigenvalues and eigenvectors for the matrix
|
|
|
\
|

=
11 4 4
8 1 4
16 8 5
A .

Solution:
The characteristic equation, |A - I| = 0
0
11 4 4
8 1 4
16 8 5
=

( + 3)[
2
+ 2 3] = 0
( + 3)
2
( 1) = 0
Hence, the eigenvalues are
1
= 3,
2
= 3 and
3
= 1.
For = -3 we have
(A -
1
I)e
1
= 0
that is,
8x
1
8x
2
16x
3
= 0
4x
1
- 4x
2
8x
3
= 0
4x
1
+ 4x
2
+ 8x
3
= 0
which form a single equation
x
1
+ x
2
+ 2x
3
= 0
We are free to choose any two of the components x
1
, x
2
, and x
3
at will, with
the remaining one determined by the above equation. Suppose we set, x
2
=
and x
3
=; then
|
|
|
\
|
+
|
|
|
\
|
=
|
|
|
\
|

=
1
0
2
0
1
1 2
X
Thus the two linearly independent eigenvectors e
1
and e
2
corresponding to
the eigenvalue

= -3 are
|
|
|
\
|
=
0
1
1
1
e and
|
|
|
\
|
=
1
0
2
2
e .
For = 1, the eigenvector is
|
|
|
\
|
=
1
1
2
3
e .

67
Example: (All the eigenvalues are the same)
Determine the eigenvalues and eigenvectors for the matrix A.

|
|
\
|
=
7 1
4 3
A .
Sol: The characteristic equation, |A - I| = 0
( ) 0 5
7 1
4 3
2
= =

Hence, the eigenvalue is
1
=
2
= 5.
(A
1
I)e = 0
2x
1
+ 4x
2
= 0
x
1
+ 2x
2
= 0
x
1
= 2x
2

Thus , we find the single eigenvector
|
|
\
|
=
1
2
e .

68
Eigenvalues and invertibility

Theorem: A square matrix A is invertible if and only if = 0 is
NOT an eigenvalue of A.

Example:
Is matrix
|
|
\
|
1 0
1 0
invertible?

Solution:
From the above theorem, we see that
The eigenvalues are 0 and 1. Since we
have 0 = , then A is not invertible.

OR,
From the formula
If
|
|
\
|
=
1 0
1 0
A , then
( )( ) ( )( )
|
|
\
|

0 0
1 1
0 1 1 0
1
1
A

Given a 2 2 matrix
|
|
\
|
=
d c
b a
A , the
inverse is
|
|
\
|
a c
b d
bc ad
A
1
1

0
1

69
Some useful properties of eigenvalues
1. The sum of the eigenvalues A is

= =
= =
n
i
ii
n
i
i
a A trace
1 1
.

Example:
Given a square matrix
|
|
|
\
|
=
3 0 0
5 2 0
4 2 1
A , find the sum of eigenvalues and
compare with trace A.

Solution:
|
|
|
\
|
=
3 0 0
5 2 0
4 2 1
A is a upper triangular matrix.
Eigenvalues are 1, 2 and 3.

Sum of eigenvalues = 1+2+3=6

trace A = 1+2+3 (the values in the diagonal of A)

2. The product of the eigenvalues of A is A det
n
i
i
=
=1
, where
det A denotes the determinant of the matrix A.

Example:
|
|
|
\
|
=
3 0 0
5 2 0
4 2 1
A , find the product eigenvalues and
compare with det A.
Solution:
( )( )( ) 6 3 2 1
3
1
= =
= i
i
.
( )( )( ) 6 3 2 1 = = A det .
Same answer
Same answer

70
3. Suppose the eigenvalues for A are
1
,
n
, ,
2
K . Then the
eigenvalues of the inverse matrix A
-1
, provided it exists, are
. ,..., ,
n

1 1 1
2 1

Example:
|
|
|
\
|
=
3 0 0
5 2 0
4 2 1
A , find the eigenvalues of
(i) A
(ii) A
-1
.

Solution:
(i) Eigenvalues of A are 1, 2 and 3.
(ii)
|
|
|
\
|
3
1
6
5
2
1
3
1
1
0 0
0
1 1
A
This an upper triangular matrix,
eigenvalues are 1, 1/2 and 1/3.

By using elementary operations
method
Or
By using classical adjoint method.

71
4. The eigenvalues of the tranposed matrix A
T
are
1
,
2
, . .,
n
as for the matrix A.

Example:
|
|
|
\
|
=
3 0 0
5 2 0
4 2 1
A , compare the eigenvalues for A and
T
A .

Solution:
Eigenvalues for A are 1, 2 and 3.
|
|
|
\
|
=
3 5 4
0 2 2
0 0 1
T
A

1
,
n
, ,
2
K . If k is a
scalar then the eigenvalues of kA are k
1
, k
2
, . ., k
n
.

Example:
|
|
|
\
|
=
3 0 0
5 2 0
4 2 1
A , find the eigenvalues for 2A.
Solution:
|
|
|
\
|
=
|
|
|
\
|
=
6 0 0
10 4 0
8 4 2
3 0 0
5 2 0
4 2 1
2 2A .

Eigenvalues for 2A are 2, 4 and 6 (which are 2(1), 2(2), 2(3)).

Same answer
1, 2 and 3 are eigenvalues of A.

72
1
,
n
, ,
2
K . If k is a
scalar and I the identity matrix then the eigenvalues of A kI
are respectively
1
k,
2
k, . . . ,
n
k.

Example:
|
|
|
\
|
=
3 0 0
5 2 0
4 2 1
A , find the eigenvalues for I A 3 + .
Solution:
|
|
|
\
|
=
|
|
|
\
|
+
|
|
|
\
|
= +
6 0 0
5 5 0
4 2 4
1 0 0
0 1 0
0 0 1
3
3 0 0
5 2 0
4 2 1
3I A .
The eigenvalues for I A 3 + are 4, 5 and 6 (which are 1+3, 2+3 and 3+3).

1
,
n
, ,
2
K . If k is a
positive integer then the eigenvalues of A
k
are
k
n
k k
, , , K
2 1
.

Example:
|
|
|
\
|
=
3 0 0
5 2 0
4 2 1
A , find the eigenvalues for
2
A .

Solution:

|
|
|
\
|
=
9 0 0
25 4 0
26 6 1
2
A
The eigenvalues for
2
A are 1, 4 and 9 (which are 1
2
, 2
2
and 3
2
).
1, 2 and 3 are eigenvalues of A.

73
Similar Matrices
Suppose A and B are n n matrices. Then A is said to be similar to
B, denote A B, if there exists a nonsingular n n matrix P such
that P
-1
AP = B.
(The matrix P is not unique.)

Example:
Suppose
|
|
\
|
=
2 6
3 7
A and
|
|
\
|
=
4 0
0 1
B . Show that B AP P =
1
where
(a)
|
|
\
|
=
1 2
1 1
P
(b)
|
|
\
|
=
2 4
2 2
P

Solution:

(a)
|
|
\
|
|
|
\
|

|
|
\
|
=
1 2
1 1
2 6
3 7
1 2
1 1
1
1
AP P

B =
|
|
\
|
=
|
|
\
|
|
|
\
|

|
|
\
|
=
4 0
0 1
1 2
1 1
2 6
3 7
1 2
1 1

(b)
|
|
\
|
|
|
\
|

|
|
\
|
=
2 4
2 2
2 6
3 7
2 4
2 2
1
1
AP P

B =
|
|
\
|
=
|
|
\
|
|
|
\
|

|
|
\
|
=
4 0
0 1
2 4
2 2
2 6
3 7
1
2
1
2
1
2
1

74
We have the following observations:
1. A A.
2. A B B A
3. A B, B C A C.

Theorem:
If A B, then
(i) the characteristic polynomial of A is equal to the
characteristic polynomial of B.
(ii) A and B have the same set of eigenvalues.

Example:
Suppose
|
|
\
|
=
2 6
3 7
A and
|
|
\
|
=
4 0
0 1
B , find
(a) the characteristic polynomial of A and B respectively.
(b) the eigenvalues of A and B respectively.

Solution:
(a) Characteristic polynomial of A = A I = 4 5
2 6
3 7
2
+ =
+

.
Characteristic polynomial of B = B I = 4 5
4 0
0 1
2
+ =

.
(b) Eigenvalues for A:
0 = A I
0 4 5
2
= +
= 1 and 4.

Eigenvalues for B:
0 = B I
0 4 5
2
= +
= 1 and 4.

From (a) and (b), matrices A and B are having the same characteristic
polynomial and eigenvalues. And from previous example, we know that A
B.

75
Diagonalization of Matrices
A square matrix A is called diagonalizable if there is an invertible
matrix P such that P
-1
AP is a diagonal matrix; the matrix P is said
to diagonalize A. We may say that A ~ a diagonal matrix.
(Useful in finding
n
A )

AP P B
1
=

1
= PBP A
( )
n
n
PBP A
1
=
( )( )( ) ( )
1
= BP B B PB A
n
P P P P P P K
1 1 1

1
= P PB A
n n

Diagonal matrix
I P P =
1

n
B can be found easily
For example,
If
|
|
\
|
=
3 0
0 2
B is diagonal
matrix then
|
|
\
|
=
|
|
\
|
=
n
n
n
n
B
3 0
0 2
3 0
0 2

A
n
is a product of 3 matrices: P, B
n
and P
-1
.

76
How to find a matrix P in Diagonalization of
Matrices?

Diagonalization algorithm
Given an n x n matrix A:
(1) Find the eigenvalues of A.
(2) Find (if possible) n linearly independent
eigenvectors p
1
, p
2
, . . . , p
n
.
(3) Form P = (p
1
, p
2
, . . . , p
n
) - the matrix with the p
i
as
columns.
(4) Then P
-1
AP is diagonal, the diagonal entries being the
eigenvalues corresponding to p
1
, p
2
, . . . , p
n

respectively.

Linearly Independent:
If
1
v ,
2
v , ,
n
v are n vectors, then the vector equation
0
2 2 1 1
= + + +
n n
v c v c v c K has at least one solution, namely 0
1
= c ,
0
2
= c , and 0 =
n
c . If this is the only solution, then
1
v ,
2
v , ,
n
v are linearly independent.

77
Example:
Diagonalize the matrix
|
|
|
\
|

=
11 4 4
8 1 4
16 8 5
A .
Sol:
The eigenvalues are found to be = -3 and = 1, with eigenvectors
|
|
|
\
|
=
|
|
|
\
|
=
|
|
|
\
|
=
1
1
2
1
0
2
0
1
1
3 2 1
p , p , p . These vectors are linearly independent.
So,
|
|
|
\
|

= =
1 1 0
1 0 1
2 2 1
3 2 1
) p , p , p ( P
is invertible and will diagonalize A. In fact,

|
|
|
\
|

=
2 1 1
3 1 1
2 0 1
1
P and
|
|
|
\
|
1 0 0
0 3 0
0 0 3
1
AP P .

78
Theorem (Distinct Eigenvalues, Linear Independent Eigenvectors)
If an n x n matrix A has distinct eigenvalues
1
, . . . ,
n
, then the
corresponding eigenvectors e
1
, . . . , e
n
are linearly independent.

Theorem
Let A be an n x n matrix.
1. A is diagonalizable if and only if it has n linearly independent
eigenvectors.
2. If A has n linearly independent eigenvectors e
1
, . . . , e
n
and
we make these the columns of Q, so that Q = [e
1
, . . . , e
n
],
then Q
-1
AQ = D is diagonal, with the eigenvalues of A as the
entries on the main diagonal.

Example:
Diagonalize the matrix
|
|
|
\
|
=
2 5 3
0 2 1
0 0 1
A .
Sol:
The characteristic polynomial of A is
2
2 1
2 5 3
0 2 1
0 0 1
) )( ( I A =

=
so the characteristic equation is (-1)(-2)
2
= 0.
Thus the eigenvalues of A are = 1 and = 2.
The eigenvectors of respective eigenvalue are
|
|
|
\
|
=
1
8
1
8
1
1
p and
|
|
|
\
|
=
1
0
0
2
p .

Since A is a 3 x 3 matrix and there are only two basis vectors, A is NOT
diagonalizable.

From the above example, we know that not every matrix is diagonalizable.

79
Theorem
Let A be any n n real matrix. Then the followings are equivalent:
1. A is nonsingular.
2. Rank(A) = n.
3. All the rows(column) are linearly independent.
4. |A| 0.
5. The eigenvalues of A are not zero.

Theorem
Every symmetric matrix is diagonalizable.

Example:
Find a matrix that diagonalizes
|
|
|
\
|
=
4 2 2
2 4 2
2 2 4
A .

Sol:
The characteristic equation of A is ( - 8)( - 2)
2
= 0.
For = 2, we have the corresponding eigenvectors,
|
|
|
\
|
=
0
1
1
1
u and
|
|
|
\
|
=
1
0
1
2
u .
For = 8, we have the corresponding eigenvectors, . u
|
|
|
\
|
=
1
1
1
3

Let
|
|
|
\
|

=
1 1 0
1 0 1
1 1 1
P . Then
|
|
|
\
|
=
8 0 0
0 2 0
0 0 2
1
AP P .

80
Applications

(1) Homogeneous systems of linear first order differential equations with
constant coefficients.
(2) Non-homogeneous systems of linear first order differential equations
with constant coefficients.

Homogeneous systems of linear first order differential equations with
constant coefficients:
Consider a system of linear first-order differential equations with constant
coefficients
x
1
= a
11
x
1
+ a
12
x
2
+ . . . + a
1n
x
n

x
2
' = a
21
x
1
+ a
22
x
2
+ . . . + a
2n
x
n

.
.
.
x
n
' = a
n1
x
1
+ a
n2
x
2
+ . . . + a
nn
x
n

with initial conditions

x
1
(0) = c
1
, x
2
(0) = c
2
, . . . , x
n
(0) = c
n
,

where a
ij
s are constants. Here we have n unknown functions x
j
= x
j
(t) in
one variable t and n differential equations.

Let A
|
|
|
|
|
\
|
=
nn n n
n
n
a a a
a a a
a a a
L
M M
L
L
2 1
2 22 21
1 12 11
, ,
2
1
|
|
|
|
|
\
|
= =
) t ( x
) t ( x
) t ( x
) t ( X X
n
M

and
2
1
.
) t ( ' x
) t ( ' x
) t ( ' x
) t ( ' X ' X
n
|
|
|
|
|
\
|
= =
M

81
Then the system can be represented in matrix equation:
X' = AX (1)

The solution of such a system of differential equations is generally tedious
but is simple if A is diagonal.

We begin the solution of (1) by making a linear change of variables from
n
x x x , , ,
2 1
L to y
1
,y
2
,,y
n
:

Y Q X = (2)

where Q is an invertible matrix. Thus, Y Q X = . Putting (2) into (1) gives
Y AQ Y Q =
Y AQ Q Y
-1
=
So the idea is to find such a matrix Q such that
D AQ Q
1
=

where D is a diagonal matrix.

Example:
Find a solution to the system
2 1 1
3x x x + =
2 1 2
2 2 x x x + =

where x
1
and x
2
are functions of t and x
1
(0) = 0, x
2
(0) = 5.

Solution:
Let
|
|
\
|
=
2 2
3 1
A ,
|
|
\
|
=
2
1
x
x
X ,
|
|
\
|
=
' x
' x
2
1
X' and X(0) = C =
|
|
\
|
5
0
.

Then this system can be written as

( ) ( ) t t AX X = , X(0) = C =
|
|
\
|
5
0
.

82
Solving for the eigenvalues and eigenvectors , we obtain
|
|
\
|
=
1
1
1
q and
|
|
\
|
=
2
3
2
q are eigenvectors corresponding to the eigenvalues 4 and
1 , respectively.
Hence with
|
|
\
|
= =
2 1
3 1
) , (
2 1
q q Q ,
|
|
\
|
1 0
0 4
AQ Q
1
= D.
Now consider new functions y
1
= y
1
(t) and y
2.
= y
2
(t).

Let Y Q X = ,
where

|
|
\
|
=
2
1
y
y
Y

ie,
|
|
\
|
|
|
\
|
=
|
|
\
|
2
1
2
1
2 1
3 1
y
y
x
x
that is ,
2 1 2
2 1 1
2
3
y y x
y y x
=
+ =

Then, from QY X = , we have Y Q X = , substitute into AX X = , we
obtain
Y AQ Y Q =
DY Y AQ Q Y = =
1 -
.

This means that

|
|
\
|
|
|
\
|
=
|
|
\
|
2
1
2
1
1 0
0 4
y
y
' y
' y
so
2 2
1 1
4
y ' y
y ' y
=
=

Solving the two first order linear differential equations, we obtain
( )
t
Ae t y
4
1
= and ( )
t
Be t y

=
2
,
where A and B are constants.

Then,
|
|
\
|
+
=
|
|
\
|
|
|
\
|
=
|
|
\
|
|
|
\
|
=
|
|
\
|
t t
t t
t
t
Be Ae
Be Ae
Be
Ae
y
y
x
x
2
3
2 1
3 1
2 1
3 1
4
4 4
2
1
2
1

83

so the general solution is
x
1
(t) =Ae
4t
+ 3Be
-t

x
2
(t) = Ae
4t
2Be
-t

Finally, the requirement in this example that x
1
(0) = 0, x
2
(0) = 5
determines the constants c and d:
0 = x
1
(0) = Ae
0
+ 3Be
0
= A + 3B
5 = x
2
(0) = Ae
0
2Be
0
= A 2B

These equations give A = 3 and 1 = B , so

x
1
(x) = 3e
4t
3e
-t

x
2
(x) = 3e
4t
+ 2e
-t
.

Example:
Find the general solution to the system
x
1
' = 5x
1
+ 8x
2
+ 16x
3

x
2
' = 4x
1
+ x
2
+ 8x
3

x
3
' = -4x
1
- 4x
2
- 11x
3

Then find a solution satisfying the boundary conditions
x
1
(0) = x
2
(0) = x
3
(0) = 1.

Solution:
The system has the form X = AX, where
|
|
|
\
|

=
11 4 4
8 1 4
16 8 5
A

The eigenvalus are 3, 3 and 1 and the corresponding eigenvectors are,
respectively,

|
|
|
\
|
=
0
1
1
1
q ,
|
|
|
\
|
=
1
0
2
2
q ,
|
|
|
\
|
=
1
1
2
3
q

84

Let ( )
|
|
|
\
|

= =
1 1 0
1 0 1
2 2 1
, ,
3 2 1
q q q Q . Then Q diagonalizes A with

Q
-1
AQ = D=
(
(
(
1 0 0
0 3 0
0 0 3
, a diagonal matrix.

Now let QY X = so ' QY X = .

From the equation X = AX, we have
AQY Y Q =
DY AQY Q Y
'
= =
1

i.e:
1 1
3y y = ,
2 2
3y y = and
3 3
y y = .
Hence we have

t
e c y
3
1 1
=

t
e c y
3
2 2
=

t
e c y
3 3
= , where c
1
, c
2
, c
3
are arbitrary constants.

Hence , QY X =
=
|
|
|
\
|

1 1 0
1 0 1
2 2 1
|
|
|
\
|
3
2
1
y
y
y

=
|
|
|
\
|
+
+

t t
t t
t t t
e c e c
e c e c
e c e c e c
3
3
2
3
3
1
3
3
2
3
1
2 2

So the general solution is

x
1
(t) = c
4
e
-3t
+ 2c
3
e
t
, where ( )
2 1 4
2c c c + =
x
2
(t) = c
1
e
-3t
+ c
3
e
t

x
3
(t) = c
2
e
-3t
- c
3
e
t

85

The boundary conditions, x
1
(0) = x
2
(0) = x
3
(0) = 1 determine the constants
c
i
.
( )
|
|
|
\
|
+
+
= =
|
|
|
\
|
3 2
3 1
3 4
2
0
1
1
1
c c
c c
c c
X

c
4
= -(c
1
+ 2c
2
)
c
4
+ 2c
3
= 1 c
1
= -3
c
1
+ c
3
= 1 c
2
= 5
c
2
- c
3
= 1 c
3
= 4

c
4
= -7

Hence,
( )
t t
e e t x 8 7
3
1
+ =

( )
t t
e e t x 4 3
3
2
+ =

( )
t t
e e t x 4 5
3
3
=

86
Non-homogeneous System of linear first order differential equations
with constant coefficients:

Consider a system of linear first-order differential equations with constant
coefficients
( ) t f x a x a x a x
n n 1 1 2 12 1 11 1
+ + + + = K
n n 2 2 2 22 1 21 2
+ + + + = K
.
.
.
n n nn n n n
+ + + + = K
2 2 1 1

with initial conditions
( )
1 1
0 c x = , ( )
2 2
0 c x = , . . . , ( )
n n
c x = 0 ,

where a
ij
s are constants. Here we have n functions ( ) t x x
j j
= in one
variable t and n differential equations.

Let A
|
|
|
|
|
\
|
=
nn
a
n
a
n
a
n
a a a
n
a a a
L
M M
L
L
2 1
2 22 21
1 12 11
,
2
1
,
) t ( x
) t ( x
) t ( x
) t ( X X
n
|
|
|
|
|
\
|
= =
M

2
1
,
) t ( ' x
) t ( ' x
) t ( ' x
) t ( ' X ' X
n
|
|
|
|
|
\
|
= =
M
and
|
|
|
|
|
\
|
=
) t ( f
) t ( f
) t ( f
) t ( F
n
M
2
1

Then the system can be represented in matrix equation:

X' = AX + F(t) ; (6)

X(0) =
|
|
|
|
|
|
\
|
=
n
c
.
.
c
c
C
2
1

The system is said to be homogeneous , if F(t) = 0 .

87

Suppose that the coefficient matrix A is diagonalizable, so there is an
invertible matrix Q such that AQ Q D
1
= is diagonal.

We choose Q to be a matrix whose columns are n linearly independent
column eigenvectors ,corresponding to the eigenvalues
1
,
2
, . . . ,
n
of A

respectively. Thus,

D = diag(
1
,
2
, . . . ,
n
)
Step 1:

Let QY X = , where

|
|
|
|
|
|
\
|
=
y
.
.
y
y
Y
2
1

Hence, Y' Q X = , since Q is a constant matrix. Substituting for X and X in
the equation (6) X' = AX + F(t) , we obtain

( ) t F AQY Y Q + = ; X(0) = C QY = ) 0 (

( ) t F Q Y AQ Q Y
1 1
'

+ |
\
|
= ; ( ) C Q Y
1
0

=

We can rewrite the above equation as:
( ) t G DY Y
'
+ = ;

where ) (
1
) ( t F Q t G

= = (g
1
(t),,g
n
(t))
T
. Let C Q K
1
= = (d
1
,,d
n
)
T
.

This is a system of linear differential equations in
n
y y y , , ,
2 1
L , the entries
of Y, which has the very simple form

( ) t g y y
1 1 1 1
+ = , ( )
1 1
0 d y =
( ) t g y y
2 2 2 2
+ = , ( )
2 2
0 d y =

.

88
( ) t g y y
n n n n
+ = , ( )
n n
d y = 0

Step 2: Solve for Y

Each of these equations is a first-order linear differential equation, it can be
solved by using the integrating factor method.

Step 3

Finally, to get X, we use X= Q Y.

========================
**The Integrating Method for solving the differential equation of the
following form :
( ) x q y
dx
dy
= +
where is a constant. We multiply the above equation by the integrating
factor

dx
e
Then we obtain

dx
e ( ) x q y
dx
dy
=
|
\
|
+

dx
e

dx
e
dx
dy
+ y

dx
e
=q(x)
dx
e

ie. (
dx
d
y

dx
e
) = q(x)
dx
e

Hence
y

dx
e
q(x)
dx
e
dx + C
We can then find y.
=====================

89
Example:
Solve the system

1
x = 6x
1
+ x
2
+ 6t

2
x = 4x
1
+3x
2
10t +4

Solution:
The system has the form
( ) t F AX X + = ,
where
|
|
\
|
=
3 4
1 6
A and ( )
|
|
\
|
+
=
4 10
6
t
t
t F .
The characteristic equation of A is
0 14 9
2
= + = A I .
The eigenvalues and eigenvectors are found to be
|
|
\
|
= =
4
1
2
1 1
e , and
|
|
\
|
= =
1
1
7
2 2
e , .
Let ( )
|
|
\
|
= =
1 4
1 1
2 1
e e Q . Then Q diagonalizes A with
|
|
\
|
= =
7 0
0 2
1
D AQ Q .
Now let QY X = , so ' QY X = .
From the equation ( ) t F AX X + = , we have
( )
( )
( ) t F Q DY
t F Q AQY Q Y
t F AQY QY
1
1 1
'
'
+ =
=
+ =

Let
( ) ( )
|
|
\
|
+
=
|
|
\
|
+
|
|
\
|
=
=
2 7
2 8
5
2
4 10
6
1 4
1 1
5
1
1
t
t
t
t
t F Q t G

So g
1
(t) =
5
2
(8t 2) , g
2
(t) =
5
2
( 7t + 2).

90

Solving
( )
( ) t g y ' y
t g y ' y
2 2 2
1 1 1
7
2
+ =
+ =

by the integrating factor method, we get

y
1
= C
1
e
2 t

5
2
(4t +1 ) and
y
2
= C
2
e
7 t

5
2
t
35
6
.

To get x
1
and x
2
, we use X = QY .

To get X, use
X=
|
|
\
|
2
1
x
x

|
|
\
|
+
+
=
|
|
\
|
|
|
\
|
=
=
2 1
2 1
2
1
4
1 4
1 1
y y
y y
y
y
QY

So, the general solution is
( )
( )
7
10
6 4
7
4
2
7
2
2
1 2
7
2
2
1 1
+ + + =
+ =
t e C e C t x
t e C e C t x
t t
t t

91
Example:
Solve the system
( )
( ) 3 0 3 2
2 0 3 4 4
2
2
2 1 2
1
2
2 1 1
= + + =
= + =
x ; t t x x x
x ; t x x x
'
'

Solution:
Let
|
|
\
|
=
1 1
4 1
A , ( )
|
|
\
|
+

=
3 2
2
3
2
4
t t
t
t F and
|
|
\
|
=
3
2
C .
The eigenvalues and eigenvectors are found to be
|
|
\
|
= =
1
2
3
1 1
e , and
|
|
\
|
= =
1
2
1
2 2
e , .
Let ( )
|
|
\
|

= =
1 1
2 2
2 1
e e Q . Then
|
|
\
|
2
1
4
1
2
1
4
1
1
Q and
|
|
\
|
= =
1 0
0 3
1
D AQ Q .

Let QY X = . Then ' QY X = .
From the equation ( ) t F AX X + = , ( ) C X = 0
QY=AQY + F(t) ,
Y=Q
1
AQY +Q
1
F(t)
( ) t G DY ' Y + = ,
where
G( ) ( )
|
|
|
|
\
|
+
+
= =

4
3
2
1
4
9
2
3
2
2
1
t t
t t
t F Q t and ( )
|
|
\
|
=
=
1
2
1
0 C Q Y .

Therefore,
( )
( ) 1 0
4
3
2
1
2 0
4
9
2
3
3
2
2
2 2
1
2
1 1
= + + =
= + + =
y ; t t y ' y
y ; t t y ' y

Solving these equations by integrating factor method, we have

92
( )
( )
4
3
2
1
4
7
4
3
2
1
4
5
2
2
2 3
1
+ =
+ + =
t e t y
t e t y
t
t

Hence,
( )
|
|
\
|
+
=
=
2 1
2 1
2
y y
y y
QY X

So, the solution is
( ) ( )
( )
t t
t t
e e t y y t x
e e y y t x
+ + = + =
+ = =
4
7
4
5
2
7
2
5
3 2
3 2
2 1 2
3
2 1 1

Example:
Find a solution to the system
4 9
1 2
2 1
+ =
+ =
x x
t sin x x

subject to the initial conditions ( ) 0 0
1
= x , and ( ) 1 0
2
= x .
Solution:
We have
|
|
\
|
+
|
|
\
|
|
|
\
|
=
|
|
\
|
4 0 9
1 0
2
1
2
1
t sin
x
x
x
x
, with
( )
( )
|
|
\
|
=
|
|
\
|
=
1
0
0
0
2
1
x
x
c .
Eigenvalues are 3 3
2 1
= = , , with eigenvectors
|
|
\
|
=
|
|
\
|
=
3
1
3
1
2 1
e , e .
Therefore
|
|
\
|
=
|
|
\
|
6
1
2
1
6
1
2
1
1
3 3
1 1
Q , Q , and as before , we obtain

( ) ( )
10
9
60 60
61
9
4
10 180 180
61
3 3
2
3 3
1
t sin e e
t x ,
t cos e e
t x
t t t t
+ = +
=

93
Appendix:

A few remarks on the Gauss-Jordan elimination method (by using
elementary row operations)

1. Suppose in solving a system of linear equations, we reduced the
augumented coefficient matrix to its reduced row echelon form as
follows:
|
|
|
\
|
7 1 0 0 0 0
5 0 3 1 0 0
3 0 4 0 2 1

Then x
2
,x
4
are the free variables and x
1
, x
3
, x
5
are the basic variables.
(They correspond to the column with the SPECIAL 1). Thus we set
x
2
= s
x
4
= t
and then solve for x
1
, x
3
, x
5
in terms of x
2
, x
4
as follows :
x
1
=3-2s-4t
x
2
=s
x
3
=5-3t
x
4
=t
x
5
=7

or
|
|
|
|
|
|
\
|
=
5
4
3
2
1
x
x
x
x
x
X =
|
|
|
|
|
|
\
|
7
0
5
0
3
+s
|
|
|
|
|
|
\
|
0
0
0
1
2
+t
|
|
|
|
|
|
\
|
0
1
3
0
4
=

2. Suppose the reduced row echelon form of the augumented cofficient
matrix of a system of linear equations is as follows:

|
|
|
\
|
0 0 0 0 0
9 0 1 0 0
7 0 0 0 1

Then
x
1
= 7
x
3
=9

94
There is no restriction on x
2
,x
4
, so they take any values, that is , they are
free.So ,we put
x
2
= s
x
4
= t
We can then write the solutions as follows:

|
|
|
|
|
\
|
+
|
|
|
|
|
\
|
+
|
|
|
|
|
\
|
=
|
|
|
|
|
\
|
=
1
0
0
0
0
0
1
0
0
9
0
7
4
3
2
1
t s
x
x
x
x
X ,
where s and t are parameters.

~END~

Linear Algebra Lecture Notes

Загружено:

Сведения о документе

Исходное описание:

Авторское право

Доступные форматы

Поделиться этим документом

Поделиться или встроить документ

Параметры публикации

Этот документ был вам полезен?

Это неприемлемый материал?

Авторское право:

Доступные форматы

Linear Algebra Lecture Notes

Загружено:

Авторское право:

Доступные форматы

1

Matrices and Determinants

Вам также может понравиться