Siri e-Kuliah PPPJJ, USM

JIM 201
Linear Algebra
Dr Siti Ainor Mohd Yatim
Chapter 2
Solving Linear Systems
2.1 Echelon Form of a Matrix

We l e a d

Objectives
At the end of this section, you should be able to
Represent a system of equations in augmented matrix or coefficient
matrix
Use the three elementary row operations, write these operations in
notation form
Recognize matrix in a row echelon form and in reduced row echelon
form
Solve a system of linear equations using the Gauss elimination
method and Gauss-Jordan elimination method
Deduce whether a system of linear equations has a unique solution
or infinitely many solutions or no solution
Solve a homogeneous system of linear equations and determine
whether the system has a trivial or nontrivial solution

We l e a d

Topics

Preliminaries
Echelon Form of a Matrix
Elementary Matrices; Finding A1
Equivalent Matrices

We l e a d

Topics

Preliminaries
Echelon Form of a Matrix
Elementary Matrices; Finding A1
Equivalent Matrices

We l e a d

Preliminaries
The primary goal is to deal with loose ends from the
first set of lectures
Develop systematic procedure for solving linear systems
that works directly with the matrix form
Establish conditions for a solution to a linear system to exist
Establish conditions for a unique solution to a linear system
Re-examine solutions to homogeneous systems
Develop a systematic procedure for computing A1
Discover equivalent conditions to nonsingularity

We l e a d

Topics

Preliminaries
Echelon Form of a Matrix
Elementary Matrices; Finding A1
Equivalent Matrices

We l e a d

Echelon Form of a Matrix

Solve a system using elimination of
variables
x1
x2
x1 2x2
3x1
x2

2x3

x3

x3

x1 x2
0 3x2
0 2x2

2x 3
x3
5x 3

x1 x2
0 6x2
0 0

2x3
2x3
13x3

1 x1
5 0

3 0

x2
3x2
2x2

1 x1 x2
4 0 6x2
6 0 6x2
1 x1
x2

8 0 6x2
26 0 0

2x3

x3
5x3

2x 3
2x 3
15x 3
2x3
2x3

x3

1
4
6

8
18
1
8
2

We l e a d

Echelon Form of a Matrix

Continuing the process on the previous slide gives x1
1, x2 2 and x3 2

In matrix form, started with

and transformed it into

1 1 2 x1

0 1 1 3 x2
0 0 1 x3

4 3
2

Row echelon form

1 1 2

1 2 1
3 1 1
1
0
0

x1

x2
x3

5
3

0 0 x 1
1

1 0 x 2 2
0 1 x 3 2

Reduced row echelon form

We l e a d

Echelon Form of a Matrix

Soon, will express the process as

Pivot Columns

1 1 2 1 1 1 2
1 2 1 5
0 3 1

1 1 3 0 2 5
3

1 1
2
1
4 0 3
1

0 0 13 3
3

Pivots

1 1 2 1 1 0 0 1
0 1 0 2
0 1 1 3 4 3

0 0 1 2 0 0 1 2

1
4

26 3

We l e a d

Echelon Form of a Matrix

Defn - An m x n matrix A is said to be in reduced row
echelon form if it has the following properties
a) If any rows consist of entirely zeros, they are at the bottom
of the matrix
b) If a row does not consist entirely of zeros, the first nonzero
entry in that row is 1
c) If rows i and i + 1 are two successive rows which do not
consist entirely of zeros, then the first nonzero entry of row i
+ 1 is to the right of the first nonzero entry of row i
d) If a column contains the first nonzero entry of some row,
then all the other entries in that column are zero

Defn - If an m x n matrix A satisfies properties (a), (b)

and (c), it is said to be in row echelon form

We l e a d

Echelon Form of a Matrix

Examples - Row Echelon Form

1
0
0
0
0

5
1
0
0
0

0
0
0
0
0

2 2 4
3 4 8
1 7 2

0 0 0
0 0 0

0
0
0
0
0

0
0
0
0
0

1
0
0
0
0

3
0
0
0
0

1
0
0
0

5 7
1 2
0 1
0 0
0 0

0
1
0
0
9

3
2

1
0

0
0
1
0

0
0

We l e a d

Echelon Form of a Matrix

Examples - Reduced Row Echelon Form

1
0
0
0

0
1
0
0

0
0
1
0

0
0

1
0
0
0
0

0
1
0
0
0

1 2 0 0 1

0 0 1 2 3
0 0 0 0 0

0
0
0
0
0

0 2 4

0 4 8
1 7 2

0 0 0
0 0 0

We l e a d

Echelon Form of a Matrix

Examples - Not Reduced Row Echelon
Form

1
0
0
0

1 0 3 4
0 2 2 5
0 0 1 2

1 2 0 4

0 0 0 0
0 0 1 3
0 3
1 2
1 2
0 0

4
5
2

1
0
0
0

2 3 4
1 2 5
0 1 2

0 0 0

We l e a d

Echelon Form of a Matrix

Comments - Row Echelon Form
Can define column echelon form and reduced
column echelon form in a similar manner.
Basically, they are just the transpose of the
corresponding row forms
Close connection between echelon forms and
the solution of linear equations

We l e a d

Echelon Form of a Matrix

Defn - An elementary row operation on
a matrix A is any one of the following
operations
a)
b)
c)

Interchange rows i and j of matrix A

Multiply row i of A by c 0
Add c times row i of A to row j of A, i j

We l e a d

Echelon Form of a Matrix

Comment
The first elementary row operation, exchange two
rows, is needed to deal with zeros in the pivot
positions. For example, consider

2 x2 1 0 2

3x1 4 x2 2 3 4

x1
x2

1

2

2 1
4 2

3x1 4 x2 2 3 4

2 x2 1 0 2

x1
x2

2

1

4 2
2 1

No multiple of the first row will remove the 3 in the

second row. The solution with equations is just to
exchange the equations. The solution with matrices is
just to exchange the rows.

This problem can also occur as an intermediate step in

solving a larger system and has the same fix

We l e a d

Echelon Form of a Matrix

Comment - The elementary row
operations on an m x n matrix A can be
accomplished by multiplying A by a
specially chosen m x m matrix. This is
useful for proofs, but not for practical
computations since too many
operations are required.

We l e a d

Echelon Form of a Matrix

Elementary Row Operations
Interchange rows i and j of matrix A

0 1 0
1 0 0
0 0 1

a11 a12 a13 a14 a21 a22 a23 a 24

a21 a22 a23 a24 a11 a12 a13 a 14

a31 a32 a33 a34 a31 a32 a33 a 34

In general, rows i and j of A can be interchanged by

multiplying A by the m x m matrix E defined as
1 if k i and k j
e pq 0 except for ekk
0 if k i or k j

and eij 1, e ji 1

E is a special case of a permutation matrix

We l e a d

Echelon Form of a Matrix

Elementary Row Operations

Multiply row i of A by c 0

1 0 0

0 c 0
0 0 1

a11 a12 a13 a14 a11 a12 a13 a 14

a21 a22 a23 a24 ca21 ca22 ca23 ca 24

a31 a32 a33 a34 a31 a32 a33 a 34

In general, row i of A can be multiplied by c by

multiplying A by the m x m matrix E defined as

e pq

0 if p q
1 if p q i
c if p q i

We l e a d

Echelon Form of a Matrix

Elementary Row Operations

Add c times row i of A to row j of A, i j
1 0 0 a11 a12 a13 a14

0 1 0 a21 a22 a23 a24

0 c 1 a
a
a
a
32
33
34

31

a11
a12
a13
a14

a21
a22
a23
a24
ca a

ca

a
ca

a
ca

a
21
31
22
32
23
33
24
34

In general, can add c times row i to row j of A

by multiplying A by the m x m matrix E defined as

e pq

c p j and q i
1 pq
0 otherwise

We l e a d

Echelon Form of a Matrix

Comments
The appropriate elementary matrix E may be
obtained by performing the desired operation on the
rows of the m x m identity matrix Im
Can define elementary matrices F to perform
elementary column operations by doing the
corresponding operations on the columns of the
identity matrix. The matrices F are applied by post
multiplying, i.e. AF

We l e a d

Echelon Form of a Matrix

Defn - An m x n matrix A is row equivalent to an m x n
matrix A if A can be obtained by applying a finite
sequence of elementary row operations to A
The basic method for solving the system AX B is to
apply elementary row operations to the augmented
matrix [ A:B ] to get a new augmented matrix [ C:D ]
such that the CX D is easy to solve
If [ C:D ] is in row echelon form, the method is called
Gaussian Elimination
If [ C:D ] is in reduced row echelon form, the method is called
Gauss-Jordan Elimination

We l e a d

Echelon Form of a Matrix

Comments
In first set of lectures, we argued that the solution to a
system of linear equations is unchanged if we do any of
the following
a) Interchange two equations
b) Multiply a given equation by a nonzero constant
c) Add a constant times one equation to another equation

When the system is viewed in terms of matrices, the

above operations become elementary row operations on
the augmented matrix.

We l e a d

Echelon Form of a Matrix

Theorem - Every nonzero m x n matrix A is row
equivalent to a matrix in row echelon form
Proof Let A = [ aij ] be a nonzero m x n matrix. Find
the first (from the left) nonzero column, s, then find
the row, r, containing the first (from the top) nonzero
entry in column s.
Define a matrix B = [ bij ] as follows. If r = 1,
B = A.
If r 1, interchange rows 1 and r of A to get B. Then
b1s 0. Multiply the first row of B by 1/b1s to get a
matrix C = [ cij ] such that c1s = 1. Consider the other
entries in column s. If chs 0, 2 h m, add chs
times the first row to row h.

We l e a d

Echelon Form of a Matrix

Proof (continued) The elements of column s in rows 2, 3,
, m of C are now zero. Call the
resulting matrix D. Consider the (m1) x
n submatrix A1 of D obtained by
deleting the first row of D. Repeat the
above process with A1 in place of A. QED
Continuing this process yields a matrix
in row echelon form that is row
equivalent to A.

We l e a d

Echelon
Form
of
a
Matrix
Theorem
- Every nonzero m x n matrix A
is row equivalent to a matrix in reduced
row echelon form
Proof - Let A be a nonzero m x n matrix.
Apply the method of the preceding
theorem to obtain a matrix H in row
echelon form that is row equivalent to
A. Suppose that rows 1, 2, , r of H are
nonzero and that the leading ones in
these rows occur in columns c1, c2, ,
c , where
c <c <L<c.

We l e a d

Echelon Form of a Matrix

Proof (continued) Starting with the last nonzero row of H, add suitable
multiples of this row to all rows above it to create
zeros in column cr above the one in row r. Repeat
this process with rows r 1, r 2, , 2 to make all
entries above a leading one equal to zero. This gives
a matrix in reduced row echelon form that is row
equivalent to H, which is row equivalent to A.
QED

Note: Can show that the reduced row echelon matrix

is unique

We l e a d

Echelon Form of a Matrix

Theorem - Let A and C be m x n matrices. If the
system AX b has an augmented matrix [ A:b ] which
is row equivalent to an augmented matrix [ C:d ],
then the systems AX b and CX d have the same
solution.
Proof - The elementary row operations on the
augmented matrices correspond to the following
operations on the equations, which leave the solution
unchanged.
a) Interchange two equations
b) Multiply a given equation by a nonzero constant
c) Add a constant times one equation to another equation

So, the two systems have the same solution

QED

We l e a d

Echelon Form of a Matrix

Corollary - If A and C are row equivalent m x n
matrices, then the homogeneous systems AX 0
and CX 0 are equivalent.
Proof - Form the augmented matrices [ A:0 ] and
[ C:0 ]. Since A and C are row equivalent, [ A:0 ] and
[ C:0 ] are row equivalent. By the preceding theorem,
AX 0 and CX 0 have the same solution.
QED

We l e a d

Echelon Form of a Matrix

Comments
If Gauss-Jordan reduction is used, the solution is just
read off. If Gaussian elimination is used, the solution
must be found by back substitution
Although we have talked about the solution, we have
not established any conditions that ensure that a
solution will exist or that it will be unique

We l e a d

Echelon Form of a Matrix

Example

C
:
D

1
0
0
0

2
1
0
0

3
2
1
0

4
3
2
1

5
1
3
2

6
7
7

9
Set x5 = r and backsolve

x1 2 x2 3x3 4 x4 5x5 6
x2 2 x3 3x4 x5 7
x3 2 x4 3x5 7
x4 2 x5 9

x1 1 10r
x2 2 5r
x3 11 r
x4 9 2r

Infinite number of solutions

We l e a d

Echelon Form of a Matrix

Example

C : D

1 2 3 4 5

0 1 2 3 6
0 0 0 0 1

The last equation is 0 x1+0 x2+0 x3+0 x4

1, which cannot be satisfied
Clearly, several things can happen and
a careful and organized analysis of the
problem is required

We l e a d

Echelon Form of a Matrix

Homogeneous Systems
Homogeneous system, AX 0, of m equations in n
unknowns occurs often applications
Example
1 0 0 0 2 0

0
0
1
0
3
0

0 0 0 1 4 0

0
0
0
0
0
0

Set x5 r, then determine x4 4r, x3 3r and x1

2r. There is no equation determining x2, so set x2 =
s. Note that the set of solutions is
determined/controlled by the pair of independent
parameters r and s. More about this later

We l e a d

Echelon Form of a Matrix

Homogeneous Systems
Theorem - A homogeneous system of m linear
equations in n unknowns always has a nontrivial
solution if m < n, i.e. if the number of unknowns exceeds
the number of equations
Proof - Let B be a matrix in reduced row echelon form
that is row equivalent to A. The homogeneous systems
AX 0 and BX 0 are equivalent. Let l be the number
of nonzero rows of B. Then l m. Since m < n, then l <
n. So, we are solving l equations in n unknowns and can
solve for l of the unknowns in terms of the remaining n
l unknowns, which can take any values. So, BX 0 and
thus AX 0 have nontrivial solutions
QED

We l e a d

Thank you