MATRIX METHODS

SYSTEMS OF
LINEAR EQUATIONS
 System of Linear Equations

Determining the values of

x1, x2, .....xn, that
simultaneously satisfy a
set of equations
 System of Linear Equations
Simultaneous equations
f1(x1, x2, .....xn) = 0
f2(x1, x2, .....xn) = 0
.............
f3(x1, x2, .....xn) = 0
Methods will be for linear equations
a11x1 + a12x2 +...... a1nxn =c1
a21x1 + a22x2 +...... a2nxn =c2
..........

an1x1 + an2x2 +...... annxn =cn

 Mathematical Background
Matrix Notation
a11 a12 a13 ... a1n
a a22 a23 ... a2 n
A 21

. . . .
a am3 ... amn
m1 am2

a horizontal set of elements is called a row

a vertical set is called a column
first subscript refers to the row number
second subscript refers to column number
 a11 a12 a13 ... a1n
a a a ... a
A 21 22 23 2n

. . . .

a m1 a m 2 a m3 ... a mn
note
This matrix has m rows an n column. subscript

It has the dimensions m by n (m x n)

 Note the consistent
column 3 scheme with subscripts
denoting row,column
a11 a12 a13 ... a1n
a a22 a23 ... a2 n
A 21
row 2
. . . .
a am3 ... amn
m1 am2
Row vector: m=1
B b1 b2 ....... bn

Column vector: n=1 Square matrix: m = n

c1
c a11 a12 a13
2
C . A a21 a22 a23

. a31 a32 a33

cm
 The diagonal consist of the elements

a11 a22 a33 a11 a12 a13

A a21 a22 a23

a31 a32 a33

Symmetric matrix
Diagonal matrix
Identity matrix
Upper triangular matrix
Lower triangular matrix
Banded matrix
Symmetric Matrix

aij = aji for all is and js

5 1 2
A 1 3 7 Does a23 = a32 ?
2 7 8
Yes. Check the other elements
on your own.
Diagonal Matrix

A square matrix where all elements off

the main diagonal are zero

a11 0 0 0
0 a22 0 0
A
0 0 a33 0
0 0 0 a44

Identity Matrix

A diagonal matrix where all elements on

the main diagonal are equal to 1
1 0 0 0
0 1 0 0
A
0 0 1 0
0 0 0 1

The symbol [I] is used to denote the identify matrix.

Upper Triangle Matrix

Elements below the main diagonal are

zero
a11 a12 a13
A 0 a22 a23

0 0 a33
Lower Triangular Matrix

All elements above the main diagonal are

zero

5 0 0
A 1 3 0
2 7 8
Banded Matrix

All elements are zero with the exception

of a band centered on the main diagonal
a11 a12 0 0
a a22 a23 0
A 21
0 a32 a33 a34
0 0 a43 a44

 Matrix Operating Rules
Addition/subtraction
add/subtract corresponding terms
aij + bij = cij
Addition/subtraction are commutative
[A] + [B] = [B] + [A]
Addition/subtraction are associative
[A] + ([B]+[C]) = ([A] +[B]) + [C]
 Matrix Operating Rules
Multiplication of a matrix [A] by a scalar g
is obtained by multiplying every element of
[A] by g
ga11 ga12 . . . ga 1n
ga ga22 . . . ga2 n
21

. . . . . .
B g A
. . . . . .

. . . . . .

gam1 gam2 . . . gamn
 Matrix Operating Rules
The product of two matrices is represented as
[C] = [A][B]

n = column dimensions of [A]

n = row dimensions of [B]

N
c ij a ik b kj
k 1
 Simple way to check whether
matrix multiplication is possible
exterior dimensions conform to dimension of resulting matrix

[A] m x n [B] n x k = [C] m x k

interior dimensions
must be equal
 Matrix multiplication
If the dimensions are suitable, matrix
multiplication is associative
([A][B])[C] = [A]([B][C])
If the dimensions are suitable, matrix
multiplication is distributive
([A] + [B])[C] = [A][C] + [B][C]
Multiplication is generally not commutative
[A][B] is not equal to [B][A]
Inverse of [A]

A A 1
A
1
A I
Inverse of [A]
A A 1
A
1
A I

Transpose of [A]
a11 a21 . . . am1
a a22 . . . am2
12
. . . . . .
A t

. . . . . .

. . . . . .

a1n a2 n . . . amn
 Determinants

Denoted as det A or A

for a 2 x 2 matrix a b
ad bc
c d
a b
ad bc
c d
 Determinants
There are different schemes used to compute the determinant.

Consider cofactor expansion

- uses minor and cofactors of the matrix

Minor: the minor of an entry aij is the determinant of the

submatrix obtained by deleting the ith row and the jth
column

Cofactor: the cofactor of an entry aij of an n x n matrix

A is the product of (-1)i+j and the minor of aij
 Minor: the minor of an entry aij is the determinant of the
submatrix obtained by deleting the ith row and the jth
column.

Example: the minor of a32 for a 3x3 matrix is:

a 11 a 12 a 13

minor of a 32 a 21 a 22 a 23
a 31 a 32 a 33
For element
a32 the ith row
is row 3
 Minor: the minor of an entry aij is the determinant of the
submatrix obtained by deleting the ith row and the jth
column.

Example: the minor of a32 for a 3x3 matrix is:

a 11 a 12 a 13

minor of a 32 a 21 a 22 a 23
a 31 a 32 a 33

For element a32 the jth column is column 2

 Minor: the minor of an entry aij is the determinant of the
submatrix obtained by deleting the ith row and the jth
column.

Example: the minor of a32 for a 3x3 matrix is:

a11 a12 a13
minor of a32 a21 a22 a23

a31 a32 a33

a11 a12 a13

a11 a13
minor of a32 a21 a22 a23 a11a23 a13a21
a21 a23
a31 a32 a33
Cofactor: Aij, the cofactor of an entry aij of an n x n matrix
A is the product of (-1)i+j and the minor of aij

i.e. Calculate A31 for a 3x3 matrix

First calculate the minor a31

a11 a12 a13
a12 a13
minor of a31 a21 a22 a23 a12 a23 a22 a13
a22 a23
a31 a32 a33
Cofactor: Aij, the cofactor of an entry aij of an n x n matrix
A is the product of (-1)i+j and the minor of aij

i.e. Calculate A31 for a 3x3 matrix

First calculate the minor a31

a11 a12 a13
a12 a13
minor of a31 a21 a22 a23 a12 a23 a22 a13
a22 a23
a31 a32 a33
Cofactor: Aij, the cofactor of an entry aij of an n x n matrix
A is the product of (-1)i+j and the minor of aij

i.e. Calculate A31 for a 3x3 matrix

First calculate the minor a31

a11 a12 a13
a12 a13
minor of a31 a21 a22 a23 a12 a23 a22 a13
a22 a23
a31 a32 a33

A31 1
3 1
a12 a23 a22 a13
Minors and cofactors are used to calculate the determinant
of a matrix.

Consider an n x n matrix expanded around the ith row

A ai1 Ai1 ai 2 Ai 2 ......ain Ain (for any one value of i)

Consider expanding around the jth column

A a1 j A1 j a2 j A2 j ......anj Anj (for any one value of j)

 a11 a12 a13
a22 a23 a21 a23 a21 a22
D a21 a22 a23 a11 a12 a13
a32 a33 a31 a33 a31 a32
a31 a32 a33

det 1 a11 a12 a23 a22 a13 1 a12 a21a33 a23a31

11 1 2

1 a13 a21a32 a22 a32

1 3
 EXAMPLE
Calculate the determinant of the following 3x3 matrix.
First, calculate it using the 1st row (the way you probably
have done it all along).
Then try it using the 2nd row.

1 7 9
4 3 2

6 1 5
 Properties of Determinants
det A = det AT
If all entries of any row or column is zero,
then det A = 0
If two rows or two columns are identical,
then det A = 0
How to represent a system of
linear equations as a matrix

[A]{X} = {C}

where {X} and {C} are both column vectors

0.3 x1 0.52 x2 x3 0.01
0.5 x1 x2 1.9 x3 0.67
0.1x1 0.3 x2 0.5 x3 0.44

A{ X } {C}

0.3 0.52 1 x1 0.01

0 .5
1 1.9 x2 0.67

0.1 0.3 0.5 x 0.44
3
 Matrix Methods
Cramers Rule
Gauss elimination
Matrix inversion
LU decomposition
a11 a12 a13
A a21 a22 a23

a31 a32 a33
 Cramers Rule
Not efficient for solving large numbers of
linear equations
Useful for explaining some inherent
problems associated with solving linear
equations.

a11 a12 a13 x1 b1

a
a22 a23 x2 b2 A x b
21
a31 a32 a33 x b
3 3
 Cramers Rule
b1 a12 a13 a11 b1 a13
1 1
x1 b2 a22 a23 x2 a21 b2 a23
A A
b3 a32 a33 a13 b3 a33

to solve for
xi - place {b} in
the ith column
 Cramers Rule
b1 a12 a13 a11 b1 a13
1 1
x1 b2 a22 a23 x2 a21 b2 a23
A A
b3 a32 a33 a13 b3 a33

to solve for
xi - place {b} in
a11 a12 b1 the ith column
1
x3 a21 a22 b2
A
a13 a32 b3
 Cramers Rule
b1 a12 a13 a11 b1 a13
1 1
x1 b2 a22 a23 x2 a21 b2 a23
A A
b3 a32 a33 a31 b3 a33

a11 a12 b1 to solve for

1 xi - place {b} in
x3 a21 a22 b2
A the ith column
a31 a32 b3
 EXAMPLE
Use of Cramers Rule
2 x1 3x2 5
x1 x2 5

2 3 x1 5
1 1 x 5
2
 Elimination of Unknowns
( algebraic approach)
a11x1 a12 x2 c1
a 21x1 a 22 x2 c2

a11x1 a12 x2 c1 a21

a 21x1 a 22 x2 c2 a11

a 21a11x1 a 2`1a12 x2 a 2`1c1 SUBTRACT

a 21a11x1 a11a22 x2 a11c2
 Elimination of Unknowns
( algebraic approach)
a21a11 x1 a2`1a12 x2 a2`1c1 SUBTRACT
a21a11 x1 a11a22 x2 a11c2

a11a21 x2 a22 a11 x2 c1a21 c2 a11

a21c1 a11c2 NOTE: same result as

x2 Cramers Rule
a12 a21 a22 a11

a22 c1 a12 c2
x1
a11a22 a12 a21
 Gauss Elimination
One of the earliest methods developed for
solving simultaneous equations
Important algorithm in use today
Involves combining equations in order to
eliminate unknowns
Two Phases of Gauss Elimination
a11 a12 a13 | c1
a Forward
a 22 a23 | c2
21
Elimination
a31 a 32 a33 | c3
Note: the prime indicates
a11 a12 a13 | c1 the number of times the
0 element has changed from
'
a 22 '
a 23 | c2'
'' ''
the original value.
0 0 a 33 | c3
Two Phases of Gauss Elimination
a11 a12 a13 | c1
0 '
a22 '
a23 | c2'
'' ''

0 0 a33 | c3 Back substitution
c3''
x3 ''
a33

x2
c '
2 a123 x3
'
a22
c1 a12 x2 a13 x3
x1
a11
 Matrix Inversion
[A] [A] -1 = [A]-1 [A] = I
One application of the inverse is to solve
several systems differing only by {c}
[A]{x} = {c}
[A]-1[A] {x} = [A]-1{c}
[I]{x}={x}= [A]-1{c}
One quick method to compute the inverse is
to augment [A] with [I] instead of {c}
Graphical Depiction of the Gauss-Jordan
Method with Matrix Inversion
A I
a11 a12 a13 1 0 0
a Note: the superscript
a22 a23 0 1 0 -1 denotes that
21
a31 a32 a33 0 0 1 the original values
1 0 0 a111 a121 a131 have been converted
1 1 1 to the matrix inverse,
0 1 0 a21 a22 a23
not 1/aij
0 0 1 1
a31 1
a 32 1
a33

I A 1
LU Decomposition Methods
 LU Decomposition
LU Decomposition is another method
to solve a set of simultaneous linear
equations
 LU Decomposition
Method
For most non-singular matrix [A] that one could conduct
forward elimination steps, one can always write it as

[A] = [L][U]
where
[L] = lower triangular matrix
[U] = upper triangular matrix
How does LU Decomposition work?
 LU Decomposition
How can this be used?

Given [A][X] = [C]

1. Decompose [A] into [L] and [U]
2. Solve [L][Z] = [C] for [Z]
3. Solve [U][X] = [Z] for [X]
Method: [A] Decompose to [L] and [U]

1 0 0 u11 u12 u13

A LU 21 1 0 0
u 22 u 23
31 32 1 0 0 u 33

[U] is the same as the coefficient matrix at

the end of the forward elimination step.
[L] is obtained using the multipliers that were
used in the forward elimination process
 Does [L][U] = [A]?

1 0 0 25 5 1
LU 2.56 1 0 0 4.8 1.56 ?
5.76 3.5 1 0 0 0.7
Using LU Decomposition to solve
Solve the following set of 25 5 1 x1 106.8
linear equations using LU 64 8 1 x 177.2
Decomposition
2
144 12 1 x3 279.2

Using the procedure for finding the [L] and [U] matrices

1 0 0 25 5 1
A LU 2.56 1 0 0 4.8 1.56
5.76 3.5 1 0 0 0.7
 Example

Set [L][Z] = [C] 1 0 0 z1 106.8

2.56 1 0 z 2 177.2

5.76 3.5 1 z 3 279.2

Solve for [Z] z1 10

2.56 z1 z 2 177.2
5.76 z1 3.5 z 2 z 3 279.2
 Example
Complete the forward substitution to solve for [Z]

z1 106.8
z 2 177.2 2.56 z1 z1 106.8
177.2 2.56106.8
96.2
Z z2 96.21
z3 279.2 5.76 z1 3.5 z 2 z3 0.735
279.2 5.76106.8 3.5 96.21
0.735
 Example
Set [U][X] = [Z]
25 5 1 x1 106.8
0 4.8 1.56 x 96.21
2
0 0 0.7 x3 0.735

Solve for [X] The 3 equations become

25a1 5a2 a3 106.8
4.8a2 1.56a3 96.21
0.7 a3 0.735
 Example
Substituting in a3 and using the
From the 3 equation
rd
second equation

0.7 a3 0.735 4.8a2 1.56a3 96.21

0.735 96.21 1.56a3
a3 a2
0.7 4.8
a3 1.050 96.21 1.561.050
a2
4.8
a2 19.70
 Example
Substituting in a3 and a2 using Hence the Solution Vector is:
the first equation
25a1 5a2 a3 106.8 a1 0.2900
106.8 5a2 a3 a 19.70
a1
25
2
106.8 519.70 1.050
a3 1.050

25
0.2900
 LU Decomposition Techniques
Doolittles Method
Choleskys Method
Crouts Method
Doolittles method ( L has ones on the
diagonal)
Choleskys method (The diagonal terms
are the same value for the L and U
matrices)
Crouts reduction (U has ones
on the diagonal)
 LU Decomposition (Doolittles method)

Matrix decomposition

1 0 0 0 u11 u12 u13 u14 a11 a12 a13 a14

l 0 0 u 24 a 21 a 22 a 24
21 1 0 u 22 u 23

a 23
l31 l32 1 0 0 0 u 33 u 34 a 31 a 32 a 33 a 34

l 41 l 42 l 43 1 0 0 0 u 44 a 41 a 42 a 43 a 44
 Doolittes method
The method alternates from solving from the
upper triangular to the lower triangular

a 12
l11 a 11 l11u12 a 12 u12
l11
l 21 a 21 a 13
l11u13 a 13 u13
l31 a 31 l11
a 14
l 41 a 41 l11u14 a 14 u14
l11
 Example
The matrix is broken into a lower and upper
triangular matrices.

3 1 2 1 0 0 3 1 2
1 2 3 1 / 3 1 0 0 7 / 3 7 / 3

2 2 1 2 / 3 4 / 7 1 0 0 1
 Choleskys method
The Cholesky decomposition is used on
symmetric positive definite matrix:
l11 0 0 0 u11 u12 u13 u14 a 11 a12 a13 a 14
l 0 0 u 24 a 21 a 22 a 24
21 l 22 0 u 22 u 23

a 23
l31 l32 l33 0 0 0 u 33 u 34 a 31 a 32 a 33 a 34

l 41 l 42 l 43 l 44 0 0 0 u 44 a 41 a 42 a 43 a 44

where, lii = uii

 Choleskys Method
The method does not alternate but does it from the
diagonal out.
l11 u11 l11 u11
a 21 a12
l 21 u12
u11 l11
a 31 a 13
l31 u13
u11 l11
a 41 a14
l 41 u14
u11 l11
 Choleskys Method
The second row in.

l 22 u 22 a 22 l 21u12 l 22 u 22 a 22 l 21u12
a 32 l31u12 a 23 l 21u13
l32 u 23
u 22 l 22
a 42 l 41u12 a 24 l 21u14
l 42 u 24
u 22 l 22
 Example
The matrices can contain imaginary values.

2 1 0 1.414 0 0 1.414 0.707 0

1 2 1 0.707 1.225 0 0 1 .225 0 .817

0 1 2 0 0.817 1.155 0 0 1.155
 Crout Decomposition
Method involves two major steps
forward elimination
back substitution
Efforts in improvement focused on
development of improved elimination
methods
One such method is Crout decomposition
 Crout Decomposition
Represents and efficient algorithm for decomposing [A]
into [L] and [U]

l11 0 0 1 u12 u13 a11 a12 a13

l l22 0 0 1 u23 a 21 a 22 a 23
21
l31 l32 l33 0 0 1 a 31 a 32 a 33
 l11 0 0 1 u12 u13 a11 a12 a13
l l22 0 0 1 u23 a 21 a 22 a 23
21
l31 l32 l33 0 0 1 a 31 a 32 a 33

Recall the rules of matrix multiplication.

The first step is to multiply the rows of [L] by the

first column of [U]

a11 l11 1 0 0 0 0 l11 Thus the first

column of [A]
a 21 l21 is the first column
a 31 l31 of [L]
 l11 0 0 1 u12 u13 a11 a12 a13
l l22 0 0 1 u23 a 21 a 22 a 23
21
l31 l32 l33 0 0 1 a 31 a 32 a 33

Next we multiply the first row of [L] by the column

of [U] to get

l11 a11
l11u12 a12
l11u13 a13
l11 a11
l11 0 0 1 u12 u13 a11 a12 a13
l11u12 a12 l
21
l22 0

0 1

u23 a 21 a 22

a 23

l31 l32 l33 0 0 1 a 31 a 32 a 33
l11u13 a13

a12
u12
l11
a13 Once the first row of [U] is established
u13
l11 the operation can be represented concisely

a1 j
u1 j for j 2 ,3,..... n
l11