Schedule

Week Date Topic Classification of Topic

1 9 Feb. 2010
Introduction to
Numerical Methods
and Type of Errors
Measuring errors, Binary representation,
Propagation of errors and Taylor series
2 14 Feb. 2010
Nonlinear Equations Bisection Method
3 21 Feb. 2010
Newton-Raphson Method
4 28 Feb. 2010 Interpolation
Lagrange Interpolation
5 7 March 2010
Newton's Divided Difference Method
6 14 March 2010
Differentiation Newton's Forward and Backward
Divided Difference
7 21 March 2010 Regression
Least squares
8 28 March 2010
Systems of Linear
Equations
Gauss-Jacobi
9 11 April 2010
Gauss- Seidel
10 18 April 2010
Integration Composite Trapezoidal and Simpson
Rules
11 25 April 2010
Ordinary Differential
Equations
Euler's Method
12 2 May 2010
Runge-Kutta 2
nd
and4
th
order Method
Gauss-Jacobi
Iterative or approximate methods
provide an alternative to the
elimination methods. The Gauss-Jacobi
method is the most commonly used
iterative method.
The system [A]{X}={B} is reshaped
by solving the first equation for x
1
, the
second equation for x
2
, and the third
for x
3
, and n
th
equation for x
n
.
Gauss-Jacobi Method
Algorithm
A set of n equations and n unknowns:
1 1 3 13 2 12 1 11
... b x a x a x a x a
n n
= + + + +
2 3 23 2 22 1 21
... b x a x a x a x a
n 2n
= + + + +
n n nn n n n
b x a x a x a x a = + + + + ...
3 3 2 2 1 1
. .
. .
. .
If: the diagonal elements are
non-zero
Rewrite each equation solving
for the corresponding unknown
ex:
First equation, solve for x
1
Second equation, solve for x
2
Gauss-Jacobi Method
Algorithm
Rewriting each equation
11
1 3 13 2 12 1
1
a
x a x a x a b
x
n n

=

nn
1 n 1 n , n 2 2 n 1 1 n n
n
1 n , 1 n
n n , 1 n 2 n 2 n , 1 n 2 2 , 1 n 1 1 , 1 n 1 n
1 n
22
n n 2 3 23 1 21 2
2
a
x a x a x a b
x
a
x a x a x a x a b
x
a
x a x a x a b
x

=

=

=




From equation 1
From equation 2
From equation n-1
From equation n
Gauss-Jacobi Method
Solve for the unknowns
Assume an initial guess for [X]
(
(
(
(
(
(

n
- n
2
x
x
x
x
1
1

Use rewritten equations to solve for

each value of x
i
.
Important: Remember to use the
most recent value of x
i
. Which
means to apply values calculated to
the calculations remaining in the
current iteration.
3
3 3 3
2
2 1 2
1
1 1 1
equations 3 3 For
c
z b x a d
z
b
z c x a d
y
a
z c y b d
x

=

=

=

Now we can start the solution process by

choosing guesses for the xs. A simple way to
obtain initial guesses is to assume that they
are zero. These zeros can be substituted into
x
1
equation to calculate a new x
0
=d
1
/a
1
.
First iteration
3
3 3 3
2
2 2 2
1
1 1 1
equations 3 3 For
c
y b x a d
z
b
z c x a d
y
a
z c y b d
x

=

=

=

2
0 3 0 3 3
1
3
0 2 0 2 2
1
1
0 1 0 1 1
1
equations 3 3 For
b
y b x a d
z
c
z c x a d
y
a
z c y b d
x

=

=

=

Applying the initial guess and solving for x, y,

and z
(
(
(

=
(
(
(

0
0
0
z
y
x
z
y
x
Gauss-Jacobi Method
Calculate the Absolute Relative Approximate Error
100

= e
new
i
old
i
new
i
i
a
x
x x
So when has the answer been found?
The iterations are stopped when the
absolute relative approximate error is less
than a prespecified tolerance for all
unknowns.
Convergence Criterion for Gauss-
Jacobi Method
The Gauss-Jacobi method has two
fundamental problems as any iterative
method:
It is sometimes nonconvergent, and
If it converges, converges very slowly.
Gauss-Jacobi Method
Why?
The Gauss-Jacobi Method allows the user to control
round-off error.
Elimination methods such as Gaussian Elimination
and LU Decomposition are prone to prone to round-off
error.
Also: If the physics of the problem are understood, a
close initial guess can be made, decreasing the
number of iterations needed.
Gauss-Jacobi Method: Example 1
The upward velocity of a rocket is given at three different
times
Time Velocity
5 106.8
8 177.2
12 279.2
The velocity data is approximated by a polynomial as:
( ) 12. t 5 ,
3 2
2
1
s s + + = a t a t a t v
( ) s t ( ) m/s v
Table 1 Velocity vs. Time data.
Gauss-Jacobi Method: Example 1
(
(
(

=
(
(
(

(
(
(

3
2
1
3
2
3
2
2
2
1
2
1
1
1
1
v
v
v
a
a
a

t t
t t
t t
3
2
1
Using a Matrix template of the form
The system of equations becomes
(
(
(

=
(
(
(

(
(
(

2 . 279
2 . 177
8 . 106
1 12 144
1 8 64
1 5 25
3
2
1
a
a
a

Initial Guess: Assume an initial guess of
(
(
(

=
(
(
(

5
2
1
3
2
1
a
a
a
Gauss-Jacobi Method: Example 1
Rewriting each equation
(
(
(

=
(
(
(

(
(
(

2 . 279
2 . 177
8 . 106
1 12 144
1 8 64
1 5 25
3
2
1
a
a
a

25
5 8 . 106
3 2
1
a a
a

=
8
64 2 . 177
3 1
2
a a
a

=
1
12 144 2 . 279
2 1
3
a a
a

=
Gauss-Jacobi Method: Example 1
Applying the initial guess and solving for a
i
(
(
(

=
(
(
(

5
2
1
3
2
1
a
a
a
6720 . 3
25
) 5 ( ) 2 ( 5 8 . 106
1
=

= a
( ) ( )
525 . 13
8
5 1 64 2 . 177
2
=

= a
( ) ( )
2 . 111
1
2 12 1 144 2 . 279
3
=

= a
Initial Guess
Gauss-Jacobi Method: Example 1
% 76 . 72 100
6720 . 3
0000 . 1 6720 . 3
1
a
=

= e x
% 212 . 85 100
525 . 13
0000 . 2 525 . 13
2
a
=

= e x
% 5 . 95 100
2 . 111
0000 . 5 2 . 111
3
a
=

= e x
Finding the absolute relative approximate error
100

= e
new
i
old
i
new
i
i
a
x
x x
At the end of the first iteration
The maximum absolute
relative approximate error is
95.5%
(
(
(

=
(
(
(

2 . 111
525 . 13
6720 . 3
3
2
1
a
a
a
Gauss-Jacobi Method: Example 1
Iteration #2
Using
(
(
(

=
(
(
(

2 . 111
525 . 13
6720 . 3
3
2
1
a
a
a
( )
881 . 2
25
2 . 111 525 . 13 5 8 . 106
a
1
=

=
( )
126 . 21
8
2 . 111 672 . 3 64 2 . 177
a
2
=

=
( ) ( )
868 . 411
1
525 . 13 12 672 . 3 144 2 . 279
a
3
=

=
from iteration #1
the values of a
i
are found:
Gauss-Jacobi Method: Example 1
Finding the absolute relative approximate error
% 455 . 227 100
881 . 2
6720 . 3 881 . 2
1
a
=

= e x
% 02 . 164 100
126 . 21
525 . 13 126 . 21
2
a
=

= e x
% 05 . 127 100
686 . 411
2 . 111 868 . 411
3
a
=

= e x
At the end of the second iteration
(
(
(

=
(
(
(

868 . 411
126 . 21
881 . 2
3
2
1
a
a
a
The maximum absolute
relative approximate error is
227.455%
Iteration a
1
a
2
a
3
1
2
3
4
3.6720
-2.881
8.2972
-42.545
72.767
227.455
134.722
119.502
13.525
21.126
44.573
162.67
85.212
164.02
147.396
127.403
111.2
-411.868
947.576
-1450.47
95.5
127.05
143.465
165.329
Gauss-Jacobi Method: Example 1
(
(
(

=
(
(
(

0857 . 1
690 . 19
29048 . 0
a
a
a
3
2
1
Repeating more iterations, the following values are obtained
%
1
a
e
%
2
a
e
%
3
a
e
Notice The relative errors are not decreasing at any significant rate
Also, the solution is not converging to the true solution of
Gauss-Jacobi Method: Pitfall
What went wrong?
Even though done correctly, the answer is not converging to the
correct answer
This example illustrates a pitfall of the Gauss-Jacobi method: not all
systems of equations will converge.
Is there a fix?
One class of system of equations always converges: One with a diagonally
dominant coefficient matrix.
Diagonally dominant: [A] in [A] [X] = [C] is diagonally dominant if:

=
=
>
n
i j
j
ij ii
a a
1

=
=
>
n
j
j
ij
a a
i
1
ii
for all i and
for at least one i
Gauss-Jacobi Method: Pitfall
| |
(
(
(

=
1 16 123
1 43 45
34 81 . 5 2
A
Diagonally dominant: The coefficient on the diagonal must be at least
equal to the sum of the other coefficients in that row and at least one row
with a diagonal coefficient greater than the sum of the other coefficients
in that row.

(
(
(

=
129 34 96
5 53 23
56 34 124
] B [
Which coefficient matrix is diagonally dominant?
Most physical systems do result in simultaneous linear equations that
have diagonally dominant coefficient matrices.
Gauss-Jacobi Method: Example 2
Given the system of equations
20 2 3 8 z y x = +
z y x 33 11 4 =
35 12 3 6 = + + z y x
(
(
(

=
(
(
(

0
0
0
z
y
x
With an initial guess of
The coefficient matrix is:
| |
(
(
(

=
12 3 6
1 11 4
2 3 8
A
Will the solution converge using the
Gauss-Jacobi method?
Gauss-Jacobi Method: Example 2
Checking if the coefficient matrix is diagonally dominant
5 1 4 11 11
23 21 22
= + = + > = = a a a
9 3 6 12 12
32 31 33
= + = + > = = a a a
5 2 3 8 8
13 12 11
= + = + > = = a a a
The inequalities are all true and at least one row is strictly greater than:
Therefore: The solution should converge using the Gauss-Jacobi Method
| |
(
(
(

=
12 3 6
1 11 4
2 3 8
A
Gauss-Jacobi Method: Example 2
(
(
(

=
(
(
(

(
(
(

35
33
20
z
y
x

12 3 6
1 11 4
2 3 8
Rewriting each equation
8
2 3 20 z y
x
+
=
11
4 33 z x
y
+
=
12
3 6 35 y x
z

=
With an initial guess of
(
(
(

=
(
(
(

0
0
0
z
y
x
( ) ( )
5 . 2
12
0 2 0 3 20
x
1
=
+
=
( )
3
11
0 0 4 33
y
1
=
+
=
( ) ( )
9166667 . 2
13
0 3 0 6 35
z
1
=

=
Gauss-Jacobi Method: Example 2
The absolute relative approximate error
% 00 . 100 100
5 . 2
0 . 0 5 . 2
1
=

= e
a
% 00 . 100 100
0 . 3
0 . 0 0 . 3
2
=

= e
a
% 100 100
916666 . 2
0 . 0 916666 . 2
3
=

= e
a
The maximum absolute relative error after the first iteration is 100%
Gauss-Jacobi Method: Example 2
(
(
(

=
(
(
(

916666 . 0
3560606 . 2
895833 . 2
2
2
2
z
y
x
After Iteration #1
( ) ( )
895833 . 2
8
9166667 . 2 2 3 3 20
2
=
+
= x
( )
3560606 . 2
11
9166667 . 2 5 . 2 4 33
2
=
+
= y
( ) ( )
916666 . 0
12
3 3 5 . 2 6 35
2
=

= z
Substituting the values into
the equations
After Iteration #2
(
(
(

=
(
(
(

9166667 . 2
0000 . 3
5000 . 2
1
1
1
z
y
x
1 1 1
, , z y x
Gauss-Jacobi Method: Example 2
Iteration #2 absolute relative approximate error
% 669055 . 13 100
895833 . 2
50000 . 2 895833 . 2
1
a
=

= e
% 33119 . 27 100
3560606 . 2
0 . 3 3560606 . 2
2
a
=

= e
% 181 . 218 100
916667 . 0
916667 . 2 916667 . 0
3
a
=

= e
The maximum absolute relative error after the second iteration is 218.181%
This is much larger than the maximum absolute relative error obtained in
iteration #1. Is this a problem?
Iteration x y z
1
2
3
4
5
2.5
2.89583
3.1543
3.04143
3.01687
100.00
13.6690
8.1942
3.7110
0.8140
3
2.3560
2.0303
1.9329
1.9696
100.00
27.3311
16.041
5.0390
1.8633
2.9166
0.91666
0.8797
0.83191
0.9127
82.193
218.181
4.2014
5.7446
8.8911
Gauss-Jacobi Method: Example 2
Repeating more iterations, the following values are obtained
%
1
a
e
%
2
a
e
%
3
a
e
(
(
(

=
(
(
(

9118312 . 0
9858987 . 1
0167424 . 3
z
y
x
The solution obtained is close to the exact solution of .
(
(
(

=
(
(
(

9118 . 0
985 . 1
0167 . 3
z
y
x
Gauss-Jacobi Method
The Gauss-Jacobi Method can still be used
The coefficient matrix is not
diagonally dominant
| |
(
(
(

=
5 3 12
3 5 1
13 7 3
A
But this is the same set of
equations used in example #2,
which did converge.
| |
(
(
(

=
13 7 3
3 5 1
5 3 12
A
If a system of linear equations is not diagonally dominant, check to see if
rearranging the equations can form a diagonally dominant matrix.