Harjadi Gunawan@2012

Gauss-Siedel and Jacobi

 Gauss-Siedel
• Iterative or approximate methods provide an
alternative to the elimination methods. The Gauss-
Seidel method is the most commonly used iterative
method.
• The system [A]{X}={B} is reshaped by solving the
first equation for x1, the second equation for x2, and
the third for x3, …and nth equation for xn. We will
limit ourselves to a 3x3 set of equations.
 Gauss-Siedel
a11 x1  a12 x2  a13 x3  b1 b1  a12 x2  a13 x3
x1 
a11

a21 x1  a22 x2  a23 x3  b2  x2 

b2  a21 x1  a23 x3
a22
b3  a31 x1  a32 x2
x1 
a31 x1  a32 x2  a33 x3  b3 a33

Now we can start the solution process by choosing

guesses for the x’s. A simple way to obtain initial
guesses is to assume that they are zero. These
zeros can be substituted into x1 equation to
calculate a new x1=b1/a11.
 Gauss-Siedel
• New x1 is substituted to calculate x2 and x3. The
procedure is repeated until the convergence
criterion is satisfied:
new
x x old
 a ,i  i
new
i
100%   s
x i
 Jacobi iteration Method

An alternative approach, called Jacobi iteration,

utilizes a somewhat different technique. This
technique includes computing a set of new x’s on the
basis of a set of old x’s. Thus, as the new values are
generated, they are not immediately used but are
retained for the next iteration.
 Gauss-Siedel

The Gauss-Seidel method The Jacobi iteration method

Convergence Criterion for Gauss-Seidel Method
• The gauss-siedel method is similar to the technique of
fixed-point iteration.
• The Gauss-Seidel method has two fundamental
problems as any iterative method:
1. It is sometimes non-convergent, and
2. If it converges, converges very slowly.
• Sufficient conditions for convergence of two linear
equations, u(x,y) and v(x,y) are:
u u
 1
x y
v v
 1
x y
 Convergence Criterion for Gauss-Seidel
Method (cont’d)
• Similarly, in case of two simultaneous equations, the
Gauss-Seidel algorithm can be expressed as:
b1 a12
u (x 1 , x 2 )   x2
a11 a11
b 2 a21
v (x 1 , x 2 )   x1
a22 a22
�u �u a12
0 
�
x1 �
x2 a11
�v a21 �v
 0
�
x1 a22 �
x2
 Convergence Criterion for Gauss-Seidel
Method (cont’d)
Substitution into convergence criterion of two linear
equations yield: a12 a21
 1, 1
a11 a22
In other words, the absolute values of the slopes must be
less than unity for convergence:
For n equations
a11  a12 n

a22  a21 a ii  
j 1
a i, j
ji

That is, the diagonal element must be greater than the off-
diagonal element for each row.
 Gauss-Siedel Method- Example 1
7.85  0.1x2  0.2 x3
 3  0.1 0.2   x1   7.85  x1  3
0.1       19.3  0.1x1  0.3 x3
 7  0.3  x2    19.3  x2 
7
0.3  0.2 10   x3   71.4  x3  71.4  0.3 x1  0.2 x2
10

• Guess x1, x2, x3= zero for the first guess

Iter. x1 x2 x3 | a,1|(%) | a,2| (%) | a,3| (%)
0 0 0 0 - - -
1 2.6167 -2.7945 7.005610 100 100 100
2 2.990557 -2.499625 7.000291 12.5 11.8 0.076
 Gauss-Seidel Method: Example 2
The upward velocity of a rocket
is given at three different times
Table 1 Velocity vs. Time data.
Time, t  s Velocity v  m/s
5 106.8
8 177.2
12 279.2

The velocity data is approximated by a polynomial as:

v t   a1t 2  a2t  a3 , 5  t  12.
 Gauss-Seidel Method: Example 2

Using a Matrix template of the form t12 t1 1  a1   v1 

 2 
t 2 t2 1  a 2   v2 
t32 t3 1  a3   v3 

 25 5 1  a1  106 .8 
The system of equations becomes  64 8 1  a   177 .2 
   2  
144 12 1  a3   279 .2 

 a1  1
 a    2
Initial Guess: Assume an initial guess of
 2  
 a3  5
 Gauss-Seidel Method: Example 2

Rewriting each equation

106.8  5a 2  a 3
a1 
25
 25 5 1  a1  106 .8 
 64 8 1  a   177 .2  177.2  64a1  a 3
   2   a2 
144 12 1  a3   279 .2  8

279.2  144a1  12a 2

a3 
1
 Gauss-Seidel Method: Example 2

Applying the initial guess and solving for ai

 a1  1 106.8  5( 2)  (5)

a1   3.6720
 a    2 25
 2  
 a3  5 177.2  64 3.6720   5
a2   7.8510
Initial Guess 8
279.2  144 3.6720  12  7.8510
a3   155.36
1

When solving for a2, how many of the initial guess values were used?
 Gauss-Seidel Method: Example 2

Finding the absolute relative approximate error

xinew  xiold At the end of the first iteration
a i  new
 100
xi
 a1   3.6720 
 a     7.8510 
3.6720  1.0000  2  
a 1  x100  72.76%
3.6720  a 3    155 .36 

 7.8510  2.0000 The maximum absolute

a  x100  125.47% relative approximate error is
2
 7.8510
125.47%

 155.36  5.0000
a 3  x100  103.22%
 155.36
 Gauss-Seidel Method: Example 2

Iteration #2
Using
 a1   3.6720  the values of ai are found:
 a     7.8510  106.8  5  7.8510  155.36
 2   a1   12.056
 a 3    155 .36  25

from iteration #1
177.2  6412.056   155.36
a2   54.882
8

279.2  14412.056  12  54.882

a3   798.34
1
 Gauss-Seidel Method: Example 2

Finding the absolute relative approximate error

12.056  3.6720 At the end of the second iteration
a 1  x100  69.543%
12.056  a   12.056 
1
a     54.882
 2  
 54.882    7.8510  a3   798.54
a 2  x100  85.695%
 54.882
The maximum absolute
 798.34    155.36 relative approximate error is
a 3  x100  80.540% 85.695%
 798.34
 Gauss-Seidel Method: Example 2
Repeating more iterations, the following values are obtained
Iteration a1 a 1 % a2 a 2 % a3 a 3 %
1 3.6720 72.767 −7.8510 125.47 −155.36 103.22
2 12.056 69.543 −54.882 85.695 −798.34 80.540
3 47.182 74.447 −255.51 78.521 −3448.9 76.852
4 193.33 75.595 −1093.4 76.632 −14440 76.116
5 800.53 75.850 −4577.2 76.112 −60072 75.963
6 3322.6 75.906 −19049 75.972 −249580 75.931

Notice – The relative errors are not decreasing at any significant rate
 a1  0.29048
Also, the solution is not converging to the true solution of a    19.690 
 2  
a 3   1.0857 
 Improvement of Convergence Using
Relaxation
x i
new
�
xi
new
 1   �
xold
i

• Where  is a weighting factor that is assigned a

value between [0, 2]
• If  = 1 the method is unmodified.
• If  is between 0 and 1 (under relaxation) this is
employed to make a non convergent system to
converge.
• If  is between 1 and 2 (over relaxation) this is
employed to accelerate the convergence.
 Gauss-Siedel Method- Example 3
 8 x1  x2  2 x3  20
 3x1  x2  7 x3  34
2 x1  6 x2  x3  38 8x 1  x 2  2x 3  20
Rearrange so that
the equations are 2x 1  6x 2  x 3  38
diagonally dominant
3x 1  x 2  7 x 3  34
 20  x2  2 x3  38  2 x1  x3  34  3x1  x2
x1  x2  x3 
8 6 7
 Gauss-Siedel Method- Example 3
unknow
iteration n value a maximum  a
0 x1 0
x2 0
x3 0
1 x1 2.5 100.00%
x2 7.166667 100.00%
x3 -2.7619 100.00% 100.00%
2 x1 4.08631 38.82%
x2 8.155754 12.13%
x3 -1.94076 42.31% 42.31%
3 x1 4.004659 2.04%
x2 7.99168 2.05%
x3 -1.99919 2.92% 2.92%
 Gauss-Siedel Method- Example 3
The same computation can be developed with relaxation where
 = 1.2
First iteration:
 20  x 2  2 x3  20  0  2(0)
x1    2.5
8 8
Relaxation yields: x1  1.2( 2.5)  0.2(0)  3
38  2x 1  x 3 38  2(3)  0
x2    7.333333
6 6
Relaxation yields: x 2  1.2(7.333333)  0.2(0)  8.8
 34  3x1  x 2  34  3(3)  8.8
x3    2.3142857
7 7
Relaxation yields: x 3  1.2( 2.3142857)  0.2(0)  2.7771429
 Gauss-Siedel Method- Example 3
Iter. unknown value relaxation a maximum  a
1 x1 2.5 3 100.00%
x2 7.3333333 8.8 100.00%
x3 -2.314286 -2.777143 100.00% 100.000%
2 x1 4.2942857 4.5531429 34.11%
x2 8.3139048 8.2166857 7.10%
x3 -1.731984 -1.522952 82.35% 82.353%
3 x1 3.9078237 3.7787598 20.49%
x2 7.8467453 7.7727572 5.71%
x3 -2.12728 -2.248146 32.26% 32.257%
4 x1 4.0336312 4.0846055 7.49%
x2 8.0695595 8.12892 4.38%
x3 -1.945323 -1.884759 19.28% 19.280%
 Gauss-Seidel Method: Pitfall
What went wrong?
Even though done correctly, the answer is not converging to the
correct answer
This example illustrates a pitfall of the Gauss-Siedel method: not all
systems of equations will converge.

Is there a fix?
One class of system of equations always converges: One with a diagonally
dominant coefficient matrix.

Diagonally dominant: [A] in [A] [X] = [C] is diagonally dominant if:

n n
aii   aij for all ‘i’ and aii   aij for at least one ‘i’
j 1 j 1
j i j i
 Gauss-Seidel Method: Pitfall
Diagonally dominant: The coefficient on the diagonal must be at least
equal to the sum of the other coefficients in that row and at least one row
with a diagonal coefficient greater than the sum of the other coefficients
in that row.
Which coefficient matrix is diagonally dominant?

 2 5.81 34 124 34 56 

 A    45 43 1  [ B]   23 53 5 
123 16 1   96 34 129

Most physical systems do result in simultaneous linear equations that

have diagonally dominant coefficient matrices.
 Gauss-Seidel Method: Example 4

Given the system of equations The coefficient matrix is:

12 x1  3 x2 - 5 x3  1 12 3  5
x1  5 x2  3x3  28  A   1 5 3 
3x1  7 x2  13x3  76  3 7 13 

With an initial guess of Will the solution converge using the

Gauss-Siedel method?
 x1  1
 x   0 
 2  
 x3  1
 Gauss-Seidel Method: Example 4

Checking if the coefficient matrix is diagonally dominant

a11  12  12  a12  a13  3   5  8
12 3  5
 A   1 5 3  a 22  5  5  a 21  a23  1  3  4
 3 7 13 
a33  13  13  a31  a32  3  7  10

The inequalities are all true and at least one row is strictly greater than:
Therefore: The solution should converge using the Gauss-Siedel Method
 Gauss-Seidel Method: Example 4

Rewriting each equation With an initial guess of

12 3  5  a1   1   x1  1
 1 5 3   a    28   x   0 
   2    2  
 3 7 13   a3  76   x3  1

1  3 x 2  5 x3 1  3 0   51
x1  x1   0.50000
12 12

28  x1  3 x3 28   0.5  31
x2  x2   4.9000
5 5

76  3 x1  7 x 2 76  3 0.50000   7 4.9000 
x3  x3   3.0923
13 13
 Gauss-Seidel Method: Example 4
The absolute relative approximate error
0.50000  1.0000
a 1   100  100.00%
0.50000

4.9000  0
a 2
  100  100.00%
4.9000

3.0923  1.0000
a 3
  100  67.662%
3.0923

The maximum absolute relative error after the first iteration is 100%
 Gauss-Seidel Method: Example 4
After Iteration #1
 x1  0.5000 
 x    4.9000 
 2  
 x3   3.0923 

Substituting the x values into the After Iteration #2

equations
 x1  0.14679 
1  3 4.9000   5 3.0923  x    3.7153 
x1   0.14679
12  2  
 x3   3.8118 
28   0.14679   3 3.0923
x2   3.7153
5

76  3 0.14679   7 4.900 
x3   3.8118
13
 Gauss-Seidel Method: Example 4

Iteration #2 absolute relative approximate error

0.14679  0.50000
a 1   100  240.61%
0.14679
3.7153  4.9000
a 2
  100  31.889%
3.7153
3.8118  3.0923
a 3
  100  18.874%
3.8118
The maximum absolute relative error after the first iteration is 240.61%

This is much larger than the maximum absolute relative error obtained in
iteration #1. Is this a problem?
 Gauss-Seidel Method: Example 4
Repeating more iterations, the following values are obtained
Iteration a1 a 1 % a2 a 2 % a3 a 3 %

1 0.50000 100.00 4.9000 100.00 3.0923 67.662

2 0.14679 240.61 3.7153 31.889 3.8118 18.876
3 0.74275 80.236 3.1644 17.408 3.9708 4.0042
4 0.94675 21.546 3.0281 4.4996 3.9971 0.65772
5 0.99177 4.5391 3.0034 0.82499 4.0001 0.074383
6 0.99919 0.74307 3.0001 0.10856 4.0001 0.00101

 x1  0.99919  x1  1 
The solution obtained  x    3.0001  is close to the exact solution of  x    3 .
 2    2  
 x3   4.0001   x3  4
 Gauss-Seidel Method: Example 5
Given the system of equations

3x1  7 x2  13x3  76 Rewriting the equations

x1  5 x2  3x3  28 76  7 x2  13 x3
x1 
12x1  3x2  5 x3  1 3
28  x1  3 x3
With an initial guess of x2 
5
 x1  1 
 x   0 1  12 x1  3 x 2
 2   x3 
 x3  1  5
 Gauss-Seidel Method: Example 5

Conducting six iterations, the following values are obtained

Iteration a1 a 1 % A2 a 2 % a3 a 3 %
1 21.000 95.238 0.80000 100.00 50.680 98.027
2 −196.15 110.71 14.421 94.453 −462.30 110.96
3 −1995.0 109.83 −116.02 112.43 4718.1 109.80
4 −20149 109.90 1204.6 109.63 −47636 109.90
5 2.0364×105 109.89 −12140 109.92 4.8144×105 109.89
6 −2.0579×105 109.89 1.2272×105 109.89 −4.8653×106 109.89

The values are not converging.

Does this mean that the Gauss-Seidel method cannot be used?