ITERATIVE TECHNIQUES

FOR SOLVING
NON-LINEAR SYSTEMS

Newton-Raphson Method for Solving

Non-Linear Equation Sysytems
Given:

f1 x1 , x 2 , x3 ,......, x n 0

f 2 x1 , x 2 , x3 ,......, x n 0
f 3 x1 , x 2 , x3 ,......, x n 0

....
f n x1 , x 2 , x3 ,......, x n 0

f1
i 1
i f1
i 1
i f1
f x , x2 ,......, xn f x , x2 ,....., xn ( x x ) ( x2 x2 ) ( xn xn )
x1
x2
xn
i 1 i 1
i 1
i
i
i
i 1
i f 2
i 1
i f 2
i 1
i f 2
f 2 x1 , x2 ,......, xn f 2 x1 , x2 ,....., xn ( x1 x1 ) ( x2 x2 ) ( xn xn )
x1
x2
xn
i 1 i 1
i 1
i
i
i
i 1
i f 3
i 1
i f 3
i 1
i f 3
f 3 ( x1 , x2 ,......, xn ) f 3 x1 , x2 ,....., xn ( x1 x1 ) ( x2 x2 ) ( xn xn )
x1
x2
xn
i 1
1 1

i 1

i 1

....................

i
1 1

....................

i 1
1

i
1

.............

.............

............

f n
i 1
i f n
i 1
i f n
f n ( x , x2 ,......, xn ) f x , x2 ,....., xn ( x x ) ( x2 x2 ) ( xn xn )
x1
x2
xn
i 1
1

i 1

i 1

i
n 1

i 1
1

i
1

If new values (x1i+1 , x2i+1 , , xni+1) are the solution of the equation set
and (x1i, x2i , , xni) are close to the new estimates, then left hand side of
the equations would be zero.

Then, the equation system will have the following form:

f1
i 1
i f1
i 1
i f1
i
i
i
( x x ) ( x2 x2 ) ( xn xn ) f1 x1 , x2 ,....., xn
x1
x2
xn
i 1
i f 2
i 1
i f 2
i 1
i f 2
i
i
i
( x1 x1 ) ( x2 x2 ) ( xn xn ) f 2 x1 , x2 ,....., xn
x1
x2
xn
i 1
i f 3
i 1
i f 3
i 1
i f 3
i
i
i
( x1 x1 ) ( x2 x2 ) ( xn xn ) f 3 x1 , x2 ,....., xn
x1
x2
xn
i 1
1

i
1

............

...............

.............

...........

f n
i 1
i f n
i 1
i f n
i
i
i
( x x ) ( x2 x2 ) ( xn xn ) f n x1 , x2 ,....., xn
x1
x2
xn
i 1
1

i
1

Then, the equation system will have the following form:

f1
i 1 f1
i 1 f1
i
i
i
(x ) (x2 )
(xn )
f1 x1 , x2 ,....., xn
x1
x2
xn
i 1i f 2
i 1 f 2
i 1 f 2
i
i
i
(x1 ) (x2 )
(xn )
f 2 x1 , x2 ,....., xn
x1
x2
xn
i 1 f 3
i 1 f 3
i 1 f 3
i
i
i
(x1 ) (x2 )
(xn )
f 3 x1 , x2 ,....., xn
x1
x2
xn

............
............
.............
...... ...........
i 1 f n
i 1 f n
i 1 f n
i
i
i
(x1 ) (x2 )
(xn )
f n x1 , x2 ,....., xn
x1
x2
xn

i 1i
1

This equation system is infact a linear system which can be also written
in matrix form:

f1

x1

2
x
1

f
n
x
1

f
1
x
2
f
2
x
2

f
n
x
2

...

...

...

f
1
x
n
f
2
x
n

f
n
x
n

Jacobian Matrix

x1
f1 x1, x2 , x3 ,......, xn

x
f x , x , x ,......, xn
2
2 1 2 3

x
f n x , x , x ,......, xn

1 2 3

Unknown
Matrix

(consequent
values of each x
variable)

Function values
evaluated at current
x values

This linear equation system can be solved by Gauss Elimination

Method:

f1

f
1
x
2
f
2
x
2

f
n
x
2

x1

2
x
1

f
n
x
1

...

...

...

f
1
x
n
f
2
x
n

f
n
x
n

x1
f1 x1, x2 , x3 ,......, xn

x
f x , x , x ,......, xn
2
2 1 2 3

x
f n x , x , x ,......, xn

1 2 3

J ( X ) F X

X J ( X
X

( i 1)

(i )

(k )

) F X
(i )

(k )

(i )

Consider the following set of non-linear equation:

x x x 1 0
2
1

2
2

2
3

1
x
x
4
2
2
x1 x 2 4 x3 0
2
1

2
3

f1 x1 , x2 , x3 x x x 1
2
1

2
2

2
3

1
f 2 x1 , x2 , x3 x
x
4
f 3 x1 , x2 , x3 x12 x22 4 x3
2
1

2
3

 f1

x1
f 2

J(X )
x
f 1
3
x1

f1

x 3 2 x
1
f 2
2 x1
x 3
2 x1

f 3
x3

f1
x 2
f 2
x 2
f 3
x 2

Make an initial guess:

(0)

1

1

2 x2
0
2 x2

2 x3
2 x3
4

(0)
F ( X ) 1.75
2

(1)

(1)

2
2 2
J ( X ( 0 ) ) 2 0
2
2 2 4

X J ( X

(0)

1 2 2 2

1 2 0 2
1 2 2 4

(0)

) F X
2
1.75

(0)

X (1)

0.79167
0.87500
0.33333

( 2)

(1)

J(X ) F X

0.50348
F ( X (1) ) 0.48785
0.05905

(1)

(1)

 1.58334 1.75000 0.66666

(1)
J ( X ) 1.58334
0 0.66666
1.58334 1.75000
4

X ( 2)

( 2)

(1)

J(X ) F X
(1)

(1)

0.79167 1.58334 1.75000 0.66666

0.87500 1.58334
0 0.66666
0.33333 1.58334 1.75000
4

0.50348
0.48785

0.05905

X ( 2)

( 4)

0.52365
0.86607
0.23810

0.44081

0.86603
0.23607

X ( 3)

0.44733
0.86603
0.23607

Example of Gauss-Seidel Iteration

x13 x12 x 2 x1 x3 6 0
e e x3 0
x1

x2

x 2 x1 x3 4
2
2

x 1

2
x 1 x 2

x 2 ln x 3 e

x 1 x 3 6

x1

x 3

2
x2

2 x 1

x 1 1
x 1 1.81712

x 2 1

x 2 0.17733

x 1 1.94859 x 2 0.0518

x 3 1
x 3 1.09199

x 3 1.0257

x 1 1.98336

x 2 0.11868

x 3 1.00484

x 1 1.9597

x 2 0.14625

x 3 1.01511

x 1 1.95112

x 2 0.13583

x 3 1.02033

Jacobi Iterative Technique

Consider the following set of equations.

10 x1
x2
x1 11x2
2 x1
x2
3 x2

2 x3
x3
10 x3
x3

6
3x4 25
x4 11
8 x4 15

Convert the set Ax = b in the form of x = Tx + c.

1
x2
10

x1
x2

1
x1
11

x3

1
x1
5

x4

1
x3
5
1
x3
11

x2
10
3
x2
8

1
x3
8

5
3
x4
11

25

11

x4
10

11

10
15

Start with an initial approximation of:

(0)
(0)
(0)
(0)
x1 0, x 2 0, x3 0 and x 4 0.
x1

(1)

x2

(1)

x3

(1)

x4

(1)

1 (0)
x2
10

1 (0)
x1
11

1 (0)
x1
5

1 (0)
x3
5
1 (0)
x3
11

1 (0)

x2
10
3 (0)
x2
8

1 (0)
x3
8

5
3 (0)
x4
11

25

11

1 (0)

x4
10

11

10
15

x1

(1)

x2

(1)

x3

(1)

x4

(1)

(1)
1

x3

(1)

1
(0)
10

1
(0)
11

1
1
(0) (0)
5
10

3
(0)
8

1
(0)
5

1
3
25
(0) (0)
11
11
11

0.6000, x 2

(1)

1
11
(0)
10
10
1
(0)
8

2.2727,

1.1000 and x 4

(1)

1.8750

15

x1

(2)

x2

( 2)

x3

( 2)

x4

( 2)

1 (1)
x2
10

1 (1)
x1
11

1 (1)
x1
5

1 (1)
x3
5
1 (1)
x3
11

1 (1)

x2
10
3 (1)
x2
8

1 (1)
x3
8

5
3 (1)
x4
11

25

11

1 (1)

x4
10

11

10
15

1 ( k 1)
x2
10

(k)

x1
x2

(k)

x3

(k)

x4

(k)

1 ( k 1 )
x1
11

1 ( k 1 )
x3
5
1 ( k 1 )
x3
11

1 ( k 1 )
x1
5

1 ( k 1 )

x2
10

3 ( k 1 )
x2
8

1 ( k 1 )
x3
8

5
3 ( k 1)
x4
11

25

11

1 ( k 1 )

x4
10

11

10
15

Results of Jacobi Iteration:

k
(k )

0
0.0000

1
0.6000

2
1.0473

3
0.9326

(k )

0.0000

2.2727

1.7159

2.0530

x3

(k )

0.0000

-1.1000 -0.8052

-1.0493

x4

(k )

0.0000

1.8750

1.1309

x1
x2

0.8852

Gauss-Seidel Iterative Technique

Consider the following set of equations.

10 x1
x2
x1 11x2
2 x1
x2
3 x2

2 x3
x3
10 x3
x3

6
3x4 25
x4 11
8 x4 15

(k)
1

x2

(k )

x3

(k )

x4

(k )

1 ( k 1 )
x2
10

1 ( k 1 )
x1
11

1 ( k 1 )
x3
5
1 ( k 1 )
x3
11

1 ( k 1 )
x1
5

1 ( k 1 )
x2
10

3 ( k 1 )
x2
8

1 ( k 1 )
x3
8

5
3 ( k 1 )
x4
11

25

11

1 ( k 1 )
x4
10

11

10
15

1 ( k 1)
x2
10

(k)

x1
x2

(k)

x3

(k)

x4

(k)

1 (k)
x1
11

1 ( k 1 )
x3
5
1 ( k 1 )
x3
11

1 (k)
x1
5

1 (k)

x2
10

3 (k)
x2
8

1 (k)
x3
8

5
3 ( k 1 )
x4
11

25

11

1 ( k 1 )

x4
10

11

10
15

Results of Gauss-Seidel Iteration:

(Blue numbers are for Jacobi iterations.)
k
(k )

0
0.0000

1
0.6000
0.6000

2
1.0300
1.0473

3
1.0065
0.9326

x2

(k )

0.0000

2.3272
2.2727

2.0370
1.7159

2.0036
2.0530

x3

(k )

0.0000

-0.9873 -1.0140
-1.1000 -0.8052

-1.0025
-1.0493

x4

(k )

0.0000

0.8789
1.8750

0.9983
1.1309

x1

0.9844
0.8852

The solution is: x1= 1, x2 = 2, x3 = -1, x4 = 1

It required 15 iterations for Jacobi method
and 7 iterations for Gauss-Seidel method
to arrive at the solution with a tolerance of
0.00001.