COMPUTATIONAL

METHODS
&
STATISTICS
MTH 2212
LECTURER
DR ZAHARAH WAHID
EXT 4514, ROOM E1-4-8-13
Applications :
Computational Methods (CMs) are mathematical
methods used to

1)Find and approximate solution for mathematical problem

2)Fit a curve (design the function) for the given tabulated data.
3)Find the roots of nonlinear algebraic equations.
4)Find optimum solution for optimization or mathematical
programming problem.
5)Find differentiation and integration numerically
6)Find the numerical solution of partial differential equations.
7)Find the numerical solution of initial value and boundary value
problems.
Computational Methods (CMs) are mathematical
methods are used to

Problem

modeling
Mathematical
Model

Analytical Computational
Methods (Numerical)
Methods

Exact
(True) Approximate
(Real) (Fixed) Solution
Solution
 Algebraic Linear Equations
An equation with variables of power one.
a11x1  a12x2  a13x3  ...  a1n xn  b1

Where a’s = constant coefficients,

b’s = constants,
. .
n = number of equation
x = variable

 There are no Trigonometric functions

 Logarithmic functions
 Exponent functions
 Hyperbolic functions
 Solution of Linear Systems of
Equation-Chap. 9

Gaussian Elimination
Pitfalls in Gauss Elimination
Pivoting
Scaling
Linear system has the general form
A set of n equations and n unknowns
a11x1  a12x2  a13x3  ...  a1n xn  b1
a21x1  a22x2  a23x3  ...  a2n xn  b2
. .
an1x1  an 2 x2  an3 x3  ...  ann xn  bn
 Naïve Gaussian Elimination
A method to solve simultaneous linear
equations of the form [A][X]=[C]

Solution two stages

1. Forward Elimination
2. Back Substitution
Linear system-Matrix Form
 Forward Elimination
The goal of forward elimination is to transform the
coefficient matrix into an upper triangular matrix

 25 5 1  x1  106.8 
 64 8 1  x   177.2 
   2  
144 12 1  x3  279.2

25 5 1   x1   106.8 
 0  4.8  1.56  x    96.21
   2  
 0 0 0.7   x3   0.735 
 Forward Elimination
A set of n equations and n unknowns
a11x1  a12x2  a13x3  ...  a1n xn  b1
a21x1  a22x2  a23x3  ...  a2n xn  b2
. .
. .
. .
an1x1  an 2 x2  an3 x3  ...  ann xn  bn

(n-1) steps of forward elimination

ER0-Elem. Row Operations

Column 1

Column 1

Column 2
ER0-Elem. Row Operations
2 x1  x2  x3  4
4 x1  x2  2 x3  10
3 x1  x2  x3  7.5

2 1 1  4 
4 1 2  
10 


3 1 1  7.5

 ER0-Elem. Row Operations
2 1  1  4  2 1  1  4  2 1  1  4 
4 1 2  10  0  1 4  2  0  1 4  2 
     
3 1 1  7.5 3 1 1  7.5 0  0.5 2.5  1.5

2 1  1  4 
0  1 4  2  Backsubst.
 
0 0 0.5  0.5
 Forward Elimination
Step 1
For Equation 2, divide Equation 1 by a11 and
multiply by a21 .

 a21 
 a (a11x1  a12 x2  a13x3  ...  a1n xn  b1 )
 11 
a21 a21 a21
a21x1  a12 x2  ...  a1n xn  b1
a11 a11 a11
 Forward Elimination
Subtract the result from Equation 2.
a21x1  a22x2  a23x3  ...  a2n xn  b2
a21 a21 a21
− a21x1  a a12 x2  ...  a a1n xn  a b1
_________________________________________________
11 11 11

 a21   a21  a21

 a22  a12  x2  ...   a2 n  a1n  xn  b2  b1
 a11   a11  a11

or a x  ...  a x  b
'
22 2
'
2n n
'
2
 Forward Elimination
Repeat this procedure for the remaining
equations to reduce the set of equations as
a11x1  a12x2  a13x3  ...  a1n xn  b1
'
a22 x2  a23
'
x3  ...  a2' n xn  b2'
'
a32 x2  a33
'
x3  ...  a3' n xn  b3'
. . .
. . .
. . .

an' 2 x2  an' 3 x3  ...  ann

'
xn  bn'

End of Step 1
 Forward Elimination
Step 2
Repeat the same procedure for the 3rd term of
Equation 3.
a11x1  a12x2  a13x3  ...  a1n xn  b1
'
a22 x2  a23
'
x3  ...  a2' n xn  b2'
"
a33 x3  ...  a3"n xn  b3"
. .
. .
. .

an" 3 x3  ...  ann

"
xn  bn"
End of Step 2
 Forward Elimination
At the end of (n-1) Forward Elimination steps, the
system of equations will look like
a11x1  a12 x2  a13 x3  ...  a1n xn  b1
'
a22 x2  a23
'
x3  ...  a2' n xn  b2'
a x  ...  a x  b
"
33 3
"
3n n
"
3
. .
. .
. .

n 1 n 1 
ann xn  bn

End of Step (n-1)

Matrix Form at End of Forward
Elimination
a11 a12 a13  a1n   x1   b1 
 0 a' a ' '
 a 2 n x2     b2' 
 22 23    
0 0 a "
33  a3n   x3    b3 
" "

    
           
 0 0 0 0 ann   xn  bn 
(n 1 )
    (n-1 )
 Back Substitution
Solve each equation starting from the last equation

25 5 1   x1   106.8 
 0  4.8  1.56  x    96.21
   2  
 0 0 0.7   x3   0.735 

Example of a system of 3 equations

Back Substitution Starting Eqns
a11x1  a12 x2  a13 x3  ...  a1n xn  b1
'
a22 x2  a23
'
x3  ...  a2' n xn  b2'
"
a33 x3  ...  an" xn  b3"
. .
. .
. .

n 1 n 1 
ann xn  bn
 Back Substitution
Start with the last equation because it has only one unknown
( n 1)
b
xn  n
( n 1)
a nn
 Back Substitution
( n 1)
b
xn  n
( n 1)
a nn

bii 1  ai,ii11 xi 1  ai,ii12 xi  2  ...  ai,in1 xn

xi  i 1 for i  n  1,...,1
aii

i 1
  aiji 1 x j
n
bi
j i 1
xi  i 1 for i  n  1,...,1
a ii
 Example 1
The upward velocity of a rocket is given at three
different times

Table 1 Velocity vs. time data.

Time, t s  Velocity, v m/s 

5 106.8
8 177.2
12 279.2

The velocity data is approximated by a polynomial as:

vt   a1t 2  a2 t  a3 , 5  t  12.

Find the velocity at t=6 seconds .
 Example 1 Cont.
Assume
vt   a1t 2  a2t  a3 , 5  t  12.

Results in a matrix template of the form:

t12 t1 1  a1   v1 
 2   a   v 
t 2 t2 1  2  2
t32 t3 1 
 a3 
 v3 

 

Using data from Table 1, the matrix becomes:

 25 5 1  a1  106.8 
 64 8 1 a   177.2 
   2  
144 12 1  a3  279.2
 Example 1 Cont.
 25 5 1  a1  106.8   25 5 1  106.8 
 64 8 1 a 2   177.2    64 8 1  177.2 
       
144 12 1  a3  279.2 144 12 1  279.2

1. Forward Elimination
2. Back Substitution
Forward Elimination
 Number of Steps of Forward
Elimination
Number of steps of forward elimination is
(n1)(31)2
 Forward Elimination: Step 1
 25 5 1  106.8  Divide Equation 1 by 25 and
 64 8 1  177.2 
  64
multiply it by 64,  2.56 .
144 12 1  279.2 25
25 5 1  106.8 2.56  64 12.8 2.56  273.408

. 64 8 1  177.2
Subtract the result from  64 12.8 2.56  273.408
Equation 2
0  4.8  1.56   96.208

 25 5 1  106.8 
Substitute new equation for  0  4.8  1.56   96.208
Equation 2  
144 12 1  279.2 
Forward Elimination: Step 1 (cont.)
 25 5 1  106.8 
Divide Equation 1 by 25 and
 0  4.8  1.56   96.208
  144
144 12 1  279.2  multiply it by 144, 25  5.76 .

25 5 1  106.8 5.76  144 28.8 5.76  615.168

Subtract the result from 144 279.2

.
12 1 
Equation 3  144 28.8 5.76  615.168
0  16.8  4.76   335.968
25 5 1  106.8 
Substitute new equation for  
Equation 3  0  4.8  1.56   96 .208 
 0  16.8  4.76   335.968
 Forward Elimination: Step 2
25 5 1  106.8  Divide Equation 2 by −4.8
 0  4.8  1.56   96.208 
  and multiply it by −16.8,
 0  16.8  4.76   335.968  16.8
 3.5 .
 4.8
0  4.8 1.56   96.208 3.5  0 16.8  5.46   336.728
0  16.8  4.76  335.968
Subtract the result from
Equation 3  0  16.8  5.46   336.728
0 0 0.7  0.76

25 5 1  106.8 
Substitute new equation for  0  4.8  1.56   96.208
Equation 3  
 0 0 0.7  0.76 
Back Substitution
 Back Substitution
25 5 1  106.8  25 5 1   a1   106.8 
 0  4.8  1.56   96.2   0  4.8  1.56 a    96.208
     2  
 0 0 0.7  0.7   0 0 0.7   a3   0.76 

Solving for a3
0.7 a3  0.76
0.76
a3 
0.7
a3  1.08571
 Back Substitution (cont.)
25 5 1   a1   106.8 
 0  4.8  1.56 a    96.208
   2  
 0 0 0.7   a3   0.76 

Solving for a2
 4.8a2  1.56a3  96.208
 96.208  1.56a3
a2 
 4.8
 96.208  1.56 1.08571
a2 
 4.8
a2  19.6905
 Back Substitution (cont.)
25 5 1   a1   106.8 
 0  4.8  1.56 a    96.2
   2  
 0 0 0.7   a3   0.76 

Solving for a1
25a1  5a2  a3  106.8
106.8  5a2  a3
a1 
25
106.8  5 19.6905  1.08571

25
 0.290472
Naïve Gaussian Elimination Solution
 25 5 1  a1  106.8 
 64 8 1 a2   177.2 
     
144 12 1  a3  279.2

 a1  0.290472
a    19.6905 
 2  
 a3   1.08571 
 Example 1 Cont.
Solution  a1  0.290472
a    19.6905 
The solution vector is
 2  
 a3   1.08571 

The polynomial that passes through the three data points is then:

vt   a1t 2  a2t  a3

 0.290472t 2  19.6905t  1.08571, 5  t  12

v6  0.2904726  19.69056  1.08571

2

 129.686 m/s.
 Pitfalls of Naïve Gauss Elimination
• Possible division by zero
• Large round-off errors
Pitfall#1. Division by zero

 Caused by coefficient value

equals zero or very close to zero.

This can be solved by using

pivoting technique
Pitfall#1. Division by zero
10 x2  7 x3  3
6 x1  2 x2  3x3  11
5 x1  x2  5 x3  9

0 10  7  x1   3 
6 2  
3 x2  11   
    
5  1 5   x3   9 
Is division by zero an issue here?
12 x1  10 x2  7 x3  15
6 x1  5 x2  3x3  14
5 x1  x2  5 x3  9

12 10  7  x1  15
6 5 3   x2   14
    
 5  1 5   x3   9 
 Is division by zero an issue here?
YES
12 x1  10 x2  7 x3  15
6 x1  5 x2  3x3  14
24 x1  x2  5 x3  28

12 10  7  x1  15 12 10  7  x1   15 

6 5 3   x2   14  0
     0 6.5  x2   6.5
    
24  1 5   x3  28 12  21 19   x3   2

Division by zero is a possibility at any step

of forward elimination
Pitfall#2. Large Round-off Errors
This may be due to some of the
Following :
 Large number of equations to be solved
due to the fact that every result is
dependent on previous results
 Errors in early steps will tend to
propagate.
Pitfall#2. Large Round-off Errors
 20 15 10  x1   45 
 3  2.249 7   x   1.751
   2  
 5 1 3   x3   9 

Solve it on a computer using 6 significant digits with chopping

 x1   0.9625 
 x    1.05 
 2  
 x3  0.999995
Pitfall#2. Large Round-off Errors
 20 15 10  x1   45 
 3  2.249 7   x   1.751
   2  
 5 1 3   x3   9 

Solve it on a computer using 5 significant digits with chopping

 x1   0.625 
 x    1.5 
 2  
 x3  0.99995

Is there a way to reduce the round off error?

Pitfall#2. Large Round-off Errors
 20 15 10  x1   45 
 3  2.249 7   x   1.751
   2  
 5 1 3   x3   9 

Exact Solution

 x1  1
 x   1
 2  
 x3  1
 Avoiding Pitfalls

Gaussian Elimination with Partial Pivoting

• Avoids division by zero
• Reduces round off error
Gauss Elimination with
Partial Pivoting
 Pivoting
This involves the following steps
• Determine the largest coefficient available in the
column below the pivot element.
• Switch the rows so that the largest element is the
pivot element. This is known as partial pivoting.
•If column as well as rows are searched for the
largest element & then switched, the process is
called complete pivoting.
 What is Different About Partial
Pivoting?
At the beginning of the kth step of forward elimination,
find the maximum of

akk , ak 1,k ,................, ank

If the maximum of the values is a pk
in the p th row, k  p  n, then switch rows p and k.
Significant x2 x1 %
figures ε x1
3 0.667 -3.33 1099
4 0.6667 0.0000 100
5 0.66667 0.30000 10
6 0.666667 0.330000 1 :
7 0.6666667 0.3330000 0.1
•This case is much less sensitive to the number
of significant figures on the result x due to x
1 2

With partial pivoting

Significant
x1 x2
figures ε x1 (%)
3 0.667 0.333 0.1
4 0.6667 0.3333 0.01
5 0.66667 0.33333 0.001
6 0.666667 0.333333 0.0001
7 0.6666667 0.3333333 0.00001
Matrix Form at Beginning of 2nd
Step of Forward Elimination
a11 a12 a13  a1n   x1   b1 
0 a '
a '
 ' 
a2 n x2   ' 
b2
 22 23    
0 a '
32 a '
33  a3n   x3   b3 
' '

    
           
 0 a '
n2 a '
n3 a '
n4 ann   xn  bn 
'
    '
 Example (2nd step of FE)

6 14 5.1 3.7 6   x1   5 
0  7 6 1 2   x2   6
    
0 4 12 1 11  x3    8 
    
 0 9 23 6 8 x
   
4 9
0  17 12 11 43  x5   3 

Which two rows would you switch?

Example (2nd step of FE)

6 14 5.1 3.7 6   x1   5 
0  17 12 11 43  x   3 
  2   
0 4 12 1 11  x3    8 
    
 0 9 23 6 8 x
   
4 9
0  7 6 1 2   x5   6

Switched Rows
 Forward Elimination
Same as naïve Gauss elimination method
except that we switch rows before each
of the (n-1) steps of forward elimination.
Example: Matrix Form at Beginning
of 2nd Step of Forward Elimination
a11 a12 a13  a1n   x1   b1 
0 a '
a '
 ' 
a2 n x2   ' 
b2
 22 23    
0 a '
32 a '
33  a3n   x3   b3 
' '

    
          
 0 a '
n2 a '
n3 a '
n4 ann   xn  bn 
'
    '
Matrix Form at End of Forward
Elimination
a11 a12 a13  a1n   x1   b1 
 0 a' a' '
 a 2 n x2     b2' 
 22 23    
0 "
0 a33  a3n   x3    b3 
" "

    
          
 0 0 0 0 ann   xn  bn 
(n 1 )
    (n-1 )
Back Substitution Starting Eqns
a11x1  a12 x2  a13 x3  ...  a1n xn  b1
'
a22 x2  a23
'
x3  ...  a2' n xn  b2'
"
a33 x3  ...  an" xn  b3"
. .
. .
. .

n 1 n 1 
ann xn  bn
 Back Substitution
( n 1)
b
xn  n
( n 1)
a nn

i 1 n
i 1
bi   aij x j
j i 1
xi  i 1 for i  n  1,...,1
a ii
Gauss Elimination with
Partial Pivoting
Example
 Example 2
Solve the following set of equations
by Gaussian elimination with partial
pivoting

 25 5 1  a1  106.8 
 64   
8 1 a 2  177.2  
     
144 12 1  a3  279.2
 Example 2 Cont.
 25 5 1  a1  106.8   25 5 1  106.8 
 64 8 1 a   177.2    
    
2   64 8 1  177.2 
144 12 1  a3  279.2 144 12 1  279.2

1. Forward Elimination
2. Back Substitution
Forward Elimination
 Forward Elimination: Step 1
• Examine absolute values of first column, first row
and below.
25 , 64 , 144
• Largest absolute value is 144 and exists in row 3.
• Switch row 1 and row 3.

 25 5 1  106.8  144 12 1  279.2

 64 8 1  177.2    64 8 1  177.2 
   
144 12 1  279.2  25 5 1  106.8 
Forward Elimination: Step 1 (cont.)
144 12 1  279.2 Divide Equation 1 by 144 and
 64 8 1  177.2  64
  multiply it by 64,  0.4444.
 25 5 1  106.8  144

144 12 1  279.2 0.4444  63.99 5.333 0.4444  124.1

.
Subtract the result from 64 8 1  177.2
Equation 2  63.99 5.333 0.4444  124.1
0 2.667 0.5556  53.10

Substitute new equation for 144 12 1  279.2

Equation 2  0 2.667 0.5556  53.10
 
 25 5 1  106.8 
Forward Elimination: Step 1 (cont.)
144 12 1  279.2 Divide Equation 1 by 144 and
 0 2.667 0.5556  53.10 25
  multiply it by 25,  0.1736.
 25 5 1  106.8  144

144 12 1  279.2 0.1736  25.00 2.083 0.1736  48.47

.
25 5 1  106.8
Subtract the result from
Equation 3  25 2.083 0.1736  48.47
0 2.917 0.8264  58.33

Substitute new equation for 144 12 1  279.2

Equation 3  0 2.667 0.5556  53.10
 
 0 2.917 0.8264  58.33
 Forward Elimination: Step 2
• Examine absolute values of second column, second row
and below.
2.667 , 2.917
• Largest absolute value is 2.917 and exists in row 3.
• Switch row 2 and row 3.

144 12 1  279.2 144 12 1  279.2

 0 2.667 0.5556  53.10   0 2.917 0.8264  58.33
   
 0 2.917 0.8264  58.33  0 2.667 0.5556  53.10
 Forward Elimination: Step 2 (cont.)
Divide Equation 2 by 2.917 and
144 12 1  279.2
 0 2.917 0.8264  58.33 multiply it by 2.667,
  2.667
 0 2.667 0.5556  53.10  0.9143.
2.917
0 2.917 0.8264  58.33 0.9143  0 2.667 0.7556  53.33

.
0 2.667 0.5556  53.10
Subtract the result from  0 2.667 0.7556  53.33
Equation 3
0 0  0.2   0.23

144 12 1  279.2 
Substitute new equation for  0 2.917 0.8264  58.33 
Equation 3  
 0 0  0.2   0.23
Back Substitution
 Back Substitution
144 12 1  279.2  144 12 1   a1   279.2 
 0 2.917 0.8264  58.33    0 2.917 0.8264 a    58.33 
     2  
 0 0  0.2   0.23  0 0  0.2   a3   0.23

Solving for a3
 0.2a3  0.23
 0.23
a3 
 0.2
 1.15
 Back Substitution (cont.)
144 12 1   a1   279.2 
 0 2.917 0.8264 a    58.33 
   2  
 0 0  0.2   a3   0.23

Solving for a2
2.917a2  0.8264a3  58.33
58.33  0.8264a3
a2 
2.917
58.33  0.8264 1.15

2.917
 19.67
 Back Substitution (cont.)
144 12 1   a1   279.2 
 0 2.917 0.8264 a    58.33 
   2  
 0 0  0.2   a3   0.23

Solving for a1
144a1  12a2  a3  279.2
279.2  12a2  a3
a1 
144
279.2  12 19.67  1.15

144
 0.2917
Gaussian Elimination with Partial
Pivoting Solution

 25 5 1  a1  106.8 
 64 8 1 a   177.2 
   2  
144 12 1  a3  279.2

 a1  0.2917
a    19.67 
 2  
 a3   1.15 
Gauss Elimination with
Partial Pivoting
Another Example
 Partial Pivoting: Example 2
Consider the system of equations
10 x1  7 x2  7
 3x1  2.099 x2  6 x3  3.901
5 x1  x2  5 x3  6
In matrix form
 10  7 0  x1   7 
 3 2.099 6  x  3.901
   2 =  
 5  1 5  x3   6 

Solve using Gaussian Elimination with Partial Pivoting using five

significant digits with chopping
 Partial Pivoting: Example
Forward Elimination: Step 1
Examining the values of the first column
|10|, |-3|, and |5| or 10, 3, and 5
The largest absolute value is 10, which means, to
follow the rules of Partial Pivoting, we switch
row1 with row1.

Performing Forward Elimination

 10  7 0  x1   7  10 7 0  x1   7 
 3 2.099 6  x   3.901

 5
 2   
 1 5  x3   6 
  0  0.001 6  x   6.001

 0 2.5
 2   
5  x3   2.5 
 Partial Pivoting: Example 2
Forward Elimination: Step 2
Examining the values of the first column
|-0.001| and |2.5| or 0.0001 and 2.5
The largest absolute value is 2.5, so row 2 is
switched with row 3

Performing the row swap

10 7 0  x1   7  10 7 0  x1   7 
 0  0.001 6  x   6.001

 0 2.5
 2   
5  x3   2.5 
 0
 2.5 5   x    2.5 
 2   
 0  0.001 6  x3  6.001
 Partial Pivoting: Example 2
Forward Elimination: Step 2

Performing the Forward Elimination results in:

10  7 0   x1   7 
 0 2.5 5   x    2.5 
  2   
 0 0 6.002  x3  6.002
 Partial Pivoting: cont..
Back Substitution
Solving the equations through back substitution

10  7
6.002
0   x1   7  x3  1
 0 2.5 5   x    2.5  6.002
  2   
 0 0 6.002  x3  6.002 2.5  5 x3
x2   1
2.5

7  7 x 2  0 x3
x1  0
10
 Partial Pivoting: Example
Compare the calculated and exact solution
The fact that they are equal is coincidence, but it
does illustrate the advantage of Partial Pivoting

 x1   0   x1   0 
X  calculated   x2    1 X  exact   x 2    1
 x3   1   x3   1 
 Scaling

 Necessary when different units are

used in the same system of equations

WHY
- to standardize the size of the
coefficient in the system of equations
- Minimize the round-off errors caused
by having larger coefficient than
others.
 Example 3

Use three significant figures to solve

(1)
(2)

1. Gauss elimination with scaling & pivoting

2. Gauss elimination without scaling and with pivoting
3. Naive Gauss elimination
1. Gauss elimination with scaling and pivoting:
Scaling:

0.00002x 1 + x2 = 0.1 (1)

x1 + x 2 = 2 (2)

Pivoting
x1+x2 =2 (1)
0.00002x 1 + x 2 = 0.1 (2)

Forward elimination
x1 + x2 = 2
x2 = 0.1
Back substitution
x1= 1.90 and x2 = 0.10
2. Gauss elimination without scaling and with
pivoting:

Pivoting
x1+x2 =2 (1)
2x1 + 100000x 2 =10000 (2)

Forward elimination
x1 + x2 = 2
100000 x2 =10000
Back substitution
x1= 1.90 and x2 = 0.10
3. Gauss elimination

Forward elimination
2 x1 +100000 x2 = 10000
100000 x2 =10000

Back substitution
x1= 1.90 and x2 = 0.10
 Exercise

Solve the unknown variables with partial

pivoting
Exercise
Solve the system using Gauss elimination

2x  y  z  7
2 x  2 y  3 z  10
 4 x  4 y  5 z  14
THE END