NUMERICAL DIFFERENTIATION

The derivative of f (x) at x

0
is:
h
x f h x f
x f
Limit
h
) (
0 0
0 0
+
=
'

An approximation to this is:
h
x f h x f
x f
) (
0 0
0
+
~
'
for small values of h.
Forward Difference
Formula
8 1
0
. and ln ) ( Let = = x x x f
Find an approximate value for ( ) 8 1. f
'
0.1 0.5877867 0.6418539 0.5406720
0.01 0.5877867 0.5933268 0.5540100
0.001 0.5877867 0.5883421 0.5554000
) . ( 8 1 f
) . ( h f + 8 1
h
f h f ) . ( ) . ( 8 1 8 1 +
h
The exact value of
5 55 0 8 1 . . =
'
f
Assume that a function goes through three points:
. , and , , ,
2 2 1 1 0 0
x f x x f x x f x
) ( ) ( x P x f ~
2 2 1 1 0 0
x f x L x f x L x f x L x P + + =
Lagrange Interpolating Polynomial
2
1 2 0 2
1 0
1
2 1 0 1
2 0
0
2 0 1 0
2 1
) )( (
) )( (

) )( (
) )( (

) )( (
) )( (
x f
x x x x
x x x x
x f
x x x x
x x x x
x f
x x x x
x x x x
x P


+


+


=
2 2 1 1 0 0
x f x L x f x L x f x L x P + + =
) ( ) ( x P x f
'
~
'
2
1 2 0 2
1 0
1
2 1 0 1
2 0
0
2 0 1 0
2 1
) )( (
2

) )( (
2

) )( (
2
x f
x x x x
x x x
x f
x x x x
x x x
x f
x x x x
x x x
x P


+


+


=
'
2
0 0 0 0
0 0 0
1
0 0 0 0
0 0 0
0
0 0 0 0
0 0 0
0
) ( ) 2 ( ) 2 (
) ( 2

) 2 ( ) ( ) (
) 2 ( 2

) 2 ( ) (
) 2 ( ) ( 2
x f
h x h x x h x
h x x x
x f
h x h x x h x
h x x x
x f
h x x h x x
h x h x x
x P
+ + +
+
+
+ + +
+
+
+ +
+ +
=
'
If the points are equally spaced, i.e.,
h x x h x x 2
0 2 0 1
+ = + = and
2
2
1
2
0
2
0
2
2
2
3
x f
h
h
x f
h
h
x f
h
h
x P

+

=
'
2 1 0 0
4 3
2
1
x f x f x f
h
x P + =
'
h x f h x f x f
h
x f 2 4 3
2
1
0 0 0 0
+ + + ~
'
Three-point formula:
h x x h x x + = =
0 2 0 1
and
If the points are equally spaced with x
0
in the middle:
2
0 0 0 0
0 0 0
1
0 0 0 0
0 0 0
0
0 0 0 0
0 0 0
0
) ( ) ( ) (
) ( 2

) ( ) ( ) (
) ( 2

) ( ( ) (
) ( ) ( 2
x f
h x h x x h x
h x x x
x f
h x h x x h x
h x x x
x f
h x x h x x
h x h x x
x P
+ +

+
+
+
+
+
+
=
'
2
2
1
2
0
2
0
2 2
0
x f
h
h
x f
h
h
x f
h
x P +

=
'
h x f h x f
h
x f + ~
'
0 0 0
2
1
Another Three-point formula:
Alternate approach (Error estimate)
Take Taylor series expansion of f(x+h) about x:
( ) ( ) ( )
( )
( )
( )
( ) + + +
'
+ = + x f
h
x f
h
x f h x f h x f
3
3
2
2
! 3 2
( ) ( ) ( )
( )
( )
( )
( ) + + +
'
= + x f
h
x f
h
x f h x f h x f
3
3
2
2
! 3 2
( ) ( )
( )
( )
( )
( )
( ) + + +
'
=
+
x f
h
x f
h
x f
h
x f h x f
3
2
2
! 3 2
.............. (1)
( ) ( )
( ) ( ) h O x f
h
x f h x f
+
'
=
+
( )
( ) ( )
( ) h O
h
x f h x f
x f
+
=
'
( )
( ) ( )
h
x f h x f
x f
+
~
'
Forward Difference
Formula
( )
( )
( )
( )
( ) + + = x f
h
x f
h
h O
3
2
2
! 3 2
( ) ( ) ( )
( )
( )
( )
( ) + + +
'
+ = + x f
h
x f
h
x f h x f h x f
3
3
2
2
! 3
8
2
4
2 2
( ) ( ) ( )
( )
( )
( )
( ) + + +
'
= + x f
h
x f
h
x f h x f h x f
3
3
2
2
! 3
8
2
4
2 2
( ) ( )
( )
( )
( )
( )
( ) + + +
'
=
+
x f
h
x f
h
x f
h
x f h x f
3
2
2
! 3
4
2
2
2
2
................. (2)
( ) ( )
( )
( )
( )
( )
( ) + + +
'
=
+
x f
h
x f
h
x f
h
x f h x f
3
2
2
! 3 2
.............. (1)
( ) ( )
( )
( )
( )
( )
( ) + + +
'
=
+
x f
h
x f
h
x f
h
x f h x f
3
2
2
! 3
4
2
2
2
2
................. (2)
2 X Eqn. (1) Eqn. (2)
( ) ( ) ( ) ( )
( )
( )
( )
( )
( )
'
=
+

+
x f
h
x f
h
x f
h
x f h x f
h
x f h x f
4
3
3
2
! 4
6
! 3
2
2
2
2
( ) ( ) ( )
( )
( )
( )
( )
( )
( )
2
4
3
3
2
! 4
6
! 3
2
2
3 4 2
h O x f
x f
h
x f
h
x f
h
x f h x f h x f
+
'
=

'
=
+ + +

( ) ( ) ( )
( ) ( )
2
2
3 4 2
h O x f
h
x f h x f h x f
+
'
=
+ + +
( )
( ) ( ) ( )
( )
2
2
3 4 2
h O
h
x f h x f h x f
x f
+ + +
=
'
( )
( ) ( ) ( )
h
x f h x f h x f
x f
2
3 4 2 + + +
~
'
Three-point Formula
( )
( )
( )
( ) = x f
h
x f
h
h O
4
3
3
2
2
! 4
6
! 3
2
Take Taylor series expansion of f(x+h) about x:
( ) ( ) ( )
( )
( )
( )
( ) + + +
'
+ = + x f
h
x f
h
x f h x f h x f
3
3
2
2
! 3 2
Take Taylor series expansion of f(x-h) about x:
( ) ( ) ( )
( )
( )
( )
( ) + +
'
= x f
h
x f
h
x f h x f h x f
3
3
2
2
! 3 2
The Second Three-point Formula
Subtract one expression from another
( ) ( ) ( )
( )
( )
( )
( ) + + +
'
= + x f
h
x f
h
x f h h x f h x f
6
6
3
3
! 6
2
! 3
2
2
( ) ( ) ( )
( )
( )
( )
( ) + + +
'
= + x f
h
x f
h
x f h h x f h x f
6
6
3
3
! 6
2
! 3
2
2
( ) ( )
( )
( )
( )
( )
( ) + + +
'
=
+
x f
h
x f
h
x f
h
h x f h x f
6
5
3
2
! 6 ! 3 2
( )
( ) ( )
( )
( )
( )
( )
+
=
'
x f
h
x f
h
h
h x f h x f
x f
6
5
3
2
! 6 ! 3 2
( )
( ) ( )
2
2
h O
h
h x f h x f
x f +
+
=
'
( )
( )
( )
( )
( ) = x f
h
x f
h
h O
6
5
3
2
2
! 6 ! 3
h
h x f h x f
x f
2
+
~
'
Second Three-point Formula
( )
( ) ( )
h
x f h x f
x f
+
~
'
Forward Difference
Formula
Summary of Errors
( )
( )
( )
( )
( ) + + = x f
h
x f
h
h O
3
2
2
! 3 2
Error term
( )
( ) ( ) ( )
h
x f h x f h x f
x f
2
3 4 2 + + +
~
'
First Three-point Formula
Summary of Errors continued
( )
( )
( )
( ) = x f
h
x f
h
h O
4
3
3
2
2
! 4
6
! 3
2
Error term
h
h x f h x f
x f
2
+
~
'
Second Three-point Formula
( )
( )
( )
( ) = x f
h
x f
h
h O
6
5
3
2
2
! 6 ! 3
Error term
Summary of Errors continued
Example:
( )
x
xe x f =
Find the approximate value of
( ) 2 f
'
1.9 12.703199
2.0 14.778112
2.1 17.148957
2.2 19.855030
x x f
with 1 0. = h
Using the Forward Difference formula:
( ) ( ) ( )
{ }
0 0 0
1
f x f x h f x
h
'
~ +
( ) ( ) ( )
{ }
{ }
1
2 2 1 2
0 1
1
17 148957 14 778112
0 1
23 708450
f f . f
.
. .
.
.
'
~
=
=
Using the 1
st
Three-point formula:
032310 . 22
855030 . 19
148957 . 17 4 778112 . 14 3
2 . 0
1

) 2 . 2 ( ) 1 . 2 ( 4 ) 2 ( 3
1 . 0 2
1
2
=

+ =
+

~
'
f f f f
h x f h x f x f
h
x f 2 4 3
2
1
0 0 0 0
+ + + ~
'
Using the 2
nd


Three-point formula:
228790 . 22
703199 . 12 148957 . 17
2 . 0
1

) 9 . 1 ( ) 1 . 2 (
1 . 0 2
1
2
=
=

~
'
f f f
The exact value of ( ) 22.167168 : is 2 f
'
h x f h x f
h
x f + ~
'
0 0 0
2
1
Comparison of the results with h = 0.1
Formula Error
Forward Difference 23.708450 1.541282
1st Three-point 22.032310 0.134858
2nd Three-point 22.228790 0.061622
( )
2 f
'
The exact value of
( )
2 f
' is 22.167168
Second-order Derivative
( ) ( ) ( )
( )
( )
( )
( ) + + +
'
+ = + x f
h
x f
h
x f h x f h x f
3
3
2
2
! 3 2
( ) ( ) ( )
( )
( )
( )
( ) + +
'
= x f
h
x f
h
x f h x f h x f
3
3
2
2
! 3 2
Add these two equations.
( ) ( ) ( )
( )
( )
( )
( ) + + + = + + x f
h
x f
h
x f h x f h x f
4
4
2
2
! 4
2
2
2
2
( ) ( ) ( )
( )
( )
( )
( )
2 4
2 4
2 2
2
2 4!
h h
f x h f x f x h f x f x + + = + +
( ) ( ) ( )
( )
( )
( )
( ) + + =
+ +
x f
h
x f
h
h x f x f h x f
4
2
2
2
! 4
2 2
( )
( )
( ) ( ) ( )
( )
( ) +
+ +
= x f
h
h
h x f x f h x f
x f
4
2
2
2
! 4
2 2
( )
( )
( ) ( ) ( )
2
2
2
h
h x f x f h x f
x f
+ +
~
NUMERICAL INTEGRATION
=
b
a
dx x f ) (
area under the curve f(x) between
. to b x a x = =
In many cases a mathematical expression for f(x) is
unknown and in some cases even if f(x) is known its
complex form makes it difficult to perform the integration.
y
x
x
0
=a
x
0
=b
f(a)
f(b)
f(x)
y
x
x
0
=a
x
0
=b
f(a)
f(b)
f(x)
Area of the trapezoid
The length of the two parallel sides of the trapezoid
are: f(a) and f(b)
The height is b-a
) ( ) (
) ( ) ( ) (
b f a f
h
b f a f
a b
dx x f
b
a
+ =
+

~
2
2
Simpsons Rule:
2
0
2
0
x
x
x
x
dx x P dx x f ) ( ) (
h x x h x x 2
0 2 0 1
+ = + = and
( ) ( )
( )
( )
2 2
0 0
2
0
2
0
1 2
0
0 1 0 2
0 2
1
1 0 1 2
0 1
2
2 0 2 1


x x
x x
x
x
x
x
( x x )( x x )
P x dx f x dx
( x x )( x x )
( x x )( x x )
f x dx
( x x )( x x )
( x x )( x x )
f x dx
( x x )( x x )

=


+


+

} }
}
}
) ( ) (
) ( ) (
2 1 0
4
3
2
0
2
0
x f x f x f
h
dx x P dx x f
x
x
x
x
y
x
x
0
=a
x
0
=b
f(a)
f(b)
f(x)
y
x
x
0
=a
x
0
=b
f(a)
f(b)
f(x)
y
x
x
0
=a
x
0
=b
f(a)
f(b)
f(x)
y
x
x
0
=a
x
0
=b
f(a)
f(b)
f(x)
Composite Numerical Integration
Riemann Sum
The area under the curve is subdivided into n
subintervals. Each subinterval is treated as a
rectangle. The area of all subintervals are added
to determine the area under the curve.
There are several variations of Riemann sum as
applied to composite integration.
In Left Riemann
sum, the left-
side sample of
the function is
used as the
height of the
individual
rectangle.
a
b
Left Riemann Sum
y
x
x A
( )
( )
1
2
3
2

1
i
x b a / n
x a
x a x
x a x
x a i x
A =
=
= + A
= + A
= + A
( ) ( )
1
n
b
i
a
i
f x dx f x x
=
~ A

}
In Right Riemann
sum, the right-side
sample of the
function is used as
the height of the
individual rectangle.
a
b
Right Riemann Sum
y
x
x A
( )
1
2
3
2
3

i
x b a / n
x a x
x a x
x a x
x a i x
A =
= + A
= + A
= + A
= + A
( ) ( )
1
n
b
i
a
i
f x dx f x x
=
~ A

}
In the Midpoint
Rule, the sample at
the middle of the
subinterval is used
as the height of the
individual
rectangle.
a
b
Midpoint Rule
y
x
x A
( )
( )( )
( )( )
( )( )
( )( )
1
2
3
2 1 1 2
2 2 1 2
2 3 1 2

2 1 2
i
x b a / n
x a x /
x a x /
x a x /
x a i x /
A =
= + A
= + A
= + A
= + A
( ) ( )
1
n
b
i
a
i
f x dx f x x
=
~ A

}
Composite Trapezoidal Rule:
Divide the interval into n subintervals and apply
Trapezoidal Rule in each subinterval.
(

+ + ~

=
) ( ) ( ) ( b f x f a f
h
dx x f
n
k
k
b
a
1
1
2
2
where
n k kh a x
n
a b
h
k
, ... , , , for and 2 1 0 = + =

=
by dividing the interval into 20 subintervals.
Find

sin
0
(x)dx
20 2 1 0
20
20
20
....., , , , ,

= = + =
=

=
=
k
k
kh a x
n
a b
h
n
k
995886 1
20
2 0
40
2
2
19
1
1
1
0
.
sin

sin sin

) ( ) ( ) sin(

=
(

+
|
.
|

\
|
+ =
(

+ + ~
=

=
k
n
k
k
k
b f x f a f
h
dx x
Composite Simpsons Rule:
Divide the interval into n subintervals and apply
Simpsons Rule on each consecutive pair of subinterval.
Note that n must be even.
=

=
) ( ) (
) ( ) (
/
) / (
b f x f
x f a f
h
dx x f
n
k
k
n
k
k
b
a
2
1
1 2
1 2
1
2
4
2
3
n k kh a x
n
a b
h
k
, ... , , , for and 2 1 0 = + =

=
where
by dividing the interval into 20 subintervals.
Find

sin
0
(x)dx
20 2 1 0
20
20
20
....., , , , ,

= = + =
=

= =
k
k
kh a x
n
a b
h n
k
000006 2
20
1 2
4
20
2
2 0
60
10
1
9
1
0
.
) sin(
) (
sin

sin sin

) sin(

=
(

(
+
|
.
|

\
|

+

|
.
|

\
|
+ ~
=
=
k
k
k
k
dx x