Numerical Differentiation

Chapter 1 Lecture 3

 At the end of this session students should be able to

estimate the derivative using forward, backward and
centered finite divided difference.

Numerical Differentiation
The Taylor Series expansion (Forward Taylor Series expansion) of the
function f at xi 1 is given by
f " ( xi ) 2
f ( n ) ( xi ) n
f ( xi 1 ) f ( xi ) f ' ( xi )h
h ...
h Rn
2!
n!
f ( n 1) ( ) n 1
h
where Rn
is a remainder term.
(n 1)!
Truncate the series after the first derivative term:

f ( xi 1 ) f ( xi ) f ' ( xi )h R1
This equation can be solved for

f ' ( xi )

f ( xi 1 ) f ( xi ) R1

h
h

Numerical Differentiation
f ( xi 1 ) f ( xi ) R1
f ' ( xi )

h
h
First-order
approximation

Truncation error

The first part of the equation is the relationship used to approximate

the derivative and the second part is used to estimate the truncation
error associated with this approximation of the derivative.

Numerical Differentiation
f (x)
f ( xi 1 )

f ' ( xi )
f ( xi )

h
xi

xi 1

f ( xi 1 ) f ( xi ) R1
f ' ( xi )

h
h

Numerical Differentiation

Here

f " ( ) 2
h
2
R1 f " ( )

h O ( h)
h
2
R1

Thus, the estimate of the derivative has a truncation error of order

h.
If we halve the step size, i.e.,
h 12 h , we would expect to halve
the error of the derivative, i.e.,

R1
f " ( )

h
h
4

Forward Difference Approx. of the First Derivative

The equation

f ( xi 1 ) f ( xi ) R1

h
h
f ( xi 1 ) f ( xi )

O ( h)
h

f ' ( xi )

is called a Forward finite difference because it utilizes

data at xi and xi 1 to estimate the derivative, where

f " ( xi )
f (3) ( xi ) 2 f ( 4 ) ( xi ) 3
O (h)
h
h
h ........
2!
3!
4!

Backward Difference Approx. of the First Derivative

f ' ( xi )

f (x)
f ( xi )

f ( xi 1 )

xi 1
Let

h xi xi 1

xi

so that

f ( xi 1 ) f ( xi h)
f " ( xi ) 2 f ( 3) ( xi ) 3
f ( xi ) f ' ( xi )h
h
h .......
2!
3!

Backward Difference Approx. of the First Derivative

Truncating this equation after the first derivative

f ( xi 1 ) f ( xi ) f ' ( xi )h R1

and rearranging yields

where

f ' ( xi )

f ( xi ) f ( xi 1 )
O ( h)
h

f " ( xi )
f (3) ( xi ) 2
O ( h)
h
h .......
2!
3!

This equation is called a Backward Finite Difference because it

utilizes data at xi 1 and xi to estimate the derivative.

Centered Difference Approx. of the First Derivative

A third way to approx. the first derivative is to subtract

the backward from the forward Taylor series expansion
as follows:

Forward :
f " ( xi ) 2 f (3) ( xi ) 3
f ( xi 1 ) f ( xi ) f ' ( xi )h
h
h ......
2!
3!
Backward :
f " ( xi ) 2 f ( 3) ( xi ) 3
f ( xi 1 ) f ( xi ) f ' ( xi )h
h
h .......
2!
3!

Centered Difference Approx. of the First Derivative

Forward Backward:

f ( 3) ( xi ) 3
f ( xi 1 ) f ( xi 1 ) 2 f ' ( xi ) h
h ......
3!

f (3) ( xi ) 2
2h f ' ( xi )
h ......
3!

f ( xi 1 ) f ( xi 1 )
f (3) ( xi ) 2

f ' ( xi )
h .......
2h
3!

f ( xi 1 ) f ( xi 1 ) f ( 3) ( xi ) 2
f ' ( xi )

h .......
2h
3!

f ( xi 1 ) f ( xi 1 )
O(h 2 )
2h

Centered Difference Approx. of the

First Derivative

where

f ( 3) ( xi ) 2 f (5) ( xi ) 4
O(h )
h
h ......
3!
5!
2

This equation is shown in the diagram below:

f (x)
f ( xi 1 )

f ' ( xi )

f ( xi )
f ( xi 1 )

xi 1

xi

xi 1

Centered Difference Approx. of the First Derivative

This equation is called a Centered Finite Difference

representation of the first derivative. Note that the
2
truncation error is of the order of h whereas the
forward and backward approximation are of the order of
h. Consequently, the centered difference is a more
accurate representation of the derivative. Because it
would approximately halve the truncation error, whereas
for centered difference, the error would be quartered.

Example 1

Use forward and backward difference approx. and

centered difference approx. to estimate the first
derivative of the function

f ( x) 0.1x 4 0.15 x 3 0.5 x 2 0.25 x 1.2

at x=0.5 using a step size h=0.5. Repeat the
computation using h=0.25. Compute your results with
the true value of the derivative.

Solution
f ( x) 0.1x 4 0.15 x 3 0.5 x 2 0.25 x 1.2
The derivative of f(x) is

f ' ( x) 0.4 x 3 0.45 x 2 1.0 x 0.25

The true value is f ' (0.5) 0.9125
For the step size h=0.5, we have

xi 1 0

f ( xi 1 ) f (0) 1.2

xi 0.5

f ( xi ) f (0.5) 0.925

xi 1 1.0

f ( xi 1 ) f (1.0) 0.2

Solution

The backward difference approx. is

f ( xi ) f ( xi 1 )
h
0.925 1.2
f ' (0.5)
0.55
0.5
0.9125 0.55
t
100% 39.7%
0.9125
f ' ( xi )

Solution

The forward difference approx. is

f ( xi 1 ) f ( xi )
h
0.2 0.925
f ' (0.5)
1.45
0.5
0.9125 1.45
t
100% 58.9%
0.9125
f ' ( xi )

Solution

The centered difference approx. is

f ( xi 1 ) f ( xi 1 )
2h
0.2 1.2
f ' (0.5)
1.0
2(0.5)
f ' ( xi )

0.9125 1.0
100% 9.6%
0.9125

Solution

For the step size h=0.25, we have

xi 1 0.25

f ( xi 1 ) f (0.25) 1.10351563

xi 0.5

f ( xi ) f (0.5) 0.925

xi 1 0.75

f ( xi 1 ) f (0.75) 0.63632813

The backward difference approx. is

f ( xi ) f ( xi 1 )
h
0.925 1.10351563
f ' (0.5)
0.714
0.25
0.9125 0.714
t
100% 21.75%
0.9125
f ' ( xi )

Solution

The forward difference approx. is

f ( xi 1 ) f ( xi )
h
0.63632813 0.925
f ' (0.5)
1.155
0.25
0.9125 1.155
t
100% 26.5%
0.9125
f ' ( xi )

Solution

The centered difference approx. is

f ( xi 1 ) f ( xi 1 )
2h
0.63632813 1.10351563
f ' (0.5)
0.934
2(0.5)
f ' ( xi )

0.9125 0.934
100% 2.4%
0.9125

Solution

h=0.5
Backward
Forward

h=0.25

t 39.7%

t 21.75%

t 58.9%

t 26.5%

t 9.6%

t 2.4%

Centered
For both step sizes, the centered difference approx. is more accurate
than forward and backward differences. Also as predicted by the
Taylor series analysis, halving the step size approximately halves the
error of backward and forward difference and quarters the error of
the centered difference.

Finite Difference Approximation of Higher Derivatives

Taylor series expansion can be used to derive numerical estimates of

higher derivatives. To do this, we write a forward Taylor series
f ( xi 2 )
expansion for f ( xi ) in terms of
:
f " ( xi )
f (3) ( xi )
2
f ( xi 2 ) f ( xi ) f ' ( xi )( 2h)
( 2 h)
(2h) 3 ....
2!
3!
(1)
From the forward Taylor series expansion

f " ( xi ) 2 f ( 3) ( xi ) 3
f ( xi 1 ) f ( xi ) f ' ( xi )h
h
h ......
2!
3!
(2)

Finite Difference Approximation of

Higher Derivatives
Eq.(1) - 2xEq.(2) gives

f ( xi 2 ) 2 f ( xi 1 )

f ( xi ) f " ( xi )h 2 f (3) ( xi )h 3 127 f ( 4 ) ( xi )h 4 ......

which can be solved for

f " ( xi )
where

f ( xi 2 ) 2 f ( xi 1 ) f ( xi )
O(h)
2
h

O(h) f (3) ( xi )h 127 f ( 4 ) ( xi )h 2 ......

This relationship is called the Second forward finite difference.

Finite Difference Approximation of

Higher Derivatives

Similar manipulations can be employed to derive a

Second backward finite difference as

f " ( xi )
where

f ( xi ) 2 f ( xi 1 ) f ( xi 2 )
O ( h)
2
h

O(h) f (3) ( xi )h 127 f ( 4) ( xi ) h 2 ......

Finite Difference Approximation of

Higher Derivatives

The Second centered finite difference is

f " ( xi )
where

f ( xi 1 ) 2 f ( xi ) f ( xi 1 )
O ( h)
h2

1
O( h) 121 f ( 4 ) ( xi )h 2 360
f ( 6 ) ( xi )h 4 ......

Alternatively, the Second Centered finite difference can

be expressed as

f ( xi 1 ) f ( xi ) f ( xi ) f ( xi 1 )

h
h
f " ( xi )
h

Example 6

Use a centered difference approximation to estimate the

second derivative of the function

f ( x) 25 x 3 6 x 2 7 x 88
at x=2 using step sizes of h=0.25 and 0.125. Compare
your estimates with the true value of the second
derivative. Interpret your results on the basis of the
remainder term of the Taylor series expansion.

Solution

f ( x) 25 x 3 6 x 2 7 x 88
f ' ( x) 75 x 2 12 x 7
f " ( x) 150 x 12
The true value of the second derivative at x=2 is

f " (2) 150(2) 12 288

Solution
xi 1 1.75

f ( xi 1 ) f (1.75) 39.85938

xi 2

f ( xi ) f (2) 102

xi 1 2.25

f ( xi 1 ) f (2.25) 182.1406

h=0.25

f ( xi 1 ) 2 f ( xi ) f ( xi 1 )
h2
f (2.25) 2 f (2) f (1.75)
f " ( 2)
288
2
(0.25)

f " ( xi )

Solution
h=0.125

xi 1 1.875

f ( xi 1 ) f (1.875) 68.82617

xi 2

f ( xi ) f (2) 102

xi 1 2.125

f ( xi 1 ) f (2.125) 139.6738

f (2.125) 2 f (2) f (1.875)

288
(0.125) 2
Both results are exact because the errors are a function
of 4th and higher derivatives which are zero for a 3rd
order polynomial function.
f " (2)