Chapter 23

Numerical Differentiation

1
 Recall - Taylor Series Expansion

h2 hn n
f (x i1 )  f (x i )  h f (x i )  f (x i )   f (x i )  O(h n 1)
2! n!

 0th order T.S. approx. (n = 0): f(xi+1) = f(xi) + O (h1)

 1st order T.S. approx. (n = 1): f(xi+1) = f(xi) + hf '(xi) + O (h2)

 2nd order T.S. approx. (n = 2):

h2
f (x i1 )  f (x i )  h f (x i )  f (x i )  O (h n 1)
2!
 nth order T.S. approximation will be exact for an nth order
polynomial

2
 Numerical Differentiation from Taylor Series Expansion
Taylor series:

h2 h3 hn n
f (x i1 )  f (x i ) hf (x i )  f (x i )  f (x i )   f (x i )  R n
2! 3! n!
h = "step size" = xi+1 – xi

Taylor series expanded backward:

3
 First order accuracy

• First derivative
• Second derivative
• Third derivative
• Fourth derivative

4
 Numerical Differentiation

1. Backward difference
f ( xi )  f ( xi 1 )
f ( xi ) 
/

h
f /
( x )  f /
( xi 1 )
f ( xi ) 
// i
h
f ( xi )  2 f ( xi 1 )  f ( xi  2 )
 2
h
f ( xi )  3 f ( xi 1 )  3 f ( xi  2 )  f ( xi 3 )
f ( xi ) 
///

h3
 First Derivatives: Backward Difference

Backward Difference Approx.:

First Derivative:

h2
f (x i1 )  f (x i )  (x i1  x i )f '(x i )  f "()
2
Letting h = xi - xi-1
h2
f (x i1 )  f (x i )  hf '(x i )  f "()
2
h2
hf '(x i )  f (x i )  f (x i1 )  f "()
2
f (x i )  f (x i1 )
f '(x)   O(h) 1st order error
h
first backward difference
6
 Second Derivatives: Backward Difference

Second Derivative:

f (x i2 )  f (x i )   x i 2  x i  f '(x) 
 x i 2  x i  2
f "(x i )
2!
+ O([xi-2– xi]3)

with h = xi– xi-1 and 2h = xi – xi-2

The 2nd order approximation to ƒ(xi-2) becomes:

ƒ(xi-2) = ƒ(xi) – 2hƒ'(xi) + 2h2 ƒ"(xi) +O (h3) [1]

2nd order approximation to ƒ(xi-1):

h2
f (x i1 )  f (x i )  hf '(x)  f "(x i )  O(h 3 ) [2]
2!
7
Second Derivatives: Backward Difference

Subtracting 2*[2] from [1] yields:

f(xi-2) – 2f(xi-1) = –f(xi) + h2f"(xi) + O (h3)
Rearranging:
h2ƒ"(xi) = f(xi) – 2f(xi-1) + f(xi-2) + O (h3)

f (x i )  2f (x i1 )  f (x i2 )  O(h 3 )

f "(x i )  1st order
h2 error
Second backward difference

8
 Numerical Differentiation
2. Forward difference
Taylor series :
f xi 1   f xi   f xi h  

f ( xi 1 )  f ( xi )
f ( xi ) 
/

h
f /
( x )  f /
( xi )
f ( xi ) 
// i 1
h
f ( xi  2 )  2 f ( xi 1 )  f ( xi )

h2
f ( xi 3 )  3 f ( xi  2 )  3 f ( xi 1 )  f ( xi )
f ( xi ) 
///

h3
 Second derivatives

f  xi  2
f  xi 1   f  xi   f  xi h  h   (1)
2!
f  xi 
f  xi  2   f  xi   f  xi 2h  
 2h   (2)
2

2!
Eq. 1 can be multiplied by 2 and subtracted from eq. 2:

f xi 2   2 f xi 1    f xi   f xi h  

2

Solve:
f xi  2   2 f xi 1   f xi 
f xi   2
 Oh 
h

Second derivative forward finite divided difference

 Numerical Differentiation

3. Centered difference

f ( xi 1 )  f ( xi 1 )
f ( xi ) 
/

2h
f ( xi 1 )  2 f ( xi )  f ( xi 1 )
f ( xi ) 
//

h2

f ( xi  2 )  2 f ( xi 1 )  2 f ( xi 1 )  f ( xi  2 )
f ( xi ) 
///
3
2h
 First Derivatives: Central Difference

Central Difference Approx.:

First Derivative:
f ( xi 1 )  f ( xi )  f ' ( xi )h 
f '' ( xi ) 2
h  Substraction
2! f ( 3) ( xi ) 3
f ( xi 1 )  f ( xi 1 )  2f ' ( xi )h  h 
f '' ( xi ) 2 3!
f ( xi 1 )  f ( xi )  f ' ( xi )h  h 
2!

f ( xi 1 )  f ( xi 1 )
f ' ( xi ) 
2h

2nd order error

12
 Numerical Differentiation: Second Derivatives
Central Difference Approx.:
Second Derivative:
f '' ( xi ) 2
f ( xi 1 )  f ( xi )  f ' ( xi )h  h  addition
2!
f '' ( xi ) 2
f ( xi 1 )  f ( xi )  f ' ( xi )h  h 
2!

2nd order error

13
 High Accuracy Differentiation Formulas

• High-accuracy finite-difference formulas can be generated

by including additional terms from the Taylor series
expansion.

• An example: High-accuracy forward-difference formula for

the first derivative.
 Derivation: High-accuracy forward-
difference formula for f`(x)
f xi  2
f xi 1   f xi   f xi h 
Taylor series expansion
h 
2!
Solve for f’(x)

f xi 1   f xi  f xi 

f xi  
h

2!
h  O h2  
f xi  2   2 f xi 1   f xi  Substitute the forward-
f xi   2
 Oh  difference approx. of f”(x)
h

High-accuracy  f xi  2   4 f xi 1   3 f xi 

forward-difference f xi  
2h
 
 O h2
formula
 Derivation: High-accuracy forward-
difference formula for f`(x)

Similar improved versions can be developed for the

backward and centered formulas as well as for the
approximations of the higher derivatives.
Higher Order Forward Divided Difference

 f ( xi  2 )  4 f ( xi 1 )  3 f ( xi )
f ( xi ) 
/

2h

 f ( xi 3 )  4 f ( xi  2 )  5 f ( xi 1 )  2 f ( xi )
f ( xi ) 
//
2
h

f ( xi ) 
///

 3 f ( xi 4 )  14 f ( xi 3 )  24 f ( xi  2 )  18 f ( xi 1 )  5 f ( xi )
2h 3
Higher Order Backward Divided Difference

3 f ( xi )  4 f ( xi 1 )  f ( xi 2 )
f ( xi ) 
/

2h

2 f ( xi )  5 f ( xi 1 )  4 f ( xi  2 )  f ( xi 3 )
f ( xi ) 
//

h2

f /// ( xi )
5 f ( xi )  18 f ( xi 1 )  24 f ( xi 2 )  14 f ( xi 3 )  3 f ( xi 4 )
 3
2h
 Higher Order Central Divided Difference

 f ( xi  2 )  8 f ( xi 1 )  8 f ( xi 1 )  f ( xi  2 )
f ( xi ) 
/

12h

f // ( xi )
 f ( xi 2 )  16 f ( xi 1 )  30 f ( xi )  16 f ( xi 1 )  f ( xi 2 )

12h 2
f /// ( xi )
 f ( xi 3 )  8 f ( xi  2 )  13 f ( xi 1 )  13 f ( xi 1 )  8 f ( xi  2 )  f ( xi 3 )

8h3
 First Derivatives - Example:
Use forward, backward and centered difference
approximations to estimate the first derivative of
f ( x )  0.1 x 4  0.15 x 3  0.5 x 2  0.25 x  1.2
at x = 0.5 with h = 0.5 and 0.25 (exact sol. = -0.9125)
• Forward Difference
 f (1)  f (0.5) 0.2  0.925
 h  0.5, f ( 0.5)    1.45,  t  58.9%
1  0.5 0.5

h  0.25, f (0.5)  f (0.75)  f (0.5)  0.63632813  0.925  1.155,   26.5%
 0.75  0.5 0.25
t

• Backward Difference
 f (0.5)  f (0) 0.925  1.2
 h  0.5, f ( 0.5)    0.55,  t  39.7%
0.5  0 0.5

h  0.25, f (0.5)  f (0.5)  f (0.25)  0.925  1.10351563  0.714,   21.7%
 0.5  0.25 0.25
t
 First Derivatives - Example:

• Central Difference

f (1)  f (0) 0.2  1.2

h  0.5, f (0.5)    1.0,  t  9.6%
1 0 1

f (0.75)  f (0.25)
h  0.25, f (0.5)  
0.75  0.25
0.63632813  1.10351563
 0.934,  t  2.4%
0.5
Forward Finite-divided differences
Backward finite-divided differences
Centered Finite-Divided Differences
 First Derivatives - Example:

Employing the high-accuracy formulas (h=0.25):

xi-2= 0.0 f(0.0) = 1.2
xi-1= 0.25 f(0.0) = 1.103516
xi = 0.5 f(0.5) = 0.925
xi+1 = 0.75 f(0.75) = 0.63633
xi+2 = 1.0 f(1.0) = 0.2
Forward Difference
 f xi  2   4 f xi 1   3 f xi 
f xi  
2h
 0.2  4(0.6363281)  3(0.925)
f 0.5   0.8594
2(0.25)
 First Derivatives - Example:

 Backward Difference
3(0.925)  4(1.035156)  1.2
f 0.5   0.8781
2(0.25)

 Central Difference
 0.2  8(0.636328)  8(1.035156)
f 0.5   0.9125
12(0.25)
 Summary

True value: f`(0.5) = -0.9125

Basic h = 0.25 Forward Backward Centered
formulas O(h) O(h) O(h2)
Estimate -1.155 -0.714 -0.934
|t| 26.5% 21.7% 2.4%

High- h = 0.25 Forward Backward Centered

Accuracy O(h2) O(h2) O(h4)
formulas Estimate -0.859375 -0.878125 -0.9125
|t| 5.82% 3.77% 0%