Numerical Methods for Initial Value Problems

Kirankumar Hiremath
CoE Systems Science, IITJ
August 29, 2013
Initial value problems (IVPs)
IVP: Given a dierential equation and initial values
dy
dt
= f(t, y), and y(t
0
) = f
0
. (1)
y: Dependent variable, t: independent variable
If f = f(y) Autonomous ODE
Not all IVPs can be solved analytically Numerical methods
Here we will focus on: First order linear IVPs. IVPs involving higher
order ODEs can be rendered as a system of rst order ODEs.
Hiremath, K. R. (IITJ) 1 / 19 August 29, 2013
Euler method (Euler explicit or Euler forward)
Given
dy
dt
= f(t, y), and y(t
0
) = y
0
, and we want to nd the solution
for t
0
t t
end
.
Discretise the interval [t
0
, t
end
] with a constant stepsize h.
t
n
= t
0
+ nh, y
n
= y(t
n
).
Knowing y at t
0
, how to compute y at t
1
= t
0
+ h?
Taylor series expansion:
y(t
1
) = y(t
0
+ h) = y(t
0
) + hy

(t
0
) + E
t
(h),
where
E
t
(h) =
h
2
2
y

(), t
0
< < (t
0
+ h) = O(h
2
)
is the local truncation error.
Hiremath, K. R. (IITJ) 2 / 19 August 29, 2013
If h is suciently small, then we can approximate
y(t
1
) = y(t
0
+ h) = y(t
0
) + hy

(t
0
)
= y(t
0
) + hf(t
0
, y(t
0
)), (using given ODE)
Similarly
y(t
2
) = y(t
1
+ h) = y(t
1
) + hf(t
1
, y(t
1
))
y(t
3
) = y(t
2
+ h) = y(t
2
) + hf(t
2
, y(t
2
))
.
.
.
y(t
n+1
) = y(t
n
) + hf(t
n
, y(t
n
))
This method is called forward Euler method, explicit Euler method, or
the standard Euler method.
Discretization Errors
Is the discretization right (consistent)?
As h 0, discrete problem continuous problem.
Is the scheme stable?
Errors dont grow unbounded.
Hiremath, K. R. (IITJ) 3 / 19 August 29, 2013
Error analysis of the Euler method: Consistency
Consistency: A numerical scheme is said to be consistent with the
ODE being discretized, if the local truncation error tends to zero as the
mesh size tends to zero.
1 From Taylor series expansion Local truncation error E
t
(h) = O(h
2
)
2 Finite precision calculation Round o errors
3 Total error/ global error
= Local truncation error propagation + Round o errors = O(h)
Hiremath, K. R. (IITJ) 4 / 19 August 29, 2013
Error analysis of the Euler method: Stability
Stability: Errors propagate Investigation of unbounded growth of
errors
Error analysis for general case can be dicult!
To get insight: Simple case
dy
dt
= ay, y(0) = y
0
, a > 0(constant)
Analytic solution: y = y
0
e
at
( asymptotically approached to 0.)
Forward Euler scheme: y
n+1
= y
n
hay
n
= (1 ah)y
n
If |1 ah| > 1 The numerical solution will grow unbounded.
The stability of the method depends on the stepsize h.
The forward Euler method is conditionally stable.
Systematic ways of analysing the stability: Von Neumann method
based on Fourier decomposition of error
Hiremath, K. R. (IITJ) 5 / 19 August 29, 2013
Example
Solve the IVP
y

(t) = 4y, y(0) = 4

over an interval [0, 2] with the Euler forward method with h = 0.5, 0.2, 0.1,
and 0.05.
Hiremath, K. R. (IITJ) 6 / 19 August 29, 2013
Improvements in the forward Euler method
Forward Euler method:
y(t
n+1
) = y(t
n
) + hf(t
n
, y(t
n
))
A new value of y(t) = y(t
n+1
) is predicted using
1 the old value of y = y(t
n
), and
2 the slope at the old value of t = t
n
(beginning of the subinterval)
to extrapolate linearly over the stepsize h.
General scheme:
y
n+1
= y
n
+ h,
: increment function How to dened it?.
Hiremath, K. R. (IITJ) 7 / 19 August 29, 2013
Improvements in the forward Euler method
Why not approximate the constant slope on [t
n
, t
n+1
] by
1 the slope at the end-point of the interval? (Backward Euler method)
y
n+1
= y
n
+ hf(t
n+1
, y
n+1
) (2)
2 the slope at the mid-point of the interval?
y
n+1
= y
n
+ hf(t
n+1/2
, y
n+1/2
) (3)
3 the average of the slopes at t
n
and t
n+1
?
y
n+1
= y
n
+ h
_
f(t
n
, y
n
) + f(t
n+1
, y
n+1
)
2
_
(4)
Lets start with the rst choice.: Solve the IVP
y

(t) = 4y, y(0) = 4

over an interval [0, 2] with the backward Euler method with h = 0.5,
and 0.2. Compare these results with that of the forward Euler method.
Hiremath, K. R. (IITJ) 8 / 19 August 29, 2013
One more example. . .
Consider
y

(t) = 1 + t + y
2
. .
f(t,y)
, y(0) = 0
The forward Euler scheme:
y
n+1
= y
n
+ h(1 + t
n+1
+ y
2
n+1
)
y
n+1
on both the sides Solve nonlinear algebraic equation.
This is a typical case when f(t, y) is nonlinear in y. Implicit scheme:
Implicit Euler method Computationally expensive
:-(( Diculty with the mid-point: y
n+1
= y
n
+ hf(t
n+1/2
, y
n+1/2
. .
unknown
)
How to overcome the above diculties?
Predictor-Corrector methods
Hiremath, K. R. (IITJ) 9 / 19 August 29, 2013
The mid-point method
Motivation: Use the slope information (= f(t
n+1/2
, y
n+1/2
)) at the
mid-point t
n+1/2
of the interval [t
n
, t
n+1
].
Diculty: We dont know y
n+1/2
.
Way out:
1 Using the standard Euler method, predict the value of y
n+1/2
:
y
n+1/2
= y
n
+
h
2
f(t
n
, y
n
)
2 Using the above predicted value of y
n+1/2
, correct the value of y
n+1
:
y
n+1
= y
n
+ hf(t
n+1/2
, y
n+1/2
)
The mid-point method: (LTE=O(h
3
), GE= O(h
2
))
y
n+1/2
= y
n
+
h
2
f(t
n
, y
n
) (Predictor)
y
n+1
= y
n
+ hf(, t
n+1/2
, y
n+1/2
) (Corrector)
Hiremath, K. R. (IITJ) 10 / 19 August 29, 2013
Heuns method (Improved/modied Euler method)
Motivation: Use the slope information at the both end points of the
interval [t
n
, t
n+1
].
Diculty: Possibility of getting an implicit scheme!
Way out: Repeat the trick used in the mid-point method :-B
Heuns method: (LTE=O(h
3
), GE= O(h
2
))
y
0
n+1
= y
n
+ hf(t
n
, y
n
) (Predictor)
y
n+1
= y
n
+ h
_
f(t
n
, y
n
) + f(t
n+1
, y
0
n+1
)
2
_
(Corrector)
Iterated Heuns method: For m = 1, 2, (+ Termination criterion)
y
0
n+1
= y
n
+ hf(t
n
, y
n
) (Predictor)
y
m
n+1
= y
n
+ h
_
f(t
n
, y
n
) + f(t
n+1
, y
m1
n+1
)
2
_
(Corrector)
Hiremath, K. R. (IITJ) 11 / 19 August 29, 2013
Example
Solve the IVP
y

(t) = 4y, y(0) = 4

over an interval [0, 2] with the mid-point method and Heuns method with
h = 0.5, 0.2, 0.1, and 0.05.. Compare these results with that of the forward
Euler method.
Hiremath, K. R. (IITJ) 12 / 19 August 29, 2013
Recap
The mid-point method:
y
n+1/2
= y
n
+
h
2
f(t
n
, y
n
)
y
n+1
= y
n
+ hf(t
n+1/2
, y
n+1/2
)
Heuns method:
y
0
n+1
= y
n
+ hf(t
n
, y
n
)
y
m
n+1
= y
n
+ h
_
f(t
n
, y
n
) + f(t
n+1
, y
m1
n
)
2
_
, m = 1, 2,
General idea:
1 Evaluate the slope function at various points within the time step.
2 Use the weighted average of all these slope values to extrapolate the y
value at the next time step.
Generalize this concept for higher order accurate methods The
Runge-Kutta (RK) methods
Hiremath, K. R. (IITJ) 13 / 19 August 29, 2013
Back to basics. . .
Suppose we want to construct a second order accurate method.
Taylor series:
y
n+1
= y(t
n+1
) = y(t
n
+h) = y(t
n
)+hy

(t
n
)+
h
2
2
y

(t
n
)+O(h
3
) (5)
From the given IVP:
y

(t
n
) = f(t
n
, y
n
) (6)
y

(t
n
) =
_
d
dt
f(t, y) +
d
dy
f(t, y)
dy
dt
_
(tn,yn)
= f
t
(t
n
, y
n
) + f
y
(t
n
, y
n
)y

(t
n
) (7)
Using Eq.(7) in Eq.(5) gives
y
n+1
= y
n
+ hf(t
n
, y
n
) +
h
2
2
_
f
t
(t
n
, y
n
) + f
y
(t
n
, y
n
)y

(t
n
)

+O(h
3
)
y
n+1
= y
n
+
h
2
f(t
n
, y
n
)
+
h
2
[f(t
n
, y
n
) + hf
t
(t
n
, y
n
) + hf
y
(t
n
, y
n
)f(t
n
, y
n
)] +O(h
3
)
Hiremath, K. R. (IITJ) 14 / 19 August 29, 2013
Inspect the equation
y
n+1
= y
n
+
h
2
f(t
n
, y
n
)
+
h
2
[f(t
n
, y
n
) + hf
t
(t
n
, y
n
) + hf
y
(t
n
, y
n
)f(t
n
, y
n
)] +O(h
3
) (8)
Multivariate Taylor series expansion:
f(t
n
+ h, y
n
+ k) = f(t
n
, y
n
) + hf
t
(t
n
, y
i
) + kf
y
(t
n
, y
n
) +
f(t
n
+h, y
n
+hf(t
n
, y
n
)) = f(t
n
, y
n
) + hf
t
(t
n
, y
n
) + hf(t
n
, y
n
)f
y
(t
n
, y
n
)+O(h
2
)
Thus Eq. (8) becomes
y
n+1
= y
n
+
h
2
f(t
n
, y
n
) +
h
2
f(t
n
+ h, y
n
+ hf(t
n
, y
n
)) +O(h
3
)
Rewriting Classical second order Runge-Kutta (RK2) method
y
n+1
= y
n
+ h
_
1
2
k
1
+
1
2
k
2
_
k
1
= f(t
n
, y
n
)
k
2
= f(t
n
+ h, y
n
+ hk
1
)
Hiremath, K. R. (IITJ) 15 / 19 August 29, 2013
The classical second order Runge-Kutta method
y
n+1
= y
n
+ h
_
1
2
k
1
+
1
2
k
2
_
k
1
= f(t
n
, y
n
)
k
2
= f(t
n
+ h, y
n
+ hk
1
)
1 The k
1
and k
2
are known as the stages of RK method.
2 They correspond to dierent estimations for the slope of the solution.
k
1
: slope estimation at the start of the interval t
n
using y
n
k
2
: slope estimation at the end of the interval t
n+1
using y
n
+ hk
1
3 This scheme can be interpreted as a predictor-corrector method.
Heuns method (with m = 1)
Hiremath, K. R. (IITJ) 16 / 19 August 29, 2013
Generalized second order Runge-Kutta methods
y
n+1
= y
n
+ h(b
1
k
1
+ b
2
k
2
)
k
1
= f(t
n
, y
n
)
k
2
= f(t
n
+
2
h, y
n
+
21
hk
1
, )
1 b
1
= b
2
=
1
2
and
21
=
2
= 1 Heuns method
2 b
1
= 0, b
2
= 1 and
21
=
2
=
1
2
The mid-point method
Conclusion: several versions of RK2 method
Four unknowns and three equations (Ref: Chapras book)
This is also true for a general nth order RK method.
Hiremath, K. R. (IITJ) 17 / 19 August 29, 2013
Generalized mth order explicit Runge-Kutta methods
y
n+1
= y
n
+ h
m

j=1
b
j
k
j
k
1
= f(t
n
, y
n
)
k
2
= f(t
n
+
2
h, y
n
+
21
hk
1
)
.
.
.
k
m
= f(t
n
+
n
h, y
n
+ h
m1

j=1

n,j
k
j
)
Most popular: Classical RK4 method
Hiremath, K. R. (IITJ) 18 / 19 August 29, 2013
Classical 4th order Runge-Kutta method
k
1
= f(t
n
, y
n
)
k
2
= f(t
n
+
1
2
h, y
n
+
1
2
k
1
h)
k
3
= f(t
n
+
1
2
h, y
n
+
1
2
k
2
h)
k
4
= f(t
n
+ h, y
n
+ k
3
h)
y
n+1
= y
n
+
1
6
(k
1
+ 2k
2
+ 2k
3
+ k
4
)h
When f = f(t): Classical RK4 is similar to Simpsons 1/3 rule
Local truncation error: O(h
5
)
Global error: O(h
4
)
Hiremath, K. R. (IITJ) 19 / 19 August 29, 2013