Elements of Ordinary Differential Initial

Value Problem Approximation

CHAPTER

1.1 INTRODUCTION
To obtain accurate numerical solutions to differential equations governing physical systems has always
been an important problem with scientists and engineers. These differential equations basically fall into
two classes, ordinary and partial, depending on the number of independent variables, present in the
differential equations: one for ordinary and more than one for partial.
The general form of the ordinary differential equation can be written as
Ly= r
(1.1.1)
where L is a differential operator and r is a given function of the independent variable t. The order of the
differential equation is the order of its highest derivative and its degree is the degree of the derivative of
the highest order after the equation has been rationalized. If no product of the dependent variable y(t)
with itself or any of its derivatives occur, the equation is said to be linear, otherwise it is nonlinear. A
linear differential equation of order m can be expressed in the form
m

Ly =

p=0

f p (t ) y ( p ) (t ) = r (t )

(1.1.2)

in which fp(t) are known functions. The general nonlinear differential equation of order m can be written
as
F (t, y, y,..., y (m1) , y (m ) ) = 0
or

(1.1.3)

y(m) (t) = f (t, y, y,..., y (m 1) )

(1.1.4)

which is called a canonical representation of differential equation (1.1.3). In such a form, the highest
order derivative is expressed in terms of the lower order derivatives and the independent variable. The
general solution of the mth order ordinary differential equation contains m independent arbitrary constants.
In order to determine the arbitrary constants in the general solution if the m conditions are prescribed at
one point, these are called initial conditions. The differential equation together with the initial conditions
is called the initial value problem. Thus, the mth order initial value problem can be expressed as
y (m) (t ) = f (t, y, y,..., y (m 1) )
y (p ) (t0 ) = y0(p ) , p = 0, 1, 2, ..., m 1

(1.1.5)

NUMERICAL SOLUTION OF DIFFERENTIAL EQUATIONS

If the m conditions are prescribed at more than one point, these are called boundary conditions.
The differential equation together with the boundary conditions is known as the boundary value problem.
We shall now discuss the basic concepts needed for the numerical solution of initial value problems.

1.2 INITIAL VALUE PROBLEMS

The mth order initial value problem of Equation (1.1.5) is equivalent to the following systems of m first
order equations:
y = v1 = v2

v1 (t0 ) = y0

v2 = v3

v2 (t0 ) = y0

v m1 = vm

vm 1 (t0 ) = y0

v m = f (t , v1 , v2 ,..., vm )

vm (t0 ) = y0

( m 2)

( m 1)

(1.2.1)

In vector notations it can be written as

dv
= f (t , v ), v (t0 ) = K
dt

where

(1.2.2)

v = [v1 , v2 ,..., vm ]T ,
f (t, v) = [v2 , v3 ... f (t , v1 , v2 ,.., vm )]T ,
h = [ y0 , y0 ,.., y0( m 1) ]T

We shall, therefore, be concerned with methods for finding out numerical approximations to the
solution of the equation
dy
= f (t , y ), y (t0 ) = y0
dt

(1.2.3)

However, before attempting to approximate the solution numerically, we must ask if the problem
has any solution. This can be answered easily in the case of initial value problem for ordinary differential
equation by Theorem 1.1.
Theorem 1.1: We assume that f (t, y) satisfies the following conditions:
(i) f (t, y) is a real function,
(ii) f (t, y) is defined and continuous in the strip
t [t0, b], y ( , ),
(iii) there exists a constant L such that for any t [t0 , b] and for any two numbers y1 and y2.
| f (t , y1 ) f (t , y2 ) | L | y1 y2 |,

where L is called the Lipschitz constant.

Then, for any y0 the initial value problem (1.2.3) has a unique solution y(t) for t [t0, b].
We will always assume the existence and uniqueness of the solution and also that f (t, y) has
continuous partial derivatives with respect to t and y of as high an order as we desire.

ELEMENTS OF ORDINARY DIFFERENTIAL INITIAL VALUE PROBLEM APPROXIMATION

1.3 DIFFERENCE EQUATIONS

In order to develop approximations to differential equations, we define the following operators:

Ey (t ) = y (t + h)

The shift operator

y (t ) = y (t + h ) y (t )

The forward difference operator

y (t ) = y (t ) y (t h)

The backward difference operator

h
y (t ) = y t + y t
2
2

The central difference operator

y (t ) =

1 h
yt + +
2 2

h
y t
2

The average operator

Dy (t ) = y (t )

The differential operator

where h is the difference interval.

Repeated applications of the difference operators lead to the following higher order differences:
n

n y (t ) =

n!

(1)k k !(n k )! y(t + (n k )h)

k =0
n

n y (t ) =
2n y (t ) =

n!

(1)k k !(n k )! y(t kh)

k =0
2n

(2n)!

(1)k k !(2n k )! y (t + (n k )h)

k =0

(1.3.1)
(1.3.2)
(1.3.3)

For linking the difference operators with the differential operator we consider Taylors formula
y (t + h) = y (t ) + h y (t ) +

h2
y (t ) + ...
2!

In operator notations we can write

(hD) 2
Ey(t) = 1 + hD +
...
+
y (t ) .

2!

The series in parentheses is the expression for the exponential and hence we have (formally)
E = ehD
(1.3.4)
Treating Equation (1.3.4) as an identity, we may derive expressions for any order derivative in
terms of the various difference operators. In Table 1.1 we list the relations between the operators. There
are many difference approximations possible for a given differential equation. The selection of a particular
difference relation is usually determined by the nature of the truncation error associated with the
approximation.
As an example, let us develop expressions for the first order derivative in terms of the forward,
backward, and central difference operators. We assume that the function y(t) may be expanded in a
Taylor series in the closed interval t h t t + h. We have
y (t h) = y (t ) h y (t ) +

h2
hn ( n )
y "(t ) ...(1)n
y (t ) + ...
2!
n!

NUMERICAL SOLUTION OF DIFFERENTIAL EQUATIONS

where a prime denotes differentiation with respect to t. We have then

y (t + h) y (t )
h
= y (t ) + y (t ) + O( h2 )
h
2
y(t ) dy
=
+ O (h )
or
h
dt
where the notation O(h) means that the first term neglected is of order h.
Similarly we obtain

(1.3.5)

y(t ) dy
=
+ O ( h)
h
dt

(1.3.6)

However,

y (t ) dy
=
+ O (h 2 )
(1.3.7)
h
dt
The forward and backward differences are accurate to order h; the average central difference is
accurate to order h2.
Thus replacing the derivatives in a differential equation by the difference approximations results
in an equation which may be considered as relating differences of an unknown function and may be
called a difference equation. The order of a difference equation is the number of intervals separating the
largest and the smallest arguments of the dependent variable. A linear difference equation is one in
which no product of the dependent variable with itself or any of its differences appear. A difference
equation is homogeneous if all nonzero terms involve the dependent variable; otherwise it is
inhomogeneous. Furthermore, a difference equation may have coefficients which are constants or
functions of the independent variable. If the coefficients are constants, we call it a difference equation
with constant coefficients; otherwise we call it an equation with variable coefficients.

Table 1.1 Relationship between the operators

E

+1

(1 )1

E1

(1 )1 1

1E1

1 (1 + )1

E1/2 E1/2

(1 + )1/ 2

(1 )1/ 2

hD
2sinh

1/ 2
1 (1 )
2

2
1 +
4

hD
cosh

log (1 )

2sinh 1
2

hD

hD

1
( E1/ 2
2

1/ 2
+ E 1/ 2 ) 1 + (1 + )
2

log E

log (1 + )

1
1 + 2 +
2

hD
2
1 +
4

ehD

2
1 2
+ 1 +
2
4

ehD 1

2
1 +
4

1 ehD

1
2 +
2

ELEMENTS OF ORDINARY DIFFERENTIAL INITIAL VALUE PROBLEM APPROXIMATION

1.3.1 Solution of Linear Difference Equation with Constant Coefficients

A kth order linear inhomogeneous difference equation with constant coefficients is of the form
a0 yn + k + a1 yn + k 1 + ... + ak yn = g n

(1.3.8)

where aj , j = 0, 1,.., k, are constants independent of n, and a0 0, ak 0.

The solution yn of Equation (1.3.8) consists of a solution to the homogeneous equation, say yn(H)
and a particular solution, say yn(P) of the inhomogeneous part.
Substituting gn = 0 in (1.3.8), we get the homogeneous difference equation
a0 yn + k + a1 yn + k 1 + ... + ak yn = 0

(1.3.9)

To find the solution of (1.3.9), we use the trial solution

(1.3.10)

yn = A n

where A 0 is a constant and is a number to be determined.

Inserting (1.3.10) in (1.3.9), we find that nontrivial solutions exist if is a root of the polynomial
a0 k + a1 k 1 + ... + ak = 0

(1.3.11)

This equation is called the characteristics equation of the difference equation (1.3.9).

Real and Distinct Roots

If 1, 2,.., k are all real and distinct, then we have
k

j =1

j =1

yn( H ) = b j nj = b j exp( n log j )

(1.3.12)

where bj are arbitrary constants.

Real and Repeated Roots

Let us assume now that 1 (= 2) is a double root of (1.3.11) and that all others roots j, j = 3, 4, , k, are
distinct. Then we would get k 1 solutions of the form 1n, 3n, ..., kn. However, it can easily be verified
by substitution that if 1, is a double root, then n 1n is also a solution (1.3.9). Thus the general solution
of (1.3.9) becomes
k

yn( H ) = b1 in + b2 n 1n + b j nj
j =3

(1.3.13)

Complex Roots
For a polynomial with real coefficients, the complex roots occur as conjugate pairs. Let 1 = + i
= rei and 2 = i = rei, where r = (2 + 2)1/2 and = tan1 (/), be the complex roots of
Eq. (1.3.11) and the all other roots 3, 4,.., k be real and distinct. Then we have
yn( H ) = r n / 2 [ b1cos( n) + b2 sin(n) ] + b3 3n + .... + bk nk
where bj are arbitrary constants.

(1.3.14)

NUMERICAL SOLUTION OF DIFFERENTIAL EQUATIONS

Particular Integral
To find the particular solution yn( P) of (1.3.8) we shall confine ourselves to the case when gn(= g) is
k

independent of n. If

a j 0 , we can then choose as particular solution

j =0

yn( P ) =

(1.3.15)

aj
j =0

The general solution of (1.3.8) becomes

yn = yn

(H )

g
k

aj

(1.3.16)

j =0

The k arbitrary parameters in yn( H ) can be determined in such a way that for n = 0, 1, 2, ..., k1,
the variable y takes the assigned values y0, y1, , yk1.

1.4 TEST DIFFERENTIAL EQUATIONS

1.4.1 First Order Differential Equation
The behaviour of the solution of the initial value problem (1.2.3) in the neighbourhood of any point ( t , y )
can be predicted by considering the homogeneous local linear form of Equation (1.2.3). The nonlinear
function f (t, y) can be linearized by expanding the function about the point ( t , y ) in the Taylor series
and truncating it after the first order terms. We have

or
where

f
f
y = f ( t , y ) + (t t )
+ ( y y )
+ O(( y y )2 )
t
y

( t , y )
( t , y )

(1.4.1)

y = y + C

(1.4.2)

f ( t , y )
f ( t , y )
f ( t , y ) and
C = f ( t , y ) + (t t )
y
.
,
t
y
y
The homogeneous local linear form
=

y = y
where =

(1.4.3)

f
is called the test equation of Eq. (1.2.3). For < 0, the solution of Eq. (1.4.3), y(t) = cet
y

decreases and tends to zero as t .

1.4.2 Second Order Differential Equation

The test equation for the second order initial value problem:

y = f (t , y, y ) ,

y (t0 ) = y0 , y (t0 ) = y0

(1.4.4)

ELEMENTS OF ORDINARY DIFFERENTIAL INITIAL VALUE PROBLEM APPROXIMATION

can also be obtained by expanding the function f (t , y , y) in Taylor series about the point ( t , y , y) and
truncating after the first order terms. We have
(1.4.5)

y = 2y w2 y + k
where
w2 =
and

f ( t , y , y )
,
y

2 =

k = f ( t , y , y ) + (t t )

f ( t , y , y )
,
y

f ( t , y , y )
f ( t , y , y )
f ( t , y , y )
y
y
t
y
y

The homogeneous local linear form

(1.4.6)

y = 2y w2 y

is called the test equation of the second order initial value problem (1.4.4). The differential equation
(1.4.6) is equivalent to the following system of equations

y = P y
where

(1.4.7)

0
y = [ y1 , y2 ]T , P = 2
w
y1 = y ,

1
,
2

y2 = y .

The nature of the solution of (1.4.6) or (1.4.7) depends on the roots 1 and 2 of the characteristic
equation of the matrix P, that is, of the equation
(1.4.8)

2 + 2 + w2 = 0

The roots of the characteristic equation are

1 = [ + 2 w2 ], 2 = [ 2 w2 ].
We now consider the following cases:
(i) > 0, w 0, > w. The characteristic roots are real, distinct and negative. Therefore, the
solutions are exponentially decreasing. For w = 0, the test equation (1.4.6) becomes

y "+ 2y = 0,

>0

(1.4.9)

(ii) < 0, w 0, | | > w. The characteristics roots are real, distinct and positive. Therefore, the
solutions are exponentially increasing. For w = 0, the test equation (1.4.6) becomes
y + 2y = 0,

<0

(1.4.10)

(iii) w > 0, and | | < w. The characteristics roots are a complex pair. Therefore, the solutions are
oscillating. If < 0, then the solution is an oscillating function whose amplitude becomes
unbounded as t . If > 0 then the solution is a damped oscillating function as t .
For = 0, the test equations (1.4.6) becomes
y + w2 y = 0

(1.4.11)

whose solution is periodic with period (2/w). We observe that if 1 and 2 ( = 0, w > 0) are pure
imaginary numbers then the solution of the test equation (1.4.3) will also be periodic if we allow to be
a pure imaginary number.

NUMERICAL SOLUTION OF DIFFERENTIAL EQUATIONS

1.4.3 System of First Order Differential Equations

Consider the system of first order initial value problems
dy
= f (t , y ), y (0) = y0
dt

(1.4.12)

where y = [y1, y2,..., ym]T and f = [ f1 , f 2 ,..., f m ]T .

Assuming that the functions fi have continuous partial derivatives, (fi / y j ) = aij , i,j = 1, 2,
, m and A denotes the m m matrix [aij], we may to terms of first order write as
dy
= A y + b( t )
(1.4.13)
dt
where b(t) is an m-dimensional vector. The matrix A is usually a variable depending on the ys and t, but
it is possible to forecast the general behaviour of the solution of (1.4.12) by treating the simple case
where A is a constant matrix.
The local linear homogeneous form of (1.4.13)

dy
= Ay
dt

(1.4.14)

with initial conditions y(0) = y0 is called the test equation of the initial value problem (1.4.12). The
general solution of (1.4.13) may be written as
m

y (t ) = ci eit yi + I (t )
i =1

(1.4.15)

where the matrix A has distinct eigenvalues i and the corresponding eigenvectors yi. The function f(t)
is the particular solution of (1.4.13). The matrix equation (1.4.13) may be decoupled by applying a
similarity transformation.
Let P = [ y1 , y2 ,..., ym ] be the matrix of eigenvectors. Set y = P z. Then,

y = P z = A P z + b(t )

(1.4.16)

Premultiplying (1.4.16) by P1, we obtain

P 1 P z = P 1 A P z + P 1b(t )
or

z = D z + h(t )

(1.4.17)

where h(t ) = P 1b(t ) and D = P 1 AP is the diagonal matrix with diagonal elements 1, 2, ...,m.
The solution of (1.4.17) is given by
zi (t ) = eit ( e i t hi (t )dt + ci ) , i = 1, 2,, m
Using a similarity transformation of A to the test equation (1.4.14), we get
z = D z

(1.4.18)
(1.4.19)

This is a set of independent equations each of the form zi = i zi . Thus the nature of the solutions
of the systems of initial value problems may be discussed by using the test equation (1.4.3) for the
following:
(i) real, (ii) pure imaginary, and (iii) complex.

ELEMENTS OF ORDINARY DIFFERENTIAL INITIAL VALUE PROBLEM APPROXIMATION

Theorem 1.2: The solutions of the system (1.4.13) with b(t) = 0 are said to be stable (asymptotically
stable), i.e., yi 0, i = 1,2,, m as t , if and only if all the eigenvalues of the matrix A have
negative real parts.

1.4.4 Stiff Differential Equations

f
Definition 1.1: The system (1.4.12) is said to be stiff if the eigenvalues of the Jacobian matrix at
y

every point t have negative real parts and differ greatly in magnitude, i.e. the eigenvalues i satisfy the
conditions:
1. real part of i < 0
2. max | real part of i | >> min | real part of i |, i = 1,2,..., m
The property 2, of the system is called stiffness and can be quantified by the stiffness ratio, SR;
SR =

max. real part of i

min. real part of i

(1.4.20)

Stiffness occurs in the problem where there are two or more very different times scale of the
independent variable on which the dependent variables are changing. Time scale or time constant is a
term refer to the rate of change. For example, if the solution is c exp (t), then the time scale is | 1/ |.
Thus, time scale is the time over which the solution increases or decreases by a factor e. Now in a
system of differential equations, if the different components have widely varying time scales, then for a
meaningful solution, we require a timestep which is smaller than the smallest of these time scales.
Example 1.1: Obtain a general solution of the system of equation
dy1
= 2000y1 + 999.75y2 + 1000.25
dt
dy2
= y1 y2
dt
with initial conditions y1(0) = 0, y2 (0) = 2.
Solution: In matrix form, we write the given system as
dy
= Ay + b (t )
dt
where

y1
2000 999.75
1000.25
A=
, y = y , and b(t) = 0
1

2
Consider the homogeneous system
y = Ay
The characteristic equation of A is

A I =

2000 999.75
= 2 + 2001 + 1000.25 = 0
1
1

The eigenvalues of A are

1= 2000.5, 2 = 0.5

10

NUMERICAL SOLUTION OF DIFFERENTIAL EQUATIONS

For 1 = 2000.5, we have the system

0.5 999.75 y1
1 1999.5 y = 0

2
The eigenvector is y1 = [1999.5 1]
For 2 = 0.5, we have the system
1999.5 999.75 y1
=0
1
0.5 y2

The eigenvector is y2 = [1 2] .
The matrix of eigenvectors is
1
1999.5 1
1 2
1
P=
, P = 4000 1 1999.5

1
2

Set y = Pz, the differential equation becomes

z =D z+h(t ), P 1 AP=D
where
0
2000.5
1999.5 1 z1
, y=
D=

0.5
2 z2
0
1

or

1999.5 z1 + z2
y=

z1 + 2 z2
h(t ) = P 1b(t ) =

1 1000.25
1 2
1 2000.5
1 1999.5 0 =

4000

4000 1000.25

0.500125
=

0.2500625
The first order differential equations become

z1 = 2000.5z1 + 0.500125
z2 = 0.5z2 + 0.2500625
The solutions of these first order equations are

z1 = c1 exp ( 2000.5t ) + 0.00025

z2 = c2 exp ( 0.5t ) + 0.500125
or

y1 = 1999.5 z1 + z2
y2 = z1 + 2 z2

Substituting the initial values, we get

y1 (0) = 0 = 1999.5 z1 (0) + z2 (0)
y2 (0) = 2 = z1 (0) + 2 z2 (0)