D.

Levy
2 Numerical Methods for Linear PDEs
2.1 Introduction and Examples
Consider the advection equation
_
u
t
+ au
x
= 0, < x < , t 0,
u(x, 0) = u
0
(x) .
(2.1)
In order to numerically approximating the solution of (2.1), we rst discretize the x t
plane: set h = x (mesh width) and k = t (time step). This generates a lattice in the
x t plane, i.e., equally spaced mesh points (x
j
, t
n
) where x
j
= jh, j = . . . , 1, 0, 1, . . .
and t
n
= nk, n = 0, 1, . . . With these notations, we have, e.g., x
j+1/2
= x
j
+ h/2 =
(j + 1/2) h.
We denote the pointwise values of the exact solution to (2.1) at the grid points,
(x
j
, t
n
), as u
n
j
, and by v
n
j
we denote an approximation of u
n
j
. We denote the forward
dierencing operator by D
+
u
n
j
= (u
n
j+1
u
n
j
)/x. Similarly, backward dierencing
and centered dierencing are denoted, respectively, by D

u
n
j
= (u
n
j
u
n
j1
)/x, and
D
0
u
n
j
= (u
n
j+1
u
n
j1
)/(2x). Often, we can nd in the literature other notations for
these operators, such as
+
or
+
for the forward dierence, which may or may not
include the factor 1/x.
There are several possible strategies for approximating the solution of (2.1). One
thing we can do is to replace the derivatives with nite-dierence approximations. Al-
ternatively, one can use Taylor based methods for deriving such approximations. Lets
briey explore both these possibilities.
Example 2.1 (Forward Euler)
Replace u
t
in (2.1) with a forward dierence approximation (in time), and replace u
x
by a centered approximation (in space). Hence
v
n+1
j
v
n
j
k
+ a
_
v
n
j+1
v
n
j1
2h
_
= 0,
i.e.,
v
n+1
j
= v
n
j

a
2
_
v
n
j+1
v
n
j1
_
,
where the mesh ratio is dened here as = k/h.
Remarks.
1. If we are given the data at time step t
n
then we can compute the data in the
next time step t
n+1
. Thus this is a two-level method.
2. This is an explicit method.
2
D. Levy 2.1 Introduction and Examples
3. In matrix form, this method can be written as
_
_
_
_
_
_
_
v
0
v
1
.
.
.
v
N1
v
N
_
_
_
_
_
_
_
n+1
=
_
_
_
_
_
_
_
1
a
2
a
2
a
2
1
a
2
.
.
.
.
.
.
.
.
.
a
2
1
a
2

a
2
a
2
1
_
_
_
_
_
_
_
_
_
_
_
_
_
_
v
0
v
1
.
.
.
v
N1
v
N
_
_
_
_
_
_
_
n
,
where the entries in the upper right and lower left appear due to the periodic
boundary conditions. Adjustments should be made for dierent types of
boundary conditions.
Example 2.2 (Backward Euler)
We repeat the same approximations we made in Example 2.1, only this time the spatial
derivatives are evaluated at t
n+1
:
v
n+1
j
v
n
j
k
+ a
_
v
n+1
j+1
v
n+1
j1
2h
_
= 0.
Hence
v
n+1
j
+
a
2
_
v
n+1
j+1
v
n+1
j1
_
= v
n
j
.
This is an implicit equation which takes the matrix form
_
_
_
_
_
_
_
X X X X X

a
2
1
a
2
.
.
.
.
.
.
.
.
.

a
2
1
a
2
X X X X X
_
_
_
_
_
_
_
_
_
_
_
_
_
_
v
0
v
1
.
.
.
v
N1
v
N
_
_
_
_
_
_
_
n+1
=
_
_
_
_
_
_
_
v
0
v
1
.
.
.
v
N1
v
N
_
_
_
_
_
_
_
n
.
Again, the additional terms in the top and bottom rows should be determined by the
boundary conditions.
The points we use in the computation of the v
n+1
j
are the stencil of the method. In
Example 2.1, the stencil is {(x
j1
, t
n
) , (x
j
, t
n
) , (x
j+1
, t
n
) , (x
j
, t
n+1
)}. See Fig. 2.1a. In
Example 2.2 the stencil is {(x
j
, t
n
) , (x
j1
, t
n+1
) , (x
j+1
, t
n+1
) , (x
j
, t
n+1
)} (see Fig 2.1b).
(a) (b)
Figure 2.1: (a) The stencil of the Forward Euler method. (b) The stencil of the Backward
Euler method.
3
2.2 The Courant-Friedrichs-Levy (CFL) Condition D. Levy
Example 2.3 (Lax-Wendro)
An example of a Taylor-based method is the Lax-Wendro (LxW) method. We rst
expand
u(x, t + k) = u(x, t) + ku
t
(x, t) +
1
2
k
2
u
tt
(x, t) +. . . (2.2)
Now the PDE we are solving is u
t
= au
x
, so
u
tt
= au
xt
= au
tx
= a (au
x
)
x
= a
2
u
xx
. (2.3)
Plugging (2.3) into (2.2), and replacing u
t
with au
x
yields
u(x, t + k) = u(x, t) kau
x
(x, t) +
1
2
k
2
a
2
u
xx
(x, t) + . . .
The LxW scheme is obtained by truncating this series after the second derivative and
replacing the derivatives with centered dierencing:
v
n+1
j
= v
n
j

ka
2h
_
v
n
j+1
v
n
j1
_
+
k
2
a
2
2h
2
_
v
n
j+1
2v
n
j
+ v
n
j1
_
. (2.4)
The LxW method, (2.4), is an explicit two-level scheme with the three-point stencil
{(x
j1
, t
n
) , (x
j
, t
n
) , (x
j+1
, t
n
) , (x
j
, t
n+1
)}, which is exactly the same stencil as the one
of the Forward Euler method (see Fig. 2.1a). It can be written in a matrix form as
_
_
_
_
_
_
_
v
0
v
1
.
.
.
v
N1
v
N
_
_
_
_
_
_
_
n+1
= A
_
_
_
_
_
_
_
v
0
v
1
.
.
.
v
N1
v
N
_
_
_
_
_
_
_
n
,
where
A =
_
_
_
_
_
_
_
1
2
a
2

a
2
+
1
2

2
a
2
X
a
2
+
1
2

2
a
2
1
2
a
2

a
2
+
1
2

2
a
2
.
.
.
.
.
.
.
.
.
a
2
+
1
2

2
a
2
1
2
a
2

a
2
+
1
2

2
a
2
X
a
2
+
1
2

2
a
2
1
2
a
2
_
_
_
_
_
_
_
.
2.2 The Courant-Friedrichs-Levy (CFL) Condition
The Courant-Friedrichs-Levy (CFL) Condition comes from work done around 1928
which used nite dierence methods to prove the existence of solutions of certain PDEs.
In order to heuristically demonstrate this condition, consider u
t
+ au
x
= 0 and
discretize it using forward dierences in space and time
v
n+1
j
v
n
j
k
+ a
_
v
n
j+1
v
n
j
h
_
= 0. (2.5)
4
D. Levy 2.2 The Courant-Friedrichs-Levy (CFL) Condition
This is a two-point scheme with a stencil {(x
j
, t
n
) , (x
j+1
, t
n
) , (x
j
, t
n+1
)} . Now, if a > 0
the characteristics of u
t
+au
x
= 0 point in the positive x direction. Therefore the data
at (x
j
, t
n+1
) depends on data to the left of x
j
. See Fig 2.2. But any interval to the
left of x
j
is not included in the stencil of this method, so the solution computed by this
method at (x
j
, t
n+1
) is based on information that does not correspond to the analytical
direction of the ow of information in the problem. Hence, there is no hope that such
an approximation would be correct in any sense.
a < 0
t t
x x
a > 0
Figure 2.2: Characteristics for the advection equation
In this example, the analytical domain of dependence of the PDE (contained in
the interval [x
j1
, x
j
] t
n
), is not contained in the numerical domain of dependence
(determined by the stencil: in this case the interval [x
j
, x
j+1
] t
n
). It is important to
note that there is a hidden assumption of continuity, i.e., if we know the approximate
solution at time t
n
at x
j
and x
j+1
, then we can dene a solution everywhere in the
interval [x
j
, x
j+1
].
The CFL condition states that a necessary condition for the convergence of a numer-
ical method is that the numerical domain of dependence contains the analytical domain
of dependence. It is important to note that the CFL condition is not a sucient con-
dition for the convergence of the approximate solution to the exact solution. In our
example, the CFL condition requires that
x ak (x, x +h) .
See Fig. 2.3. Hence 0 ak h, i.e., 0 > a > 1, which implies that a < 0 and
|a| < 1. This is the CFL condition for (2.5).
k
x x+h
x ak
Figure 2.3: Numerical and analytical domains of dependence for a < 0.
5
2.3 Von-Neumann Stability Analysis D. Levy
The quantity a is often called the Courant number and measures the numerical
speed.
2.3 Von-Neumann Stability Analysis
For a periodic function u(x, t) we can write the Fourier series
u(x, t) =
1

=
u(, t) e
ix
,
where
u(, t) =
1

2
_

u(x, t) e
ix
dx.
It is a well-known fact that a translation in space amounts to a phase shift in the Fourier
space. In other words, if we dene the shift (translation) operator as
E
j
u(x) = u(x + jh) ,
then

E
j
u() =
1

2
_

u(x +jh) e
ix
dx
=
1

2
_
+jh
+jh
u(x) e
i(xjh)
dx = e
ijh
u () .
At this point we have enough tools for studying the stability of numerical schemes for
approximating solutions of linear problems with constant coecients and with periodic
boundary conditions. Consider, e.g., the Forward Euler (FE) scheme for u
t
= au
x
(+periodic boundary conditions)
v (x, t + k) v (x, t)
k
= a
v (x + h, t) v (x h, t)
2h
,
which we rewrite as
v (x, t + k) = v (x, t) +
a
2
[v (x + h, t) v (x h, t)] . (2.6)
Fourier transforming (2.6) gives
v (, t + k) = v (, t) +
a
2
_
e
ih
e
ih

v (, t)
= [1 +ai sin (h)] v (, t) .
We dene the amplication factor (or symbol )

Q by v (, t + k) =

Q v (, t). In this
example,

Q = 1+ai sin (h) . Then if the initial data is u(x, 0) = f (x), then the Fourier
transform of the approximate solution at time t
n
equals v (, t
n
) =

Q
n
v (, 0) =

Q
n

f ().
6
D. Levy 2.4 Articial Viscosity
A numerical method is stable in the time interval [0, T] for a sequence k, h 0 if
for some constant K (T) ,
sup
0t
n
T;,k,h

Q
n

K (T) .
In the case of (2.6),

Q = 1 +ai sin (h) so

=
_
1 +
2
a
2
sin
2
(h).
Therefore,

> 1 whenever sin

2
(h) = 0 and

n
will diverge in the limit k, h 0.
We can conclude that the Forward Euler method is unconditionally unstable when the
mesh ratio = k/h is held xed. It is left as a simple exercise to check that the FE
methods is stable when the ratio k/h
2
is held xed. Is there anything wrong with such
stability condition? Yes. Such stability requirement forces the time-step to be too small
(for a hyperbolic problem).
2.4 Articial Viscosity
A natural question at this point is the following: given an unstable numerical method can
we stabilize it? In general, for rst-order linear equations, the answer to this question is
positive. All we have to do is to add a suciently large quantity of articial viscosity.
Numerically, this can be done by adding a dissipative term to the scheme, in the form
of a second derivative. For simplicity, lets consider the case where the velocity is a = 1
so that the Cauchy problem becomes u
t
= u
x
, u(x, 0) = f (x). Consider a numerical
scheme that is given by
_
v
n+1
j
= (I + kD
0
) v
n
j
+khD
+
D

v
n
j
v
0
j
= f
j
(2.7)
Here, is a constant that we still need to determined. We will see that there is a
non-unique choice of that will result with a stable method. First, we rewrite (2.7) as
v
n+1
j
v
n
j
k
= D
0
v
n
j
+hD
+
D

v
n
j
. (2.8)
In the limit k 0, equation (2.8) is a consistent approximation of
u
t
= u
x
+hu
xx
.
As h 0 we return to u
t
= u
x
. Hence, (2.8), is a consistent approximation of u
t
= u
x
.
Our goal is to choose , k, and h such that the resulting scheme is stable in the sense
that

1 (reference: Gustafsson, Kreiss and Oliger p. 44-46). A straightforward

computation shows that the amplication factor of (2.7) is

Q = 1 +isin 4sin
2

2
,
7
2.5 Stability and Convergence D. Levy
where = h and = k/h. Hence

2
= 1
_
8 4
2
_
sin
2

2
+
_
16
2
4
_

2
sin
4

2
.
There are two ways to proceed:
1. Suppose 2 1, then 16
2
4 0. Then if 8 4
2
0 then

1. This
implies that 0 < 2 1. Example: take = k/2h = /2. With this choice of
parameters we recover the Lax-Wendro scheme:
v
n+1
j
= v
n
j
+ kD
0
v
n
j
+
k
2
2
D
+
D

v
n
j
.
What we just proved is that the LxW method is stable, assuming that the CFL
condition 1 is satised.
2. Suppose 2 1, so if 2 1 then

1. For example, take = h/2k = 1/2

so
v
n+1
j
= v
n
j
+ kD
0
v
n
j
+
1
2
h
2
D
+
D

v
n
j
,
i.e.,
v
n+1
j
=
1
2
_
v
n
j+1
+ v
n
j1
_
+ kD
0
v
n
j
. (2.9)
Equation (2.9) is the Lax-Friedrichs (LxF) scheme. The LxF scheme is also stable
for 1. Note that the only dierence between the LxF scheme and the FE
scheme is that the term v
n
j
in (2.6) is replace with the average
1
2
_
v
n
j+1
+ v
n
j1
_
in
(2.9). This averaging has a stabilizing eect; the dissipation we introduced leads
to stability.
2.5 Stability and Convergence
Consider the 2-periodic initial value problem
_
u
t
(x, t) = P
_
x, t,

x
_
u(x, t) ,
u(x, 0) = f (x) ,
(2.10)
for h =
2
N+1
, k > 0. A general dierence approximation of (2.10) can be written in the
form
_
Q
1
v
n+1
=

q
=0
Q

v
n
, n = q, q + 1, . . .
v

= f
()
, = 0, 1, . . . , q
(2.11)
where Q

are dierence operators. Dene the discrete solution operator S

h
that evolves
the solution from time t

to t
n
through the relation
v
n
= S
h
(t
n
, t

) v

.
Let (u, v)
h
=

N
j=1
u
j
v
j
h, and u
2
h
= h

N
j=1
|u
j
|
2
. We then have
8
D. Levy 2.5 Stability and Convergence
Denition 2.4 The dierence approximation (2.11) is stable for 0 < h h
0
if there
exists constants
S
, c,
S
such that for all h
_
_
Q
1
1
_
_
h
c,
S
h
(t
n
, t

)
h

S
e

S
(t
n
t

)
.
A stable numerical scheme satises for all initial conditions f the following:
v
n

h
(t
n
) f
h
,
where (t
n
) =
S
e

S
t
n
. The exponential factor is required so that we can treat approx-
imations to dierential equations with solutions that grow in time, such as u
t
= u
x
+u.
Denition 2.5 Let u(x, t) be a smooth solution of (2.10). Then the local truncation
error is dened by
k
n
j
= Q
1
u
_
x
j
, t
n+1
_

=0
Q

u
_
x
j
, t
n
_
.
The local truncation error is the extent to which the solution to the PDE fails to satisfy
the dierence approximation.
Denition 2.6 A dierence approximation is accurate of order (p
1
, p
2
) if for all su-
ciently smooth solutions u(x, t) of the PDE if there is a function L(t
n
) such that for
h h
0

h
L(t
n
) (h
p
1
+ k
p
2
) ,
where L(t
n
) is bounded on every nite time interval. If p
1
> 0 and p
2
> 0 then the
approximation is called consistent.
Example 2.7
The Leap-frog scheme for u
t
= u
x
is obtained by approximating both derivatives with
a centered approximation, i.e., for = h/k,
v
n+1
j
= v
n1
j
+
_
v
n
j+1
v
n
j1
_
. (2.12)
The local truncation error for the Leap-frog scheme (2.12) is given by
k
n
j
= u
_
x
j
, t
n+1
_
[u(x
j+1
, t
n
) u(x
j1
, t
n
)] u
_
x
j
, t
n1
_
. (2.13)
9
2.5 Stability and Convergence D. Levy
Expanding the terms in (2.13) around (x
j
, t
n
), we have

n
j
=
1
k
_
u +ku
t
+
k
2
2
u
tt
+
k
3
6
u
ttt
+
k
4
24
u
tttt
+ O
_
k
5
_

_
u ku
t
+
k
2
2
u
tt

k
3
6
u
ttt
+
k
4
24
u
tttt
+ O
_
k
5
_
_

_
u +hu
x
+
h
2
2
u
xx
+
h
3
6
u
xxx
+
h
4
24
u
xxxx
+ O
_
h
5
_

_
u hu
x
+
h
2
2
u
xx

h
3
6
u
xxx
+
h
4
24
u
xxxx
+ O
_
h
5
_
___
=
1
k
_
2ku
t
+ 2
k
3
6
u
ttt
+ O
_
k
5
_

_
2hu
x
+ 2
h
3
6
u
xxx
+ O
_
h
5
_
__
= 2 (u
t
u
x
) +
k
2
3
u
ttt
+
h
2
3
u
xxx
+ O
_
k
4
, h
4
_
=
k
2
3
u
ttt
+
h
2
3
u
xxx
+ O
_
k
4
, h
4
_
.
We can therefore conclude that the Leap-frog approximation is accurate of order (2, 2).
Stability by itself is not sucient to give a good approximation (for example, v
n
j
= 1
for all j and n is a stable but wrong approximation to the solution of u
t
= u
x
). We have
already seen that consistency is also not sucient for a good approximation (forward
Euler is consistent but unstable: the approximation will diverge). Fortunately, requiring
both stability and consistency does guarantee convergence to the solution, as stated by
Theorem 2.8 (Lax Equivalence Theorem) Assume that the solution of (2.10) is
smooth and that the approximation (2.11) is stable. Assume that the approximation and
its initial data are accurate of order (p
1
, p
2
) . Then on any nite interval [0, T], the error
satises
v
n
u(, t
n
)
h

S
_
e

S
t
n
_
_
v
0
u(, 0)
_
_
h
+
_
_
Q
1
1
_
_
h

h
(, t
n
) max
0jn1
_
_

j
_
_
h
_
= O(h
p
1
+ k
p
2
) .
That is, the solutions of the dierence approximation converge to the solution of the
dierential equation as h 0.
In other words, stability and consistency implies convergence. It is important to
emphasize that we can expect such a result to hold only for linear problems. There is
no such general theorem for nonlinear problems.
10