Lec 13

Introduction to Simulation - Lecture 13
Convergence of Multistep Methods

Jacob White
Thanks to Deepak Ramaswamy, Michal Rewienski, and

Karen Veroy
Outline
Small Timestep issues for Multistep Methods
Local truncation error
Selecting coefficients.
Nonconverging methods.
Stability + Consistency implies convergence
Next Time Investigate Large Timestep Issues
Absolute Stability for two time-scale examples.
Oscillators.
Basic Equations
Multistep Methods
General Notation
Nonlinear Differential Equation:

k-Step Multistep Approach:
x
x
l k
l 2
x l 1 x l
d
x(t ) = f ( x(t ), u (t ))
dt
j =0
j =0
l j
l j
x
=
f
x
, u ( tl j )
j
j
Multistep coefficients
Solution at discrete points
tl k
tl 3 tl 2 tl 1 tl
Time discretization
Basic Equations
Multistep Methods
Common Algorithms
Multistep Equation:
Multistep Coefficients:
BE Discrete Equation:
Trap Discrete Equation:
j =0
j =0
l j
l j
x
=
f
x
, u ( tl j )
j
j
Forward-Euler Approximation:
FE Discrete Equation:
x ( tl ) x ( tl 1 ) + t f ( x ( tl 1 ) , u ( tl 1 ) )
xl x l 1 = t f ( x l 1 , u ( tl 1 ) )
k = 1, 0 = 1, 1 = 1, 0 = 0, 1 = 1
x l x l 1 = t f ( x l , u ( tl ) )
k = 1, 0 = 1, 1 = 1, 0 = 1, 1 = 0
t
f ( x l , u ( tl ) ) + f ( x l 1 , u ( tl 1 ) )
2
1
1
k = 1, 0 = 1, 1 = 1, 0 = , 1 =
2
2
x l x l 1 =
Multistep Methods
Basic Equations
Definitions and Observations
Multistep Equation:
j =0
j =0
l j
l j
x
=
f
x
, u ( tl j )
j
j
1) If 0 0 the multistep method is implicit

2) A k step multistep method uses k previous x ' s and f ' s
3) A normalization is needed, 0 = 1 is common
4) A k -step method has 2k + 1 free coefficients
How does one pick good coefficients?

Want the highest accuracy
Simplified Problem for

Analysis
Multistep Methods
d
Scalar ODE:
v ( t ) = v(t ), v ( 0 ) = v0
dt
Why such a simple Test Problem?
Nonlinear Analysis has many unrevealing subtleties

Scalar is equivalent to vector for multistep methods.
multistep
d
x ( t ) = Ax(t ) discretization
dt
Let Ey (t ) = x(t )
Decoupled
Equations
l j
l j
x
=
Ax
j
j
k
j =0
l j
l j
1
y
=
E
AEy
j
j
j =0
j =0
l j
1
= t j
j =0
j =0
j =0
y l j
Simplified Problem for

Analysis
Multistep Methods
Scalar ODE:
d
v ( t ) = v(t ), v ( 0 ) = v0
dt
k
k
Scalar Multistep formula:
l j
l j
v
=
v
j
j
j =0
j =0
Must Consider ALL

Im ( )
Decaying
Solutions
O
s
c
i
l
l
a
t
i
o
n
s
Growing
Solutions
Re ( )
Multistep Methods
Convergence Analysis
Convergence Definition
Definition: A multistep method for solving initial value

problems on [0,T] is said to be convergent if given any
initial condition
max
T
l0,
t
vl v ( l t ) 0 as t 0
v l computed with t
t
l
v computed with
2
vexact
Multistep Methods
Order-p convergence
Definition: A multi-step method for solving initial value

problems on [0,T] is said to be order p convergent if
given any and any initial condition
max
v v ( l t ) C ( t )
l
T
l0,
t
for all t less than a given t0

Forward- and Backward-Euler are order 1 convergent
Trapezoidal Rule is order 2 convergent
Multi-step Methods
10
M
a
x
E
r
r
o
r
10
-2
10
-4
10
10
Reaction Equation Example
10
Backward-Euler
Trap rule
Forward-Euler
-6
-8
10
-3
10
-2
Timestep
10
-1
10
For FE and BE, Error t For Trap, Error ( t )
Multistep Methods
Two Conditions for
Convergence
1) Local Condition: One step errors are small

(consistency)
Typically verified using Taylor Series
2) Global Condition: The single step errors do not grow
too quickly (stability)
All one-step (k=1) methods are stable in this sense.
Multi-step (k > 1) methods require careful analysis.
Multistep Methods
Global Error Equation

k
Multistep formula:
Exact solution Almost
satisfies Multistep Formula:
j v
l j
j =0
k
t j vl j = 0
j =0
d
l
=
v
t
t
v
t
e
(
j ( l j )
j
l j )
dt
j =0
j =0
Global Error: E l v ( tl ) v l
Local Truncation Error

(LTE)
Difference equation relates LTE to Global error

l
l 1
l k
l
E
+
E
+
+
E
=
e
( 0
( 1
( k
0)
1)
k)
Forward-Euler
Consistency for Forward Euler
Forward-Euler definition
l +1
v t v = 0
l
l t , ( l + 1) t
Substituting the exact v ( t ) and expanding

2
dv ( l t ) ( t ) d 2 v ( )
v ( ( l + 1) t ) v ( l t ) t
=
2
l
d
v = v
dt
dt
dt
el
where e is the LTE and is bounded by

2
d
v ( )
2
l
e C ( t ) , where C = 0.5 max [0,T ]
2
dt
Forward-Euler
Global Error Equation
Forward-Euler definition
l +1
l
l
v = v + t v
Using the LTE definition
v ( ( l + 1) t ) = v ( l t ) + t v ( l t ) + e
Subtracting yields global error equation

l +1
l
l
E = ( I + t ) E + e
l
Using magnitudes and the bound on e
E
l +1
I + t E + e (1 + t ) E + C ( t )
l
Forward-Euler
A helpful bound on difference

equations
A lemma bounding difference equation solutions
If
Then
l +1
(1 + ) u + b, u = 0, > 0
l
e
l
u
b
l
l
To prove, first write u as a power series and sum

l 1
u (1 + )
l
j =0
1 (1 + )
b=
b
1 (1 + )
l
One-step Methods
A helpful bound on difference
equations cont.
To finish, note (1 + ) e (1 + ) e
l
1
1
+
1
+
1
(
)
(
)
e
l
u
b=
b
b
1 (1 + )
One-step Methods
Back to Forward Euler

Convergence analysis.
Applying the lemma and cancelling terms
l t
2
l +1
l
2
E 1 + t E + C ( t ) e
C ( t )
b
Finally noting that l t T ,
max l[0, L] E e
l
Forward-Euler
Observations about the

forward-Euler analysis.
max l[0, L] E e
l
forward-Euler is order 1 convergent

Bound grows exponentially with time interval.
C related to exact solutions second derivative.
The bound grows exponentially with time.
Forward-Euler
Exact and forward-Euler(FE)

Plots for Unstable Reaction.
12
10
RFE
Rexact
6
4
TempExact
TFE
2
0
0
0.5
1.5
Forward-Euler Errors appear to grow with time
2.5
Forward-Euler
forward-Euler errors for

solving reaction equation.
1.2
1
E
0.8
r
r 0.6
o0.4
r
Rexact-RFE
0.2
Texact - TFE
0
-0.2
0.5
Time
1.5
2.5
Note error grows exponentially with time, as bound

predicts
Forward-Euler
Exact and forward-Euler(FE)

Plots for Circuit.
v1exact
0.8
v1FE
0.6
0.4
v2FE
0.2
0
0
v2exact
0.5
1.5
2.5
3.5
Forward-Euler Errors dont always grow with time
Forward-Euler
forward-Euler errors for

solving circuit equation.
0.03
v1exact - v1FE
E 0.02
r
r 0.01
o
0
r
-0.01
v2exact-v2FE
-0.02
-0.03
0.5
1.5
Time2
2.5
3.5
Error does not always grow exponentially with time!

Bound is conservative
Making LTE Small
Multistep Methods
Exactness Constraints
k
d
l
=
v
t
t
v
t
e
(
Local Truncation Error: j ( l j )

j
l j )
dt
j =0
j =0
Can't be from
d
v (t ) = v (t )
dt
LTE
d
p 1
If v ( t ) = t v ( t ) = pt
dt
p
( ( k j ) t )
v (t )
j =0
k j
t j p ( ( k j ) t )
j =0
d
v ( tk j )
dt
p 1
=e
Making LTE Small
Multistep Methods
Exactness Constraints Cont.

k
( ( k j ) t )
j =0
( t )
If
t j p ( ( k j ) t )
p 1
j =0
k
k
p
p 1
k
j (l j ) j p (l j ) = e
j =0
j =0
k
k
p
p 1
j (( k j )) j p ( k j ) = 0
j =0
j =0
then e k = 0 for v(t ) = t p
As any smooth v(t) has a locally accurate Taylor series in t:
if
k
k
p
p 1
j ( k j ) j p ( k j ) = 0 for all p p0
j =0
j =0
k
k
l
d
p0 +1
Then j v ( tl j ) j v ( tl j ) = e = C ( t )
dt
j =0
j =0
Multistep Methods
Making LTE Small

Exactness Constraint k=2
Example
k
k
p
p 1
Exactness Constraints: j ( k j ) j p ( k j ) = 0
j =0
j =0
For k=2, yields a 5x6 system of equations for Coefficients

p=0
p=1
p=2
p=3
p=4
1
2
8
16
1
1
1
1
1
1 0
0 1
0 4
0 12
0 32
0
1
2
3
4
0
0 0
1
1
0
2
0 = 0
0
0
0
1
0 0
2
Note
i = 0
Always
Making LTE Small
Multistep Methods
Exactness
Constraints for k=2
1
2
8
16

Example Continued
0
0 0 0
1
1 1 0
2
1 0 4 2 0 = 0
0

1 0 12 3 0 0
1
1 0 32 4 0 0
2
1 1
1 0
0
1
Forward-Euler 0 = 1, 1 = 1, 2 = 0, 0 = 0, 1 = 1, 2 = 0,
2
FE satisfies p = 0 and p = 1 but not p = 2 LTE = C ( t )
Backward-Euler 0 = 1, 1 = 1, 2 = 0, 0 = 1, 1 = 0, 2 = 0,
2
BE satisfies p = 0 and p = 1 but not p = 2 LTE = C ( t )
Trap Rule 0 = 1, 1 = 1, 2 = 0, 0 = 0.5, 1 = 0.5, 2 = 0,
3
Trap satisfies p = 0,1, or 2 but not p = 3 LTE = C ( t )
Multistep Methods
Making LTE Small

example, generating methods
First introduce a normalization, for example 0 = 1

1
1
1
1
0 0 1 1
1 1 2 2
0 4 2 0 0 = 4

0 12 3 0 1 8
0 32 4 0 2 16
1
0
0
1
Solve for the 2-step method with lowest LTE
0 = 1, 1 = 0, 2 = 1, 0 = 1/ 3, 1 = 4 / 3, 2 = 1/ 3
Satisfies all five exactness constraints LTE = C ( t )
Solve for the 2-step explicit method with lowest LTE

0 = 1, 1 = 4, 2 = 5, 0 = 0, 1 = 4, 2 = 2
Can only satisfy four exactness constraints LTE = C ( t )
Making LTE Small
Multistep Methods
LTE Plots for the FE, Trap, and

Best Explicit (BESTE).
10
d
v (t ) = v (t )
d
FE
-5
L 10
T
E
Trap
-10
10
Beste
Best Explicit Method

has highest one-step
accurate
-15
10
-4
10
-3
10
Timestep
-2
10
-1
10
10
Making LTE Small
Multistep Methods
Global Error for the FE, Trap,

and Best Explicit (BESTE).
10
M
d
a -2 d v (t ) = v (t )
10
x
E -4
r 10
r
-6
10
o
r
t [0,1]
FE
Wheres BESTE?
Trap
-8
10 -4
10
-3
10
-2
10
Timestep
-1
10
10
Multistep Methods
Global Error for the FE, Trap,

and Best Explicit (BESTE).
worrysome
200
M 10
a
x 100
E 10
r
r 0
10
o
r
10
Making LTE Small
Best Explicit Method has

lowest one-step error but
global errror increases as
timestep decreases
d
v (t ) = v (t )
d
Beste
FE
Trap
-100
10
-4
10
-3
10
Timestep
-2
10
-1
10
Multistep Methods
Stability of the method

Difference Equation
Why did the best 2-step explicit method fail to

Converge?
Multistep Method Difference Equation
l
l 1
E
+
E
+
( 0
( 1
0)
1)
v ( l t ) v
+ ( k t k ) E l k = el
LTE
Global Error
We made the LTE so small, how come the Global
error is so large?
An Aside on Solving Difference Equations

Consider a general kth order difference equation
a0 x + a1 x
l
l 1
+ ak x
l k
=u
Which must have k initial conditions

1
x = x0 , x = x1 ,
0
,x
= xk
As is clear when the equation is in update form
1
0
x = ( a1 x +
a0
1
+ ak x
k +1
Most important difference equation result

l
x can be related to u by x = h u
l
j =0
l j
An Aside on Difference Equations Cont.
If a0 z + a1 z
k
k 1
+ ak = 0 has distinct roots

1 , 2 , , k
k
Then x = h u where h = j ( j )
l
l j
j =0
j =1
To understand how h is derived, first a simple case
Suppose x = x + u and x = 0
1
0
1
1
2
1
2
1
2
x = x + u = u , x = x + u = u + u
l 1
x =
l
j =0
l j
An Aside on Difference Equations Cont.

Three important observations
If i <1 for all i, then x C max j u

where C does not depend on l
l
If i >1 for any i, then there exists

j
l
a bounded u such that x
If i 1 for all i, and if i =1, i is distinct
then x Cl max j u
l
Multistep Methods

Difference Equation
Multistep Method Difference Equation

l
l 1
E
+
E
+
( 0
( 1
0)
1)
+ ( k t k ) E l k = el
Definition: A multistep method is stable if and only if

As t 0 max
T
E C max T el
l0,
t
t
l
T
l0,
t
for any el
Theorem: A multistep method is stable if and only if

The roots of 0 z k + 1 z k 1 + + k = 0 are either
Less than one in magnitude or equal to one and distinct
Multistep Methods

Stability Theorem Proof
Given the Multistep Method Difference Equation

l
l 1
E
+
E
+
( 0
( 1
0)
1)
+ ( k t k ) E l k = el
If the roots of
k j
z
j =0
j =0
are either
less than one in magnitude

equal to one in magnitude but distinct
Then from the aside on difference equations
E Cl max l e
l
From which stability easily follows.
Multistep Methods

Stability Theorem Proof
roots of
j =0
k j
=0
-1
roots of
Im
As t 0, roots
move inward to
match polynomial
Re
k j
z
= 0 for a nonzero t
( j
j)
j =0
Multistep Methods

The BESTE Method
Best explicit 2-step method
0 = 1, 1 = 4, 2 = 5, 0 = 0, 1 = 4, 2 = 2
Im
roots of z + 4 z 5 = 0
2
-5
-1
Method is Wildly unstable!
Re
Multistep Methods

Dahlquists First Stability
Barrier
For a stable, explicit k-step multistep method, the

maximum number of exactness constraints that can be
satisfied is less than or equal to k (note there are 2k
coefficients). For implicit methods, the number of
constraints that can be satisfied is either k+2 if k is even
or k+1 if k is odd.
Multistep Methods
Conditions for convergence,

stability and consistency
1) Local Condition: One step errors are small (consistency)

Exactness Constraints up to p0 (p0 must be > 0)
max
T
l0,
t
C1 ( t )
p0 +1
for t < t0
2) Global Condition: One step errors grow slowly (stability)
roots of
k j
z
j = 0 inside or simple on unit circle
j =0
max
T
E C2 max T el
l0,
t
t
l
T
l0,
t
Convergence Result:
max
E CT ( t )
l
T
l 0,
t
p0
Summary
Small Timestep issues for Multistep Methods
Local truncation error and Exactness.
Difference equation stability.
Stability + Consistency implies convergence.
Next time
Absolute Stability for two time-scale examples.
Oscillators.
Maybe Runge-Kutta schemes

Lec 13

Загружено:

Сведения о документе

Исходное описание:

Оригинальное название

Авторское право

Доступные форматы

Поделиться этим документом

Поделиться или встроить документ

Параметры публикации

Этот документ был вам полезен?

Это неприемлемый материал?

Авторское право:

Доступные форматы

Lec 13

Загружено:

Авторское право:

Доступные форматы

Introduction to Simulation - Lecture 13

Convergence of Multistep Methods

Thanks to Deepak Ramaswamy, Michal Rewienski, and

Nonlinear Differential Equation:

Solution at discrete points

1) If 0 0 the multistep method is implicit

How does one pick good coefficients?

Simplified Problem for

Nonlinear Analysis has many unrevealing subtleties

Simplified Problem for

Scalar Multistep formula:

Must Consider ALL

Definition: A multistep method for solving initial value

Definition: A multi-step method for solving initial value

for all t less than a given t0

Reaction Equation Example

For FE and BE, Error t For Trap, Error ( t )

1) Local Condition: One step errors are small

Global Error Equation

Local Truncation Error

Difference equation relates LTE to Global error

Consistency for Forward Euler

Substituting the exact v ( t ) and expanding

where e is the LTE and is bounded by

Global Error Equation

Subtracting yields global error equation

A helpful bound on difference

A lemma bounding difference equation solutions

To prove, first write u as a power series and sum

Back to Forward Euler

Applying the lemma and cancelling terms

Observations about the

forward-Euler is order 1 convergent

Exact and forward-Euler(FE)

Forward-Euler Errors appear to grow with time

forward-Euler errors for

Note error grows exponentially with time, as bound

Exact and forward-Euler(FE)

Forward-Euler Errors dont always grow with time

forward-Euler errors for

Error does not always grow exponentially with time!

Making LTE Small

Local Truncation Error: j ( l j )

Making LTE Small

Exactness Constraints Cont.

then e k = 0 for v(t ) = t p

As any smooth v(t) has a locally accurate Taylor series in t:

Making LTE Small

For k=2, yields a 5x6 system of equations for Coefficients

Making LTE Small

Exactness Constraint k=2

Making LTE Small

First introduce a normalization, for example 0 = 1

Solve for the 2-step method with lowest LTE

Satisfies all five exactness constraints LTE = C ( t )

Solve for the 2-step explicit method with lowest LTE

Can only satisfy four exactness constraints LTE = C ( t )

Making LTE Small

LTE Plots for the FE, Trap, and

Best Explicit Method

Making LTE Small

Global Error for the FE, Trap,