DRAFT DRAFT DRAFT DRAFT DRAFT DRAFT DRAFT

Comparison of time-advancement algorithms for gyrokinetic

plasma turbulence and other stiff oscillatory initial value problems
G. W. Hammett∗
Princeton University, Princeton Plasma Physics Laboratory†
(Dated: July 8, 2007 (formatting update 2020.03.04))
Abstract
While general purpose algorithms for integrating initial value differential equations are fairly
mature, there has been interesting recent research on optimal algorithms for Navier-Stokes fluid
turbulence, sometimes including chemical reactions. Such computer simulations tend to be limited
more by stability than by accuracy, and so benefit from low-order mixed algorithms which treat
some terms implicitly and others explicitly. In plasma turbulence, stiffness of the equations can
be due not only to strongly damped modes but also to high frequency modes (which might not
be important for the dynamics of interest but which could set a severe Courant condition for an
explicit method). We show how a unitary Crank-Nicolson implicit algorithm for stiff oscillatory
terms can be combined with a backward-differentiation implicit treatment of stiff damped terms.
We compare various algorithms, extending a table of comparisons from a recent book on geophysical
fluid dynamics to include various recent algorithms. One of the best methods appears to be Hulsen’s
variant of a method proposed by Karniadakis, Israeli, and Orszag, though implicit-explicit Runge-
Kutta methods may also be attractive in some cases.

∗
Electronic mail: hammett@princeton.edu
†
http://w3.pppl.gov/~hammett

1
 Caution: this is a rough draft, incomplete, out-of-order, may have errors...

I. MOTIVATION AND BACKGROUND

There are a huge number of numerical methods that have been developed for various
problems. While sometimes the plethora of algorithms can be mind-numbing, it can be
interesting to understand why different techniques are optimal for different applications.
Algorithms may be judged not only in terms of speed or accuracy but also in terms of
robustness, ease of use, or preservation of various properties of the exact solutions.
In many ways, the general theory of numerical integration of initial value problems in
ordinary differential equations is very well established, with many excellent books[? ? ?
? ? ] and online tutorials (?? list some?). There are also a number of high-quality
general-purpose packages available for integrating ODE’s (CVODE, netlib etc.??). But
there are some unique features of computational fluid dynamics (CFD) (in applications such
as weather and climate modeling or turbulent dynamics in regular fluids, reacting fluids,
or plasmas) that have motivated continued research on time-advancement algorithms that
are optimal for this class of problems. In particular, they tend to be mixed implicit-explicit
algorithms, allowing implicit treatment of stiff terms for stability and explicit treatment
of other terms for speed. Here we will compare various methods including some of these
recently developed algorithms[? ? ? ? ? ? ]. This provides an extension of the interesting
Tables 2.1 and 2.2 of Durran[? ] (see also [? ]), which recently compared various methods
commonly used in geophysical fluid dynamics. We will be considering factors that are often
important for gyrokinetic simulations of plasma turbulence in fusion energy research, but
will be using a very generic test problem here, so these comparisons may also be useful for
other ares of CFD facing stiff damped and/or stiff oscillatory dynamics.
Codes for computational fluid dynamics have tended to directly implement their own
simpler, lower-order, time-advancement algorithms for several reasons. One is that the errors
are often dominated by difficulties in the spatial differencing algorithms (leading to high-
order spatial differencing or spectral methods, various shock-capturing schemes or adaptive
mesh refinement in some cases, etc.), so the time advancement algorithms have been less
critical and received less attention (sometimes the spatial differencing algorithm may also
influence the choice of time advancement algorithms). The hardest part of improved time-
advancement algorithms is efficiently solving the inversion problems that arise from treating
some terms implicitly. Thus efficient preconditioners and solvers has been the focus of much
of the effort in CFD. Optimization of the details of the time-advancement algorithm, which
we consider here, has less potential payoff, but there are potential factors of 2-3 improvements
in speed (or more if higher accuracy is required) that are possible by choosing the optimal
algorithm for the problem at hand.
CFD problems are usually driven-dissipative systems (driven by external forcing or phys-
ical instabilities) that often exhibit chaotic or turbulent dynamics. One is often interested in
statistical time-averages rather than the details of a particular solution. The butterfly effect
means that chaotic systems will have an exponential sensitivity to small errors , making
it impossible to calculate the exact long-term solution accurately, though presumably the

2
long-time statistical averages of the approximate solution are still useful. Even in laminar
flow calculations, often only one or two significant digits is needed for engineering appli-
cations, so that, according to [? ], many codes just use 1st or 2cd order time integration
methods without error estimation or step-size control (though [? ] conclude it is better
to use at least second order algorithms and often best to use higher-order implicit-explicit
Runge-Kutta algorithms, even for these cases that only need low accuracy, because of the
larger time steps they allow for the same level of accuracy).

A. Previous widely-used algorithms

To get a sampling of some of the most widely used algorithms in CFD, and how the
choice of algorithms has evolved some over time, one may consider the following references.
Canuto et al., in their book on the 1980’s state of the art for spectral methods in fluid
dynamics, states that a very popular choice for advection-diffusion equations or the full
Navier-Stokes equations is to use Adams-Bashforth for advection and Crank-Nicolson for
diffusion (p.105 of [? ]). They state that 2cd order Adams-Bashforth is popular, but describe
the advantages of 3rd order Adams-Bashforth (AB2 is marginally or weakly unstable for
oscillatory modes, which is usually coped with by just using a small time step, while AB3
has an extended range of absolute stability along the real ω axis, making it one of the most
efficient algorithms in terms of operation at a stability limit[? ? ]). They note that while
2cd order Crank-Nicolson is absolutelty stable for viscous terms, it has the disadvantage
that it damps short wavelength components very weakly, while in reality these components
should decay very quickly. The only remedies they give for this are first-order backward
Euler, or a decentered version of Crank-Nicolson, and they don’t mention the alternative of
higher-order backwards differentiation.
?? Lighten up the critique of 1st order Euler? Sometimes this is useful. Should compare
with BDF on the real axis? At least 1st order Euler maintains positivity for all ∆t? Does
BDF3 or BDF4 maintain positivity at least on the imaginary ω∆t axis?
Ascher et al.[? ] confirm that the most popular implicit-explicit scheme by 1995 was still
2cd order Adams-Bashforth for explicit advection terms and Crank-Nicolson for implicit
diffusion terms, though they go on to present a number of better alternatives. One of
these improved alternatives appears to have been first proposed by Karniadakis, Israeli,
and Orszag[? ], who present a 2cd order and a 3rd order method based on backwards
differentiation formulas (BDF) for implicit terms, and a version of Adams-Bashforth for
explicit terms. The Adams-Bashforth treatment is modified to be compatible with the
choice of time-centering in backwards-differentiation formulas. BDF has been made popular
for stiff problems by the work of Gear[? ] and others. One of the advantages of BDF for
the implicit damping terms is that, unlike Crank-Nicolson, they will give rapid damping of
modes that should be strongly damped, even with large time steps, while retaining second
order accuracy for well-resolved modes.
?? Should define above the model problem dφ/dt = F (φ) = −iω0 φ, so that we can discuss
what A-stable and L-stable mean more clearly? Should go ahead and define formulas for
CN etc. formulas?

3
 It can be shown by Dahlquist’s theorems that multistep algorithms, such as Adams-
Bashforth or BDF, can not be “A-stable” (stable for arbitrary ∆t for modes with =(Ω) < 0,
which should be damped) and higher than 2cd order. Because of this, Hulsen[? ] proposed a
variant of the Karniadakis et al. algorithm, employing 2cd order BDF for implicit damping
terms for robustness, with a BDF-compatible version of 3rd order Adams-Bashforth for
explicit advection terms. We will confirm that this appears to be a very good algorithm for
many cases.
BDF2/AB3 is beginning to be used by a number of codes, including codes by Bruce
Scott, Naulin and Nielsen, and “Semi-Implicit Time-Integrators for a Scalable Spec-
tral Element Atmospheric Model”, Francis X. Giraldo, Q.J.R. Meteorol. Soc. (2005)
http://www.nrlmry.navy.mil/ giraldo/postscript/nseam.pdf
Implicit Runge-Kutta methods are not subject to the Dahlquist 2cd order limit, and can
in principle be made simultaneously higher-order accurate and A-stable or even L-stable
(L-stability insures that the amplification factor goes to zero as =(ω)∆t gets large, for any
=(ω) < 0). One often doesn’t need to treat all of the terms implicitly, and so there has been
a lot of work in recent years on mixed higher-order Implicit-Explicit Runge-Kutta (IMEX
RK) algorithms[? ? ? ]. IMEX Runge-Kutta schemes can suffer from reduction in order
of accuracy in some stiff cases (because of the way implicit and explicit terms are treated
differently and the lower order of accuracy of the intermediate RK steps compared to the
final RK step), and this recent research has focussed on methods that are stiffly accurate,
asymptotic-preserving, maximize conservation properties, and have a structure that avoids
further complicating the implicit solver steps. Bijl et al.[? ] acknowledge that “there is
much less experience with implicit RK methods than BDF methods in the computation of
large-scale engineering flows”. Kennedy and Carpenter[? ] provide a very comprehensive
survey of recent IMEX RK methods, and present 3rd and 4th order methods that look very
good for many cases. However, they found one very stiff class of chemistry problems with
nonequilibrium initial conditions for which even those methods had difficulties. For that
problem, they found it was better to use BDF for implicit terms and Adams-Bashforth for
explicit terms (see p. 160-161 and table 8-10 of [? ]).
?? This suggests there might be some advantage of using some kind of predictor-corrector
algorithm in which BDF2/AB2 or BDF2/AB3 was used for intermediate stages, but get
combined in a final RK-like step... (each stage of an explicit RK step looks like first-order
Euler...).
Computational plasma physics problems are often stiff, not only because of strongly
damped modes but also because of the presence of very high frequency modes, which (usu-
ally) do not affect the dynamics of interest but which would set a severe Courant limit on
the time step if treated explicitly. Sometimes these fast waves can be analytically averaged
or ordered out of the equations to be solved, though in many cases a computational implicit
treatment of the difficult terms can be just as effective. In other cases (perhaps to simplify
implementation on parallel computers), one just chooses a fully explicit method with the
largest possible stable time step for the stiff terms, in which case the time step is often
so small for the dynamics of interest that it gives sufficiently high accuracy even with a
low-order algorithm.

4
 One should note that BDF algorithms of higher than 2cd order developed for stiff damped
problems are usually not appropriate for stiff oscillatory problems, because these higher order
BDF algorithms have a region of instablity for oscillatory modes near the real ω∆t axis.

B. Existing gyrokinetic plasma turbulence algorithms

Jenko’s code, GENE, the first drift-kinetic code for electron dynamics of short wavelength
drift waves (with simpler ion dynamics, no particle trapping, 4-D initially for electrons,
including δAk but not δBk ), employed explicit Lax-Wendroff.
Dorland’s code, GS2, the first fully electromagnetic nonlinear implementation of gyroki-
netics in general toroidal geomtry (in the local flux-tube limit), is a nonlinear extension of an
earlier linear gyrokinetic code by Kotschenreuther. The linear dynamics are treated fully im-
plicitly (2cd order Crank-Nicolson for advection and field terms, 1st order backwards Euler
for the collisional damping term), which helped it be the first code to succesffully handle the
disparate time scales of ions and electrons with full electromagnetic fluctuations. Kotschen-
reuther developed a trick for calculating the fields implicitly at each time step before the
particle distribution function is calculated. (This greatly reduces the time it takes for the
implicit inversion.) The nonlinear terms are evaluated pseudo-spectrally, and advanced in
time with 2cd order Adams-Bashforth at first, though we are extending this to 3rd order
AB. (We are investigating a fast iterative implicit algorithm for GS2, which is part of what
motivated the present work.)
Candy and Waltz’s code, GYRO, is the first full-torus nonlinear gyrokinetic code. Early
version of the code were fully explicit, using 4th order Runge-Kutta[? ] and just using a
small time step. In larger-sized tokamaks with full electron dynamics, there can be some
high frequency modes for which implicit treatment is advantageous. Recent versions of
GYRO thus treat the parallel electron dynamics (and their resulting effect on the electric
and magnetic fields) implicitly, employing a 2cd order IMEX RK algorithm.

C. Conclusions

Our conclusions will be that:

For a simple robust algorithm, the best choice is probably BDF2 for implicit terms and
AB3 for explicit terms.
In some cases, it is useful to treat some implicit terms differently than other implicit
terms. Terms that are stiff due to strong damping they cause are best treated by BDF2 (or
even higher order BDF), while terms that are stiff oscillatory (i.e., have dynamics that lead
to very high frequency oscillations or waves) might be better treated by 2cd order Crank-
Nicolson, which has the nice feature for oscillations of introducing no artificial damping or
growth for arbitrary time step. It is not obvious how to combine BDF2 for some terms
with Crank-Nicolson for others, because they are usually centered at different times, but we
present in Eq.(??) a way to do this, including AB3 for explicit terms.
There has been a lot of work in just the last couple of years on more advanced techniques,

5
employing Implicit-Explicit Runge-Kutta algorithms. For explicit terms, these IMEX RK
algorithms have stability-based and accuracy-based efficiencies which are similar to AB3, as
shown in Table I. (the higher-order Runge-Kutta algorithms would become more attractive
for problems that require higher accuracy than assumed in Table I.) For implicit terms,
the 4th order Kennedy-Carpenter algorithm has an accuracy-based efficiency factor that is
about twice as good as for BDF2. This implies that a code using this 4th order algorithm
would be about twice as fast than a BDF2 code, if the time step was limited by accuracy of
the implicit terms. However, in many general CFD cases and in plasma turbulence cases,
the time step is limited more by the Courant stability limit of the explicit terms than by the
accuracy of the implicit terms, in which case the two approaches are roughly comparable.
Also, Kennedy and Carpenter found at least one type of problem where existing IMEX RK
algorithms had to take very small time steps, while BDF/AB algorithms were robustly able
to handle even this case (p.160-161 and Table 8-10 of [? ]). (Need to explain somewhere
that this “stiffness leakage” problem doesn’t show up in the simple cases we consider here,
and that it can be due to the way the implicit and explicit operators are treated differently,
for cases where the implicit and explicit operators don’t commute, and that Kennedy and
Carpenter usually found that BDF/AB algorithms don’t suffer from this (did they say that
BDF/AB don’t suffer from order reduction or from stiffness leakage, or both...).)

II. ALGORITHMS

Karniadakis91 proposed using BDF2 for stiff terms and a modification of AB2 (to be
consistent with the shifted time centering of BDF) for explicit terms. They also presented
a BDF3/AB3 algorithm for higher order accuracy. While this may be very useful in many
cases, it is known that multistep algorithms such as BDF or Adams-Moulton can not be
A-stable if they are higher than 2cd order. Higher order BDF algorithms have a region
of numerical instability near the origin along the imaginary λ axis (cite online stability
boundary?). This can be problematic for cases where the stiffness is due to high-frequency
oscillatory dynamics and not just to strongly damped modes. To fix this problem and
provide a more robust algorithm, Hulsen96 proposed a combination of
For example, some algorithms treat linear diffusive terms implicitly, and nonlinear con-
vective terms explicitly. But there can be cases where convective terms
??
While general purpose algorithms for integrating initial value differential equations are
fairly mature, there has been interesting recent research on optimal algorithms for Navier-
Stokes fluid turbulence, sometimes including chemical reactions. Such computer simulations
tend to be limited more by stability than by accuracy, and so benefit from low-order mixed
algorithms which treat some terms implicitly and others explicitly. In plasma turbulence,
stiffness of the equations can be due not only to strongly damped modes but also to high
frequency modes (which might not be important for the dynamics of interest but which could
set a severe Courant condition for an explicit method). We show how a unitary Crank-
Nicolson implicit algorithm for stiff oscillatory terms can be combined with a backward-
differentiation implicit treatment of stiff damped terms. We compare various algorithms,

6
extending a table of comparisons from a recent book on geophysical fluid dynamics to include
various recent algorithms. One of the best methods appears to be Hulsen’s variant of a
method proposed by Karniadakis, Israeli, and Orszag, though implicit-explicit Runge-Kutta
methods may also be attractive in some cases.

III. A ROBUST EFFICIENT ALGORITHM

The algorithm that appears to look best overall for our types of problems is a variation
of a class of algorithms proposed by Karniadakis, Israeli, and Orszag[? ]. The variation
proposed by Hulsen, in
http://www.mate.tue.nl/ hulsen/papers open/1043.pdf
is to combine the 3rd order explicit Adams-Bashforth part of the calculation with a 2cd
order Backwards-Differentiation-Formula for the implicit terms, because only 1st or 2cd
order implicit multistep methods can maintain absolute stability for all non-growing modes
as it should (this is called A-stable, stable for ω0 ∆t with Im(ω0 ) ≤ 0).
Splitting the RHS of the ODE into terms that will be treated implicitly and explicitly:
∂φ
= F (φ) = G(φ) + E(φ) (1)
∂t
the resulting algorithm is:
3φn+1 /2 − (2φn − φn−1 /2)
= G(φn+1 ) + (8E(φn ) − 7E(φn−1 ) + 2E(φn−2 ))/3 (2)
∆t
The form of the LHS is the standard 2cd order backwards differentiation formula (ref.
Gear, etc.). The coefficients of the explicit terms on the RHS are slightly different than the
standard 3rd order Adams-Bashforth, because they are extrapolating to estimate E(φn+1 )
while the standard Adams-Bashforth is extrapolating to estimate E(φn+1/2 ).
At every time step, there is one inversion (involving G(φn1 )) that has to be done, and one
evaluation of an explicit term. Compared to the present AB2 algorithm in GS2, one would
need to have storage to save the history of one more extra explicit term E(φn−2 ) and also
save φn=1 . (Here φ is the complete state vector of the system, i.e., the particle distribution
function in the gyrokinetic equation, not just the electrostatic potential.)

IV. DISCUSSION OF RESULTS

Results are summarized in the tables, which extend similar comparisons done in Durran’s
Table 2.2. The model problem considered here is
∂φ
= F (φ, t) = λφ = −iω0 φ (3)
∂t
and the normalized step size s = ω0 ∆t.
The stability-based efficiency factor is defined to the maximum stable normalized step
s = ω0 ∆t (for real ω0 ) divided by the number of evaluations of F per time step.

7
 Some algorithms contain a weak instability for any s, which is usually just coped with by
using a small time step because the growth rate for this instability scales as a power of δt.
We thus also have defined an accuracy-based efficiency factor, which one can roughly
think of as the value of s needed to obtain 0.1% accuracy when integrating the model
equation for a time T = 1/|λ|. Define the (complex) amplification factor for the algorithm
as:
A = φn+1 /φn (4)
so that the error per time step can be found from a series expansion in s of

A/Aexact = A/exp(−is) = 1 + cn+1 sn+1 + O(n + 2) (5)

for an n’th order accurate algorithm. The total error after N time steps will be approximately

(1 + cn+1 sn+1 )N ≈ 1 + N cn+1 sn+1 (6)

Taking N = T /∆t = 1/(|λ|∆t) = 1/s, then to have a total error of tot we must use a
normalized type step of
 1/n
tot
s= (7)
cn+1
We have set tot = 10−3 in the tables, which one can think of as integrating the model
equations for 1 “eddy turnover time” T = 1/λ with an accuracy of 0.1%, or 10 eddy times
with a 1% accuracy, or integrating for 10 wave periods 2π/|λ| with an accuracy of ≈ 6%.
(For multi-step algorithms, there are several roots for A, and we consider the physical root
here, for which A(s = 0) = 1.)
Low-order time integration algorithms are often used in fluid turbulence calculations (cite
Bijl, Carpenter et al. JCP 179 (2002) 313 et al.) because only relatively low accuracy is
required (engineering fluid dynamics calculations often need only 1 or 2 digits of accuracy in
final calculations), and because spatial discretization is often the dominant source of errors.
So our accuracy-based efficiency calculation above uses a relatively coarse tolerance for tot .
If higher accuracy is required, then higher-order methods become relatively more attractive
than the results in the tables would suggest.
Note that AB3 has a lot of advantages over AB2, with a 3 times greater accuracy-based
efficiency factor and the best stability-based efficiency factor of any explicit method in Table
I (there are some alternate predictor-corrector algorithms with leapfrog-like prediction steps
that have slightly higher stability-based efficiency).
Main conclusions
The main conclusion I would draw from Table I is that if one wishes to maximize stability
and robustness for stiff oscillatory and damped types of problems, then the Hulsen version
of the Karniadakis-Israeli-Orszag algorithm appears to be optimal. This basically employs
BDF2 for implicit terms and AB3 for explicit terms (with the coefficients for AB3 modified
to be appropriate for combining with BDF2).
Some slight further optimization might be possible by averaging AB3-BF2 with AB3-
CN2 and/or Leapfrog-CN2, particularly depending on whether one wants to reduce numer-
ical damping of unresolved purely oscillatory waves at large ω0 ∆t. One could introduce

8
 Method # of F Acc.-based Stab.-based Amp. Relative Max s
evaluations Efficiency Efficiency Factor phase
per step Factor Factor
2 s2
Forward 1 0.002 0.0 1 + s2 1− 3 0.0
2 s2
Backward 1 0.002 ∞ 1 − s2 1− 3 ∞
s2
Trapezoidal 1 0.11 ∞ 1 1− 12 ∞
Crank-Nicolson
s4 s2
Leapfrog- 2 0.055 0.71 1− 4 1− 12 1.41
Trapezoidal
s4 5 2
Adams- 1 0.049 0.0 1+ 4 1+ 12 s 0.0
Bashforth-2
Adams- 1 0.14 0.72 1 − 38 s4 1+ 289 4
720 s 0.72
Bashforth-3
Karniadakis AB2/BDF2 1 0.039 0.0 1 + 34 s4 1 + 23 s2 0.0
(explicit part)
1 4 7 2
Modified Crank-Nicolson 1 0.083 ∞ 1 − 32 s 1 − 48 s ∞
BDF2 1 0.055 ∞ 1 − 14 s4 1 2
1 − 3s ∞
1 2
Unitary BDF2 1 0.11 ∞ 1 1 − 12 s ∞
Karniadakis BDF3/AB3 1 0.11 0.63 1 − 34 s4 6 2
1 + 5s 0.63
(explicit part)
BDF3 1 0.16 0 1 + 14 s4 1− 3 4
10 s 0.0
7 4 83 4
Hulsen BDF2/AB3 1 0.12 0.72 1 − 12 s 1+ 90 s 0.72
(explicit part)
RK2 2 0.039 0 1 + 18 s4 1 + 16 s2 0.0
1 4 1 4
RK3 3 0.096 0.57 1 − 24 s 1 + 30 s 1.73
1 6 1 4
RK4 4 0.15 0.71 1 − 144 s 1 − 120 s 2.83
IMEX-SSP2(3,2,2) 2 0.039 0.0 1 + 18 s4 1 + 16 s2 0.0
(expl. part)
1 4 1 2
IMES-SSP2(3,3,2) 3 0.037 0.0 1+ 24 s 1− 12 s 0.0
(expl. part)
1 4 1 2
IMEX-SSP2(3,2,2) 3 0.052 ∞ 1− 16 s 1+ 24 s ∞
(impl. part)
1 4 1 2
IMEX-SSP2(3,3,2) 3 0.052 ∞ 1− 288 s 1− 24 s ∞
(impl. part)
ARK3(2)4L[2]SA-ESDIRK 3 0.11 ∞ 1 − 0.026s4 1 − 0.015s4 ∞
(Carpenter imp.)
ARK3(2)4L[2]SA-ERK 4 0.11 0.621 1 − 0.013s4 1 − 0.0048s4 2.48
(Carpenter exp.)

TABLE I. Comparison of various algorithms, extending Durran’s Table 2.2. The normalized time
step s = ∆tω0 , where dφ/dt = F (φ) = −iω0 φ for the simple wave test case. ?? Need to double check
all numbers since I fixed bug on 7/8/2006, particularly the RK4 and AB3 numbers? ?? Might be useful to
add a column indicating the asymptotic value of all of the roots as ν∆t → ∞ (i.e., whether the algorithm is
L-stable like BDF2, or at least somewhat damped like Modified Crank-Nicolson, or completely undamped
in that limit, like Crank-Nicolson.)

9
 Method # of F Acc.-based Stab.-based Amp. Relative Max s
evaluations Efficiency Efficiency Factor phase
per step Factor Factor
7 4 73 4
Leapfrog predictor 2 0.11 0.79 1− 72 s 1+ 540 s 1.59
Adams-Moulton-3 corr.
25 4 1 2
LP-GK2 Leapfrog pred. 0.86 1− 128 s 1− 48 s 1.71
Gear-Karniadakis-2 corr.
LP-GK3 Leapfrog pred. 0.80 1 − 0.1666s4 1 + 0.47s4 1.61
Gear-Karniadakis-3 corr.

TABLE II. Like previous table, but considering some predictor-corrector types of algorithms with
leapfrog predictors. The normalized time step s = ∆tω0 , where dφ/dt = F (φ) = −iω0 φ for the
simple wave test case. The LP-GK2 algorithm has the highest efficiency factor, allowing time steps
that are 36% longer than the 3rd order stiffly-stable scheme. Cannuto gives a reference for this
and/or similar algorithms, and point out that enhanced stability on the real ω0 ∆t axis is offset by
reduced stability for imaginary ω0 ∆t.

an adjustable parameter, like the θ parameter in the usual CN/forward/backward Euler

algorithm, to control the degree of numerical damping while still maintaining formal second
order accuracy for modes with small ω0 ∆t, unlike a backward-biased CN. Ascher 95 consid-
ers the general 3-level multistep method, and proposes a modified CN2, perhaps one of his
is optimal. One has to be careful: Leapfrog looks very good on the imaginary λ axis, but is
unstable on either side of it...
Runge-Kutta methods can formally get around the Dahlquist stability theorem that lim-
its A-stablility to second order methods for multistep methods, but higher-order implicit-
explicit (IMEX) versions of Runge-Kutta are a current research topic and special formula-
tions are required to avoid reduced order of accuracy in the mixed system or other propoerties
such as asymptotic preserving, asymptotically accurate, strong-stability-preserving, and L-
stability. The 2cd order IMEX-SSP2 schemes from Pareschi and Russo (2002) presented in
Table I are not quite as good as the AB3/BDF2 algorithm. However, higher-order IMEX
Runge-Kutta schemes might be advantageous. Explicit RK4 has a comparable stability-
based efficiency to AB3, and its higher order gives it a roughly 3 times greater accuracy-
based efficiency than GDB2, so an IMEX extensions of RK4 that was L-stable in its implicit
part could be advantageous. Perhaps some of the higher-order algorithms in the recent
paper by Kennedy and Carpenter (2003) or Ascher et al. 1997 should be investigated. It
would be nice at least in some cases to be able to get around the Dahlquist stability barrier
that limits A-stability to second order methods.

V. STATUS

Current conclusion as summarized above: for stability-limited problems AB3/BDF2 best,

or perhaps combined with AB3/CN2 (or some weighted average).
Am I correct in characterizing Crank-Nicolson as 1 evaluation per step (one can save the
result from the prevous step)? Or is that unstable?
Leapfrog explicit and CN implicit appears tempting for problems where you are sure all

10
the eigenvalues for ω0 are real, but leapfrog is unstable if there is damping... Still, would
some mixture of leapfrog and other things be useful, as I started to explore in Table II?
Should check the issue of order of the combined system:

dφ/dt = Lφ = (A + B)φ

where A and B are the parts of L that will be treated implicitly and explicitly. Are there
any subtletites in the fact that A and B might not commute? Problem of losing high order
accuracy is stated to be a potential problem for IMEX RK methods (and special versions of
IMEX RK are developed to avoid it), but is it not a problem for AB3+BDF2?
Also, the recent Runge-Kutta IMEX work pays a lot of attention to issues like asymptotic
preserving and avoiding loww of order of accuracy, but I haven’t seen them explicitly men-
tioned for the multiple step (AB/BDF) work (some references imply those special problems
apply only to RK). Might be useful to check at least on some simple test cases (2x2 matrices
with one stiff and one non-stiff eigenvalue?).
Should make bar charts comparing efficiencies of various methods.
Should have some acknowledgement that this may not be appropriate for all systems,
and another class of algorithms is widely used for cases where shock capturing is important,
etc.
——————————————————-
The rest of the paper is an older draft of some thoughts and may change
more...:

VI. SUMMARY

Summary of a report, perhaps titled “Comparison of numerical methods for continuum

gyrokinetic plasma turbulence simulations”.
Literature on numerical methods in general and on ODES in particular is vast and sophis-
ticated, terminology for similar techniques differs in many different applications, methods
specialized/optimized for each problem. Rich and interesting field.
We are interested in numerical methods for plasma simulations, in particular using con-
tinuum (or Eulerian, as opposed to particle-in-cell types of algorithms) methods.
For simulations of turbulence, do not need extremely high-order integration methods.
The system is chaotic (sensitive dependence on initial conditions), and one is interested
in time-averaged statistics of the dynamics, not the precise behavior of any one realization.
(Because instabilities are present in the system, i.e., there is a large Lyaponov exponent, and
the “Butterfly effect” occurs, so that a high resolution calculation of the exact dynamics
would have to include many small physical corrections to really be precise. But if we
are averaging over many oscillations, and not concerned with the exact phases after many
oscillations, then a lower-order integration method is sufficient.)
Also, errors are often dominated by spatial discretization algorithms, so excessive accu-
racy in time integration algorithm isn’t necessary.
Review popular methods:

11
 2cd-order Adams-Bashforth for the explicit terms and Crank-Nicolson for the implicit
terms (most popular method according to Ascher, Ruuth, and Wetton, SIAM J. on Numer-
ical Analysis, 32 (1995), 797-823 “Implicit-Explicit Methods for Time-Dependent Partial
Differential Equations”.
Main problem with this is the Crank-Nicolson is “A-stable” but not “L-stable”. To get
around this, Karniadakis, Israeli, and Orszag (1991) propose to use:
Stiffly-stable Backwards Differentiation Formulas for the implicit terms, and a modified
Adams-Bashforth treatment of the implicit terms. Fairly straightforward to implement.
(They were also interested in issues of imposing the incompressibility constraint for low
Mach number Navier-Stokes with adequate accuracy).
Karniadakis et al. gave several versions of this algorithm, including 2cd order and 3rd
versions. The 3rd order version has the nice feature that 3rd order Adams-Bashforth has a
domain of stability that includes the imaginary axis (for the classic f’=lambda f problem,
where lambda is the eigenvalue) for small time steps, while lower order methods have a weak
instability (in practice, the time step for a 2cd order method can be made small enough that
the amplification due to this weak instability is negligible during the course of the simulation,
but the use of a 3rd order method can allow a larger time step before having to worry about
a weak instability.
However, a problem with this algorithm is that 3rd order BDF formulas are not A-stable.
For Navier-Stokes this isn’t important, as the implicit term is just used for the linear viscosity
terms, which are Re(lambda) very negative and the BFD formulas are stable. But for plasma
problems, we are dealing with systems where there are physically irrelevant high-frequency
waves. We want algorithms that are stable for these high frequency waves (we don’t care
about accuracy for them, as long as they are stable and remain at a low amplitude), while
still having accurate calculations of the low frequency waves.
Martien Hulsen (MEAH-138, 1996), recommends mixed order modification of Karnaidakis
et al. for this problem: 2cd order BDF for the implicit terms (so that they remain A-stability)
with 3rd order Adams-Bashforth for the explicit terms (so that they have a finite regime of
stability dt*omega for waves, i.e. are “asymptotically stable”).
This seems to have the right mix of capabilities that we need.
Produce a table or figures comparing various methods, including an error-based efficiency
factor?
Gaffney also noted similar issues (ACM TOMS 10 (1984), 68). He tended to use canned
(general purpose) ODE packages with higher-order algorithms, while 2cd or 3rd order algo-
rithms programmed directly into a turbulence code are probably sufficient for our purposes.
Another option: Implicit-Explicit Runge-Kutta Schemes
—————————————————–
Gaffney studied some. Candy recommends some very recent ones.
Few comparisons of the relative merits of multistep AB/BDF methods vs. IMEX RK.
Seem to be comparable?
Another constraint of what we do: it is hard to treat our problem fully implicitly, as we
are dealing with a very large sparse system with nonlinearities and it is very hard to invert
the full system. We can however use a “physics-based preconditioner” approach: using our

12
understanding of the physics to identify parts of the problem that can are most important
for an implicit treatment and to identify ways to efficently do so. Kotschenreuther’s work
important in showing how to do this, by showing how to efficiently treat the full linear
parallel dynamics implicitly...
————————————————————-

VII. INTRODUCTION

I’ve been filling in some gaps in my numerical knowledge (accreted over a couple of
decades by random readings and writing of codes) by reading the very interesting book
by Dale R. Durran, Numerical Methods for Wave Equations in Geophysical Fluid
Dynamics (Springer, 1999).
Have also read about higher-order stiffly-stable implicit/explicit algorithms in an inter-
esting paper by G. Em. Karniadakis, M. Israeli, and S.A. Orszag, “High-Order Splitting
Methods for the Incomrpessible Navier-Stokes Equations”, JCP 97, 414 (1991).
I wrote a Maple script to verify all of the Durran’s Table 2.2. Durran defines an efficiency
factor, as the ratio of the maximum time step (for the simple wave test problem) to the
number of evaluations of the RHS of the ODE per time step.
I found 3 algorithms that have a (slightly) higher efficiency algorithm than the best
algorithms in the table. [Perhaps more importantly, if one wishes to use implicit methods
for some terms, to cope with stiffness for example, these algorithms are easily combined with
high-order implicit algorithms to give stiffly-stable results.]
These 3 algorithms are:

A. Leapfrog predictor, Adams-Moulton corrector

This is a variant of Durran’s “ABM Predictor-Corrector” algorithm, but uses a Leapfrog

predictor instead of an AB2 predictor:

φ1 = φn−1 + 2h∆tF (φn ) (8)

h∆t 
φn+1 = φn + 5F (φ1 ) + 8F (Φn ) − F (Φn−1 )

(9)
12

B. Leapfrog predictor, Karniadakis-Gear 2cd order corrector

This is like the 2cd order stiffly-stable implicit-explicit scheme given by Karniadakis et
al. (Eq. 15 and Table IV). The implicit part of this algorithm is based on Gear’s backward
differentiation formulas, which have the advantage over Crank-Nicolson of providing damp-
ing for all modes that should be strongly damped). Rather than exactly solving the implicit
term, we use a Leapfrog prediction for it, so the final algorithm is fully explicit (though it is
easy to treat part of the RHS operator fully implicitly if it is easily invertible). The result

13
is:

φ1 = φn−1 + 2hF (φn ) (10)

3 n+1
φ − (2φn − 21 φn−1 )
2
= gI F (φ1 ) + (1 − gI )[2F (φn ) − F (φn−1 )] (11)
∆t
where gI = 11/16 was found to be a (near) optimum value.

C. Leapfrog predictor, Karniadakis-Gear 3rd order corrector

This is a 3rd order extension of the previous algorithm. It is like the 3rd order stiffly-
stable implicit-explicit scheme given by Karniadakis et al. (Eq. 15 and Table IV). Again,
a Leapfrog prediction is used to approximate the implicit term, so the overall algorithm is
explicit (though one can easily modify it to treat any terms implicitly that one wishes). The
result is:

φ1 = φn−1 + 2hF (φn ) (12)

11 n+1 3 1
6
φ − (3φn − 2 F (φn−1 ) + 3 F (φn−2 )
= gI F (φ1 ) (13)
∆t
+ (1 − gI )[3F (φn ) − 3F (φn−1 ) + F (φn−2 )] (14)

where gI = 7/8 was found to be a (near) optimum value.

VIII. DISCUSSION

As is well known, implicit methods can be stable for arbitrary time step, as the trapezoidal
(or centered implicit) method is in Durran’s table 2.2. For explicit methods, there is probably
an ideal stability limit related to the Courant condition, which makes the maximum possible
efficiency factor of ∼ 1, close to what these best algorithms are achieving.
Should plot the full stability boundary in the complex s = ∆tω0 plane?
Should also mention or compare with the Implicit-Explicit Runge-Kutta algorithms that
Candy cites?
Be cautious about conclusions:
There is a huge number of papers in this area, these techniques may be known already...
Many different algorithms with different properties, “best” algorithm is different for dif-
ferent problems, depends on which properties of the exact equations are most important to
preserve or approximate most accurately for the problem at hand.
Another possible variation: Crank-Nicolson2/BDF2 mix for implicit terms, with AB3 for
explicit terms. (Ascher 95 consider this mix?). This is because CN2 has higher accuracy-
based efficiency, but there are times when you want to introduce some damping for high
dt*omega modes where CN2 has none...

14
 ACKNOWLEDGMENTS

I thank Dr. Bruce Scott for first telling me about the interesting properties of the Kar-
niadakis, Israeli, Orszag (KIO) algorithm that Naulin et al. were using, and I thank Dr.
Volker Naulin and Dr. Anders Nielsen for referring me to the unpublished report by Dr.
Martien Hulsen on useful modifications of the KIO algorithm. Dr. Jeff Candy provided
useful pointers to recent literature on implicit-explicit Runge-Kutta methods and other al-
gorithms. I also thank Emily Belli, Dr. Jeff Candy, Dr. Bill Dorland, and Dr. Frank Jenko
for stimulating discussions on algorithms for gyrokinetics.
Supported by DOE Contract # DE-AC02-76CH03073.

15