See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/225426727

EASY-FIT: A software system for data fitting in dynamical systems

Article  in  Structural and Multidisciplinary Optimization · March 2002

DOI: 10.1007/s00158-002-0174-6

CITATIONS READS

46 311

1 author:

Klaus Schittkowski
University of Bayreuth
146 PUBLICATIONS   4,774 CITATIONS   

SEE PROFILE

All content following this page was uploaded by Klaus Schittkowski on 14 April 2014.

The user has requested enhancement of the downloaded file.

EASY-FIT: A Software System for Data Fitting in
Dynamic Systems

Klaus Schittkowski
Universität Bayreuth

Abstract
EASY-FIT is an interactive software system to identify parameters in explicit model functions,
dynamic systems of equations, Laplace transformations, systems of ordinary differential equa-
tions, differential algebraic equations, or systems of one-dimensional time-dependent partial dif-
ferential equations with or without algebraic equations. Proceeding from given experimental
data, i.e. observation times and measurements, the minimum least squares distance of measured
data from a fitting criterion is computed, that depends on the solution of the dynamic system.
The software system is implemented in form of a relational database under MS-Access 97
running under MS-Windows. The underlying numerical algorithms are coded in Fortran and are
executable independently from the interface. Model functions are either interpreted and evaluated
symbolically by a program called PCOMP, that allows in addition automatic differentiation of
nonlinear model functions, or by user-provided Fortran subroutines. In the latter case interfaces
for the Fortran compilers Watcom F77/386, Salford FTN77, Lahey F77L-EM/32 and Microsoft
Fortran PowerStation are provided.

1 Introduction
Data fitting plays an important role in many natural science, engineering and
other disciplines. The key idea is to estimate unknown parameters in a mathe-
matical model that describes a real life situation, by minimizing the distance
of some known experimental data from the theoretical model function val-
ues. Thus also model parameters that cannot be measured directly, can be

73
identified by a least squares fit and analysed subsequently in a quantitative
way. To sum up, parameter estimation or data fitting, respectively, is ex-
tremely important in all practical situations, where a mathematical model and
corresponding experimental data are available to describe the behaviour of a
dynamic system.
The purpose of the documentation is to introduce a software system called
EASY-FIT that can be used to compute parameters by a least squares fit in-
teractively. The mathematical model that must be provided by the system
analyst, has to belong to one of the following categories:
– explicit model functions
– dynamic systems of equations (steady-state systems)
– Laplace equations
– ordinary differential equations
– differential algebraic equations
– one-dimensional, time-dependent partial differential equations
– one-dimensional partial differential algebraic equations
The general mathematical model to be analysed and the corresponding
software contains additional features with the goal to apply it to a large variety
of different applications:
1. More than one fitting criterion can be defined, i.e. more than one experi-
mental data set can be fitted within a model formulation.
2. The fitting criteria are arbitrary functions depending on the parameters to
be estimated, the solution of the underlying dynamic system, and the time
variable.
3. The model may possess arbitrary equality or inequality constraints and
upper and lower bounds for parameters to be estimated. Constraints are
allowed to depend also on the solution of the dynamic equation at given
time and concentration or spatial values, respectively.
4. Dynamic models may possess so-called switching points where the in-
tegration is restarted, e.g. for non-continuous input functions or model
changes.
5. An additional independent model parameter called concentration can be
introduced.
6. Differential algebraic equations can be solved up to index 3. Consistent
initial values for index-1-formulations are computed by the system.
7. In case of partial differential equations, also coupled ordinary differential
equations and non-continuous transitions between different areas can be
taken into account.
8. Partial differential equations may possess algebraic equations e.g. to solve
higher order systems. Consistent initial values are computed internally.
9. The model functions may be defined by their Laplace transforms. The

74
 backtransformation is done internally.
10. Gradients can be evaluated by automatic differentiation, i.e. without ad-
ditional round-off, truncation or approximation errors, and without com-
pilation of Fortran code.
11. Partial differential equations may be defined by their flux functions to
facilitate the input. Also hyperbolic equations can be solved successfully
by special variants of the upwind formula.
12. Alternatively data can be fitted w.r.t. the L 1 (sum of absolute values) or
the L∞ (maximum of absolute values) norm.
Only for illustration purposes we denote the first independent model vari-
able the time variable of the system, the second one the concentration variable
and the dependent data as measurement values of an experiment. By this the
probably most frequent application is described. On the other hand, these
terms may have any other meaning within a model depending on the under-
lying application.
Nonlinear model functions can be evaluated symbolically. Thus any com-
pilation and link of Fortran subroutines is not required whenever model func-
tions are defined or altered in this way. A particular advantage of this ap-
proach is the automatic differentiation of model functions to avoid numerical
truncation errors. The corresponding program is called PCOMP, see Dob-
mann, Liepelt and Schittkowski [13], and is part of the executable codes.
The data fitting model, alternative phrases are parameter estimation or sys-
tem identification, is outlined in Section 2. Is is shown, how the dymamic
systems have to be adapted to fit into the least squares formulation required
for starting an optimization algorithm.
A brief review of available numerical algorithms is presented in Section 3.
Only some basic features of the underlying ideas are presented. More details
are found in the references. The codes allow the numerical identification of
parameters in any of the seven situations under investigation. The executable
files are called MODFIT.EXE and PDEFIT.EXE.
EASY-FIT is in permanent use to solve real life problem, i.e. problems
with some realistic practical background. Application areas are for example
pharmacy, biochemistry, chemical engineering, physics, and mechanical en-
gineering. A few applications are listed in Section 4. Moreover we present
a case study by which we investigate the acidification of groundwater, where
the diffusion is modeled by convection and dispersion. The example shows
how a typical application problem must be implemeted for EASY-FIT.
The interactive system EASY-FIT proceeds from a relational database for
storing model information, experimental data and results. A context sensitive
help is included containing additional technical and organizational informa-
tion e.g. about the input of data and optimization tolerances. The organization
of the software system is outlined in Section 5.

75
 Fig. 1: EASY-FIT: Creation of a New Problem

Fig. 2: EASY-FIT: Input of Experimental Data

76
2 Parameter Estimation Models
EASY-FIT is an interactive software system to identify parameters in explicit
model functions, dynamic systems of equations, Laplace transformations,
systems of ordinary differential equations, differential algebraic equations or
systems of one-dimensional, time-dependent partial differential equations op-
tionally with additional algebraic equations. Proceeding from given experi-
mental data, i.e. observation times and measurements, the minimum least
squares distance of measured data from a fitting criterion is computed, that
depends on the solution of the dynamic system. The model function may
depend also on an additional independent variable that could represent for
example a concentration value of an experiment.
The basic mathematical model is the least squares problem, i.e. the prob-
lem of minimization of a sum of squares of nonlinear functions of the form

l
min i=1 fi (p)2
gj (p) = 0 , j = 1, ..., me ,
(1)
p ∈ IRn : gj (p) ≥ 0 , j = me + 1, ..., m ,
pl ≤ p ≤ pu .

Here we assume that the parameter vector p is n-dimensional and that all
nonlinear functions are continuously differentiable with respect to p. Upper
and lower bounds are treated independently from the remaining constraints.
All least squares parameter estimation algorithms proceed from the above
formulation, although in the one or other case different approaches are avail-
able to define the objective functions. The assumption, that all problem func-
tions must be smooth, is essential. The efficient numerical algorithms under
consideration are based more or less on the Gauss-Newton method, that re-
quires first derivatives.
Alternatively the L 2 -norm may be changed to the L 1 -norm, i.e.,

l
min i=1 |fi (p)|
gj (p) = 0 , j = 1, ..., me ,
(2)
p ∈ IRn : gj (p) ≥ 0 , j = me + 1, ..., m ,
pl ≤ p ≤ pu
or the L∞ -norm leading to

min maxi=1,...,l |fi (p)|

gj (p) = 0 , j = 1, ..., me ,
(3)
p ∈ IRn : gj (p) ≥ 0 , j = me + 1, ..., m ,
pl ≤ p ≤ pu .

77
 Fig. 3: EASY-FIT: Discretization and Integration Tolerances

Fig. 4: EASY-FIT: Model Functions

78
2.1 Explicit Model Functions
In this case we investigate parameter estimation problems, where one vector-
valued model function is available in explicit form, the so-called fitting crite-
rion function, with one additional variable called time, and optionally with an-
other one called concentration. We proceed from r measurement sets, given
in the form
k
(ti , cj , yij ) , i = 1, . . . , nt , j = 1, . . . , nc , k = 1, . . . , r, (4)
where nt time values, nc concentration values and n t nc r corresponding ex-
perimental measurement values are defined. Together with a vector-valued
model function

h(p, t, c) = (h1 (p, t, c), . . . , hr (p, t, c))T ,

we get the above least squares formulation by

k k
fs (p) = wij (hk (p, ti , cj ) − yij ), (5)
where s runs from 1 to l = n t nc r in any order. Moreover we assume that
k
there are suitable weighting factors w ij > 0 given by the user, that are to
reflect the individual influence of a measurement on the whole experiment.
The underlying idea is to minimize the distance between the model func-
tion at certain time and concentration points and the corresponding measure-
ment values. This distance is denoted the residual of the problem. In the ideal
case the residuals are zero indicating a perfect fit of the model function by the
measurements.
Since we assume that h(p, t, c) does not depend on an additional dynamic
system function, we call it an explicit model function. Otherwise h(p, t, c)
may depend in addition on the solution vector y(p, t, c) of an auxiliary prob-
lem, e.g. an ordinary differential equation, that is implicitly defined. Models
of this kind are considered in the subsequent sections.

2.2 Dynamic Systems of Equations

Our special goal is to estimate parameters in systems of nonlinear equations,
which may depend also on the time variable. Typical applications are for
example steady-state system as obtained by neglecting the transient terms
of an ordinary differential equation. To simplify the notation we omit the
concentration variable c. In this case the model function h(p, t) depends in
addition on the solution vector of a system of nonlinear equations, i.e.

h(p, t) = h(p, z(p, t), t) , (6)

79
where z(p, t) ∈ IRm is implicitly defined by the solution z of the system

s1 (p, z, t) = 0 ,
··· (7)
sm (p, z, t) = 0 .
The system functions are assumed to be continuously differentiable with
respect to variables p and z. Moreover we require the regularity of the system,
i.e. that the system is solvable and that the derivative matrix
 
∂si (p, z, t)
∂zj
has full rank for all p with p l ≤ p ≤ pu and for all z, for which a solution
z(p, t) exists. Consequently the function z(p, t) is differentiable with respect
to all p in the feasible domain.
Now let t be fixed and let z(p, t) be a solution of the system of equations.
If we denote S(p, z) := (s 1 (p, z, t), . . . , sm (p, z, t))T for all p and z, we
get from the identity S(p, z(p, t)) = 0, which is to be satisfied for all p, the
derivative

∇p S(p, z(p, t)) + ∇z S(p, z(p, t))∇z(p, t) = 0 .

Here ∇p S(p, z) and ∇z S(p, z) denote the partial derivative matrices of

S(p, z) with respect to the parameters p and z, respectively. In other words,
the desired Jacobian matrix ∇z(p, t) is obtained from the linear system

∇p S(p, z(p, t)) + ∇z S(p, z(p, t))V = 0 , (8)

where V is a m × n-matrix. Note that we describe here the implicit function
theorem. Since ∇ z S(p, z(p)) is nonsingular, the above system is uniquely
solvable.

2.3 Laplace Models

In many practical applications, the model is available in form of a Laplace
formulation to simplify the underlying analysis. The numerical algorithms
described in this paper, are able to proceed directly from the Laplace trans-
form and to compute its inverse internally by a quadrature formula.
The advantage of a Laplace formulation is that the numerical complex-
ity of nonlinear systems can be reduced to a lower level. Linear differential
equations, e.g., can be transformed into algebraic equations, and linear partial
differential equations with constant coefficients can be reduced to ordinary
differential equations. The simplified systems are often solvable by analyti-
cal considerations.

80
 If the model function is given in form of a Laplace transform, say
H(p, s, c), the inversion is performed numerically by a method proposed
by Stehfest [42]. Proceeding from coefficients which can be evaluated before
starting the parameter estimation algorithm, we compute the expression
2q
ln 2  i ln 2
h(p, t, c) = vi H(p, , c) . (9)
t i=1 t
h(p, t, c) is the numerical inverse Laplace transform of H(p, s, c) subject to
an accuracy given by the number q, and defines our fitting criterion in this
situation. The function may depend in addition on a concentration variable
c. It is recommended to use q = 5 or q = 6. Any smaller value decreases
the required accuracy, any larger value introduces additional round-off errors.
For the practical models we have in mind, the numerical instabilities induced
e.g. by oscillating function values, do not appear.
Note that another advantage of the above formula is, that we get easily
the gradient of the fitting function w.r.t. the parameters to be estimated, if
derivatives of the Laplace transform ∇ p H(p, s, c) are available.

2.4 Systems of Ordinary Differential Equations

We proceed now from r measurement sets, given in the form (4) and suppose
that the fitting criteria depend in addition on the solution y(p, t, c) of a system
of ordinary differential equations, i.e.

h(p, t, c) = h(p, y(p, t, c), t, c) , (10)

where y(p, t, c) is implicitly defined by a system of s ordinary differential
equations

ẏ1 = F1 (p, y, t, c) , y1 (0) = y10 (p, c) ,

···
(11)
ẏs = Fs (p, y, t, c) , ys (0) = ys0 (p, c) .

Without loss of generality we assume that the initial time is zero. The
initial values of the differential equation system y 10 (p, c), . . ., ys0 (p, c) may
depend on one or more of the system parameters to be estimated, and on the
concentration parameter c.
There are many practical situations where a model changes during inte-
gration over the time variable, and where initial values are to be adopted.
A typical example is a pharmacokinetic application with an initial infusion
and subsequent applications of drug doses by injection. In a formal way we
describe the model changes by

81
 ẏ10 = F10 (p, y 0 , t, c) , y10 (0) = y 01 (p, c) ,
··· (12)
ẏs0 = Fs0 (p, y 0 , t, c) , ys0 (0) = y 0s (p, c) ,
for 0 ≤ t ≤ τ1 and

ẏ1i = F1i (p, y i , t, c) , y1i (τi ) = y i1 (p, c, y1i−1 (p, τi , c)) ,

··· (13)
ẏsi = Fsi (p, y i , t, c) , ysi (τi ) = y is (p, c, ysi−1 (p, τi , c)) ,

for τi ≤ t ≤ τi+1 , i = 1, . . . , nb . nb is the number of break or switching

points τi with 0 < τ1 < . . . < τnb < T , where τnb +1 := T is the last ex-
perimental time. The initial values of each subsystem are given by functions
y ij (p, c, y) depending on the parameters to be estimated, the actual concen-
tration value, and the solution of the previous interval at the break point τ i .
Internally the integration of the differential equation is restarted at a break
point. It is possible that break points become variables to be adapted during
the optimization process, if they are not known in advance.
Constraints of the form
gj (p) = 0 , j = 1, ..., me ,
(14)
gj (p) ≥ 0 , j = me + 1, ..., m

are allowed, where the restriction functions may depend on the solution of the
dynamic system at predetermined time and concentration values, i.e.

gj (p) = gj (p, y(p, tj , cj ), tj , cj ) (15)

for j = 1, ..., m. The m predetermined time and, if available, concentration

values must coincide with some of the given measurement values. If not, the
given data are rounded to the nearest experimental value. If constraints are to
be defined independently from given measurement data, it is recommended to
insert dummy experimental values with zero weights at the desired time and
concentration points t j and cj , respectively.
In some cases it is possible that multiple shooting reduces the number of
iterations of the least squares algorithm drastically, in particular if measure-
ments are available for all state variables. So-called shooting points are intro-
duced which may coincide with the experimental time values. Then the inte-
gration is performed successively from one shooting point to the subsequent
one, cf. Ascher and Petzold [2]. To guarantee that a continuous numerical so-
lution of the ODE is obtained, we have to add initial values for each shooting
interval to the parameters to be optimized, and nonlinear equality constraints
obtained by the requirement that a solution variable at the end of a shooting

82
interval must coincide with the initial value of the subsequent one. A special
advantage is that starting values needed for the optimization algorithm are
easily obtained from the measurement values, see Bock [7].

2.5 Systems of Differential Algebraic Equations

Parameter estimation problems based on differential algebraic equations, are
very similar to those based on ordinary differential equations. The only dif-
ference is that we allow additional algebraic equations in the system, i.e. we
get additional system variables

ẏ1 = F1 (p, y, z, t, c) , y1 (0) = y10 (p, c) ,

···
ẏs1 = Fs1 (p, y, z, t, c) , ys1 (0) = ys01 (p, c) ,
(16)
0= G1 (p, y, z, t, c) , z1 (0) = z10 (p, c) ,
···
0= Gs2 (p, y, z, t, c) , zs2 (0) = zs02 (p, c) .
Now y(p, t, c) and z(p, t, c) are solution vectors of a joint system of s 1 +s2
differential and algebraic equations (DAE). The system is called an index-1-
problem or an index-1-DAE, if the algebraic equations can be solved w.r.t. z,
i.e. if the matrix
∇z G(p, y, z, t, c) (17)
possesses full rank. In all other cases we get DAEs with a higher index, see
e.g. Hairer and Wanner [18] for a suitable definition and more details.
Similar to systems of ordinary differential equations, it is possible to define
switching or break points, and to modify initial values at these points, that
may depend on the parameters to be estimated, the concentration variable
and the solution w.r.t. the previous interval.
For simplicity we consider now only problems of index one, although the
numerical algorithms are capable to treat also higher index models. Moreover
problems with higher index can be transformed to problems of index one by
successive differentiation of the algebraic equations.
We have to be very careful when defining the inital values of the model,
since they must satisfy the consistency equation

G1 (p, y 0 (p, c), z 0 (p, c), t, c) = 0 ,

... (18)
Gs2 (p, y 0 (p, c), z 0 (p, c), t, c) = 0 .
Here the intial values y 10 (p, c), . . . , ys01 (p, c) and z10 (p, c), . . . , zs02 (p, c) are
functions depending on the parameters to be estimated, and the concentration
variable as declared by the user.

83
 Parameter estimation problems based on DAE models, can be solved by
the program MODFIT. The code checks whether the consistency condition
is satisfied or not when starting an integration cycle. In the latter case con-
sistent initial values are computed by solving the above nonlinear system of
equations w.r.t. z, if the intial values for the differential equations are in-
serted. Also if break points exist, consistent initial values are evaluated before
restarting the integration procedure.
Again we have to assume that the observation times are strictly increasing,
and get a fitting criterion of the form

h(p, t, c) = h(p, y(p, t, c), z(p, t, c), t, c) .

As before, constraints of the form

gj (p) = 0, j = 1, ..., me ,
(19)
gj (p) ≥ 0, j = me + 1, ..., m

are allowed, where the restriction functions may depend on the solution of the
dynamic system at predetermined time and concentration values, i.e.

gj (p) = g j (p, y(p, tj , cj ), z(p, tj , cj ), tj , cj ) (20)

for j = 1, ..., m.
As in case of ODEs, also the shooting technique and internal numerical
evaluation of gradients can be applied.

2.6 Systems of One-Dimensional Time-Dependent Partial Dif-

ferential Equations
Partial differential equations are extensions of ordinary differential equations,
where an additional space or spatial variable is introduced together with its
first and optionally also some higher partial derivatives. Again we assume
without loss of generality, that the initial time is zero. This assumption fa-
cilitates the description of the mathematical model and is easily satisfied in
practice by a suitable linear transformation of the time variable.
The model we want to investigate now, is defined by a system of n p one-
dimensional partial differential equations in one or more spatial intervals, see
also Schittkowski [36]. These intervals that could describe e.g. certain areas
with different diffusion coefficients, are given by the outer boundary values
xL and xR that define the total integration interval w.r.t. the space variable x,
and optionally some additional internal transition points x a1 , . . . , xana −1 . Thus
we get a sequence of n a + 1 boundary and transition points

xa0 := xL < xa1 < . . . < xana −1 < xana := xR . (21)

84
For each integration interval, we define a one-dimensional, time-dependent
partial differential equation of the form

uit = F i (x, t, f i (x, t, v, ui , uix ), fxi (x, t, v, ui , uix ), v, ui , uix , uixx , p) , (22)
where x ∈ IR is the spatial variable with x ai−1 ≤ x ≤ xai for i = 1, . . . , na ,
t ∈ IR the time variable with 0 < t ≤ T , v ∈ IR no the solution vector of the
coupled system of ordinary differential equations, u i ∈ IRnp the system vari-
able we want to compute, and p ∈ IR n the parameter vector to be identified
by the data fitting algorithm.
Optionally the right-hand side may depend also on a so-called flux function
f i (x, t, v, ui , uix ) which is introduced either to facilitate the declaration of the
model functions or for being able to apply special upwind formulae in case
of hyperbolic equations, when usual approximation schemes break down e.g.
when shocks propagate over the integration interval. In this case only scalar
equations can be treated and the underlying equation is of the form

uit = (f i (x, t, ui , p))x , i = 1, . . . na . (23)

A solution of the coupled system depends on the spatial variable x, the
time variable t, the parameter vector p, and is therefore written in the form
v(t, p) and ui (x, t, p) for i = 1, . . . , na .
For both end points x L and xR we allow boundary conditions of the form
u1 (xL , t, p) = uL (t, p) ,
una (xR , t, p) = uR (t, p) ,
(24)
u1x (xL , t, p) = ûL (t, p) ,
unx a (xR , t, p) = ûR (t, p)
for 0 < t ≤ ts . It is very essential to understand, that we do not require the
evaluation of all boundary functions. Instead a user may omit some of them
depending on the structure of the PDE model. Note that boundary informa-
tion is also contained in coupled ordinary differential equations introduced
subsequently.
Transition conditions between the different areas may be defined in addi-
tion. They are allowed at most at a transition point x ai and have the form
ui (xai , t, p) = cR
i (t, u
i+1 a
(xi , t, p), p) ,
ui+1 (xai , t, p) = cL i a
i (t, u (xi , t, p), p) ,
(25)
uix (xai , t, p) = ĉR
i (t, u
i+1 a
(xi , t, p), ui+1 a
x (xi , t, p), p) ,

ui+1 a
x (xi , t, p) = ĉL i a i a
i (t, u (xi , t, p), ux (xi , t, p), p)

85
with 0 < t ≤ ts , i = 1, . . . , na − 1. The transition conditions are required
to obtain a unique solution, and depend on the underlying model structure
similar to boundary conditions.
Since the starting time is assumed to be zero, initial conditions must have
the form
ui (x, 0, p) = ui0 (x, p) , i = 1, . . . , na (26)
 a 
and are defined for all x ∈ xi−1 , xai , i = 1, . . . , na .
If the partial differential equations are to be coupled to ordinary differential
equations, we proceed from an additional ODE-system of the form

v̇j = Gj (t, v, uij (xj , t, p), uixj (xj , t, p), uixx

j
(xj , t, p), p) (27)

for j = 1, . . . , no with initial values

v(0, p) = v0 (p) , (28)

that may depend again on the parameters to be estimated. The system has
no components, i.e. v = (v 1 , . . . , vno )T . Coupling of ordinary differential
equations is allowed at arbitrary points within the integration interval and the
corresponding area is denoted by the index i j . The spatial variable value
xj belongs to the j-th area, i.e. x j ∈ [xaij −1 , xaij ) or xj ∈ [xana −1 , xana ],
respectively, j = 1, . . . , no , and is called coupling point.
Coupling points are rounded to their nearest line when discretizing the
system. The right-hand side of the coupling equation may depend on the cor-
responding solution of the partial differential equation and its first and second
derivative w.r.t. the space variable at the coupling point under consideration.
Similar to systems of ordinary differential equations, it is possible to define
break points and modified initial values at these points, that may depend on
the parameters to be estimated, the spatial variable and the solution w.r.t. the
previous interval.
To indicate that the fitting criteria h k (p, t) depend also on the solution of
the differential equation at the corresponding fitting point, where k denotes
the index of a measurement set, we use the notation

hk (p, t) = hk (p, t, v(t, p), uik (xk , t, p), uixk (xk , t, p), uixx
k
(xk , t, p)) (29)

and insert hk instead of hk into the data fitting function. Again the fitting cri-
teria may depend on solution values at a given spatial variable value w.r.t. to
an integration interval defined by the index i k . The spatial
  xk belongs
variable 
to the ik -th integration area, i.e. x k ∈ xaik −1 , xaik or xk ∈ xana −1 , xana ,
respectively, k = 1, . . . , r, where r denotes the total number of measurement
sets. The fitting criterion may depend on the solution of the partial differen-
tial equation and its first and second derivative w.r.t. the space variable at the

86
fitting point. Fitting points are rounded to their nearest line when discretizing
the system.
As for ordinary differential equations, dynamic constraints of the form
gj (p) = 0 , j = 1, ..., me ,
(30)
gj (p) ≥ 0 , j = me + 1, ..., m
are allowed, where the restriction functions may depend on the solution of the
partial differential equation and its first and second spatial derivatives at pre-
determined time and spatial values, and the solution of the coupled ordinary
differential equation at predetermined time values, i.e.
gj (p) = g j (xj , tj , p, v(tj , p), ujk (xj , tj , p), ujxk (xj , tj , p), ujxx
k
(xj , tj , p))
(31)
for j = 1, ..., m. Here u jk denotes the corresponding integration area that
contains the spatial parameter x j . The m predetermined time values must
coincide with some of the given measurement values. If not, the given data are
rounded to the nearest experimental data. The corresponding spatial values
must coincide with a line obtained by the equidistant discretization of the
underlying integration interval.
In order to achieve smooth fitting criteria and constraints, we assume that
all model functions depend continuously differentiable on the parameter vec-
tor p. Moreover we assume that the discretized system of differential equa-
tions is uniquely solvable for all p with p l ≤ p ≤ pu . A collection of 20
examples of partial differential equations that can be solved by the presented
approach, and comparative numerical results are found in Schittkowski [36].

2.7 Systems of One-Dimensional Partial Differential Algebraic

Equations
PDAEs are based on the same model structure as one-dimensional, time-
dependent partial differential equations. The only difference is that additional
algebraic equations are permitted as in case of DAEs. Typical examples are
higher order partial differential equations, e.g.,
ut = f (x, t, u, uxx, uxxxx, p) ,
transformed into a second order equation by introducing additional variables,
e.g.,
ut = f (x, t, u, w, wxx , p) ,
0 = w − uxx ,
or distributed systems of the form
u1t = f1 (x, t, u1 , u2 , p) ,
u2x = f2 (x, t, u1 , u2 , p) ,

87
with initial values u1 (x, 0, p) = u1 (x, p) , u2 (0, t, p) = u2 (t, p), transformed
into the PDAE
u1t = f1 (x, t, u1 , u2 , p) ,
0 = u2x − f2 (x, t, u1 , u2 , p)
or
u2x = f2 (x, t, u1 , u2 , p) ,
0 = u1t − f1 (x, t, u1 , u2 , p) ,
respectively.
Thus we proceed again from a sequence of n a + 1 boundary and transition
points
xa0 := xL < xa1 < . . . < xana −1 < xana := xR (32)
and consider the PDAE-system
uit = F1i (x, t, f i (x, t, v, ui , uix ), fxi (x, t, v, ui , uix ), v, ui , uix , uixx , p) ,
0 = F2i (x, t, f i (x, t, v, ui , uix ), fxi (x, t, v, ui , uix ), v, ui , uix , uixx , p) ,
(33)
i = 1, . . . na , where x ∈ IR is the spatial variable with x ai−1 ≤ x ≤ xai for
i = 1, . . . , na , t ∈ IR the time variable with 0 < t ≤ T , v ∈ IR no the solution
vector of the coupled system of ordinary differential equations, u i ∈ IRnp the
system variable we want to compute, and p ∈ IR n the parameter vector to be
identified by the data fitting algorithm.
But now the state variables u i are divided into so-called differential vari-
ables and algebraic variables, i.e. u i = (ui1 , ui2 )T , where the number of
algebraic variables is identical to the number of algebraic equations sum-
marized by the vector F 2 . Also we allow flux functions, switching points,
constraints for parameters and state functions, and coupled ordinary differen-
tial equations. All these extensions are treated in the same way as for partial
differential equations without algebraic equations.
However we must handle initial and boundary conditions with more care.
We have to guarantee, that at least the boundary and transition conditions
satisfy the algebraic equations
0 = F2i (xai , t, f i (xai , t, v, ui , uix ), fxi (xai , t, v, ui , uix ), v, ui , uix , uixx, p)
(34)
for i = 1, . . . na . If initial conditions for discretized algebraic equations
are violated, then the corresponding system of nonlinear equations is solved
internally proceeding from initial values given. In other words, consistent ini-
tial values are computed automatically, where the given data serve as starting
parameters for the nonlinear programming algorithm applied.
Consequently we allow only index-1-systems unless it is guaranteed, that
consistent initial values for the discretized DAE are available. Jumps or dis-
continuities at initial values of algebraic equations do not make sense.

88
3 Numerical Algorithms
EASY-FIT serves as a user interface for the parameter estimation programs
MODFIT and PDEFIT, that are also executable independently from the user
interface of EASY-FIT. One of its features is the automatic generation of input
files in ASCII format for the codes mentioned above. Model functions are
either defined symbolically to be executed by the automatic differentiation
tool PCOMP, or must be given in form of Fortran codes.
The corresponding data and code organization is outlined in Section 5. In
this section we describe very briefly the underlying numerical algorithms that
are implemented.

3.1 Data Fitting Algorithms

The parameter estimation programs contain interfaces for four different least
squares algorithms, where only one code is also capable to solve L 1 - and L∞ -
problems, i.e. problems, where the sum of absolute residual or the maximum
of absolute residual values is to be minimized.
DFNLP: By transforming the original problem into a general nonlinear pro-
gramming problem in a special way, typical features of a Gauss-Newton and
quasi-Newton least squares method are retained, see Schittkowski [34] for
details. In case of minimizing a sum of absolute function values or the max-
imum of absolute function values, the problem is transformed into a smooth
nonlinear programming problem by introducing additional variables and con-
straints. The resulting optimization problem is solved by a standard sequen-
tial quadratic programming code called NLPQL, cf. Schittkowski [33].
DN2GB: The subroutine is a frequently used unconstrained least squares al-
gorithm and was developed by Dennis, Gay and Welsch [12]. The mathema-
tical method is also based on a combined Gauss-Newton and quasi-Newton
approach.
DSLMDF: First successive line searches are performed along the unit vectors
by comparing function values only. The one-dimensional minimization is
based on a quadratic interpolation. After the search cycle the Gauss-Newton-
type method DFNLP is executed with a given number of iterations. If a solu-
tion is not obtained with sufficient accuracy, the search cycle is repeated, see
Nickel [27] for details.
NELDER: This is a very simple search method developed by Nelder and
Mead [26]. Successively edges of a simplex are exchanged by comparing
function values only. The edge with highest function value is replaced by an-
other one generated by a move through the center of the actual simplex to the
other side.

89
 Fig. 5: EASY-FIT: Optimization Parameters and Tolerances

Fig. 6: EASY-FIT: Simulation Plot

90
 Note that only DFNLP is able to take linear or nonlinear constraints into
account. However all other algorithms satisfy upper and lower bounds of the
parameters to be estimated. The algorithms are also capable to solve problems
with large residuals.
The choice of algorithm NELDER is only useful when the other algorithms
fail because of non-differentiable model functions, a very bad starting point
or large round-off errors in the function evaluation.

3.2 Solving Systems of Dynamic Equations

The program MODFIT is executed to solve parameter estimation problems
based on dynamic systems of equations or steady-state systems, respectively.
To solve the corresponding systems of nonlinear equations, they are treated
as general nonlinear programming problems and solved by the Fortran code
NLPQL, see Schittkowski [33]. Objective function is the sum of squares of
the system parameters, and the constraints are identical to the nonlinear sys-
tem of equations given. The algorithm proceeds from a successive quadratic
approximation of the Lagrangian function and linearization of constraints. To
get a search direction, a quadratic programming problem must be solved in
each iteration step, a subsequent line search stabilizes the algorithm.
Also the initial values required for the start of an optimization cycle, must
be predetermined by the user in a suitable way. They may depend on the para-
meters of the outer optimization problem. The system of nonlinear equations
must be solved for each experimental time value. Moreover the gradients of
the model function h(p, z(p, t), t) are calculated analytically by the implicit
function theorem. In this case a system of linear equations must be solved for
each time value by numerically stable Householder transformations.
As mentioned above, the parameter estimation algorithm DFNLP does also
require the execution of subroutine NLPQL for solving the modified nonlin-
ear programming problem. To avoid execution conflicts, DFNLP uses reverse
communication for evaluating function and gradient values.

3.3 Numerical Laplace-Back-Transformation

The executable program MODFIT is also executed to solve parameter estima-
tion problems, where the model functions are defined in the Laplace space.
In this case, constraints are not allowed.
Since we suppose that analytical back-transformation is not available, we
have to apply a numerical quadrature formula, see Bellman et al. [3] for de-
tails. In our case we use the quadrature formula of Stehfest [42]. Proceeding
from a given Laplace transform H(p, s, c) of an unknown model function
h(p, t, c) subject to the parameter vector p to be optimized, we compute the

91
coefficients
min(i,q)
 k q+1 (2k)!
vi := (−1)q+i , (35)
(q − k)!k!(k − 1)!(i − k)!(2k − i)!
k=(i+1)/2

which are independent of H and which can be evaluated before starting the
main procedure, where function values are to be computed. Then
2q
ln 2  i ln 2
h(p, t, c) = vi H(p, , c) (36)
t i=1 t

is a numerical approximation formula for h(p, t, c).

The parameter for controlling the accuracy is the number q. When working
with double precision arithmetic, it is recommended to use q = 5 or q = 6.
Any smaller value decreases the required accuracy, any larger value intro-
duces additional round-off errors. For the practical models we have in mind,
the numerical instabilities induced e.g. by oscillating function values, do not
appear.
A particular advantage of the formula is, that the derivative of h(p, t, c)
w.r.t. the parameters to be estimated, i.e. p, are easily obtained from the
derivatives of H(p, s, c).

3.4 Numerical Algorithms for Ordinary Differential Equations

Within MODFIT it is possible to select among seven different solvers:
DOPRI5 Explicit Runge-Kutta method of order 4/5 based on Dormand
and Prince [30] formula, cf. Hairer, Nørsett and Wanner [19].
Steplength is adapted internally.
DOP853 Explicit Runge-Kutta method of order 8 based on Dormand
and Prince [30] formula, cf. Hairer, Nørsett and Wanner [19].
Local error estimation and step size control is based on em-
bedded formulae of order 5 and 3.
ODEX Extrapolation method based on GBS algorithm with variable
order and stepsize, see Hairer, Nørsett and Wanner [19].
RADAU5 Implicit Radau-type Runge-Kutta method of order 5 for stiff
equations, cf. Hairer and Wanner [18].
SDIRK4 Diagonally implicit Runge-Kutta method with 5 stages, cf.
Hairer and Wanner [18].
SEULEX Extrapolation algorithm based on linearly implicit Euler
method, cf. Hairer and Wanner [18].
IND-DIR Runge-Kutta-Fehlberg method of order 4 to 5, cf. Shampine
and Watts [39] with additional sensitivity analysis imple-
mented by Benecke [4].

92
 Note that the first six codes use dense output, i.e. the integration is per-
formed over the whole interval given by first and last time value, and inter-
mediate solution values are interpolated. In these cases gradients w.r.t. pa-
rameters to be estimated, are obtained by external numerical differentiation.
The codes RADAU5, SDIRK4 and SEULEX are able to solve also stiff equa-
tions.
The last algorithm IND-DIR is capable to evaluate derivatives of the so-
lution of the ODE internally w.r.t. the parameters to be estimated, i.e. by
analytical differentiation of the Runge-Kutta scheme.
It is possible that the right-hand side of an ODE is non-continuous w.r.t.
integration time, e.g. if non-continuous input functions exist. Especially in
case of short peaks, the integration routine might not realize the peak at all
because of a big time step. Moreover the numerical approximation of gradi-
ents could become unstable in case of discontinuities. Thus MODFIT allows
to supply an optional number of time values, so-called break points, where
the integration of the ODE is restarted with initial tolerances, e.g. with the
initially given stepsize. The integration in the preceeding interval is stopped
at the time value given minus a relative error in the order of the machine
precision.
Restarts of the integration routine are needed only for the codes DO-
PRI5,..., SEULEX because of their dense output facilities. In case of
IND-DIR, the integration is restarted at each experimental time value. By
introducing artificial time values with zero weight, additional restarts at dis-
continuities can be simulated easily. Note also that break points can be treated
as optimization variables, i.e. may vary from one iteration step to the other.

3.5 Numerical Algorithms for Differential Algebraic Equa-

tions
Algebraic differential equations are solved by any of the following implicit
methods:
RADAU5 Implicit Radau-type Runge-Kutta method of order 5 for stiff
equations, cf. Hairer and Wanner [18].
SDIRK4 Diagonally implicit Runge-Kutta method with 5 stages, cf.
Hairer and Wanner [18].
SEULEX Extrapolation algorithm based on linearly implicit Euler
method, cf. Hairer and Wanner [18].
DAEs with an index higher than one can be solved only by RADAU5. If
consistent initial values cannot be provided by the user, the corresponding
nonlinear system of equations is treated as general nonlinear programming
problem with equality constraints. A minimum norm solution is computed

93
by the sequential quadratic programming method NLPQL of Schittkowski
[33]. The initial values given for the algebraic variables are used as starting
values for NLPQL.
For reasons outlined in the previous section, it is possible that the right-
hand side of a DAE becomes non-continuous w.r.t. integration time. Thus it
is possible to supply an optional number of time values, where the integration
of the DAE is restarted.

3.6 Numerical Algorithms for Partial Differential Equations

The underlying idea is to transform the partial differential into a system of
ordinary differential equations by discretizing the model functions w.r.t. the
spatial variable x. This approach is known as the method of lines, see e.g.
Schiesser [32].
For the i-th integration interval of the spatial variable, we denote the num-
ber of discretization points by n i , i = 1, . . . , na . We proceed from uniform
grid points within each interval and get a discretization of the whole space
interval from xL to xR . To approximate the first and second partial deriva-
tives of u(x, t, p) w.r.t. the spatial variable at a given point x, several different
alternatives have been implemented in PDEFIT, see also Schittkowski [38]:
a) Polynomial approximation: We compute an interpolating polynomial sub-
ject to some neighbouring values. The number of interpolation points de-
pends on the polynomial degree selected. Polynomials of order 3, 5, 7 or
9 are computed from Newton’s formula. By differentiating the interpolation
formulae, first and second order approximations are obtained. In case of Neu-
mann boundary conditions, Hermite interpolation is applied for being able to
exploit known derivative values.
b) Difference Formulae: First and second derivatives can be approximated
by difference formulae, see e.g. Schiesser [32]. Difference formulae with 3
and 5 points for first derivatives are available, that can be applied recursively
to get also the second derivatives. Alternatively a 5-point difference formula
for second derivatives is implemented as well. The difference formulae are
adapted at the boundary to accept given function and gradient values.
c) Upwind Formulae for Hyperbolic Equations: In case of one hyperbolic
equation, i.e.

uit = (f i (x, t, ui , p))x (37)

with a so-called flux function f , approximation by polynomials or difference
formulae might become unstable especially if non-continuous boundary con-
ditions are supplied to describe e.g. the propagation of shocks, see again
Schiesser [32] for some numerical examples. The following update formulae
are available for solving hyperbolic equations:

94
 -simple upwind formula
-TVD with upwind factor -1/3
-Fromm’s TVD-scheme
-third order upwind-biased TVD-scheme
-second order TVD-scheme
-central difference TVD-scheme
-first order high resolution TVD-upwind
-second order central-difference high resolution
For more information, see the original literature, e.g. Yee [50], Chakravarthy
and Osher [9], [10], [11], Sweby [44], Wang and Richards [47], and Yang and
Przekwas [49]. TVD stands for total variation diminishing and the correpond-
ing one parameter family of upwind formulae was proposed by Chakravarthy
and Osher [9]. In this case a certain stability criterion requires that the inter-
nal time stepsizes of the ODE-solver do not become too small compared to
the spatial discretization accuracy. Because of the black box approach used,
the stepsizes however cannot be modified and we have to suppose that the
criterion remains satisfied.
Whenever a boundary condition in Dirichlet-form is given, cf. (26) and
(27), then we know the value of the boundary function and use it to interpolate
or approximate the function u(x, t, p) as decribed above. In other words the
corresponding function value in the right-hand side of the discretized system
is replaced by the value given. Alternatively a boundary condition may appear
in Neumann-form, see (26) or (27). In this case the derivative values at the
boundary are replaced by the given ones before evaluating the second order
spatial derivative approximations.
Ordinary differential equations are added to the discretized system without
any further modification. Since arbitrary coupling points are allowed, they
are rounded to the nearest line of the discretized system. In the same way
fitting criteria can be defined at arbitrary values of the spatial variable.
When defining the transition function, it is important to have the under-
lying flux direction in mind. If, for example, the flux is in the direction
of the spatial variable and we want to define a continuous transition at x ai ,
then we have to formulate the corresponding transition function in the form
ui+1 a i a
k (xi , t, p) = uk (xi , t, p) in order to guarantee, that the boundary values
w.r.t. xL are spread over the interval.
For the same reasons outlined in the previous sections, it is possible that
the right-hand side of a PDE becomes non-continuous w.r.t. integration time.
Thus it is possible to supply an optional number of time values, where the in-
tegration of the ODE is restarted with initial tolerances, e.g. with the initially
given stepsize. The integration in the preceeding interval is stopped at the
time value given minus a relative error in the order of the machine precision.

95
Break or switching points are either constant or optimization variables to be
adapted by the optimization code.
In many application models we need to compute an integral w.r.t. the spa-
tial variable x e.g. to evaluate a mass balance, i.e.
 xa
j
ui (x, t, p)dx ,
xa
j−1

where the integral is taken over the j-th area where the PDE is defined, j =
1, . . . , na , and where i = 1, . . . , n p . The integral is evaluated by Simpson’s
rule and and can be retrieved from a common block or, alternatively, through
a special construct of the PCOMP language.

3.7 Numerical Algorithms for Partial Differential Algebraic

Equations

The basic idea is now to transform the partial differential into a system of
differential algebraic equations by discretizing the model functions w.r.t. the
spatial variable x. It is assumed that always the last n ae equations of the n p
given ones are algebraic.
Again we denote the number of discretization points by n i , i = 1, . . . , na ,
for the i-th integration interval of the spatial variable. We proceed from uni-
form grid points within each interval and get a discretization of the whole
space interval from x L to xR . To approximate the first and second partial
derivatives of u(x, t, p) w.r.t. the spatial variable at a given point x, we may
apply polynomial approximation or difference formulae as outlined in the
previous section. Thus we get a system of differential algebraic equations,
that can be solved then by any of the available integration routines.
Boundary conditions have to satisfy the algebraic equations. Consistent
initial values are computed within the code PDEFIT, where the given data
serve as starting parameters for the nonlinear programming algorithm ap-
plied. Consequently we allow only index-1-systems unless it is guaranteed,
that consistent initial values for the discretized DAE are available. Also any
jumps or discontinuities at initial values of algebraic equations do not make
any sense.

3.8 Statistical Error Analysis

Proceeding from the assumption that the model is linearized in a neighbour-

hood of an optimal solution and that all experimental data are Gaussian and
independent, some statistical data can be evaluated:

96
 – Variance/covariance matrix of the problem data
– Correlation matrix of the problem data
– Estimated variance of residuals
– Confidence intervals for the individual parameters subject to the signifi-
cance levels 1%, 5% or 10%, respectively.

4 Case Study

4.1 Application Problems

The author’s database of EASY-FIT contains more than 500 real life and aca-
demic data fitting problems based on about 400 different mathematical mod-
els. In some cases it is also possible to transform an optimal control problem
to a parameter estimation problem. A few practical applications solved by
EASY-FIT and the underlying numerical routines are:
1. Substrate diffusion in a metabolically active cutaneous tissue, see Schitt-
kowski [37], Boderke, Schittkowski, Wolf, Merkle [8], Steingässer [43],
or Blatt, Schittkowski [6] for an optimal control model
2. Acetylene reactor, see Wansbrough [48] or Birk, Liepelt, Schittkowski,
Vogel [5] for an optimal control approach
3. Drying of maltodextrin in a convection oven, see Frias, Oliveira, Schitt-
kowski [15]
4. Receptor-ligand binding study, see Rominger, Albert [31] or Schitt-
kowski [35]
5. Multibody system of a truck, see Simeon, Grupp, Führer, Rentrop [41],
Simeon, Rentrop [40] or Führer, Leimkuhler [16]
6. Manutec r3 robot, see Otter, Türk [29]
7. Mass transfer in sorbing porous media, see Hoch [22], Van Genuchten,
Wierenga [45], or Andersson, Olsson [1]
8. Ammonium and nitrate fertilization in forest soils, see Fischer [14]
9. Distillation column, see Jourdan [23], or Kuhn, Schmidt [24]
10. Cooling during rolling-mill operations, see Hedrich [21] or Groch [17]
11. Signal evaluation in periodic mechanical systems, see Zschieschang and
Rockhausen [51]
12. Design of horn radiators for satellite communication by least squares opti-
mization, see Hartwanger and Schittkowski [20]
13. Isomerization in protein unfolding and refolding, see Odefey, Mayr,
Schmid [28], or Mayr, Odefey, Schutkowski, Schmid [25]
14. Thermodynamic coupling for GroEL-mediated unfolding, see Walter,
Lorimer, Schmid [46]

97
4.2 Groundwater Flow
The mathematical model to be introduced in form of a case study, describes a
tracer experiment that was conducted at the lake Gårdsjön in Sweden, see An-
dersson and Olsson [1], to investigate acidification of groundwater pollution.
To perform an experiment, a catchment of 1.000 m 2 was covered by a roof. A
tracer impulse consisting of Lithium-Bromide was applied with steady-state
flow conditions. Tensiometer measurements of the tracer concentration are
documented in a distance of 40 m from the center of the covered area.
The diffusion equations proposed by Van Genuchten and Wierenga [45]
were chosen by Hoch [22] to analyze the diffusion process and to get a simu-
lation model. A two-domain approach was selected in form of two equations,
to model the mobile and the immobile part of the system. The first one de-
scribes the diffusion of the flow through the soil by convection and dispersion,
the second one the diffusion orthogonal to the flow direction.

∂cm ∂cim ∂ 2 cm ∂cm

θm (x, t) + θim (x, t) = θm Dm 2
(x, t) − θm Vm (x, t)
∂t ∂t ∂x ∂x
∂cim
θim (x, t) = α (cm (x, t) − cim (x, t))
∂t

for 0 < t ≤ 2.55 and 0 < x < L = 80 with boundary conditions

Dm ∂cm 5800 , if t < t0 ,
cm (0, t) − (0, t) =
Vm ∂x 0, otherwise,

Dm ∂cm
cm (L, t) + (L, t) = 0
Vm ∂x
for 0 ≤ t ≤ 2.55. Initial conditions are c m (x, 0) = 0 and cim (x, 0) = 0 for
0 < x < L.
Here we are interested in fitting the function

Dm ∂cm 1
h(p, t) = cm ( 12 L, t) − ( L, t)
Vm ∂x 2
defined for 0 < t ≤ 2.55. We let t 0 = 0.01042, θim = 1 − θm , and
Vm = 100.
Our intention is to give an impression how a typical application problem
can be formulated by the PCOMP interpreter language of EASY-FIT. The
order of variables and model functions is mandatory, whereas their names
can be chosen arbitrarily. In this particular case, the order of the parameters
and state variables is:

98
1. parameters to be estimated (p)
2. state variables identifying solution of partial differential equations (u)
3. variables identifying first spatial derivatives of solution of partial differen-
tial equations (u x )
4. variables identifying second spatial derivatives of solution of partial differ-
ential equations (u xx)
5. spatial variable (x)
6. time variable (t)
In a similar way the order of model function is prescribed:
1. functions defining right-hand side of system of partial differential equa-
tions (F (x, t, u, ux , uxx, p), see (22))
2. initial values of system of partial differential equations at t = 0 (u 0 (x, p),
see (26))
3. boundary derivatives w.r.t. x (ĉ L (t, p), ĉR (t, p), see (24))
4. fitting criterion (h(p, t, u, u x , uxx ), see (29))
Boundary functions are defined first at left boundaries of all areas, then at
right boundaries of all integration areas. In the present example we do not
have to take flux functions, transition functions between different integration
areas, Dirichlet boundary conditions, coupled ordinary differential equations,
or constraints into account. The model is implemented then in the PCOMP
language as follows:
* REAL CONSTANT
t0 = 0.0104167
C0 = 5800
Vm = 100
C
C------------------------------------------------------
C
* VARIABLE
Tm, Dm, alpha
Cim, Cm, Cim_x, Cm_x, Cim_xx, Cm_xx
x, t
C
C------------------------------------------------------
C
* FUNCTION Cim_t
Tim = 1 - Tm
Cim_t = alpha/Tim*(Cm - Cim)
C
* FUNCTION Cm_t
Cm_t = Dm*Cm_xx - Vm*Cm_x - alpha/Tm*(Cm - Cim)
C
C------------------------------------------------------
C
* FUNCTION Cim0
Cim0 = 0.0
C
* FUNCTION Cm0
Cm0 = 0.0
C
C------------------------------------------------------

99
C
* FUNCTION Cm_x_left
IF (t.LT.t0) THEN
Cm_x_left = (Cm - C0)*Vm/Dm
ELSE
Cm_x_left = Cm*Vm/Dm
ENDIF
C
* FUNCTION Cm_x_right
Cm_x_right = -Cm*Vm/Dm
C
C------------------------------------------------------
C
* FUNCTION Cm_fit
Cm_fit = Cm - Dm/Vm*Cm_x
C
C------------------------------------------------------
C
* END

Parameters to be estimated are θ m , α, and Dm . Initial and final optimiza-

tion parameters are shown in the subsequent table together with their signifi-
cance intervals subject to a significance level of 5 %.

parameter initial value final value confidence interval

lower upper
θm 0.5 0.347704 0.337491 0.357818
Dm 100.0 348.981 326.485 371.476
α 1.0 3.86361 3.39786 4.32936

25 lines are used to discretize the spatial interval. The resulting system
of 50 ordinary differential equations is solved by RADAU5 subject to an er-
ror tolerance of RELERR=ABSERR=1.0D-6. After 24 iterations the least
squares code DN2GB computes the scaled residual value 0.0067, see Figure
4.2. The corresponding surface plots for mobile and immobile part are shown
in Figure 4.2. Statistical significance intervals are quite tight at least subject
to the first two parameters.
A couple of typical screen shots of EASY-FIT for generating a new model,
starting an optimization run, and interpreting results are shown in figures 1 to
6 on pages 76, 78 and 90.

100
 50
45
22222222 data 2
22
22 2 22222222
40 fit criterion
2
35
2 22
2 2222
2 222
30
22 222
22
h(p, t)

25
2 22 2222
20
2 2 2 2222222
15 2 2 222
22 22222
10 2222
2222222
5
0 2
0 0.5 1 1.5 2 2.5 3
t

Fig. 7: Final Trajectories for Groundwater Flow

5 Software Organization
EASY-FIT consists of a database containing models, data and results, and of
underlying numerical algorithms for solving the parameter estimation prob-
lem depending on the mathematical structure, i.e.
MODFIT parameter estimation in explicit functions, steady-state sys-
tems, Laplace transforms, ordinary differential and differen-
tial algebraic equations
PDEFIT parameter estimation in one-dimensional, time-dependent
partial differential algebraic equations
EASY-FIT requires a lot of system resources to run in a smooth and ef-
ficient way. Especially the run time system of the MS-Access 97 1 database
needs as much CPU memory as possible. Recommended hardware is a Pen-
tium processor with 32 MB memory on board and at least 200 MHz. A full
installation requires about 40 MB on hard disk. The system runs under Win-
dows 95 and Windows NT.
EASY-FIT comes with the run-time and royalty-free version of MS-
Access. Plots are generated either by internal plot facilities or optionally by
the external graphic systems MS-Graph5 and GNUPLOT 2 . Numerical data
1 MS, Windows, Microsoft Access are registered trademarks of Microsoft Corp.
2
1986-93
c by T. Williams, C. Kollin

101
 model function

cm (x, t)

100
50
0
2.5
2
0 10 1.5
20 1
30 40 0.5 t
50 60 70 0
x 80

model function

cim (x, t)

100
50
0
2.5
2
0 10 1.5
20 1
30 40 0.5 t
50 60 70 0
x 80

Fig. 8: Surface Plots for Groundwater Flow

102
describing problems and models are kept in a separate database with file
name EASY_DAT.MDB to allow easier maintenance and updates.
Parameter estimation problems are solvable without a Fortran compiler, if
the model functions are defined in form of the PCOMP modeling language.
In this case nonlinear model functions are interpreted and evaluated during
run time together with their derivatives. The full version of EASY-FIT allows
also the most flexible input of the underlying model in form of Fortran code,
and has an interface for the Watcom F77/386 3, the Salford FTN77 4 , the Lahey
F77L-EM/32, and the Microsoft Fortran PowerStation 5 compiler, where the
compiler and linker options can be altered and adapted to special needs.
To install EASY-FIT, one has to insert the CD-ROM and to execute the
setup program SETUP.EXE. Installation assumes that the run-time version of
MS-Access 97 is to be loaded together with EASY-FIT (typical and custom
setup).
EASY-FIT is delivered with Fortran source codes for the main numerical
codes MODFIT.FOR and PDEFIT.FOR, and for interfaces to the PCOMP in-
terpreter in the files MODFITEX.FOR and PDEFITEX.FOR, respectively. It
is then possible to link numerical algorithms with user-provided dimension-
ing parameters or to solve problems where model functions are defined in
form of a Fortran code. These files together with additional object codes of
internal numerical algorithms e.g. for solving the optimization problem or
for integrating differential equations, are copied to a separate directory by the
setup program, if required by the user. If the submitted default compiler in-
terface is to be changed e.g. from the Watcom to Salford or Lahey compiler,
one has to set the corresponding compiler name, some path names, and the
compiler and linker execution commands.
For more information about EASY-FIT and its capabilities, contact the
author under
klaus.schittkowski@uni-bayreuth.de
A demo version is available through the URL
http://www.uni-bayreuth.de/departments/math/˜kschittkowski/home.htm

3 WATCOM is a registered trademark of WATCOM Systems Inc.

4 FTN77 is a trademark of Salford Software Ltd.
5 PowerStation is a trademark of Microsoft Corporation

103
 References
[1] Andersson F., Olsson B. eds. (1985): Lake Gådsjön. An acid forest lake and its catchment,
Ecological Bulletins, Vol. 37, Stockholm
[2] Ascher U.M., Petzold L.R. (1998): Computer Methods for Ordinary Differential Equations and
Differential-Algebraic Equations, SIAM, Philadelphia
[3] Bellman R.E., Kalaba R.E., Lockett J. (1966): Numerical Inversion of the Laplace Transform,
American Elsevier, New York
[4] Benecke C. (1993): Interne numerische Differentiation von gewöhnlichen Differentialgleichun-
gen, Diploma Thesis, Department of Mathematics, University of Bayreuth
[5] Birk J., Liepelt M., Schittkowski K., Vogel F. (1998): Computation of optimal feed rates and
operation intervals for turbular reactors, to appear: Journal of Process Control
[6] Blatt M., Schittkowski K. (1998): Optimal Control of One-Dimensional Partial Differential
Equations Applied to Transdermal Diffusion of Substrates, in: Optimization Techniques and
Applications, L. Caccetta, K.L. Teo, P.F. Siew, Y.H. Leung, L.S. Jennings, V. Rehbock eds.,
School of Mathematics and Statistics, Curtin University of Technology, Perth, Australia,
Volume 1, 81 - 93
[7] Bock H.G. (1983): Recent advantages in parameter identification techniques for ODE, Pro-
ceedings of the International Workshop on Numerical Treatment of Inverse Problems in
Differential and Integral Equations, Birkhäuser, 95-121
[8] Boderke P., Schittkowski K., Wolf M., Merkle H.P. (1998): A mathematical model for diffusion
and concurrent metabolism in metabolically active tissue, submitted for publication
[9] Chakravarthy S.R., Osher S.(1984): High resolution schemes and the entropy condition, SIAM
Journal on Numerical Analysis, Vol. 21, No. 5, 955-984
[10] Chakravarthy S.R., Osher S. (1984): Very high order accurate TVD schemes, ICASE Report
No. 84-44
[11] Chakravarthy S.R., Osher S. (1985): Computing with high resolution upwind schemes for hy-
perbolic equations, Lectures in Applied Mathematics, Vol. 22, 57-86
[12] Dennis J.E.jr., Gay D.M., Welsch R.E. (1981): Algorithm 573: NL2SOL-An adaptive nonlinear
least-squares algorithm, ACM Transactions on Mathematical Software, Vol. 7, No. 3, 369-
383
[13] Dobmann M., Liepelt M., Schittkowski K. (1995): Algorithm 746: PCOMP: A FORTRAN
code for automatic differentiation, ACM Transactions on Mathematical Software, Vol. 21,
No. 3, 233-266
[14] Fischer P. (1996): Modellierung und Simulation der Ammonium- und Nitrat-Dynamik in struk-
turierten Waldböden unter besonderer Berücksichtigung eines dynamischen, hierarchischen
Wurzelsystems, Diploma Thesis, Department of Mathematics, University of Bayreuth
[15] Frias J.M., Oliveira J.C, Schittkowski K. (1998): Modelling of maltodextrin DE12 drying
process in a convection oven, submitted for publication
[16] Führer C., Leimkuhler B. (1991): Numerical solution of differential-algebraic equations for
constrained mechanical motion, Numerische Mathematik, Vol. 59, 55-69
[17] Groch A.G. (1990): Autmatic control of laminar flow cooling in continuous and reversing hot
strip mills, Iron and Steel Engineer, 16-20
[18] Hairer E., Wanner G. (1991): Solving Ordinary Differential Equations II. Stiff and Differential-
Algebraic Problems, Springer Series Computational Mathematics, Vol. 14, Springer
[19] Hairer E., Nørsett S.P., Wanner G. (1993): Solving Ordinary Differential Equations I: Nonstiff
Problems, Springer Series Computational Mathematics, Vol. 8, Springer
[20] Hartwanger C., Schittkowski K. (1998): Computer Aided Design of Horn Radiators for Satel-
lite Communication by Least Squares Optimization, submitted for publication
[21] Hedrich C. (1996): Modellierung, Simulation und Parameterschätzung von Kühlprozessen in
Walzstraßen, Diploma Thesis, Department of Mathematics, University of Bayreuth

104
[22] Hoch R. (1995): Modellierung von Fließwegen und Verweilzeiten in einem Einzugsgebiet unter
stationären Fließbedingungen, Diplomarbeit, Fakultät für Biologie, Chemie und Geowis-
senschaften, Universität Bayreuth
[23] Jourdan, M. (1997): Simulation und Parameteridentifikation von Destillationskolonnen,
Diploma Thesis, Department of Mathematics, University of Bayreuth
[24] Kuhn U., Schmidt G. (1987): Fresh look into the design and computation of optimal output
feedback controls for linear multivariable systems, International Journal on Control, Vol.
46, No. 1, 75-95
[25] Mayr L.M., Odefey C., Schutkowski M., Schmid F.X. (1996): Kinetic analysis of the unfolding
and refolding of ribonuclease T1 by a stopped-flow double-mixing technique, Biochemistry,
Vol. 35, No. 17, 5550-5561
[26] Nelder J.A., Mead R. (1965): A simplex method for function minimization, The Computer
Journal, Vol. 7, 308
[27] Nickel B. (1995): Parameterschätzung basierend auf der Levenberg-Marquardt-Methode in
Kombination mit direkter Suche, Diploma Thesis, Department of Mathematics, University
of Bayreuth
[28] Odefey C., Mayr L.M., Schmid F.X. (1995): Non-prolyl cis-trans peptide bond isomerization
as a rate-determining step in protein unfolding and refolding, Journal of Molecular Biology,
Vol. 245, 69-78
[29] Otter M., Türk S. (1988): The DFVLR models 1 and 2 of the Manutec r3 robot, DFVLR-Mitt.
88-3, DFVLR, Oberpfaffenhofen
[30] Prince P.J., Dormand J.R. (1981): High order embedded Runge-Kutta formulae, Journal on
Computational Applied Mathematics, Vol. 7, 67-75
[31] Rominger K.L., Albert H.J. (1985): Radioimmunological determination of Fenoterol. Part I:
Theoretical fundamentals, Arzneimittel-Forschung/Drug Research, Vol.35, No.1a, 415-420
[32] Schiesser W.E. (1991): The Numerical Method of Lines, Academic Press, San Diego
[33] Schittkowski K. (1985/86): NLPQL: A FORTRAN subroutine solving constrained nonlinear
programming problems, Annals of Operations Research, Vol. 5, 485–500
[34] Schittkowski K. (1988): Solving nonlinear least squares problems by a general purpose SQP-
method, in: Trends in Mathematical Optimization, K.-H. Hoffmann, J.-B. Hiriart-Urruty,
C. Lemarechal, J. Zowe eds., International Series of Numerical Mathematics, Vol. 84,
Birkhäuser, 295-309
[35] Schittkowski K. (1994): Parameter estimation in systems of nonlinear equations, Numerische
Mathematik, Vol. 68, 129-142
[36] Schittkowski K. (1997): Parameter estimation in partial differential equations, Optimization
Methods and Software, Vol. 7, No. 3-4, 165-210
[37] Schittkowski K. (1998): Parameter Estimation in a Mathematical Model for Substrate Diffu-
sion in a Metabolically Active Cutaneous Tissue, to appear: Proceedings of the Optimization
Day, Perth, Australia, June 29-30
[38] Schittkowski K. (1998): PDEFIT: A FORTRAN code for parameter estimation in partial dif-
ferential equations, to appear: Optimization Methods and Software
[39] Shampine L.F., Watts H.A. (1979): The art of writing a Runge-Kutta code, Applied Mathema-
tics and Computations, Vol. 5, 93-121
[40] Simeon B., Rentrop P. (1993): An extended descriptor form for the simulation of constrained
mechanical systems, in: Advanced Multibody System Dynamics, W. Schiehlen ed., Kluwer
Academic Publishers, p. 469-474
[41] Simeon B., Grupp F., Führer C., Rentrop P. (1994): A nonlinear truck model and its treatment
as a multibody system, Journal of Computational and Applied Mathematics, Vol. 50, 523-
532
[42] Stehfest H. (1970): Algorithm 368: Numerical inversion of Laplace transforms, Communica-
tions of the ACM, Vol. 13, 47-49
[43] Steingässer I. (1994): The organized HaCaT cell culture sheet: A model approach to study
epidermal peptide drug metabolism, Dissertation, Pharmaceutical Institute, ETH Zürich

105
 [44] Sweby P.K. (1984): High resolution schemes using flux limiters for hyperbolic conservation
laws, SIAM Journal on Numerical Analysis, Vol. 21, No. 5, 995-1011
[45] Van Genuchten M.T., Wierenga P.J. (1976): Mass transfer studies in sorbing porous media. 1.
Analytical solutions, Soil Sci. Soc. Am. Journal, Vol. 44, 892-898
[46] Walter S., Lorimer G.H., Schmid F.X. (1996): A thermodynamic coupling mechanism for
GroEl-mediated unfolding, Biochemistry, Vol. 93, 9425-9430
[47] Wang Z., Richards B.E. (1991): High resolution schemes for steady flow computation, Journal
of Computational Physics, Vol. 97, 53-72
[48] Wansbrough R.W. (1985):Modeling chemical reactors, Chemical Engineering, Vol. 5
[49] Yang H.Q., Przekwas A.J. (1992): A comparative study of advanced shock-capturing schemes
applied to Burgers’ equation, Journal of Computational Physics, Vol. 102, 139-159
[50] Yee H.C. (1985): Construction of a class of symmetric TVD schemes, Lectures in Applied
Mathematics, Vol. 22, 381-395
[51] Zschieschang T., Rockhausen L. (1996): Zur Signalauswertung und Modellierung bei peri-
odischen Vorgängen mit transienten Anteilen, Report, Institute of Mechanics, Technical
University of Chemnitz-Zwickau, Chemnitz, Germany

106

View publication stats