GRADIENT DESCENT OPTIMIZATION OF PENICILLIN PRODUCTION: A

CONSTANT-VOLUME STRATEGY

Matej Pcolka
Department of Control Engineering
Faculty of Electrical Engineering
Prague, Czech Republic
email: pcolkmat@fel.cvut.cz
Abstract
Fermentation processes as a class of biological processes
containing the growth of the biomass (bacteria, yeasts) resulting from the consumption of essential substrate supplies (source of carbon, nitrogen, oxygen, etc.) constitute
a very delicate challenge from the control point of view.
Nonlinearities and complicated dynamics of the biomass
growth followed by the production of various products
(from which especially the variety of antibiotics makes
the fermentation processes attractive for the industrial utilization) come hand in hand with the attractivity and going along with high level of uncertainty and difficult online measurement of the process variables turn attempts on
optimal control of the fermentation process into a rather
complicated task. Gradient method whose theory (partially
adapted and fully implemented by authors of this paper)
suggests a possible way of handling these issues combined
with a fresh control strategy of fixed volume proves a significantly better performance on a set of numerical experiments than other known methods. Moreover, model structure used in the previous work has been modified so that
it corresponds with a fresh optimization strategy which
stands for the main contribution of this paper.
KEY WORDS
Fermentation process, penicillin, gradient method, optimization

Introduction

Although the industrial productivity of antibiotics has been

increasing over the last few decades rapidly, this can be
credited mostly to a massive improvement in an area of
production technologies. Nevertheless, the control background of the production has been paid less attention which
has resulted in a suboptimal operation manners achieving
final product concentrations deep below maximum reachable values. The important point in the endeavour after production enhancement is to realize the contribution, pros and

Sergej Celikovsk
y
Department of Control Engineering
Faculty of Electrical Engineering
Prague, Czech Republic
email: celikovs@utia.cas.cz
cons of used control strategy. A wide variety of ways how
to operate the input feed flow (which influences the formation of the final product especially by the amount of the
substrate nutrient supplied to system through it) has been
discussed in literature so far. The simplest of them (mentioned in [1]) are indirect feedback methods for nutrient
feeding based on pH or dissolved oxygen measurements the substrate concentration is then maintained at predetermined setpoint by either a simple open loop controller [2]
or an on/off [3] or a PID type controller. More involved are
fuzzy approaches which appeared in the 1990s [4] and have
been revitalized at the beginning of the millennium [5].
However, the most impressive results have been reached
using model predictive control (MPC) approach based on
the given criterion minimization [6]. The main drawback
of this method is the fact, that it is usually performed either
with an approximately or exactly linearized mathematical
model of the controlled process. Approximate linearization
when performed in certain operating point can be invalid
for operating points far away from that which linearization
was performed in (and it is known that the operating points
range varies a lot during the cultivation) and each-step approximate linearization can be prohibitively time consuming. Moreover, no stability assumptions can be made for
the approximate models obtained in each step and even one
unstable model obtained by approximate linearization can
degrade the MPC performance vastly. On the other hand,
exact linearization blows all MPC problems away - unfortunately, in the area of fermentation processes, the existence of exact linearization is rather rare and occasional.
Therefore, a proper alternative is needed - gradient descent
method which has already proved encouraging results in
various research areas [7], [8], [9] is a strong candidate as it
can handle even a nonlinear process model very effectively.
The crucial point for this model based method is the availability of a mathematical model describing the biochemical process and determining an adequate cost functional to
be optimized. This paper is tied up with the authors previous work and modifies previously used model structure.

Enhanced optimization strategy is presented and obtained

results suggest using this method in industrial practice.
The paper is organized as follows: Section 2 introduces nonlinear unstructured dynamical model of the fermentation process which is used for the optimization purposes. The penicillin cultivation is chosen to represent the
fermentation processes, modification of the previously used
model (which is crucial for the use of new control strategy introduced later in this paper) is explained. Section
3, the optimization issue of the final product concentration maximization including the constraints specification
is formulated. The gradient method is introduced, its theoretical background is clarified and a fresh control strategy (a constant-volume strategy) is proposed. In Section 4,
results of a constant-volume strategy are presented, compared to those obtained using strategy presented earlier and
discussed. Section 5 concludes the paper.

MODEL OF THE FERMENTATION

PROCESS

Let us consider a fed-batch fermentation process of penicillin cultivation [10], [6] described by the following
model:
dV
dt
dCX
dt
dCS
dt
dCP
dt

= u V (e

Toper Tf
Tb Tf

1),

dV CX
,
dt V
CS,in u dV CS
= CX +

,
V
dt V
dV CP
= CX KH CP
.
dt V
= ( KD )CX

(1)

Here V (l) refers to cultivation broth volume,

CX (gl1 ) represents biomass concentration, CS (gl1 )
stands for the limiting substrate concentration (let us consider carbon to be the limiting substrate) and CP (gl1 )
represents the final product (penicillin) concentration. The
substrate feed flow rate u (lh1 ) is considered to be the operated input to the model.
Parameters (h1 ), and w () are specific vaporization constants, Toper (K) represents empirically obtained
operational temperature Topt (K) [11], and Tf (K) and
Tb (K) refer to the freezing temperature and the boiling
temperature of the broth, respectively, which are considered to be the same as those of the water [6].
A simple constant term KD (h1 ) models biomass
death kinetics while the total of specific biomass

growth rate (h1 ) and the specific production rate

(ggDW 1 h1 ) weighted by biomass on substrate yield
coefficient YX/S and the product on substrate yield coefficient YP/S gives specific substrate consumption rate
1
1
(ggDW 1 h1 ) = YX/S
+YP/S
. In this paper, Contois
kinetics of the biomass growth [12] and Haldane kinetics
[13] of the product formation are considered, which results
in the following expressions for the and :
CS
,
KX CX + CS
CS
,
= max
KP + CS + CS2 /KI
= max

(2)

where max (ggDW 1 h1 ), max (ggDW 1 h1 ) are

the maximum specific growth and production rates,
KX (ggDW 1 ) is the Contois saturation constant,
KP (gl1 ) is product formation saturation constant and
KI (gl1 ) is product formation inhibition constant for
product formation.
Input substrate concentration CS,in (gl1 ) reflects the
effect of the input flow u on the substrate concentration CS .
Finally, penicillin hydrolysis is modeled by a degradation
constant KH (h1 ). At this point, more interested readers
are referred to [10] and [6] where the model is described in
more details.
Now, let us assume, that the cultivation volume V can
be not only increased by exogenous input u, but a volume
withdrawal can be performed as well. This requires a new
input variable to be introduced and the volume differential
equation changes into:
Toper Tf
dV
w
= u1 V (e Tb Tf 1) u2 ,
dt

(3)

where u1 corresponds with the old input variable and

u2 (lh1 ) stands for volume withdrawal.
This little change of volume differential, however, effects the differential equations of other state variables as
well. Let us remind that the state variables are in form
of concentration which cannot be increased nor decreased
by volume withdrawal. Therefore, terms including volume
differential should be modified as follows: dV /dt
dV /dt + u2 . It is clearly visible that adding u2 to volume
differential term eliminates the effect of volume withdrawal
in accordance with physical expectation.
For the needs of optimization and in order to follow
the conventional notation, let us rewrite the model 1 into
the ordinary form using xT = [V, CX , CS , CP ] and uT =
[u1 , u2 ]:

Toper Tf

x 1 = u1 (e Tb Tf 1)x1 u2 ,


 Toper Tf
 

x2
x3
w
KD x2 u1 e Tb Tf 1 x1
,
x 2 = max
KX x2 + x3
x1

 


 Toper Tf
max
x3
x3
x3
max
CS,in u1
w T T
b
f
x 3 =

1
x1
+
x
+

e
,
2
1
YX/S KX x2 + x3
YP/S KP + x3 + x23 /KI
x1
x1
 

 Toper Tf
x4
x3
w T T
b
f
1 x1
x 4 = max
x2 KH x4 u1 e
.
KP + x3 + x23 /KI
x1

OPTIMAL CONTROL DESIGN

In authors previous work, an optimal feeding strategy coming out of a gradient projection method has been introduced. Theoretical complication given by the fact that the
saturations may depend on the states has been successfully
addressed as the projection has been replaced by a dynamic
procedure and assumption on sufficiently large cultivation
tank volume has been made. However, in industrial application, the cultivation tank may be filled up with an initial
volume so large that applying computed input feed flow
rate leads to tank overflow in short horizon. A perspective
offering solution to this problem has been tackled in the
previous section. Here, we propose a new control strategy
which makes use of the second input u2 and brings interesting results enhancement.
3.1

Fixed-volume strategy

By applying another exogenous input u2 , one can avoid

tank overflow, yet another problem occurs. Thoroughful
reader has surely already noticed that having introduced
two input variables, the first differential equation of mathematical description of the system does not comply with
physical laws. It can be shown that at certain finite time
point the volume can reach zero value and further withdrawal can theoretically cause negative volume, which is

(4)

physically impossible. One way of avoiding this is to set a

dynamical constraint on the second input u2 which ensures
that at the point of zero volume V the withdrawal does not
exceed the inlet flow. However, looking at the issue from
the engineering point of view, it is not either convenient
to decrease the volume below certain too low value as the
final product amount equals to product concentration CP
multiplied by the volume V .
Let us introduce an idea leading to a strategy solving
the sketched negative volume difficulty. It consists in an
assumption that the second input u2 is used to compensate
the effect of the first input u1 on the volume V . In other
words, we require the second input to be able to hold the
volume constant - from the first differential equation of the
model 4 which should be equal to zero, the second input u2
can be calculated directly as:
dV
= u1 Kvap V u2 = 0
dt
u2 = u1 Kvap V0 ,

(5)

where V0 is the initial state and Kvap is the overall vaporT

Tf
ization constant, Kvap = exp(w oper
1). As the
Tb Tf
first differential equation is identically equal to zero and
the second input u2 effects only the first state equation, the
system gets simplified into the following three-state model
with T = [CX , CS , CP ]:



2
1
max
KD 1 ( Kvap V0 ) ,
KX 1 + 2
V0


max
2
max
2
CS,in
2
=
+
1 +
( Kvap V0 ) ,
YX/S KX 1 + 2
YP/S KP + 2 + 22 /KI
V0
V0
2
3
= max
1 KH 3 ( Kvap V0 ) .
KP + 2 + 22 /KI
V0

1 =
2
3

We can see, that the three state model 6 involves

one optimization variable which corresponds with u1
in model 4. Model 6 supplemented by the corresponding
model parameters (tab. 1) offers an engineer a tool to de-

(6)

sign the optimal control minimizing a properly chosen criterion. Let us remark that V0 is a parameter that can be
specified individually.

Table 1. Model parameters.

3.2

Table 2. Optimization constraints.

Parameter

Value

Parameter

Value

Parameter

Value

max
max
KP
YX/S
KD
KX
KH

0.11
0.004
0.1
0.47
0.0136
0.06
0.01

YP/S
CS,in
Kvap
KI
Toper
Tf
Tb

1.2
500
6.23 104
0.1
298
273
373

1,0
2,0
3,0
max

1.5
6
0
0.05

3.3

Optimization task formulation

From the optimization point of view, penicillin production

optimization can be viewed as fixed initial state, free time
interval and free final state issue. Without any loss of generality and due to upper cultivation duration constraint,
let us now consider multiple optimization routines with
fixed time intervals of length tj,end {200, 300, 400, 500}
where j denotes certain optimization routine. This helps us
to simplify the optimization procedure and avoid difficulties with general time interval solution.
For the purpose of the optimization, the objective
functional reflecting the optimization effort needs to be formulated. As the main goal is to maximize the final product
concentration, the following criterion in the Mayer form
which is to be minimized is formulated:
J = 3 (tend ).

(7)

Regarding state optimization constraints, it can be

shown that the new model 6 satisfies physical constraints
(state variables nonnegativity) and no further attention is
needed to be paid to low state constraints. Moreover,
fixed-volume strategy eliminates the need for upper volume constraint handling. Thus, input saturation constraint
0 max and input piecewise constant character
d
dt = 0 for ml t < (m + 1)l, m = 0, 1, . . . (accomplished by sampling of the input with sampling period
l = 4 h) are the only static constraints related to this optimization task.
Having properly defined the system equations, the input constraints and the objective functional, the optimization problem for t [t0 , tend ] (without any loss of generality, let us consider t0 = 0 h) can be summarized:
(t) = arg min J ((t)) ,
(t)

(8)

such that the following constraints hold:

This method belongs to family of the optimal control methods [14]. For the problem stated by 8 and the constraints
given in the form of 9, the optimal input is searched iteratively. First of all, the initial input vector 0 is estimated
(in our case, a zero vector has been chosen). Then, the following procedure is applied:

k+1
= k

(t0 ) = 0 ,
(9)

Here, f ((t), (t)) refers to the model 6. The values of 0

and max are summarized in tab. 2.

J
,

(10)

where k = 0, 1, 2, . . . is the number of the iteration

and is the search step parameter. Here it should be noted
that direct calculation of J
is quite complicated due to
the fact, that 3 depends on via a differential equation.
Therefore, let us rather introduce a Hamiltonian H in the
following form:
H = pT f.

(11)

Here, f refers to the model 6 and p is the adjoint state vector

solved back in time. To compute the gradient of 7 with
respect to (t), set first:
H
,

H
,
=
p
(t0 ) = 0 ,


d
p(tend ) =
|t=tend ,
d
p =

(12)

where is the terminal term of the optimization criterion. In our case, = 3 (tend ) from which it follows p(tend ) = [0, 0, 1]T . It can be shown (mathematically rigorous proof is beyond the scope of this paper) that
J
H
H
= and thus, gradient can be used in iterative
procedure 10, which changes into:

k+1
= k +

= f ((t), (t)),
(t)
0 (t) max .

Nonlinear gradient method

H
.

(13)

Due to the overall nonconvexity of the problem, a

constant search step parameter has been chosen = 0.002.
Input saturation constraints are handled by mapping the
iterated input vector k+1 on an admissible input set
admiss = {, 0 max } by a simple saturation.

4.5

90

OPTIMIZATION RESULTS

80

V0

3.5

70

60

2.5

50

P (g)

CP (gl )

Requirement of piecewise constant nature of the input is

satisfied by it sampling with sampling period l = 4 h.
The iterative procedure described by 13 terminates at
the moment when negatively taken value of the decrease
between the k-th and the k + 1-st criterion value (Jk+1
Jk ) is less than a chosen tolerance.

40

1.5

30

20

0.5

V0

10

0
0

80

160

240

320

0
0

400

80

160

t (h)

In this section, results obtained by constant-volume strategy are presented and compared to those obtained by original one-input gradient method optimization presented in
authors previous work. The optimization results have been
simulated with the penicillin cultivation model in MATLAB environment.
4.1

Constant-volume strategy results

First, constant-volume strategy has been tested on a simulation with initial volume V0 = 7 l. Figure 1 shows
very satisfactory results of the optimization. It is obvious,
that cultivation period that contributes the most to the final product concentration CP (tend ) takes approximately
the last 75 h. A rapid increase product concentration can
be observed, however, the biomass concentration decreases
strongly. This has a simple biological explanation - as can
be seen from the characters of both and (see eq. 2),
increasing one of them, the second one decreases, which
corresponds to the fact that either the biomass population
growth or the penicillin production is being preferred at one
time.

320

400

Figure 2. Volume-dependency of CP and P profiles.

It can be seen that with increasing V0 , product concentration CP over the cultivation horizon decreases. This can
be explained as follows: the feeding effect of on the controlled system is always inversely proportional to the actual
amount of broth in tank, which in case of constant volume
V0 does not change over the cultivation period. With increase of V0 , the effect of decreases which then leads to
performance aggravation.
On the other hand, the total amount of product P increases with increasing fixed volume V0 . This is due to
the fact that the total product amount P is proportional not
only to product concentration CP but also to broth volume, P = CP V0 . Although product concentration CP
decreases, its decrease is compensated by fixed volume V0
increase.
4.5

90

80

100
C (gl1)

240
t (h)

70
3.5

0
0

50

100

150

200
t (h)

250

300

350

400

P (g)

CP (gl1)

50

50

C (gl1)

60
40

2.5

40

20
0
0

50

100

150

200
t (h)

250

300

350

400

CP (gl )

6
1

60

13

19

V0

25

31

37

30

13

19

V0

25

31

37

4
2
0
0

50

100

150

200
t (h)

250

300

350

400

Figure 1. State profiles - constant-volume strategy.

4.2

Volume dependency

Next in this paper, constant-volume strategy has been tested

on multiple simulations with various initial volume V0 . Initial volume conditions have been chosen as linearly increasing, V0 (l) = 7 + 3k, k {0, 1, . . . , 10}.
Figure 2 shows dependency of profiles of both product concentration CP and product amount P on initial volume V0 .

Figure 3. CP (tend ) and P (tend ) dependency.

Yet, another interesting tendency is to be observed

from fig. 4 - it is a convergency of input profile to a highsaturation-valued vector with V0 increase. From technoV0
logical point of view, this is caused by an increase of max
ratio - the higher the volume is, the more feed is needed to
V0
keep the whole system developing and the higher the max
ratio is, the longer a high-saturated input must be applied.
In order to inspect the effect of various V0 more in details, another set of simulations has been performed, however, with a constant ration V0 /max = 7/0.05. The initial
volume V0 has been set linearly growing as in the previous
simulation set.
Figures 5 and 6 show that holding the V0 /max ratio

0.05

0.25

0.045
0.04

0.2
u (lh1)

u (lh1)

0.035
0.03
0.025

0.15

0.02

0.1
V

0.015

0.01

0.05

V0

0.005
40

80

120

160

200
t (h)

240

280

320

360

160

140

3.5

P (g)

C (gl1)

100
2.5

80

2
60
1.5
40

20

0.5

160

240

320

V0

0
0

400

80

160

t (h)

240

320

400

t (h)

Figure 5. Volume-dependency of CP and P profiles, fixed

V0 /max ratio.

4.48

160

4.46
140
4.44
120

4.42

100
P (g)

CP (gl1)

4.4
4.38

80

4.36
4.34

60

4.32
40
4.3
4.28

13

19

V0

25

31

240

320

400

Dependence of the optimized profiles on cultivation length

120

80

160
t (h)

4.3

0
0

80

Figure 7. Volume-dependency of input profile, fixed

V0 /max ratio.

Figure 4. Volume-dependency of input profile.

4.5

0
0

400

37

20

13

19

V0

25

31

37

Figure 6. CP (tend ) and P (tend ) dependency, fixed

V0 /max ratio.

As has already been mentioned, cultivation length is considered to be constant, yet it can be chosen from a set
{200, 300, 400, 500} h. Figure 8 compares cultivation with
classical gradient method (presented in authors previous
work) to the constant-volume strategy. For every chosen
cultivation length, it is obvious that constant-volume strategy achieves better results than the classical one and the
product concentration at the final time CP (tend ) is higher.
Looking at figure 9, convergence of input profiles to a certain superprofile can be seen. Similar kind of convergence has been mentioned in [9] as well. However, although both profiles are stable backward in time, they do
not settle down at the same value (here, we assume settling down in negative march of time). In negative time,
classical gradient method settles down on a zero value
while constant-volume strategy obtained input profile settles down on upper saturation. The fact that volume is held
constant by the second virtual input (virtual due to the fact
that it is not considered in optimization) and it cannot dynamically aggravate the product concentration profile enables to deliver more feed into cultivation tank without negative effect of volume increase and thus, with better fed
biomass population, the product concentration obtained at
the end of the cultivation is higher.
CP (gl1)

0
0

4
2

gradient method
constantvolume strategy

0
0

50

100

150

200

4
2
0

4
2

50
t (h)

100

150

200

250

300

100

150
t (h)

200

250

300

350

400

200

250
t (h)

300

350

400

450

500

gradient method
constantvolume strategy

0
0

CP (gl1)

fixed, CP profiles aggravation is not as drastic as in the previous case and the P profiles improvement is much more
superior. Besides a linear P (tend ) increase, figure 6 uncovers a little bit strange stair-like CP (tend ) tendency. However, looking at Figure 7 where input profiles for various
V0 are shown, it is obvious that the stair-like character is
caused by the requirement of piecewise constant input profile.

CP (gl1)

CP (gl1)

t (h)
gradient method
constantvolume strategy

4
2
0
0

50

gradient method
constantvolume strategy

50

100

150

Figure 8. Product concentration profiles for various cultivation lengths, comparison.

u (lh )

0.05
gradient method
constantvolume strategy
0
0

50

100

150

200

100

150

200

250

300

0.05

u (lh )
u (lh )

0.05

u (lh )

t (h)
0.05
gradient method
constantvolume strategy
0

0
0

50
t (h)

gradient method
constantvolume strategy
50

100

150
t (h)

200

250

300

350

400

200

250
t (h)

300

350

400

450

500

gradient method
constantvolume strategy
0
0

50

100

150

Figure 9. Input profiles for various cultivation lengths,

comparison.

CONCLUSION

This paper improves the patterns suggested in previous

work of authors. A new model of the controlled system
involving the second input variable is derived, a fresh fixedvolume control strategy is introduced and the model is
adapted so that it comports with the strategy requirements.
Results of the optimization routine are verified by a set of
numerical experiments and they are compared to those of
previously introduced classical gradient method. The comparison can be summarized as a very encouraging as the
new strategy achieves better results than the old one. Moreover, as one of the former states (volume V ) is required to
be a parameter, result dependence on this parameter is examined and discussed. A superprofile convergence (observed in earlier publications) occurs in this case as well
and this supports the claim that it is a property of the optimization issues where the fixed time, fixed initial condition
and free terminal condition are considered.

References
[1] S. Lee, High cell-density culture of Escherichia
coli, Trends in biotechnology, vol. 14, no. 3, pp. 98
105, 1996.
[2] M. Gregory and C. Turner, Open-loop control of specific growth rate in fed-batch cultures of recombinant
E. coli, Biotechnology Techniques, vol. 7, no. 12, pp.
889894, 1993.
[3] T. Suzuki, T. Yamane, and S. Shimizu, Phenomenological background and some preliminary trials of automated substrate supply in pH-stat modal fed-batch
culture using a setpoint of high limit, Journal of Fermentation and Bioengineering, vol. 69, no. 5, pp.
292297, 1990.

[4] T. Siimes, P. Linko, C. von Numers, M. Nakajima,

and I. Endo, Real-time fuzzy-knowledge-based control of Bakers yeast production, Biotechnology and
bioengineering, vol. 45, no. 2, pp. 135143, 1995.
[5] J. Horiuchi, Fuzzy modeling and control of biological processes, Journal of bioscience and bioengineering, vol. 94, no. 6, pp. 574578, 2002.
[6] A. Ashoori, B. Moshiri, A. Khaki-Sedigh, and
M. Bakhtiari, Optimal control of a nonlinear fedbatch fermentation process using model predictive approach, Journal of Process Control, vol. 19, no. 7,
pp. 11621173, 2009.
[7] Y. Dai and Y. Yuan, A nonlinear conjugate gradient
method with a strong global convergence property,
SIAM Journal on Optimization, vol. 10, no. 1, pp.
177182, 2000.
[8] E. Birgin and J. Martnez, A spectral conjugate gradient method for unconstrained optimization, Applied Mathematics and optimization, vol. 43, no. 2,
pp. 117128, 2001.
[9] S. Celikovsky, S. Papacek, A. Cervantes-Herrera, and
J. Ruiz-Leon, Singular perturbation based solution
to optimal microalgal growth problem and its infinite time horizon analysis, Automatic Control, IEEE
Transactions on, vol. 55, no. 3, pp. 767772, 2010.
[10] J. Van Impe and G. Bastin, Optimal adaptive control of fed-batch fermentation processes with multiple substrates, in Control Applications, 1993., Second IEEE Conference on. IEEE, 2002, pp. 469474.
[11] A. Constantinides, J. Spencer, and E. Gaden Jr, Optimization of batch fermentation processes. II. Optimum temperature profiles for batch penicillin fermentations, Biotechnology and Bioengineering, vol. 12,
no. 6, pp. 10811098, 1970.
[12] D. Contois, Kinetics of bacterial growth: relationship between population density and specific growth
rate of continuous cultures, Microbiology, vol. 21,
no. 1, p. 40, 1959.
[13] G. Briggs and J. Haldane, A note on the kinetics of
enzyme action, Biochemical journal, vol. 19, no. 2,
p. 338, 1925.
[14] A. Bryson and Y. Ho, Applied optimal control, New
York: Blaisdell, 1969.