Solution Techniques for Periodic Control Problems :

A Case Study in Production Planning

H. MAURER , CH. BU SKENS
Westfalische Wilhelms-Universitat Munster
Institut fur Numerische Mathematik
Einsteinstrasse 62, D{48149 Munster, Germany
e-mail: maurer@math.uni-muenster.de
AND
G. FEICHTINGER
Institute for Econometrics, Operations Research and Systems Theory,
University of Technology, Argentinierstrasse 8 , A{1040 Wien, Austria
e-mail: or@e119ws1.tuwien.ac.at

SUMMARY
Two numerical techniques for solving optimal periodic control problems with
a free period are developed. The f irst method uses shooting techniques for
solving an appropriate boundary value problem associated with the necessary
conditions of the minimum principle. A convenient form of the transver-
sality condition for the free period is incorporated. The second method is a
direct optimization method that applies nonlinear programming techniques to
a discretized version of the control problem. Both numerical methods are il-
lustrated in detail by a nonconvex economic production planning problem. In
this model, the  -test reveals that the steady{state operation is not optimal.
The optimal periodic control is computed such that a complete set of neces-
sary conditions is veried. The solution techniques are extended to obtain the
optimal periodic control under various state constraints. A sensitivity analy-
sis of the optimal solution is performed with respect to a specic parameter
in the model.
Keywords: periodic optimal control, boundary value problem, direct opti-
mization method, production planning, sensitivity analysis

1
1 Introduction
Periodic control problems have been studied extensively in the literature.
Many applications of periodic control problems have been reported in che-
mical reaction engineering (cf. the survey in References 1, 2 ), electrical
engineering, 3 6 ight path engineering 7 10 and, more recently, in various
economic planning problems.11 14 For autonomous problems, the existence
of an optimal periodic control that improves the steady state solution can be
derived from the -test developed in References 4, 15.
The theory of necessary conditions (minimum principle) for periodic cont-
rol problems is well developed in the autonomous case.5;15 19 Surprisingly,
however, one can hardly nd the complete numerical solution of a practical
problem for which all necessary conditions have actually been checked. From
this numerical status we draw the main motivation for this paper. We shall
present two numerical solution techniques for computing periodic controls
with a free period. Both numerical methods are illustrated by the solution of
an economic production planning problem. The numerical techniques can be
extended to include state constraints. This fact is demonstrated by imposing
bounds on storage and production in the production planning problem.
The organization of the paper is as follows. Section 2 discusses the necessary
conditions for autonomous periodic control problems with free final time.
When solving the complete boundary value problem (BVP) arising from the
minimum principle it is convenient to use a transversality condition for the
free period in a form that is dierent from the one in References 18, 19.
We show that e.g. shooting techniques 10;20;21 can be applied to BVP after
introducing appropriate modifications of the periodic boundary conditions.
Numerical experience with shooting methods has indicated that these me-
thods provide a highly accurate solution but suer from the drawback that
they need a rather precise initial guess of the adjoint variables. As an al-
ternative method, we propose a direct optimization method that dispenses
with the adjoint variables. This direct optimization method operates in the
spirit of References 22{25 and applies nonlinear programming methods to a
discretized version of the periodic control problem.
Section 3 discusses the numerical solution of a production planning problem
that extends the one discussed in References 11, 12. We present both the nu-
merical solution of the model with nonzero discount rate 11;12 and with zero
discount rate. Using the idea of the -test 4;15 in a direct way, we show that
the steady{state solution is proper (i.e. non{optimal) for all discount-rates
in a suitable range of system parameters. In case of a nonzero discount rate,

2
this analysis is related to the Hopf bifurcation analysis given in References 11,
12. Having established the existence of a periodic control, we compute the
optimal periodic control employing both methods from Section 2. Quite sur-
prisingly, the numerical analysis reveals that the periodic planning problem
with zero discount-rate is very similar to the `sailboat example' considered
in References 10, 26.
In Section 4, we consider the optimal periodic solution of the economic plan-
ning problem under state constraints. A complete numerical solution is ob-
tained by extending both numerical methods from Section 2. When state
constraints are present, solution techniques using the full BVP arising from
the minimum principle are rather tedious. The alternative direct optimiza-
tion method is much easier to handle but provides less accurate informations
on junction points with the boundary of the state constraints.
In recent years, there is an increasing interest in the stability of the optimal
solution with respect to parameters or perturbations in the system. Re-
ferences 27{30 develop verifiable conditions which ensure that the optimal
solution is a differentiable function of any parameter. In Section 5, we per-
form a sensitivity analysis of the optimal periodic control with respect to
a specific parameter in the production planning problem. This sensitivity
analysis is based on the observation that the first order derivatives of the
optimal periodic solution with resp. to the parameter satisfy a linear BVP.
We indicate how to use sensitivity differentials in a first order approximation
of the perturbed solution.

2 Numerical methods for optimal periodic

control problems
2.1 Necessary conditions
We restrict the discussion to a special version of the general optimal periodic
control problem (OPC) considered in Ref. 17. Let L : IRn  IRm ! IR and
f : IRn  IRm ! IRn be C k {fuctions with k  1 and let tf > 0 denote the
free period. Then problem (OPC) consists in determining a control function
u 2 L1m [0; tf ] and a period tf > 0 that minimize the average cost
Z tf
J (u; tf ) = t1 0 L(x(t); u(t)) dt (1)
f

3
subject to
x_ (t) = f (x(t); u(t)) for a.e. t 2 [0; tf ] ; (2)
x(0) = x(tf ) : (3)
Following References 18, 19, the Hamiltonian for (OPC) becomes
H (x; ; bfu) = L(x; u) +  f (x; u) ;  2 IRn : (4)
Let the pair x( : ); u( : ) be an optimal periodic solution that is assumed to
be normal. Then the necessary conditions state that there exists an adjoint
function  2 Wn1;1[0; tf ] satisfying
_ (t) = Hx (x(t); (t); u(t)) for a.e. t 2 [0; tf ] ; (5)
(0) = (tf ) ; (6)
Z tf
(t)f (x(t); u(t)) dt = 0 ; (7)
0
u(t) 2 argmin u H (x(t); (t); u) for a.e. t 2 [0; tf ] : (8)
Condition (7) constitutes the transversality condition associated with the free
period tf . Note that condition (7) is stated in a form that is slightly dierent
from the one in References 18, 19. The proof of the transversality condition
is closely connected with numerical techniques for solving problem (1){(3).
Introduce the new time variable s 2 [0; 1] defined by the transformation
t = s
tf ; 0  s  1 ; (9)
and identify the new state x(s) resp. control u(s) with x(s
tf ) resp.
u(s
tf ) . Then problem (OPC) in (1){(3) is equivalent to the following
control problem (OPC)' with xed time interval [0; 1] : Minimize
Z1
J (u; tf ) = L(x(s); u(s)) ds (10)
0
subject to
x_ (s) = tf
f (x(s); u(s)) ; t_f = 0 ; (11)
x(0) = x(1) : (12)
The variable (x; tf ) in problem (OPC)' is treated as the augmented state
variable. When applying the usual Pontryagin minimum principle to (10){
(12), the transversality condition for the adjoint variable associated with tf
yields the transversality condition (7).

4
2.2 Shooting methods for solving the periodic boun-
dary value problem
For numerical purposes it is convenient to rewrite the necessary conditions
(5){(7) in terms of the new time variable s 2 [0; 1] dened by (9). Identifying
also (s) with (t) = (s
tf ) , we have to solve the following boundary value
problem BVP in the time interval [0; 1] :
x_ (s) = tf
f (x(s); u(s)) ; t_f = 0 ; (13)
_ (s) = tf
Hx (x(s); (s); u(s)) ; (14)
x(0) = x(1) ; (0) = (1) ; (15)
y_ (s) = (s)f (x(s); u(s)) ; y(0) = y(1) = 0 : (16)
This BVP with 2(n+1) ODE's and boundary conditions is not yet suitable for
numerical computation. It determines only the orbit but does not specify the
initial point of the underlying periodic trajectory. To compute the trajectory
we replace the 2n boundary conditions (15) by the following conditions. Fix
a component k 2 f1; :::; ng and let xk0 be a value that is supposed to lie
in the range of the periodic solution xk (s) . Then we replace the n periodic
conditions x(0) = x(1) by n + 1 conditions
xk (0) = xk0 ; xk (1) = xk0 ; and xi(0) = xi(1) for i 6= k : (17)
The additional condition for xk requires that one deletes the corresponding
periodic condition in (0) = (1) . Hence, we consider only n 1 periodic
conditions
i (0) = i (1) for i 6= k : (18)
For solving the BVP it remains to express the control variable u in terms of
x and  via the minimum condition (8). We restrict the discussion to the
case of a regular Hamiltonian and assume that the strict Legendre{Clebsch{
condition Huu > 0 holds along the optimal solution. In addition, suppose
that there exists a locally unique C1{solution u(x; ) of the equation
Hu(x; ; u(x; ))  0 : (19)
Hence, in summary we have to compute the solution of the BVP (13), (14),
(16){(18) in 2n +1 variables x; ; tf where the control u(x; ) is substituted
from (19).
A very efficient method for solving a BVP of such type is the multiple shoo-
ting method in Ref. 20 as implemented in the code BNDSCO of Ref. 21.
This method produces a solution of high accuracy both in state and adjoint

5
variables. However, the user of this method is often faced with the difficulty
of finding a rather precise initial guess for the adjoint variables. For that
reason, we introduce an alternative method in the next section.

2.3 The direct optimization method

The direct method uses nonlinear programming methods to solve the problem
(10), (11), (17) in the interval [0; 1]. Applying the discretization techniques in
References 22{25, we proceed as follows. Let N > 0 be a positive integer and
let h = 1=N be the mesh spacing with mesh points i = ih; i = 0; 1; : : : ; N .
Approximations of the values x(i) and u(i) are denoted by xi and ui. Let us
restrict the discussion to Euler's method. In Euler's method, the functional
(10) and the ODE (11) are approximated by
NX1
h L(xi; ui) ; (20)
i=0
x = xi + h
tf
i +1
f (xi; ui) ; i = 0; : : :; N 1 : (21)
Having in mind the boundary conditions (17) we choose the following opti-
mization variables
y = (u0; u1; : : :; uN 1; x~ 0; tf ) ; (22)
where x~ 0 is the free initial state x0 with component x0k deleted in view of
(17). In contrast to many direct optimization approaches, we do not treat
(21) as explicit equations but rather as recursive equations to compute xi,
i = 1; : : : ; N , for given data (22). Then the NLP-problem with optimization
variables (22) consists in minimizing (20) subject to the constraints
xNj = x0j for j = 1; : : : ; n : (23)
Instead of Euler's method (20) and (21), one can use any higher order single
step method, e.g. a higher order Runge-Kutta scheme. There are many
reliable optimization codes for solving the NLP-problem (20){(23). We have
employed the sequential quadratic programming code E04UCF in the NAG-
library.
Several of these options have been implemented in the code NUDOCCCS of
Ref. 23. This code provides a direct optimization method for solving optimal
control problems subject to control and state inequality constraints.

6
3 Optimal production planning with inten-
sity splitting
We consider the production planning model studied in References 11, 12 and
treat also the case of zero discount rate. The model consists of the following
variables and functions :
z(t) : stock of inventory at time t 2 [0; tf ],
v(t) : rate of production,
u(t) : rate of change in the production (control),
d : constant demand,
K (v) : cost of production,
1 hz 2 : inventory costs , h > 0 ,
2
1 cu2 : production adjustment costs, c > 0 ,
2
r  0 : discount rate .
There is empirical evidence that total production costs K (v) can show an S-
shaped form. The minimization of non-convex production costs may lead to a
splitting of the production intensity between zero and a level corresponding to
minimal per unit costs. If the adjustment of production intensity is charged
with costs and inventories are allowed, then even a constant demand can lead
to persistent oscillations, i.e. to gradually intensity splitting of production
rates. We choose production costs that are given by the following convex{
concave function depicted in Figure 1 for 1  v  3 :
K (v) = [ (v 3)(v 1)2 + 2:5 ] (v 1) + 3 ; (24)
K 00(v) = 12(v 1)(v 2) < 0 for 1 < v < 2 :

Figure 1 : Convex{concave productions costs

7
All subsequent computations are performed with the nominal values taken
from References 11, 12 :
c = 0:8 ; d = 1:6 ; h = 2:252 ; r = 0:005 resp. r = 0 :
Note that K 00(d) = 0:288 < 0 holds for d = 1:6. For nonzero discount
rate r > 0 , we consider the following production problem with infinite time
horizon: Minimize
Z1
J (u) = ( K (v(t)) + 1 h z(t)2 + 1 c u(t)2 ) exp( rt) dt (25)
0 2 2
subject to
z_ (t) = v(t) d ; v_ (t) = u(t) ; 0  t  tf ; (26)
z(0) = z0 ; v(0) = v0 : (27)
The choice of initial values z0 and v0 will be specified later.
In case of a zero discount rate r = 0 we replace the infinite time horizon
cost function by the average cost over a free period tf > 0 . This leads to
the following optimal periodic control problem (OPC) of minimizing
Z tf
J (u; tf ) = t1 ( K (v(t)) + 21 h z(t)2 + 21 c u(t)2 ) dt (28)
f 0
subject to
z_ (t) = v(t) d ; v_ (t) = u(t) ; 0  t  tf ; (29)
z(0) = z(tf ) ; v(0) = v(tf ) : (30)
After computing the optimal periodic solution of problem (28){(30) with
production costs (24) we realized that this control problem is very similar to
the `sailboat-problem' considered in References 10, 26, 31.
In practice, both the restriction of the production rate and the stock of
inventory makes economic sense. Hence, we shall add to the periodic control
problem (28){(30) the following bounds on production and inventory:
vmin  v(t)  vmax ; zmin  z(t)  zmax : (31)
Numerical techniques for computing the optimal solution under the state
constraints (31) will be discussed in Section 5 .

8
3.1 The canonical system and Hopf bifurcation
The current value Hamiltonian for applying the Minimum Principle to the
discounted problem (25) - (27) becomes; cf. Ref. 32 :
H (z; v; z ; v ) = K (v) + 12 h z2 + 21 c u2 + z (v d) + v u ; (32)
where z and v denote the adjoint variables associated with z and v. This
Hamiltonian agrees with the one for the autonomous periodic control problem
(OPC) in (28){(30); compare (4).
The control u minimizing the Hamiltonian is computed from
Hu = cu + v = 0 ; i.e. u = v =c :
The state equations and the adjoint equations _ z = rz Hz , _ v = rv Hv
yield the following system where we suppress the explicit time argument:
z_ = v d ; v_ = v =c ; (33)
_ z = rz h
z ; _ v = rv K 0(v) z : (34)
The stationary or equilibrium point for (33) and (34) is given by
z = rK 0(d)=h ; v = d ; z = K 0(d) ; v = 0 : (35)
References 11, 12 have carried out a bifurcation analysis of the system (33),
(34) with respect to the parameter d for nonzero discount rate r > 0. We
extend this bifurcation analysis and consider also the case r = 0 . The
Jacobian of the system (33) and (34) evaluated at the equilibrium point (35)
is the matrix
0 1
0 1 0 0
BB 0 0 0 1=c CC
BB CC ;  := K 00(d) : (36)
@ h 0 r 0 A
0  1 r
It is convenient to introduce  = K 00(d) as bifurcation parameter. First we
compute the parameter 0 for which at least two eigenvalues become purely
imaginary. This parameter 0 is characterized by the equation
2 2cr2  4ch = 0 (37)
that has the negative solution
p2
0 = cr2 c r4 + 4ch < 0 : (38)
9
Then the four eigenvalues of the matrix (36) are given by
r s
1;2() = 2  2c + i 02c  r4 ;
r 0 2
(39)
r s
3;4() = 2  2c0 i 02c  r4 :
r 2

Note that for r = 0 all four eigenvalues i(0) are purely imaginary. For
r > 0 the eigenvalues 1;2(0) are purely imaginary while Re(3;4(0)) =
r > 0 is nonzero. Moreover, for r > 0 one easily obtains
d (Re( ()))j = 1 > 0 :
1 =0
d 2cr
Hence, a Hopf bifurcation occurs at the parameter 0 dened by (38), cf.
Ref. 33. A more detailed analysis reveals that the emanating limit cycles are
asymptotically stable. 11;12 Note that the existence of a limit cycle is not in
con ict with Theorem 2 of Ref. 17 which holds only for autonomous systems.
In terms of the original bifurcation parameter d we get 0 = K 00(d) =
= 12(d 1)(d 2) which leads to the bifurcation parameters
 q 
d1;2 = 21 3  1 + 0=3 : (40)
For the nominal values h = 2:252, c = 0:8 and r = 0:005 we find
0 = 2:684453878 ; d1 = 1:337841093 ; d2 = 1:662158907 :
Thus the system (33) and (34) has a limit cycle for all d1 < d < d2 when
r > 0 . In contrast to this behavior, Hopf bifurcation does not occur for
r = 0 . The numerical computations in the next section demonstrate that
the equilibrium (35) behaves like a centrum. Hence, the autonomous case
r = 0 provides a numerical example illustrating Theorem 2 in Ref. 17.

3.2 The -test

For autonomous control problems, the -test 4;15;34 provides conditions for
testing the optimality of the equilibrium. The equilibrium is called locally
proper if a neighboring periodic control provides a better functional value.
It is shown in Ref. 34 that local properness is intimately related to Hopf
bifurcation. However, a similar theory for nonautonomous control problems
has not yet been developed.
Our concern is to apply a -test to the system (33), (34) for all discount-rates
r  0. Consider the periodic control
u(t; ") = " sin(!t) ; " > 0 ;
10
with frequency ! > 0 and period tf = 2=!. The ODE's z_ = v d,
v_ = u(t; ") have the periodic solution
v(t; ") = d !" cos(!t) ; z(t; ") = z !"2 sin(!t) (41)
where z = K 0(d) r=h . The corresponding functional value over a period is
2Z=! 
J (") := K (v(t; ")) + 21 h z(t; ")2 + 21 c u(t; ")2 e rtdt :
0
Hence, local properness follows from the fact that J (") < J (0) holds for " >
0 small and a suitable frequency ! > 0. The last property is a consequence
of the relations J 0(0) = 0 and J 00(0) < 0 which we are going to show next.
Denoting partial derivatives by subscripts, the first and second derivative of
J (") are given by
2Z=!
J 0(") = fK 0(v)v" + hzz" + cuu"g e rtdt ;
0
2Z=!n o
J 00(") = K 00(v)v"2 + hz"2 + cu2" e rt dt :
0
Using the integral values
2Z=!
sin(wt)e rtdt = !2 !+ r2 (1 e r2=! ) ;
0
2Z=!
cos(!t)e rtdt = !2 +r r2 (1 e r2=! ) ;
0
2Z=!
sin(!t)2e rtdt = 4!22!+ r2
Ir ;
2

0 81
< (1 e r2=! ) for r > 0
Ir := : r
2=w for r = 0
we find that
J 0(0) = 0 for all ! > 0 ; r  0 ;
J 00(0) = !2(4!I2r + r2) F (!) ;
where
F (!) := K 00(d) ( 2!2 + r2 ) + 2c !4 + 2h :
11
Hence, the condition J 00(0) < 0 holds if F (!) < 0 is satisfied for a suitable
frequency ! > 0 . This leads us to determine the minimum of the function
F (!) which is attained at the frequency !0 > 0 satisfying
0 00
q
F (!0) = 4!0(K (d) + 2c!0 ) = 0 ; !0 = K 00(d)=2c :
2

This frequency gives the value

00 2
F (!0) = K 2(cd) + K 00(d)r2 + 2h :
Setting F (!0) = 0 , we rediscover equation (37) with  = K 00(d) < 0. A
simple analysis shows in fact that F (!0) < 0 holds for all d1 < d < d2
with d1;2 defined in (40). Note that this analysis is valid also for r = 0.
To summarize our analysis we have proved local properness of the equilibrium
(35) for all constants r, c, h, d satisfying the relations
 q   q 
1 + 0=3 > 0 ; 21 3 1 + 0=3 < d < 12 3 + 1 + 0=3
p24
with 0 = cr2 c r + 4ch . In particular, these inequalities hold for the
nominal values d = 1:6, c = 0:8, h = 2:252 , and r = 0 resp. r = 0:005 .

4 Optimal periodic control in production plan-

ning: the unconstrained case
4.1 Boundary value methods
We intend to solve the complete BVP arising from the minimum principle.
The discussion on the Hopf bifurcation in Section 3 indicates that the beha-
vior of the solution for r > 0 is different from that for r = 0. In all follo-
wing computations we shall employ the nominal constants d = 1:6, c = 0:8,
h = 2:252.
Case r > 0 : We wish to determine the stable limit cycle of system (33) and
(34). The period tf of the emanating limit cycle at the Hopf bifurcation d1
or d2 is determined by Hopf's theorem according to
s
2 = 2 2c  4:85 ;
j1(0)j 0
where the values 0 and 1(0) are given in (38) and (39). This provides a
good initial guess for the unknown period tf corresponding to the nominal
constants.
12
Consider now the new time variable s 2 [0; 1] defined by t = s
tf in (9).
The canonical system (33) and (34) leads to the following BVP of order five
in the interval [0; 1] :
z_ = tf
(v d) ; v_ = tf
v =c ; t_f = 0 ; (42)
_ z = tf
(rz h z) ; _ v = tf
(rv K 0(v) z ) ; (43)
z(0) = z(1) ; v(0) = v0 ; v(1) = v0 ; (44)
z (0) = z (1) ; v (0) = v (1) : (45)
The boundary conditions are the modified boundary conditions introduced
in (17) with k = 2 . The specific initial value v0 in (44) will be chosen to lie
in the range of the solution v(t) . With the choice of the equilibrium v0 =
v = d , the BVP (42){(45) includes the equilibrium (35) as a special solution.
Here one risks that numerical methods tend to reproduce the equilibrium.
For that reason, we propose the initial value v0 = 1:55 . Using the multiple
shooting routine BNDSCO of Ref. 21, the following results are obtained for
the discount rate r = 0:005 :
z(0) = 0:19964412 ; v(0) = 1:55 ; tf = 4:86196395 ;
z (0) = 1:32042160 ; v (0) = 0:28307252 :
The limit cycle for inventory z and production v is not shown here since it
is very similar to the optimal periodic solution obtained for zero discount
r = 0 ; cf. Figure 2. Moreover, there is numerical evidence that the limit
cycle for r > 0 converges to the optimal periodic solution for r = 0 .
The functional (25) with r = 0:005 computed in one period [0; tf ] becomes
J = 4:1440018 < J  = 4:1470052 ;
where J  denotes the functional (25) evaluated at the stationary point (35).
Of course, the functional value (25) in one period depends on the initial point
on the limit cycle. The smallest functional value J = 4:143996 is obtained
approximately for the initial point
z(0) = 0:196238 ; v(0) = 1:53 :
Now we are in a position to describe solutions to the innite-horizon control
problem (25){(27). If the initial point (z0; v0) is chosen on the limit cycle,
then the innitely repeated limit cycle satises all necessary conditions of the
minimum principle and yields a better functional value than the steady state
solution. To substantiate this claim it remains to verify the innite-horizon
transversality condition for the adjoint variable (cf. Ref. 32):
exp( rt)(z (t); v (t)) ! 0 for t ! 1 :
13
Obviously, this condition is satised for the periodic solution z (z); v (t)
on the limit cycle. To our knowledge, second order sufficient optimality
conditions in the innite{horizon case are not yet available in the literature.
Hence, we are not able to prove the optimality of the limit-cycle and have to
content ourselves with verifying necessary conditions.
Case r = 0 : The canonical equations agree with (42) and (43) for r = 0 :
z_ = tf
(v d) ; v_ = tf
v =c ; t_f = 0 ; (46)
_ z = tf
h
z ; _ v = tf
(K 0(v) + z ) : (47)
We noted in Section 3 that the equilibrium z = 0; v = d; v = 0; z =
K 0(d) behaves like a centrum. In fact, for every initial point (z0; v0) near
(z; v) = (0; d) there exists a periodic solution of (46) and (47) satisfying
the following boundary conditions
z(0) = z0; z(1) = z0; v(0) = v0; v(1) = v0; z (0) = z (1) : (48)
The periodic condition v (0) = v (1) is automatically satisfied for every
solution obeying (48). Figure 2 shows four periodic solutions (dotted lines).
In this family of periodic solutions, the optimal periodic control is filtered
out by the transversality condition (7) that becomes here
Z1
(z (s)(v(s) d) v (s)2=c) ds = 0 :
0
Then the BVP (13){(16) with index k = 2 in (17) and (18) consists of
equations (46) and (47) supplemented by the transversality condition:
z(0) = z(1) ; v(0) = v0 ; v(1) = v0 ; z (0) = z (1) ; (49)
y_ = z (v d) 2v =c ; y(0) = y(1) = 0 : (50)
The code BNDSCO in Ref. 21 yields the following solution for v0 = 1:55 :
z(0) = 0:2024097082 ; tf = 4:86194472 ;
z (0) = 1:321750668 ; v (0) = 0:2831306809 :
The functional value (28) becomes
J = 4:19587379 < J  = K (d) = 4:1976 :
To enhance a numerical comparison, we also give the solution for v0 = 1:6 :
z(0) = 0:20593698 ; tf = 4:86194472 ;
z (0) = 1:25662199 ; v (0) = 0:28136785 :
14
The optimal periodic solution is depicted in Figure 2.
Second order sufficient optimality conditions for nonconvex periodic control
problems are given in References 19, 35. The rst part of the sufficient
condition consists in checking that a matrix Riccati
! equation (Ref. 19, eq.
(47)) for the symmetric 22 matrix R = rr1 rr2 has a bounded solution.
2 4
The matrix Riccati equation leads to the following equations
r_1 = h + r22=c ;
r_2 = r1 + r2r3=c ;
r_3 = 2r2 K 00(v) + r32=c ;
for which we obtain bounded solutions in the interval [0; tf ] by choosing e.g.
the initial values r1(0) = 0 ; r2 = 1:35 ; r3(0) = 0 . In a similar way, the
properties of the monodromy matrix associated with the free period (Ref.
19, Proposition 5) can be checked. Hence, the periodic solution in Figure 2
provides a local minimum for problem (28){(30). Also, in a more heuristic
way it is easy to verify that the computed period control is indeed optimal.
Fix v0 = 1:55 and compute the periodic solution of BVP (46){(48) for every
z0  0 . Then the functional J (z0) in (28) attains a maximum at z0 = 0
and a strict minimum at z0 = 0:2024097082 .

v*=

z*= z

Figure 2 : Periodic solution of the system (46){(48) (dotted lines) and

optimal periodic solution with transversality condition (50) (solid line)

15
4.2 The direct optimization method
We outline the direct optimization technique described in Section 2.3 only in
case r = 0. The optimization variable (22) becomes
y = (u0; u1; : : :; uN 1; z0; tf ) ;
while the boundary conditions are
zN = z0 ; v0 = vN = v0 with v0 = d or v0 = 1:55 :
The cost function and the ODE's are computed via a fourth order Runge
Kutta scheme. The control variable is approximated by a continuous and
piecewise linear function with u(i) = ui .
Choosing the mesh size h = 1=50 and the equilibrium z = 0, v = d,
u = 0 as initial guess, the optimal periodic solution is computed by the
direct method (code NUDOCCCS in Ref. 23) with an accuracy of ve deci-
mals. This numerical result is remarkable in view of the fact that the direct
method dispenses with the adjoint variables and, hence, does not need the
sophisticated transversality condition (50).

5 Optimal periodic control under state const-

raints
In this section, state constraints of the form (31)
vmin  v(t)  vmax ; zmin  z(t)  zmax
are imposed for production and inventory. It was argued 11;12 that such state
constraints can not be handled in the framework of Hopf bifurcation. Howe-
ver, we intend to show that arguments based on Hopf bifurcation are no
longer needed once the optimal period control has been computed. Nume-
rical methods for the state constraints (31) can be developed on the basis
of necessary conditions and junction results developed for state constrained
problems; cf. References 36, 37 and the recent survey on state constrained
problems in Ref. 38.
We restrict the discussion to the case r = 0 ; solution methods for r > 0 are
similar.

16
5.1 State constraint vmin  v(t)  vmax
We begin with discussion the minimum principle for the state constraint
v(t)  vmax ; 0  t  tf : (51)
The state constraint (51) is of rst order 36;38 because v_ = u . Upon inspec-
tion of Figure 2 it can be seen that the state constraint (51) becomes active
for the nominal constraint vmax = 1:7 . The Hamiltonian (32) is regular and
admits the unique minimum u = v =c . We infer from a result on first
order state constraints that the optimal periodic solution of (28){(30) and
(51) does not contain a contact point with the boundary; cf. e.g., Ref. 38.
Rather we can expect a boundary arc [ t1; t2]  [0; tf ] with
v(t) = vmax for 0 < t1  t  t2 < tf : (52)
The boundary control is determined by 0 = v_ = u which yields
v (t) = 0 for t1  t  t2 :
Moreover, the control u = v =c and the adjoint variables are continuous
at t1 and t2. This property leads to the junction conditions
v (t1 ) = v (t+2 ) = 0 :
Here, the second condition is automatically satisfied when leaving the boun-
dary at t2. The junction points t1 and t2 are considered as free variables such
that the following conditions hold on the boundary arc:
z_ = vmax d ; v_ = 0 ; t1  t  t2 ; (53)
v(t1) = vmax ; v (t1) = 0 : (54)
After expressing the problem in the new time variable s 2 [0; 1] we have to
solve the multipoint BVP composed by equations (46), (47), (49), (50) in the
interval [0; 1] n [s1; s2] , ti = si
tf , and equations (53), (54) in the interval
[s1; s2] . The shooting method in Ref. 21 yields the following solution with
vmax = 1:7 :
z(0) = 0:08878748 ; v(0) = v0 = 1:6 ;
z (0) = 1:22911044 ; v (0) = 0:12263938 ;
tf = 4:88586795 ; t1 = 0:70418879
tf ;
t2 = 0:77289330
tf ; J = 4:19707624 :
The corresponding periodic solution is shown in Figure 3.

17
 v

vmax=

Figure 3 : Optimal periodic control under state constraint v(t)  1:7

When applying the direct optimization method to this problem, we do not
need a detailed discussion of the minimum principle. Here, the NLP-problem
(20){(22) is augmented by N + 1 inequality constraints for the control ap-
proximation vi = v(i) at the mesh points i :
vi  vmax ; i = 0; : : : ; N :
With N = 50 the period tf and the optimal solutions z, v and u are
reproduced with an accuracy of 5 decimals. The junction points are accurate
up to the mesh spacing h = 1=N .

5.2 State constraint zmin  z (t)  zmax

We begin with the discussion of the lower bound for the inventory z :
zmin  z(t) ; 0  t  tf :
This state constraint is of second order 36;38 because z = u. Typical examples
show that the solution may contain contact points as well as boundary arcs.
For all zmin  0 , the production planning example exhibits only one isolated
contact point t1 2 [0; tf ] with the boundary such that
z(t1) = zmin ; z_(t1) = v(t1) d = 0 :
The minimum principle yields a jump in the adjoint variable z according to
z (t+1) = z (t1 ) + 1 ; 1  0 ;
18
while v is continuous at t1. Choosing v(0) = d = 1:6 it is clear that the
contact point is t1 = 0 with z(0) = zmin. The routine BNDSCO 21 gives the
following solution for the lower bound zmin = 0:1 :
z(0) = zmin = 0:1 ; v(0) = v0 = 1:6 ;
z (0) = 1:18910928 ; v (0) = 0:13881374 ;
1 = 0:06071147 ; tf = 4:73623831 ;
J = 4:19681672 :
The optimal periodic solution is shown in Figure 4.
Also in this case, it is much easier to implement the direct optimization
method. The inequality constraints
zmin  zi ; i = 0; : : : ; N
are added to the NLP-problem (20){(22). The code NUDOCCCS in Ref. 23
yields a solution that agrees with the above solution within five decimals.
The discussion of the minimum principle for state constraints on both pro-
duction and inventory, e.g.
1:55  v(t)  1:65 ; 0:04  z(t)  0:04 ;
becomes rather elaborated. Here again, the direct optimization procedure
readily yields the solution shown in Figure 5 .

z min = z

Figure 4 : Optimal periodic solution with zmin = 0:1  z(t)

19
 v

vmax =1.65

vmin =1.55

z min = z max = z

Figure 5 : Optimal periodic solution with state constraints

1:55  v(t)  1:65 and 0:04  z(t)  0:04

6 Sensitivity analysis
In the preceding sections, optimal periodic solutions have been computed
for a set of fixed nominal data in the system. In practical applications, it
is of great importance to determine the sensitivity of the optimal solution
with respect to perturbations in the nominal data. References 27{30 develop
conditions ensuring that the optimal solution is a dierentiable function of pa-
rameters in the system. Conditions for this type of solution differentiability
require the check of second order sufficient conditions for the nominal solu-
tion. On the basis of such theoretical results, a numerical sensitivity analysis
can be performed. It has been shown 28;30 that the sensitivity differentials
of the optimal state and costate variables satisfy a linear boundary value
problem associated with the nominal (unperturbed) solution.
We illustrate sensitivity analysis by the production planning problem (28){
(30). Let us study the dependence of the optimal periodic solution on the
parameter c in the cost functional (28). Denote solutions of the BVP (46),
(47), (49), (50) by
z(s; c) ; v(s; c) ; z (s; c) ; v (s; c) ; tf (c) ; y(s; c) : (55)
Choose the reference value c0 = 0:8 . The shooting method for solving
the BVP is basically a Newton{type procedure. The code BNDSCO 21 eva-
luates the Jacobian associated with this Newton procedure. It turns out
20
that the Jacobian for the production planning problem (28){(30) is a regular
matrix. Hence, by the implicit function theorem the functions in (55) are
C1{functions with respect to both variables s and c . Then the sensitivity
dierentials

zc(s) = @z @c ( s; c c
0 ) ; v (s) =
@v (s; c ) ; c (s) = @z (s; c ) ;
@c 0 z @c 0
cv (s) = @ @c
v
( s; c c dtf (c ) ; y c (s) = @y (s; c )
0) ; tf =
dc 0 @c 0
satisfy a linear (BVP) that is obtained by differentiating (46), (47), (49) and
(50) formally with respect to c . This leads to the following (BVP) with six
ODE:
z_ c = tf
vc + tcf
(v d) ; (56)
v_ c = [ tf
(cv =c0 v =c20)tcf
v =c0 ] ; (57)
_ cz = [ tf
h
zc + tcf
h
z ] ; (58)
_ cv = [ tf
(K 00(v)vc + cz ) + tcf
(K 0(v) + z ) ] ; (59)
t_cf = 0 ; (60)
y_ c = cz (v d) + z vc 2v cv =c0 + 2v =c20 ; (61)
and boundary conditions
zc(0) = zc (1) ; vc(0) = vc(1) = 0 ; (62)
c c c c
z (0) = z (1) ; y (0) = y (1) = 0 : (63)
This BVP has to be solved along the nominal solution of (46), (47), (49),
(50) with reference values c0 = 0:8 and v0 = 1:55 . The shooting method in
Ref. 21 yields the following initial values:
zc(0) = 0:8467118029 ; vc(0) = 0 ;
cz (0) = 0:3698160271 ; cv (0) = 1:409682167 ;
tcf = 1:421158894 ; yc(0) = 0 :
In view of u = v =c , the sensitivity differential of the optimal periodic
control is given by (see Figure 6) :
uc (s) = @u
@c ( s; c c
0 ) = v (s)=c0 + v (s)=c0 :
2

The computation of the sensitivity differentials can be used to generate a

first order approximation of the perturbed optimal control according to
u(s; c)  u(s; c0) + uc(s)(c c0) : (64)
21
The right{hand side in (64) thus provides an easily implementable approxi-
mation which is considered in the literature as the near{optimal feedback
approximation of the perturbed control. Similar approximations are valid
for the other variables v; z; z ; v .

u c(s)

Figure 6 : Sensitivity differential uc(s) of the optimal control

At first sight, the asymmetry of the sensitivity differential uc(s) may surprise
the reader. This asymmetry is caused by the asymmetry of the optimal
periodic solution in Figure 2 .

7 Conclusion
There are very few examples in the literature containing the complete state
and costate solution of an optimal periodic control problem. This fact has
motivated us to develop two numerical methods for solving periodic control
problems. The first method is based on shooting methods for solving the
boundary value problem associated with the minimum principle. The state
and costate solution is obtained by implementing a convenient form of the
transversality condition associated with the free period. Special care is taken
to discuss the appropriate form of the boundary conditions.
The second method is a direct optimization method that applies nonlinear
programming techniques to a discretized version of the periodic control pro-
blem. This approach dispenses with the costate variables and enjoys favo-
rable convergence properties. The algorithm provides a posteriori estimates
for the costate variables. When state constraints are imposed, the direct
optimization method detects the correct structure of the optimal solution
more easily than the shooting method. We have compared both methods
22
by solving the production planning problem with bounds on production and
inventory.
The  test provides a test for checking if the steady state solution can be
improved by a periodic solution. So far, this test has only been developed for
autonomous problems. We have extended the  test in a straightforward
manner to a non-autonomous production planning problem with nonzero
discount rate. The computations for this specific example indicate that a
 test could probably be designed for general discounted control problems.
When looking at the numerical results in Section 4, the relative improvement
achieved by the periodic optimal against the steady state control is indeed
very small. A more visible improvement can be obtained by choosing a smal-
ler constant c in the cost funtion (28). However, the numerical techniques
do not change with smaller constants. For this reason, we have decided to
use the constants from References 11, 12 for the numerical analysis.
The contribution of this paper may help to revive the interesting area of
periodic control problems by passing from qualitative discussions of models
to concrete quantitative numerical solutions. The advance in numerical com-
puting techniques developed in recent years allows for treating more realistic
problems by including constraints both on control and state.
Acknowledgement: The authors would like to thank Andrea Rosenbaum
and Nicola Victor for their computational assistance.

References
[1] Bailey, J. E., 'Periodic operations of chemical reactors: a review', Che-
mical Engineering Communications, 1, 111-124 (1973).
[2] Matsubara, M., Y. Nishimura, N. Watanabe and K. Onogi, 'Periodic
control theory and applications', Research Reports of Automatic Control
Laboratory, Vol. 28, Faculty of Engineering, Nagoya University, Japan,
June 1981.
[3] Bittanti, S., G. Fronza and G. Guardabassi, 'Periodic control: a fre-
quency domain approach', IEEE Transactions on Automatic Control,
AC-18, No. 1, 33-38 (1973).
[4] Guarbadassi, G., 'Optimal steady-state versus periodic optimization: a
circle criterion', Ricerce di Automatica, 2, No. 3 (1971).

23
 [5] Guarbadassi, G., A. Locatelli and S. Rinaldi, `Status of periodic opti-
mization of dynamical systems', Journal of Optimization Theory and
Applications, 12, 1-20 (1974).
[6] Marzollo, R., Periodic Optimization, Vols. 1, 2, Springer Verlag, New
York, 1972.
[7] Chuang, C.-H. and J. L. Speyer, `Periodic optimal hypersonic scramjet
cruise', Optimal Control Applications and Methods, 8, 231-242 (1987).
[8] Sachs, G. and T. Christotoulou, `Reducing fuel consumption of subsonic
aircraft by optimal cyclic cruise', AIAA Journal of Aircraft, 24, 616-622
(1987).
[9] Sachs, G., K. Lesch, H. G. Bock and M. Steinbach, `Periodic optimal
trajectories with singular control for aircraft with high aerodynamics
efficiency', International Series of Numerical Mathematics, Vol. III,
Birkhauser Verlag, Basel, 1993.
[10] Speyer, J. L. and R. T. Evans, `A shooting method for the numerical
solutions of optimal periodic control problems', Proceedings of the 20th
IEEE Conference on Decision and Control, Dec. 1981.
[11] Feichtinger, G., K.-P. Kistner and A. Luhmer, `Ein dynamisches Mo-
dell des Intensitatssplitting', Zeitschrift fur betriebswirtschaftliche For-
schung, 11, 1242-1258 (1988).
[12] Feichtinger, G., `Limit cycles in dynamic economic systems', Annals of
Operations Research, 37, 313-344 (1992).
[13] Feichtinger, G. and A. Novak, 'Optimal pulsing in an advertising dif-
fusion model', Optimal Control Applications & Methods, 15, 267-276
(1994).
[14] Feichtinger, G., A. Novak and F. Wirl, 'Limit cycles in intertemporal
adjustment models', J. of Economic Dynamics and Control, 18, 353-380
(1994).
[15] Bernstein, D. S. and E. G. Gilbert, `Optimal periodic control: The 
test revisited', IEEE Transactions on Automatic Control, AC{25, No.
4, 673-684 (1980).
[16] Bittanti, S, G. Fronza, G. Guardabassi, and C. Maezzoni, 'A maximum
priciple for periodic optimization', Ricerce di Automatica, 3, No.2, 170-
179 (1972).

24
[17] Bittanti, S, C. Maezzoni, 'Structural properties of a Hamiltonian sys-
tem in a periodic optimization problem', Automatica i Telemekhanika,
6, 5-13 (1975). Translated in Automation and Remote Control, 36, 877-
884 (1975).
[18] Gilbert, E. G., `Optimal periodic control: A general theory of necessary
conditions', SIAM J. Control and Optimization, 15, 717-746 (1977).
[19] Wang, Q., and J. L. Speyer, `Necessary and sufficient conditions for
local optimality of a periodic process', SIAM Journal on Control and
Optimization, 28, 482-497 (1990).
[20] Bulirsch, R. `Die Mehrzielmethode zur numerischen Losung von nicht-
linearen Aufgaben der optimalen Steuerung', Report of the Carl{Cranz
Gesellschaft, Oberpfaenhofen, Germany, 1971.
[21] Oberle, H. J. and W. Grimm, `BNDSCO - A program for the numerical
solution of optimal control problems', Internal Report No. 515-89/22,
Institute for Flight Systems Dynamics, DLR, Oberpfaenhofen, Ger-
many, 1989.
[22] Betts, J. T. and W. P. Humann, `Path constrained trajectory optimi-
zation using sparse sequential quadratic programming', Applied Mathe-
matics and Statistics Group, Boeing Computer Services, Seattle, USA,
1991.
[23] Buskens, Ch., `Direkte Optimierungsmethoden zur numerischen Berech-
nung optimaler Steuerungen', Diploma thesis, Institut fur Numerische
Mathematik, Universitat Munster, Munster, Germany, 1993.
[24] von Stryk, O. and R. Bulirsch, `Numerical solution of optimal cont-
rol problems by direct collocation', Schwerpunktprogramm der Deut-
schen Forschungsgemeinschaft \Anwendungsbezogene Optimierung und
Steuerung", Technische Universitat Munchen, Report No. 322, 1991.
[25] Teo, K. L., C. J. Goh and K. H. Wong, A Unied Computational Ap-
proach to Optimal Control Problems, Longman Scientic and Technical,
New York, 1991.
[26] Chuang, C.-H., J. L. Speyer and J. V. Breakwell, `An asymptotic ex-
pansion for an optimal relaxation oscillator', SIAM J. on Control and
Optimization, 26, 678-696 (1988).
[27] Malanowski, K., `Regularity of solutions in stability analysis of opti-
mization and optimal control problems`, Control and Cybernetics, 23,
61-86 (1994).
25
[28] Malanowski, K. and H. Maurer, `Sensitivity analysis for parametric cont-
rol problems with control{state constraints', Computational Optimiza-
tion and Applications, 5, 253-283 (1996).
[29] Maurer, H. and H. J. Pesch, `Solution dierentiability for parametric
nonlinear control problems', SIAM Journal on Control and Optimiza-
tion, 32, 1542-1554, 1994.
[30] Maurer, H. and H. J. Pesch, `Solution dierentiability for parametric
nonlinear control problems with control-state constraints', Control and
Cybernetics, 23, 201-227 (1994).
[31] Evans, R. T., J. L. Speyer and C.-H. Chuang, `Solution of a periodic
optimal control problem by asymptotic series', Journal of Optimization
Theory and Applications, 52, 343-364 (1987).
[32] Feichtinger, G. and R. F. Hartl, Optimale Kontrolle okonomischer Pro-
zesse, deGruyter, Berlin, 1986.
[33] Guckenheimer, J. and P. Holmes, Nonlinear Oscillations, Dynamical
Systems and Bifurcations of Vector Fields, Springer, New York, 1983.
[34] Colonius, F., Optimal Periodic Control, Lecture Notes in Mathematics,
Vol. 1313, Springer Verlag, Berlin, 1988.
[35] Bittanti, S., A. Locatelli and C. Maezzoni, 'Second-variation methods
in periodic optimization', Journal of Optimization Theory and Applica-
tions, 14, 31-49 (1974).
[36] Jacobson, D. H., M. M. Lele and J. L. Speyer, `New necessary condi-
tions with state variable inequality constraints', J. Math. Analysis and
Applications, 35, 255-284 (1971).
[37] Maurer, H. and W. Gillessen, 'Application of multiple shooting to the
numerical solution of optimal control problems with bounded state va-
riables', Computing, 15, 105-126 (1975).
[38] Hartl, R. F., S. P. Sethi and R. G. Vickson, 'A survey of the maximum
principles for optimal control problems with state constraints', SIAM
Review, 37, 181-218 (1995).

26