Integration of real-time optimization and model predictive control

V. Adetola, M. Guay
*
Department of Chemical Engineering, Queens University, Kingston, Ontario, Canada K7L 3N6
a r t i c l e i n f o
Article history:
Received 6 April 2009
Received in revised form 3 September 2009
Accepted 5 September 2009
Keywords:
Adaptive control
Real-time optimization
Model predictive control
a b s t r a c t
This paper proposes a controller design approach that integrates RTO and MPC for the control of
constrained uncertain nonlinear systems. Assuming that the economic function is a known function of
constrained systems states, parameterized by unknown parameters and time-varying, the controller
design objective is to simultaneously identify and regulate the system to the optimal operating point.
The approach relies on a novel set-based parameter estimation routine and a robust model predictive
controller that takes into the effect of parameter estimation errors. A simulation example is used to dem-
onstrate the effectiveness of the design technique.
2009 Elsevier Ltd. All rights reserved.
1. Introduction
One of the key challenges in the process industry is how to best
operate the plant under different conditions such as feed composi-
tions, production rates, energy availability, feed and product prices
that changes all the time. Such a task is usually tackled using a
supervisory control technique. One such technique that has re-
ceived considerable attention in the process industry is real-time
optimization (RTO) [6,21,15]. Real-time optimization (RTO), which
refers to the online economic optimization of a process plant, is a
widely employed technology to meet this challenge. RTO attempts
to optimize process performance (usually measured in terms of
prot or operating cost) thereby enabling companies to push the
protability of their processes to their true potential as operating
conditions change.
The popular RTO is based on the assumption that model and
disturbance transients can be neglected if the optimization execu-
tion time interval is long enough to allow the process to reach and
maintain steady-state. A typical RTO system includes components
for steady-state detection, data acquisition and validation, process
model updating, optimization calculations and optimal operating
policies transfer to advanced controllers. In the conventional lay-
ered optimization-control strategy, an upper (or supervisory)
real-time optimization (RTO) layer computes optimal operating
values and a lower layer (or control) implements them. The RTO
is based on a steady-state plant model and it is only executed when
the plant is stabilized. The control problem is solved apart from the
optimization problem at different frequencies and using different
models. Since the two layers may not be dealing exactly with the
same information, there may be some conict and the predicted
optimal operational point is often sub-optimal.
The integration of RTO and control for an optimal plant opera-
tion is still an open research problem. There have been attempts
in the literature to address the integration task in a systematic
way. The only results are focused on specic applications. No the-
oretical foundation such as stability and performance analysis of
the integrated scheme exists. The proposed techniques can be clas-
sied into two main classes, based on the frequency of RTO execu-
tion and the type of model employed.
(1) Single level strategy, integration of steady-state RTO and
Linear MPC: In this approach, the economic optimization
and control problems are solved simultaneously in a single
algorithm. model-based predictive control is formulated to
perform both functions by adding an economic objective
term to standard MPC objective function [8,25]. The result-
ing problem involves solving a nonlinear objective function
subject to dynamic and steady-state constraints. Implemen-
tation of the extended controller relies on extensive tuning
of the weighting factors for stability and performance.
Extensive simulations are needed to select appropriate val-
ues for the weighting factors [25] unlike in linear MPC where
robust tuning techniques are available. The one-layer
approach could respond to changes in the plant optimal con-
ditions faster than the two-layer approach. However, the
method is only applicable to plants that have relatively short
settling time and can be effectively described with a linear
model.
(2) Two level strategy, integration of dynamic RTO and non-
linear MPC: This method attempts to account for dynamic
nonlinear behaviour of production plants. It involves an
0959-1524/$ - see front matter 2009 Elsevier Ltd. All rights reserved.
doi:10.1016/j.jprocont.2009.09.001
* Corresponding author. Tel.: +1 613 533 2788; fax: +1 613 533 6637.
E-mail address: martin.guay@chee.queensu.ca (M. Guay).
Journal of Process Control 20 (2010) 125133
Contents lists available at ScienceDirect
Journal of Process Control
j our nal homepage: www. el sevi er. com/ l ocat e/ j procont
upper level dynamic economic optimization and a lower
level nonlinear MPC [20,19,26]. The RTO problem need not
be solved at each sample time, but based on disturbance
dynamics or plant condition. A technique based on distur-
bance sensitivity analysis is embedded into the two level
strategy to trigger the dynamic RTO computation when per-
sistent disturbances have been detected that have high sen-
sitivities [20,19]. This approach considers dynamical models
in the RTO and recomputes feasible setpoints only if eco-
nomic benets are possible. Since the performance of the
approach depends on the interaction between the two lay-
ers, the scheme may still suffer some disadvantages due to
the different dynamical models used in the two layers
approach. To alleviate the problems associated with the
use of different models in the two layers, an alternative
approach was presented in [17,18] where empirical grey-
box models are utilized as part of the integration
mechanism.
In this paper, a formal design technique is proposed that inte-
grates RTO and MPC for constrained uncertain nonlinear systems.
The framework considered assumes the economic function is a
known function of constrained systems states, parameterized by
unknown parameters. The objective and constraint functions may
explicitly depend on time, which means that our proposed method
is applicable to both dynamic and steady state economic optimiza-
tion. The control objective is to simultaneously identify and regu-
late the system to the operating point that optimizes the
economic function. The control input may also be required to sat-
isfy some constraints. The main objective of this work is to provide
the theoretical framework to study RTO/MPC integration such that
the overall objectives (e.g., stability, transient performance and
optimality) of RTO/MPC are achieved.
The method proposed solves the control and optimization prob-
lem at the same frequency. This eliminates the ensuing interval of
no-feedback that occurs between economic optimization and
thereby improving disturbance attenuation. The RTO layer is tack-
led via a computational efcient approach. The constrained eco-
nomic optimization problem is converted to an unconstrained
problem and Newton based optimization method is used to devel-
op an update lawfor the optimumvalue. The integrated design dis-
tinguishes between the extremum seeking and the adaptive
tracking of the reference trajectory.
While many advances have been made in nonlinear systems for
the stabilization of one xed operating point, few attempts have
been made to address the stabilization problem for time-varying
or non-xed setpoints. In [22], a stabilizing nonlinear MPC algo-
rithm was developed for asymptotically constant reference signals.
By selecting a prediction horizon that is longer than the time the
reference setpoint is assumed to have converged, the constant
pre-programmed value is used to design the stabilizing controller
parameters, i.e., the terminal stability constraint X
f
and terminal
penalty W. The result is limited to reference signals that converge
to a-priori known constant setpoint. The method proposed in [14],
combines a pseudo-linearization technique with a nonlinear MPC
strategy to stabilize a family of (known and constant) setpoints.
The linearization technique was used to obtain a constant lineari-
zation family that is independent of the setpoint value. Fixed con-
troller parameters were then developed to achieve stability for the
whole setpoint family. While the method provides a possible solu-
tion for tracking changing setpoints, such pseudo-linearization
transformation and feedback is in general difcult to obtain and in-
volve cumbersome computation.
A major challenge of extremum-seeking control is that the opti-
mum reference setpoint is not known in advance. The cost function
and the dynamical system depend on unknown parameters and
hence the optimal operating point can only be determined in
real-time. The future behaviour of the setpoint trajectory, which
is crucial for calculating model predictive control law is unavail-
able. Therefore, a single terminal penalty function and constraint
set is insufcient to guarantee stability of the reference trajectory.
The stability result, based on the adaptive MPC approach in [1], an-
chors on the design of time-varying stability parameters that de-
pend on the current state measurement, setpoint trajectory and
parameter estimation error.
The paper is organized as follows. The formal problem descrip-
tion is given in Section 2. Section 3 discusses the problem of
designing an extremum-seeking technique for real-time set-point
generation. The one-layer RTO-MPC integration is presented in
Section 4, followed by the two-layer approach in Section 5. A sim-
ulation example is considered in Section 6. The paper concludes in
Section 7 where brief conclusions and recommendations for future
research are given.
2. Problem description
Consider a constrained optimization problem of the form
min
x2R
nx
pt; x; h; 1a
s:t: c
j
x 6 0 j 1. . . m
c
1b
with h representing unknown parameters, assumed to be uniquely
identiable and lie within an initially known convex set
H
0
,Bh
0
; z
0
h
. The functions p and c
j
are assumed to be C
2
in all of
their arguments (with locally Lipschitz second derivatives), uni-
formly for t 2 [0, 1). The constraint c
j
6 0 must be satised along
the systems state trajectory x(t).
Assumption 1. The following assumptions are made about (1).
(1) There exists e
0
> 0 such that
@
2
p
@x
2
Pe
0
I and
@
2
c
@x
2
P0 for all
t; x; h 2 R

R
nx
H

, where H

is an neighborhood
of H.
(2) The feasible set
X x 2 R
nx
j max
j
c
j
x 6 0
_ _
has a non-empty interior.
Assumption 1 states that the cost surface is strictly convex in x
and X is a non-empty convex set. Standard nonlinear optimization
results guarantee the existence of a unique minimizer x

t; x; h 2 X
to problem (1). In the case of non-convex cost surface, only local
attraction to an extremum could be guaranteed. The control objec-
tive is to stabilize the nonlinear system
_ x f x; n; u gx; n; uh,Fx; n; u; h; 2a
_
n f
n
x; n 2b
to the optimum operating point or trajectory given by the solution
of (1) while obeying the input constraint u 2 U 2 R
nu
in addition to
the state constraint x 2 X 2 R
nx
. The dynamics of the state n is as-
sumed to satisfy the following input to state stability condition with
respect to x.
Assumption 2. If x is bounded by a compact set B
x
#X, then there
exists a compact set B
n
#R
nn
such that n 2 B
n
is positively invariant
under (2).
Remark 3. Assumption 2 can be checked using various Lyapunov
based techniques, such as constructive nonlinear design tech-
niques as outlined in [24]. In practice, this assumption requires
that the n dynamics are rendered robust to changes in x. For
126 V. Adetola, M. Guay / Journal of Process Control 20 (2010) 125133
example, this would hold if the n dynamics represent controlled
dynamics in closed-loop. Of course, some care must be taken in
the design of controllers in interconnected systems. The mono-
graph [24] provides many useful stability results for intercon-
nected nonlinear systems.
The design proposed in this paper contains three components.
The rst component attempts to solve the optimization problem
(1) using an extremum-seeking setpoint design. The objective of
the extremum-seeking algorithm is to update the desired setpoint
to be tracked by the control system. Since the cost function of the
RTO problem (1) depends on the unknown parameters, a parame-
ter identication algorithm must be considered as a second com-
ponent in the design. In this paper, a nite-time parameter
identier [3] is considered. This identication technique provides
a relaxed persistency of excitation condition that can be removed
in nite-time when the parameter estimation routine is known
to have converged. The particular structure of this routine allows
one to develop a set-based estimation routine that provides an
estimate of the uncertainty of the unknown parameter values. This
set-based estimation forms the basis of adaptive control algorithm
proposed. The third component of the design is an adaptive model
predictive control algorithm that guarantees robust feasibility of
the closed-loop system and stabilizes the unknown optimum set-
point of the optimization problem (1) subject to a persistency of
excitation condition.
3. Extremum-seeking setpoint design
In this section, the extremum-seeking setpoint generation de-
sign is presented. This design incorporates an interior point des-
cent algorithm along with the nite-time parameter identier.
3.1. Finite-time parameter identication
Let ^x denote the state predictor for (2), the dynamics of the state
predictor is designed as
_
^x f x; n; u gx; n; u
^
h k
w
te w
_
^
h; 3
where
^
h is a parameter estimate generated via any update law
_
^
h; k
w
> 0 is a design matrix, e x ^x is the prediction error and w
is the output of the lter
_ w gx; n; u k
w
w; wt
0
0: 4
Denoting the parameter estimation error as
~
h h
^
h, it follows
from (2) and (3) that
_ e gx; n; u
~
h k
w
e w
_
^
h: 5
The use of the lter matrix w in the above development provides di-
rect information about parameter estimation error
~
h without
requiring a knowledge of the velocity vector _ x. This is achieved by
dening the auxiliary variable
g e w
~
h 6
with g, in view of 4, 5, generated from
_ g k
w
g; gt
0
et
0
: 7
The auxiliary variable is thus guaranteed to vanish exponentially to
the origin. This property is exploited in the design of a nite-time
parameter identier. This identication algorithm provides the
mechanism through which the unknown optimum of the parameter
dependent problem (1) can be reached. It is considered as an inte-
gral part of the extremum-seeking setpoint generation algorithm.
Based on the dynamics 3, 4 and (7), the following result provides
an effective parameter identier (12) that can recover the unknown
parameter estimations in nite-time.
Theorem 4 [3]. Let Q 2 R
nhnh
and C 2 R
nh
be generated from the
following dynamics:
_
Q w
T
w; Qt
0
0; 8a
_
C w
T
w
^
h e g; Ct
0
0: 8b
Suppose there exists a time t
c
and a constant c
1
> 0 such that Q(t
c
) is
invertible i.e.
Qt
c

_
tc
t
0
w
T
swsds 1 c
1
I; 9
then
h Qt
1
Ct forall t Pt
c
: 10
The result in Theorem 4 is independent of the control u and param-
eter identier
_
^
h structure used for the state prediction (Eq. (3)).
Moreover, the result holds if a nominal estimate h
0
of the unknown
parameter (no parameter adaptation) is employed in the estimation
routine. In this case,
^
h is replaced with h
0
and the last part of the
state predictor (3) is dropped (
_
^
h 0). Let
h
c
,Qt
c

1
Ct
c
: 11
The nite-time (FT) identier is given by
^
h
c
t
^
ht; if t < t
c
;
h
c
; if t Pt
c
:
_
12
Note that nominally, there is no requirement on the exact nature of
the parameter update law for
^
ht for t < t
c
. The main feature of the
identier (12) is that any transient parameter update law
_
^
h can be
turned off at t = t
c
.
3.2. Constraint removal
An interior point barrier function method is used to enforce the
inequality constraint. The state constraint is incorporated by aug-
menting the cost function p as follows:
p
a
t; x; h,pt; x; h
1
g
c

mc
j1
lnc
j
x 13
with g
c
> 0, a xed constant. The augmented cost function (13) is
strictly convex in x and the unconstrained minimization of p
a
there-
fore has a unique minimizer in intfXg which converges to that of
(1) in the limit as g
c
?1 [7]. Thus the optimizer of (13) provides
an approximation of the optimizer of the original problem (1) for
xed values of g
c
.
3.3. Setpoint update law
Let x
r
2 R
nx
denote a reference setpoint to be tracked by x and
^
h
denote an estimate of the unknown parameter h. A setpoint update
law _ x
r
can be designed based on Newtons method, such that x
r
(t)
converges exponentially to the (unknown)
^
h dependent optimum
value of (13). To this end, consider an optimization Lyapunov func-
tion candidate
V
r

1
2
@p
a
@x
t; x
r
;
^
h
_
_
_
_
_
_
_
_
2
,
1
2
kz
r
k
2
: 14
For the remainder of this section, omitted arguments of p
a
and its
derivatives are evaluated at t; x
r
;
^
h. Differentiating (14) yields
V. Adetola, M. Guay / Journal of Process Control 20 (2010) 125133 127
_
V
r

@p
a
@x
@
2
p
a
@x@t

@
2
p
a
@x
2
_ x
r

@
2
p
a
@x@h
_
^
h
_ _
: 15
Using the update law
_ x
r

@
2
p
a
@x
2
_ _
1
@
2
p
a
@x@t

@
2
p
a
@x@h
_
^
h k
r
@p
T
a
@x
_ _
,f
r
t; x
r
;
^
h 16
with k
r
> 0 and r0 r
0
2 intfXg results in
_
V
r
6 k
r
kz
r
k
2
; 17
which implies that the gradient function z
r
converges exponentially
to the origin.
Lemma 5 Suppose h;
^
h is bounded, the optimal setpoint x
r
(t)
generated by (16) is feasible and converges to x

pa

^
h, the minimizer
of (13) exponentially.
Proof. Feasibility follows from the boundedness of h;
^
h and
Assumption 1 while convergence follows from (17) and the fact
that z
r
is a diffeomorphism. h
4. One-layer integration approach
Since the true optimal setpoint depends on h, the actual desired
trajectory x

r
t; h is not available in advance. However, x
r
t;
^
h can
be generated from the setpoint update law (16) and the corre-
sponding reference input u
r
(x
r
) can be computed online.
Assumption 6. Given an x
r
t;
^
h is such that
0 f
r
t; x
r
;
^
h 18
and there exists u
r
t;
^
h that can generates x
r
t;
^
h.
Formally, this assumption requires invertibility and hence,
strong relative degree, of the nonlinear system (2). The require-
ment that f
r
t; x
r
;
^
h 0 restricts the points where the gradient of
the augmented cost with respect to x
r
vanishes.
The design objective is to design a model predictive control law
such that the true plant state x tracks the reference trajectory
x
r
t;
^
h. Given the desired time-varying trajectory (x
r
, u
r
), an attrac-
tive approach is to transform the tracking problem for a time-
invariant system into a regulation problem for an associated
time-varying control system in terms of the state error x
e
= x x
r
and stabilize the x
e
= 0 state. The formulation requires the MPC
controller to drive the tracking error x
e
into the terminal set
X
e
f

~
h at the end of the horizon. Since the dynamics of the system
are uncertain, we use the nite-time identier (12) for online
parameter adaptation and incorporate robust features in to the
adaptive controller formulation to account for the impact of the
parameter estimation error
~
h in the design. For ease of presenta-
tion, only the full state information case is considered. The design
of an output feedback approach can be achieved using output feed-
back MPC techniques [16,13,2]. Note that the work presented in [5]
provides an initial study of the class of problems treated in this pa-
per for linear control systems.
4.1. Minmax adaptive MPC
Feedback minmax robust MPC is employed to provide robust-
ness for the MPCcontroller during the adaptationphase. The control-
ler maximizes a cost function with respect to h and minimizes it over
feedback control policies j. The integrated controller is given as
u j
mpc
t; x
e
;
^
h,j

0; x
e
;
^
h; 19a
j

,arg min
j;;;
Jt; x
e
;
^
h; j; 19b
where Jt; x
e
;
^
h; j is the (worst-case) cost associated with the opti-
mal control problem:
Jt; x
e
;
^
h; j,max
h2H
_
T
0
Ls; x
p
e
; u
p
; u
r
ds 20a
Ws; x
p
e
T;
~
h
p
T; 20b
s:t: 8s 2 0; T;
_ x
p
f x
p
; n
p
; u
p
gx
p
; n
p
; u
p
h; x
p
0 x; 20c
_
n
p
f x
p
; n
p
; n
p
0 n; 20d
_ x
p
r
f
r
t; x
r
; h; x
p
r
0 x
r
; 20e
x
p
e
x
p
x
p
r
; 20f
_
w
p
bg
T
x
p
; n
p
; u
p
k
w
w
p
; w
p
0 w; 20g
_
Q
p
bw
p
^
T
w
p
; Q
p
0 Q; 20h
_
^
h
p
CQ
p
~
h
p
;
~
h
p
h
^
h
p
;
^
h
p
0
^
h; 20i
u
p
s,js; x
p
e
s;
^
h
p
s 2 U; 20j
x
p
e
s 2 X
e
; x
p
e
T 2 X
e
f

~
h
p
T; 20k
where X
e
x
p
e
: x
p
2 X
_ _
; X
e
f
is the terminal constraint and
b 2 {0, 1}. The effect of the future parameter adaptation is incorpo-
rated in the controller design via (20a) and (20k), which results in
less conservative worst-case predictions and terminal conditions.
4.2. Uncertainty set update
One of the most important parameter of the MPC controller for-
mulation is the parameter uncertainty set H. Given the nite-time
parameter identier, it is possible to evaluate the uncertainty set
and update its value given the properties of the matrix Q(t) com-
puted for the implementation of the parameter identier. The up-
date algorithm (originally proposed in [9]) is given as follows.
Algorithm 1. Let Er k
min
CQr, beginning from time
t
i1
= t
0
, the parameter and set adaptation is implemented itera-
tively as follows:
(1) Initialize z
h
t
0
z
0
h
; Ht
0
B
^
ht
0
; z
h
t
0
;

E Et
0
0
(2) Implement the following adaptation law over the interval
s 2 [t
i1
, t
i
)
_
z
h
s

Ez
h
s 21
(3) At time t
i
, perform the updates

E
Et
i
; if Et
i
PEt
i1

Et
i1
; otherwise
_
22
^
h; H
_ _

^
ht
i
; Ht
i

_ _
; if z
h
t
i
z
h
t
i1

6 k
^
ht
i

^
ht
i1
k
^
ht
i1
; Ht
i1

_ _
; otherwise
_

_
23
(4) Iterate back to step 2, incrementing i = i + 1.
The main property of the update algorithm is that given H(t
i1
)
the updated set H(t
i
) is guaranteed to contain the unknown
parameter value h if that value is contained in H(t
i1
). Further-
more, the size of the uncertainty set is updated following (21)
using the smallest value of the excitation index Et.
With the knowledge of the uncertainty set, the integrated RTO
and MPC design can be implemented as follows.
Algorithm 2. The nite-time minmax MPC algorithm performs
as follows: At sampling instant t
i
128 V. Adetola, M. Guay / Journal of Process Control 20 (2010) 125133
(1) Measure the current states of the plant x = x(t
i
), n = n(t
i
) and
obtain the current value of the desired setpoint x
r
= x
r
(t
i
) via
the update law (16)
(2) Obtain the current value of matrices w, Q and C from
_ w gx; u k
w
w; wt
0
0; 24
and
_
Q w
T
w; Qt
0
0; 25a
_
C w
T
wh
0
x ^x g; Ct
0
0; 25b
respectively,
(3) If det(Q) = 0 or cond(Q) is not satisfactory update the param-
eter estimates
^
h and the uncertainty set Ht,B
^
ht; z
h
t
according to Algorithm (1) in the Appendix.
Else if det(Q) > 0 and cond(Q) is satisfactory, set b = 0 and
update
^
h Q
1
t
i
Ct
i
; z
h
0
End
(4) Solve the optimization problem 19a,20k and apply the
resulting feedback control law to the plant until the next
sampling instant
(5) Increment i = i + 1. If z
h
> 0, repeat the procedure from step
1 for the next sampling instant. Otherwise, repeat only steps
1 and 4 for the next sampling instant.
Since the algorithm is such that the uncertainty set H contracts
over time, the conservatism introduced by the robustness feature
in terms of constraint satisfaction and controller performance re-
duces over time and when H contracts upon h, the minmax
adaptive framework becomes that of a nominal MPC. The draw-
back of the nite-time identier is attenuated in this application
since the matrix invertibility condition is checked only at sam-
pling instants. The benet of the identier, however, is that it al-
lows an earlier and immediate elimination of the robustness
feature.
4.3. Lipschitz-based adaptive MPC
While the minmax approach provides the tightest uncertainty
cone around the actual systems trajectory, its application is lim-
ited by the enormous computation required to obtain the solution
of the minmax MPC algorithm. To address this concern, the robust
tracking problem is re-posed as the minimization of a nominal
objective function subject to robust constraints.
A simpler so-called Lipschitz-based approach is proposed in
the following. This technique does not rely on a worst-case for-
mulation and relies on robust bounds that can approximate the
uncertain behaviour of the system dynamics and hence pre-
serve the robust stability and feasibility of the more complex
minmax approach. The main advantage of the Lipschitz-based
approach is that it can be implemented using standard nonlin-
ear model predictive control techniques. The formulation does
not require the solution of a minmax problem and relies only
on the nominal model prediction subject to Lipschitz bounds
on the cone of predictions subject to parameter estimation
error.
The model predictive feedback is dened as
u j
mpc
t; x
e
;
^
h; z
h
u

0; 26a
u

:,arg min
u
p
0;T
Jt; x
e
;
^
h; z
h
; u
p
; u
r
; 26b
where Jt; x
e
;
^
h; z
h
; u
p
; u
r
is given by the optimal control problem:
Jt; x
e
;
^
h; z
h
; u
p
; u
r

_
T
0
Lt; x
p
e
; u
p
; u
r
ds 27a
Wx
p
e
T; z
p
h
T; 27b
s:t: 8s 2 0; T;
_ x
p
f x
p
; u
p
gx
p
; u
p

^
h; x
p
0 x; 27c
_
n
p
f n
p
; x
p
; n
p
0 n; 27d
_ x
p
r
f
r
t; x
r
;
^
h; x
p
r
0 x
r
; 27e
x
p
e
x
p
x
r
; 27f
_ z
p
e
bL
f
L
g
Pz
p
e
kgx
p
; n
p
; u
p
kz
h
; 27g
z
p
x
0 0; 27h
X
p
e
s,Bx
p
e
s; z
p
e
s #X
e
; u
p
s 2 U; 27i
X
p
e
T #X
e
f
z
p
h
T: 27j
Since the Lipschitz-based robust controller is implemented in open-
loop, there is no setpoint trajectory x
r

^
h feedback during the inter-
sample implementation. Therefore, the worst-case deviation
z
p
e
Pmax
h2H
kx
e
x
p
e
k max
h2H
kx x
p
k. Hence z
p
e
given in (27g) fol-
lows from
_
z
p
x
L
f
L
g
Pz
p
x
kgx
p
; ukz
h
; z
p
x
t
0
0; 28
where P z
h
k
^
hk. We assume an appropriate knowledge of Lips-
chitz bounds as follows:
Assumption 7. A set of functions L
j
: XR
nn
U !R

; j 2 ff ; gg
are known which satisfy
L
j
X; n; u Pmin L
j
j sup
x
1
;x
2
2X
kjx
1
; n; u jx
2
; n; uk
_
L
j
kx
1
x
2
k
_
6 0
_
; 29
4.4. Implementation algorithm
Algorithm 3. The nite-time Lipschitz-based MPC algorithm per-
forms as follows: at sampling instant t
i
(1) Measure the current states of the plant x = x(t
i
),n = n(t
i
) and
obtain the current value of the desired setpoint x
r
= x
r
(t
i
)
via the update law (16)
(2) Obtain the current value of matrices w, Q and C from (24)
and (25)
(3) If det(Q) = 0 or cond(Q) is not satisfactory, set b = 1 and
update the parameter estimates
^
h
^
ht
i
and uncertainty
bounds z
h
= z
h
(t
i
) and z
p
h
T z
p
h
t
i
T via Eq. (30)
_
^
h CC Q
^
h;
^
ht
0
h
0
; 30
Eqs. (21) and (31)
z
p
h
s exp

Est
i

z
h
t
i
s 2 t
i
; t
i
T; 31
where

E PEt
i
k
min
CQt
i
:
Else if det(Q) > 0 and cond(Q) is satisfactory, set b = 0 and
update
^
h Q
1
t
i
Ct
i
; z
h
0
End
(4) Solve the optimization problem (26,27) and apply the
resulting feedback control law to the plant until the next
sampling instant
(5) Increment i = i + 1. If z
h
> 0, repeat the procedure from step
1 for the next sampling instant. Otherwise, repeat only steps
1 and 4 for the next sampling instant.
V. Adetola, M. Guay / Journal of Process Control 20 (2010) 125133 129
Implementing the adaptive MPC control law according to Algo-
rithm 3 ensures that the uncertainty bound z
h
reduces over time
and hence, the error margin z
p
x
imposed on the predicted state also
reduces over time and shrinks to zero when the actual parameter
estimate is constructed in nite-time.
In addition, the Lipschitz-based approach is similar to a stan-
dard MPC approach subject to one additional ODE for the uncer-
tainty cone. It is therefore easily implemented using
computational MPC techniques. Thus, the Lipschitz-based ap-
proach can be implemented using any real-time MPC technique
such as [11,12,23] for example. The purpose of this manuscript is
to provide a theoretical framework to integrate MPC and RTO with
guaranteed robust feasibility. There is no claim of improved com-
putational performance in this manuscript. This aspect of the prob-
lem will be considered in the context of future work and in the
specic applications.
4.5. Robust stability
Following the results presented in [1], robust stability is guar-
anteed under the standard assumptions that X
e
f
#X
e
is an invari-
ant set, W is a local robust CLF for the resulting time varying
system and the decay rate of W is greater than the stage cost L
within the terminal set X
e
f
in conjunction with the requirement
for W to decrease and X
f
to enlarge with decreased parametric
uncertainty. The reader is referred to [1] for further details and a
proof of the robust stability result.
4.6. Enhancing parameter convergence
In minmax adaptive formulation, the terminal penalty is
parameterized as a function of
~
h. This ensures that the algorithm
will seek to reduce the parameter error in the process of optimizing
the cost function and will automatically inject some excitation in
the closed-loop system, when necessary, to enhance parameter
convergence. However, this is not the case in the Lipschitz-based
approach since the control calculation only uses nominal model.
To improve the quality of excitation in the closed-loop the pro-
posed excitation cost is
J
E

b
1 E
p
h
T
; 32
where
E
p
h
s k
min
fQ
p
sg or E
p
h
s m
T
Q
p
sm 33
with m 2 R
n
h
a unit vector. Note that any reduction in the cost func-
tion due to J
E
implies an improvement in the rank of Q
p
. Though, the
predicted regressor matrix Q
p
differs from the actual matrix Q, a suf-
cient condition for Q > 0 is for Q
p
> z
Q
PkQ Q
p
k.
The cost associated with the injection of external excitation sig-
nals is not negligible in practice. However, the proposed approach
integrates the cost of excitation directly in the closed-loop perfor-
mance cost. Thus, in cases where the precise estimation of the
parameters yields signicant improvements of the closed-loop
cost, the algorithm will react by providing input signals that im-
prove parameter convergence. This special feature eliminates the
need to design an external signal. We note that since parameter
convergence is a necessary requirement for the solution of the
RTO task, one cannot remove the need for additional excitation.
However, the approach proposed addresses several new features
that have not appeared in the literature. These features reduce
the need for excessive external excitation that can severely affect
the closed-loop performance of adaptive control systems. The
adaptive control algorithm is inspired by a nite-time identica-
tion approach that provides mechanisms to remove additional
excitation. The controller formulation is designed to behave like
a standard MPC routine when parameter estimates are close to
the true parameter values.
5. Two-layer integration method
The integration task can also be posed as a two degree of free-
dom paradigm where the problem is divided into two phases. The
rst phase deals with generating a state trajectory that optimizes a
given objective function while respecting the system dynamics and
constraints, and the second phase deals with the design of a con-
troller that would regulate the system around a specic setpoint.
At each sampling instant, the current value of the setpoint,
x
r
t;
^
h given by the extremum-seeking setpoint update (16) is
passed down to the tracking controller for implementation. The ap-
proach can be viewed as a sampling and zero-hold implementation
of the setpoint update law.
In this approach, the adaptive MPC controller and the extre-
mum-seeking setpoint update are viewed as separate mechanisms.
The MPC controller design is implement following (19) and (26).
The only difference is that rather than solving the setpoint differ-
ential equation (16) inside the MPC loop, the measurement of x
r
obtained at sampling instants is used as the desired setpoint to
be tracked, that is, Eqs. (20e) and (27e) are replaced by
_ x
p
r
0; x
p
r
0 x
r
: 34
The adaptive controllers are implemented according to Algorithms
2 and 3. There is no technical difference arising form the two-layer
approach as compared to the one-layer method presented above.
Remark 8. While this paper is directed to systems with parametric
uncertainty, the procedure developed for achieving the integration
task is well suited for solving the nominal integration problem (in
the absence of uncertainty). In this case, the setpoint trajectory is
generated with
_
^
h 0. Rather than using robust adaptive NMPC,
standard state feedback NMPC will be employed to drive the
tracking error to zero. Since the setpoint trajectory is known a-
priori in this case, the terminal cost and terminal set can be
designed ofine. Moreover, the computation of the reference
generator and the MPC becomes more manageable.
Remark 9. For systems with nonparametric uncertainties, one
must consider a signicant reformulation of the adaptive MPC for-
mulation as presented in [4] where a robust adaptive MPC design is
proposed for systems with unknown but bounded disturbances.
The presence of disturbances would limit the adaptive MPC by
robustly stabilizing a set containing the unknown optimal setpoint
rather than converging to the point exactly as is proven in the next
section.
6. Main result
The integration result is provided in the following:
Theorem 10. Consider problem (1) subject to system dynamics (2),
and satisfying Assumption 1. Let the controller be (19)or(26) with
setpoint update law (16) and parameter identier (12). If the
invertibility condition (9) is satised, then for any . > 0, there exists
constant g
c
such that lim
t!1
kxt x

t; hk 6 ., with x
*
(t,h) the
unique minimizer of (1). In addition x 2 X; u 2 U for all t P0.
Proof. We know from triangle inequality that
kx x

hk 6 kx x
r

^
hk kx
r

^
h x

pa

^
hk kx

pa

^
h x

^
hk
kx

^
h x

hk; 35
130 V. Adetola, M. Guay / Journal of Process Control 20 (2010) 125133
where x

pa

^
h denotes the unique minimizer of the unconstrained
problem (13) for h
^
h. Since the MPC controllers guarantees
asymptotic convergence of x
e
to the origin, we have
lim
t!1
kx x
r

^
hk 0. Also, it follows from Lemma 5, that
kx
r

^
h x

pa

^
hk converges exponentially to the origin. Moreover, it
is well established that x

pa

^
h converges continuously to x

^
h as
g
c
?1 [7, Proposition 4.1.1]. Therefore there exists a class Kfunc-
tion
1
a
c
() such that
lim
t!1
kx

pa

^
h x

^
hk 6 a
c
1
g
c
_ _
: 36
The nite-time identication procedure employed ensures that
^
h h for all t Pt
c
, with t
c
< 1 and thus lim
t!1
kx

^
h x

hk 0.Fi-
nally, we have
lim
t!1
kxt x

t; hk 6 a
c
1
g
c
_ _
37
and the result follows for sufciently large g
c
. The constraint satis-
faction claim follows from the feasibility of the adaptive model pre-
dictive controllers. h
7. Simulation example
Consider the parallel isothermal stirred-tank reactor in which
reagent A forms product B and waste-product C [10]. The reactors
dynamics are given by
dA
i
dt
A
in
F
in
i
V
i
A
i
F
out
i
V
i
k
i1
A
i
2k
i2
A
2
i
;
dB
i
dt
B
i
F
out
i
V
i
k
i1
A
i
;
dC
i
dt
C
i
F
out
i
V
i
k
i2
A
2
i
;
where A
i
, B
i
, C
i
denote concentrations in reactor i, k
ij
are the reaction
kinetic constants, which are only nominally known. The inlet ows
F
in
i
are the control inputs, while the outlet ows F
out
i
are governed by
PI controllers which regulate reactor volume to V
0
i
.
The economic cost function is the net expense of operating the
process at steady-state
pA
i
; s; h

2
i1
p
i1
s
i
P
A
P
B
k
i1
A
i
V
0
i
p
i2
s
i
2P
A
k
i2
A
2
i
V
0
i
; 38
where P
A
, P
B
denote component prices, p
ij
is the net operating cost of
reaction j in reactor i. Disturbances s
1
, s
2
reect changes in the oper-
ating cost (utilities, etc.) of each reactor. The control objective is to
robustly regulate the process to the optimal operating point that
optimizes the economic cost (38) while satisfying the following
state constraints 0 6 A
i
6 3; c
v
A
2
1
V
0
1
A
2
2
V
0
2
15 6 0 and input
constraint 0:01 6 F
in
i
6 0:2. The reaction kinetics are assumed to
satisfy 0.01 6 k
i
6 0.2.
The two-layer approach was used for the simulation. The set-
point value available at sampling instant is passed down to the
MPC controller for implementation. The sampling time is take to
be 0.1 s. The robustness of the adaptive controller is guaranteed
via the Lipschitz bound method. The stage cost is selected as a qua-
dratic cost Lx
e
; u
e
x
T
e
Q
x
x
e
u
T
e
R
u
u
e
, with Q
x
> 0 and R
u
P0. This
choice is completely arbitrary.
7.1. Terminal penalty and terminal set design
Let x = [A
1
, A
2
]
T
, h = [k
11
, k
12
, k
21
, k
22
]
T
and u F
in
1
; F
in
2

T
, the
dynamics of the system can be expressed in the form:
_ x
x
1
k
V1
n
1
V
0
1
n
3

n
1
x
2
k
V2
n
2
V
0
2
n
4

n
2
_
_
_
_
..
f
p1

A
in
n
1
0
0
A
in
n
2
_
_
_
_
..
f
p2
u
x
1
2x
2
1
0 0
0 0 x
2
2x
2
2
_ _
..
g
h;
where n
1
, n
2
are the two tank volumes and n
3
, n
4
are the PI integra-
tors. The system parameters are V
0
1
0:9; V
0
2
1:5; k
v
1
k
v
2
1;
P
A
5; P
B
26; p
11
p
21
3 and p
12
= p
22
= 1.
A Lyapunov function for the terminal penalty is dened as the
input to state stabilizing control Lyapunov function (iss-clf):
Wx
e

1
2
x
T
e
x
e
: 39
Choosing a terminal controller
u k
f
x
e
f
1
p
2
f
p
1
k
1
x
e
k
2
gg
T
x
e
_ _
40
with design constants k
1
, k
2
> 0, the time derivative of (39) becomes
_
Wx
e
k
1
x
T
e
x
e
x
T
e
g h k
2
x
T
e
g g
T
x
e
; 41
6 k
1
kx
e
k
2

1
4k
2
khk
2
: 42
Since the stability condition requires
_
Wx
e
T LT 6 0. We
choose the weighting matrices of L as Q = 0.5I and R = 0. The termi-
nal state region is selected as
X
e
f
fx
e
: Wx
e
6 a
e
g 43
such that
k
f
x
e
2 U;
_
WT LT 6 0; 8h; x
e
2 H; X
e
f
: 44
Since the given constraints requires the reaction kinetic h and con-
centration x to be positive, it follows that
_
W L k
1
0:5kx
e
k
2
x
T
e
gh k
2
x
T
e
gg
T
x
e
6 0 45
for all k
1
> 0.5 and x
e
> 0. Moreover, for x
e
< 0, the constants k
1
and
k
2
can always be selected such that (45) is satised "h 2 H. The
task of computing the terminal set is then reduced to nding the
largest possible a
e
such that for k
f
: 2 U for all x 2 X
e
f
.
The terminal cost (39) is used for this simulation and the termi-
nal set is re-computed at every sampling instant using the current
setpoint value.
The system was simulated subject to a ramping measured eco-
nomic disturbance in s
2
from t = 6 to 10. The simulation results are
presented in Figs. 15. The phase trajectories displayed in Fig. 1
show that the reactor states x
1
and x
2
obey the imposed con-
straints. The concentration of A in reactor x
1
is shown to hit the
constraint.
Fig. 2 shows that the actual, unknown setpoint cost p(t, x
r
, h)
converges to the optimal, unknown p
*
(t, x
*
, h). The initial effect of
the parameter estimation is observed initially but vanishes quickly
with the help of the nite-time identier. As soon as the parameter
estimates reach their unknown true values, the extremum seeking
converges quickly to the unknown optimum, as expected.
Fig. 3 conrms the effectiveness of the adaptive MPC in tracking
the desired setpoint. It can be seen that the state x
1
hits the upper
bound at t = 2.5 approximately. Fig. 4 demonstrates that the con-
vergence of the parameter estimates to their true values. This is de-
spite the absence of an exogenous dither signal. Note that, in
principle, the adaptive MPC has a self exciting feature that penal-
izes large estimation errors. The simulation results show that this
1
A continuous function l : R

!R

is of class K if it is strictly increasing and

l(0) = 0.
V. Adetola, M. Guay / Journal of Process Control 20 (2010) 125133 131
approach is extremely successful for the design of the adaptive
control system.
The control variables are shown in Fig. 5. The required control
action is implementable and satises the given constraints.
Overall, the simulation example demonstrates the many fea-
tures of the proposed integrated RTO and MPC. The technique pro-
vides an effective mechanism to solve problems normally
encountered in RTO while providing robust stability and robust
feasibility that are the main performance benets of MPC.
8. Conclusions
This paper provides a formal design technique for integrating
RTO and MPC for constrained nonlinear uncertain systems. The
solution is based on the tools and strategies developed in [1,9]. A
single layer and two-layer approaches are presented. The con-
strained RTO problem is solved by the use of interior point barrier
functions and Newtons method. Adaptive nonlinear MPC based on
minmax framework and Lipschitz bounds proposed drive the sys-
tems state to the minimizer of the uncertain objective function. In
future work, the computational requirement of the proposed ap-
proach will be considered. An adaptive MPC approach for systems
subject to exogenous disturbances has been recently proposed.
This technique would be suitable for application in robust inte-
grated systems.
Acknowledgment
The authors would like to acknowledge the nancial support of
the Natural Sciences and Engineering Research Council of Canada.
0 5 10 15
7.5
7
6.5
6
5.5
5
4.5
4
p
r
o
f
i
t

f
u
n
c
t
i
o
n
time
p*(t,x
*
,)
p(t,x
r
,)
Fig. 2. Optimal and actual prot functions.
1 1.5 2 2.5 3
0.5
1
1.5
2
2.5
3
x
1
x
2
c
v
= 0
Fig. 1. Phase diagram and feasible state region. The thick line shows the closed-
loop trajectories for the concentration of A in reactor 1, x
1
, and in reactor 2x
2
. Also
shown are the state constraints x
1
= 3, x
2
= 3 and c
v
= 0.
0 5 10 15
0.5
1
1.5
2
2.5
3
3.5
time
s
t
a
t
e
s
x
x
r
reactor 1
reactor 2
Fig. 3. Reference trajectories and closed-loop states.
0 5 10 15
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
time
p
a
r
a
m
e
t
e
r
s
k
11
k
21
k
12
k
22
Fig. 4. Unknown parameters and estimates.
0 5 10 15
0
0.05
0.1
0.15
0.2
u
1
0 5 10 15
0
0.05
0.1
0.15
0.2
time
u
2
Fig. 5. Closed-loop systems inputs.
132 V. Adetola, M. Guay / Journal of Process Control 20 (2010) 125133
References
[1] V. Adetola, D. DeHaan, M. Guay, Adaptive model predictive control for
constrained nonlinear systems, Systems and Control Letters 58 (5) (2009)
320326.
[2] V. Adetola, M. Guay, Nonlinear sampled-data output feedback receding
horizon control, Journal of Process Control 15 (2005) 469480.
[3] V. Adetola, M. Guay, Finite-time parameter estimation in adaptive control of
nonlinear systems, IEEE Transactions on Automatic Control 53 (3) (2008) 807
811. April.
[4] V. Adetola, M. Guay, Adaptive robust MPC of nonlinear systems with
exogenous disturbances, in: Proceedings of the IFAC ADCHEM, Istanbul,
Turkey, 2009.
[5] V.A. Adetola, M. Guay, Adaptive output feedback extremum seeking receding
horizon control of linear systems, Journal of Process Control 16 (5) (2006) 521
533.
[6] K. Basak, K.S. Abhilash, S. Ganguly, D.N. Saraf, Online optimization of a crude
distillation unit with constraints on product properties, Industrial Engineering
Chemistry Research 41 (2002) 15571568.
[7] D.P. Bertsekas, Nonlinear Programming, Athena Scientic, Belmont, MA, 1995.
[8] M.T. de Gouva, D. Odloak, One-layer real time optimization of lpg production
in the fcc unit: procedure advantages and disadvantages, Computers &
Chemical Engineering 22 (1998) S191S198.
[9] D. DeHaan, V. Adetola, M. Guay, Adaptive robust MPC: an eye towards
computational simplicity, in: Proceedings of the IFAC Symposium on Nonlinear
Control systems, South Africa, 2007.
[10] D. DeHaan, M. Guay, Extremum seeking control of state constrained nonlinear
systems, Automatica 41 (9) (2005) 15671574.
[11] D. DeHaan, M. Guay, A real-time framework for model-predictive control of
continuous-time nonlinear systems, IEEE Transactions on Automatic Control
52 (11) (2007) 20472057.
[12] M. Diehl, R. Findeisen, F. Allgwer, H.G. Bock, J.P. Schlder, Nominal stability of
the real-time iteration scheme for nonlinear model predictive control, IEE
PROCEEDINGS Control Theory and Applications 152 (3) (2005) 296308.
May.
[13] R. Findeisen, F. Allgower, Robustness properties and output feedback of
optimization based sampled-data open-loop feedback, in: 44th IEEE
Conference on Decision and Control and European Control Conference ECC
2005, Omnipress, 2005, p. 12.
[14] R. Findeisen, H. Chen, F. Allgwer, Nonlinear predictive control for setpoint
families, in: American Control Conference, 2000, pp. 260264.
[15] J.F. Forbes, T.E. Marlin, Design cost: a systematic approach to technology
selection for model-based real-time optimization, Computers and Chemical
Engineering 20 (1996) 717734.
[16] L. Imsland, R. Findeisen, E. Bullinger, F. Allgwer, B.A. Foss, On output feedback
nonlinear model predictive control using high gain observers for a class of
systems, in: 6th IFAC Symposium on Dynamic and Control of Process Systems,
DYCOPS-6, Jejudo, Korea, 2001, pp. 9196.
[17] J. Kadam, W. Marquardt, Sensitivity-based solution updates in closed-loop
dynamic optimization, in: DYCOPS, vol. 7, Cambridge, USA, 2004.
[18] J. Kadam, W. Marquardt, in: Integration of Economical Optimization and
Control for Intentionally Transient Process Operation, Assessment and Future
Directions of Nonlinear Model Predictive Control, vol. 358, Springer, Berlin/
Heidelberg, 2007, pp. 419434.
[19] J. Kadam, W. Marquardt, M. Schlegel, T. Backx, O.H. Bosgra, P.-J. Brouwer, G.
Dnnebier, D. van Hessem, A. Tiagounov, S. de Wolf, Towards integrated
dynamic real-time optimization and control of industrial processes, in: I.E.
Grossmann, C.M. McDonald (Eds.), FOCAPO 2003, Proceedings of the
Conference, 2003, pp. 593596.
[20] J. Kadam, M. Schlegel, W. Marquardt, R.L. Tousain, D.H.v. Hessem, J.v.d. Berg,
O.H. Bosgra, A two-level strategy of integrated dynamic optimization and
control of industrial processes a case study, in: J. Grievink, J.v. Schijndel
(Eds.), European Symposium on Computer Aided Process Engineering, vol. 12,
2002, pp. 511516.
[21] U.E. Lauks, R.J. Vasbinder, P.J. Valkenburg, C. van Leeuwen, On-line
optimization of an ethylene plant, Computers & Chemical Engineering 16
(1992) S213S220.
[22] L. Magni, On robust tracking with nonlinear model predictive control,
International Journal of Control 75 (6) (2002) 399407.
[23] T. Ohtsuka, A continuation/gmres method for fast computation of nonlinear
receding horizon control, Automatica 40 (4) (2004) 563574.
[24] R. Sepulchre, M. Jankovic, P. Kokotovic, Constructive Nonlinear Control,
Springer-Verlag, 1997.
[25] A.C. Zanin, M.T. de Gouva, D. Odloak, Integrating real-time optimization into
the model predictive controller of the fcc system, Control Engineering Practice
10 (2002) 819831.
[26] X. Zhu, W. Hong, S. Wang, Implementation of advanced control for a heat-
integrated distillation column system, in: Industrial Electronics Society, 2004,
IECON 2004, 30th Annual Conference of IEEE, vol. 3, 26 November 2004, pp.
20062011.
V. Adetola, M. Guay / Journal of Process Control 20 (2010) 125133 133