Model Predictive Control

Predictive Control
for linear and hybrid systems

F. Borrelli, A. Bemporad, M. Morari
March 7, 2014
2
i
to Maryan, Federica and Marina
and all our families
ii
Preface
Dynamic optimization has become a standard tool for decision making in a wide
range of areas. The search for the most fuel-ecient strategy to send a rocket into
orbit or the most economical way to start-up a chemical production facility can
be expressed as dynamic optimization problems that are solved almost routinely
nowadays.
The basis for these dynamic optimization problems is a dynamic model, for
example,
x(k + 1) = g(x(k), u(k)), x(0) = x
0
that describes the evolution of the state x(k) with time, starting from the initial
condition x(0), as it is aected by the manipulated input u(k). Here g(x, u) is
some nonlinear function. Throughout the book we are assuming that this discrete-
time description, i.e., the model of the underlying system, is available. The goal
of the dynamic optimization procedure is to nd the vector of manipulated inputs
U
N
= [u(0)
, ..., u(N 1)
such that the objective function is optimized over some

time horizon N, typically
min
UN
N1
k=0
q(x(k), u(k)) +p(x(N))
The terms q(x, u) and and p(x) are referred to as the stage cost and terminal cost,
respectively. Many practical problems can be put into this form and many algo-
rithms and software packages are available to determine the optimal solution vector
U
N
, the optimizer. The various algorithms exploit the structure of the particular
problem, e.g., linearity and convexity, so that even large problems described by
complex models and involving many degrees of freedom can be solved eciently
and reliably.
One diculty with this idea is that, in practice, the sequence of u(0), u(1), ...,
which is obtained by this procedure cannot be simply applied. The model of the
system predicting its evolution is usually inaccurate and the system may be aected
by external disturbances that may cause its path to deviate signicantly from the
one that is predicted. Therefore, it is common practice to measure the state after
some time period, say one time step, and to solve the dynamic optimization problem
again, starting from the measured state x(1) as the new initial condition. This
feedback of the measurement information to the optimization endows the whole
procedure with a robustness typical of closed-loop systems.
What we have described above is usually referred to as Model Predictive Control
(MPC), but other names like Open Loop Optimal Feedback and Reactive Schedul-
ing have been used as well. Over the last 25 years MPC has evolved to dominate
the process industry, where it has been employed for thousands of problems [239].
The popularity of MPC stems from the fact that the resulting operating strategy
respects all the system and problem details, including interactions and constraints,
which would be very hard to accomplish in any other way.
Indeed, often MPC is used for the regulatory control of large multivariable
linear systems with constraints, where the objective function is not related to an
economical objective, but is simply chosen in a mathematically convenient way,
iii
namely quadratic in the states and inputs, to yield a good closed-loop response.
Again, there is no other controller design method available today for such systems,
that provides constraint satisfaction and stability guarantees.
One limitation of MPC is that running the optimization algorithm on-line at
each time step requires substantial time and computational resources. Today, fast
computational platforms together with advances in the eld of operation research
and optimal control have enlarged in a very signicant way the scope of appli-
cability of MPC to fast-sampled applications. One approach is to use tailored
optimization routines which exploit both the structure of the MPC problem and
the architecture of the embedded computing platform to implement MPC in the
order of milliseconds.
The second approach is to have the result of the optimization pre-computed
and stored for each x in the form of a look-up table or as an algebraic function
u(k) = f(x(k)) which can be easily evaluated. In other words, we want to determine
the (generally nonlinear) feedback control law f(x) that generates the optimal
u(k) = f(x(k)) explicitly and not just implicitly as the result of an optimization
problem. It requires the solution of the Bellman equation and has been a long
standing problem in optimal control. A clean simple solution exists only in the case
of linear systems with a quadratic objective function, where the optimal controller
turns out to be a linear function of the state (Linear Quadratic Regulator, LQR).
For all other cases a solution of the Bellman equation was considered prohibitive
except for systems of low dimension (2 or 3), where a look-up table can be generated
by gridding the state space and solving the optimization problem o-line for each
grid point.
The major new contribution of this book is to show how the nonlinear opti-
mal feedback controller can be calculated eciently for some important classes of
systems, namely linear systems with constraints and switched linear systems or,
more generally, hybrid systems. Traditionally, the design of feedback controllers for
linear systems with constraints, for example anti-windup techniques, was ad hoc
requiring both much experience and trial and error. Though signicant progress
has been achieved on anti-windup schemes over the last decade, these techniques
deal with input constraints only and cannot be extended easily.
The classes of constrained linear systems and linear hybrid systems treated in
this book cover many, if not most, practical problems. The new design techniques
hold the promise to lead to better performance and a dramatic reduction in the
required engineering eort.
The book is structured in four parts. Chapters marked with a star () are
considered advanced material and can be skipped without compromising the
understanding of the fundamental concepts presented in the book.
In the rst part of the book (Part I) we recall the main concepts and results of
convex and discrete optimization. Our intent is to provide only the necessary
background for the understanding of the rest of the book. The material of this
part has been extracted from the following books and lecture notes: Con-
vex optimization by Boyd and Vandenberghe [64], Nonlinear Programming
Theory and Algorithms by Bazaraa, Sherali, Shetty [26], LMIs in Control
by Scherer and Weiland [256] and Lectures on Polytopes by Ziegler [294].
iv
Continuous problems as well as integer and mixed-integer problems are pre-
sented in Chapter 1. Chapter 2 discusses the classical results of Lagrange
duality. In Chapter 3 Linear and Quadratic programs are presented together
with their properties and some fundamental results. Chapter 4 introduces
algorithms for the solution of unconstrained and constrained optimization
problems. We only discuss those that are important for the problems en-
countered in this book and explain the underlying concepts. Since polyhedra
are the fundamental geometric objects used in this book, Part I closes with
Chapter 5 where we introduce the main denitions and the algorithms, which
describe standard operations on polyhedra.
The second part of the book (Part II) is a self-contained introduction to
multi-parametric programming. In our framework, parametric programming
is the main technique used to study and compute state feedback optimal
control laws. In fact, we formulate the nite time optimal control problems
as mathematical programs where the input sequence is the optimization vec-
tor. Depending on the dynamical model of the system, the nature of the
constraints, and the cost function used, a dierent mathematical program is
obtained. The current state of the dynamical system enters the cost function
and the constraints as a parameter that aects the solution of the mathemati-
cal program. We study the structure of the solution as this parameter changes
and we describe algorithms for solving multi-parametric linear, quadratic and
mixed integer programs. They constitute the basic tools for computing the
state feedback optimal control laws for these more complex systems in the
same way as algorithms for solving the Riccati equation are the main tools for
computing optimal controllers for linear systems. In Chapter 6 we introduce
the concept of multiparametric programming and we recall the main results
of nonlinear multiparametric programming. Then, in Chapter 7 we describe
three algorithms for solving multiparametric linear programs (mp-LP), multi-
parametric quadratic programs (mp-QP) and multiparametric mixed-integer
linear programs (mp-MILP).
In the third part of the book (Part III) we introduce the general class of
optimal control problems studied in the book. Chapter 8 contains the basic
denitions and essential concepts. Chapter 9 presents standard results on
Linear Quadratic Optimal Control while in Chapter 10 unconstrained optimal
control problems for linear systems with cost functions based on 1 and
norms are analyzed.
In the fourth part of the book (Part IV) we focus on linear systems with
polyhedral constraints on inputs and states. We start with a self-contained
introduction to controllability, reachability and invariant set theory in Chap-
ter 11. The chapter focuses on computational algorithms for constrained
linear systems and constrained linear systems subject to additive and para-
metric uncertainty.
In Chapter 12 we study nite time and innite time optimal control problems
with cost functions based on 2, 1 and norms. We rst show how to trans-
form them into LP or QP optimization problems for a xed initial condition.
v
Then we show that the solution to all these optimal control problems can
be expressed as a piecewise ane state feedback law. Moreover, the optimal
control law is continuous and the value function is convex and continuous.
The results form a natural extension of the theory of the Linear Quadratic
Regulator to constrained linear systems.
Chapter 13 presents the concept of Model Predictive Control. Classical feasi-
bility and stability issues are shown through simple examples and explained
by using invariant set methods. Finally we show how they can be solved with
a proper choice of the terminal constraints and the cost function.
The result in Chapter 12 and Chapter 13 have important consequences for
the implementation of MPC laws. Precomputing o-line the explicit piece-
wise ane feedback policy reduces the on-line computation for the receding
horizon control law to a function evaluation, therefore avoiding the on-line
solution of a mathematical program. However the number of polyhedral re-
gions of the explicit optimal control laws could grow exponentially with the
number of constraints in the optimal control problem. Chapter 14 discusses
approaches to dene approximate explicit control laws of desired complexity
that provide certicates of recursive feasibility and stability.
Chapter 15 focuses on ecient on-line methods for the computation of RHC
control laws. If the state-feedback solution is available explicitly, we present
ecient on-line methods for the evaluation of explicit piecewise ane control
laws. In particular, we present algorithms to reduce its storage demands and
computational complexity. If the on-line solution of a quadratic or linear
program is preferred, we briey discuss how to improve the eciency of a
mathematical programming solver by exploiting the structure of the RHC
control problem.
Part IV closes with Chapter 16 where we address the robustness of the opti-
mal control laws. We discuss min-max control problems for uncertain linear
systems with polyhedral constraints on inputs and states and present an ap-
proach to compute their state feedback solutions. Robustness is achieved
against additive norm-bounded input disturbances and/or polyhedral para-
metric uncertainties in the state-space matrices.
In the fth part of the book (Part V) we focus on linear hybrid systems. We
give an introduction to the dierent formalisms used to model hybrid systems
focusing on computation-oriented models (Chapter 17). In Chapter 18 we
study nite time optimal control problems with cost functions based on 2, 1
and norms. The optimal control law is shown to be, in general, piecewise
ane over non-convex and disconnected sets. Along with the analysis of the
solution properties we present algorithms that eciently compute the optimal
control law for all the considered cases.
Berkeley Francesco Borrelli
Lucca Alberto Bemporad
Zurich Manfred Morari
March 2014
vi
Acknowledgements
Large parts of the material presented in this book is extracted from the work
of the authors Francesco Borrelli, Alberto Bemporad and Manfred Morari.
Several sections contain the results of the PhD Theses of Miroslav Baric, Mato
Baotic, Fabio Torrisi, Domenico Mignone, Pascal Grieder, Eric Kerrigan,
Tobias Geyer and Frank J. Christophersen.
We are extremely grateful to Martin Herceg for his meticulous work on the
matlab examples and all the book gures.
A special thanks goes to Michal Kvasnica, Stefan Richter, Valerio Turri and
Thomas Besselmann, Rick Meyer for their help on the construction of the
examples.
Our gratitude also goes to the several people who have carefully read pre-
liminary versions of the book and gave us suggestions on how to improve
them. They include David Mayne, Per Olof Gutman, Diethard Klatte, Paul
Goulart, Bill Levine, Miroslav Fikar and all the students of our class on Model
Predictive Control (ME290J at UC Berkeley, 227-0221-00 at ETH).
The authors of the LaTex book style les and macros are Stephen Boyd
and Lieven Vandenberghe. We are most grateful to Stephen and Lieven for
sharing them with us.
The authors of Chapter 4 are Dr. Alexander Domahidi and Dr. Stefan Richter
(domahidi@embotech.com, richters@alumni.ethz.ch).
The author of Chapter 14 is Prof. Colin N. Jones (colin.jones@ep.ch).
Contents
I Basics of Optimization 1
1 Main Concepts 3
1.1 Optimization Problems . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.1.1 Continuous Problems . . . . . . . . . . . . . . . . . . . . . . 4
Active, Inactive and Redundant Constraints . . . . . . . . . . 5
Problems in Standard Forms . . . . . . . . . . . . . . . . . . 5
Eliminating Equality Constraints . . . . . . . . . . . . . . . . 5
Problem Description . . . . . . . . . . . . . . . . . . . . . . . 6
1.1.2 Integer and Mixed-Integer Problems . . . . . . . . . . . . . . 6
1.2 Convexity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
Operations preserving convexity . . . . . . . . . . . . . . . . 7
Linear and quadratic convex functions . . . . . . . . . . . . . 8
Convex optimization problems . . . . . . . . . . . . . . . . . 8
2 Optimality Conditions 11
2.1 Optimality Conditions for Unconstrained Problems . . . . . . . . . . 11
2.2 Lagrange Duality Theory . . . . . . . . . . . . . . . . . . . . . . . . 12
2.2.1 Strong Duality and Constraint Qualications . . . . . . . . . 14
2.2.2 Certicate of Optimality . . . . . . . . . . . . . . . . . . . . . 15
2.3 Complementary Slackness . . . . . . . . . . . . . . . . . . . . . . . . 15
2.4 Karush-Kuhn-Tucker Conditions . . . . . . . . . . . . . . . . . . . . 16
2.4.1 Geometric Interpretation of KKT Conditions . . . . . . . . . 18
3 Linear and Quadratic Optimization 21
3.1 Polyhedra and Polytopes . . . . . . . . . . . . . . . . . . . . . . . . . 21
3.2 Linear Programming . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
3.2.1 Graphical Interpretation and Solution Properties . . . . . . . 22
3.2.2 Dual of LP . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
3.2.3 KKT condition for LP . . . . . . . . . . . . . . . . . . . . . . 25
3.2.4 Active Constraints and Degeneracies . . . . . . . . . . . . . . 25
3.2.5 Convex Piecewise Ane Optimization . . . . . . . . . . . . . 28
3.3 Quadratic Programming . . . . . . . . . . . . . . . . . . . . . . . . . 29
3.3.1 Graphical Interpretation and Solution Properties . . . . . . . 30
3.3.2 Dual of QP . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
3.3.3 KKT conditions for QP . . . . . . . . . . . . . . . . . . . . . 31
3.3.4 Active Constraints and Degeneracies . . . . . . . . . . . . . . 32
viii Contents
3.3.5 Constrained Least-Squares Problems . . . . . . . . . . . . . . 32
3.4 Mixed-Integer Optimization . . . . . . . . . . . . . . . . . . . . . . . 32
4 Numerical Methods for Optimization 35
4.1 Unconstrained Optimization . . . . . . . . . . . . . . . . . . . . . . . 38
4.1.1 Gradient Methods . . . . . . . . . . . . . . . . . . . . . . . . 38
Classic Gradient Method . . . . . . . . . . . . . . . . . . . . 38
Fast Gradient Method . . . . . . . . . . . . . . . . . . . . . . 41
Strongly Convex Problems . . . . . . . . . . . . . . . . . . . . 42
Preconditioning . . . . . . . . . . . . . . . . . . . . . . . . . . 44
Stopping criteria . . . . . . . . . . . . . . . . . . . . . . . . . 44
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
4.1.2 Newtons Method . . . . . . . . . . . . . . . . . . . . . . . . . 46
Preconditioning and Ane Invariance . . . . . . . . . . . . . 47
Newtons Method with Equality Constraints . . . . . . . . . 47
4.1.3 Line Search Methods . . . . . . . . . . . . . . . . . . . . . . . 48
4.2 Constrained Optimization . . . . . . . . . . . . . . . . . . . . . . . . 49
4.2.1 Gradient Projection Methods . . . . . . . . . . . . . . . . . . 49
Solution in Dual Domain . . . . . . . . . . . . . . . . . . . . 51
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
4.2.2 Interior Point Methods . . . . . . . . . . . . . . . . . . . . . . 54
Primal barrier methods . . . . . . . . . . . . . . . . . . . . . 54
Primal-dual methods . . . . . . . . . . . . . . . . . . . . . . . 59
Relation of Primal-Dual to Primal Barrier Methods . . . . . 64
4.2.3 Active Set Methods . . . . . . . . . . . . . . . . . . . . . . . 64
Active Set Method for LP . . . . . . . . . . . . . . . . . . . . 64
Active Set Method for QP . . . . . . . . . . . . . . . . . . . . 66
5 Polyhedra and P-collections 69
5.1 General Set Denitions and Operations . . . . . . . . . . . . . . . . 69
5.2 Polyhedra and Representations . . . . . . . . . . . . . . . . . . . . . 71
5.3 Polytopal Complexes . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
5.3.1 Functions on Polytopal Complexes . . . . . . . . . . . . . . . 76
5.4 Basic Operations on Polytopes . . . . . . . . . . . . . . . . . . . . . 76
5.4.1 Minimal Representation . . . . . . . . . . . . . . . . . . . . . 76
5.4.2 Convex Hull . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
5.4.3 Envelope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
5.4.4 Vertex Enumeration . . . . . . . . . . . . . . . . . . . . . . . 78
5.4.5 Chebychev Ball . . . . . . . . . . . . . . . . . . . . . . . . . . 80
5.4.6 Projection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
5.4.7 Set-Dierence . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
5.4.8 Pontryagin Dierence . . . . . . . . . . . . . . . . . . . . . . 84
5.4.9 Minkowski Sum . . . . . . . . . . . . . . . . . . . . . . . . . . 84
5.4.10 Polyhedra Union . . . . . . . . . . . . . . . . . . . . . . . . . 85
5.4.11 Ane Mappings and Polyhedra . . . . . . . . . . . . . . . . 87
5.5 Operations on P-collections . . . . . . . . . . . . . . . . . . . . . . . 88
5.5.1 Set-Dierence . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
Contents ix
5.5.2 Polytope Covering . . . . . . . . . . . . . . . . . . . . . . . . 90
5.5.3 Union of P-collections . . . . . . . . . . . . . . . . . . . . . . 91
II Multiparametric Programming 93
6 Multiparametric Nonlinear Programming 95
6.1 Introduction to Multiparametric Programs . . . . . . . . . . . . . . . 95
6.2 General Results for Multiparametric Nonlinear Programs . . . . . . 98
7 Multiparametric Programming: a Geometric Approach 107
7.1 Multiparametric Programs with Linear Constraints . . . . . . . . . . 107
7.1.1 Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
7.1.2 Denition of Critical Region . . . . . . . . . . . . . . . . . . . 108
7.1.3 Reducing the Dimension of the Parameter Space . . . . . . . 109
7.2 Multiparametric Linear Programming . . . . . . . . . . . . . . . . . 110
7.2.1 Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
7.2.2 Critical Regions, Value Function and Optimizer: Local Prop-
erties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
7.2.3 Propagation of the Set of Active Constraints . . . . . . . . . 115
7.2.4 Nonunique Optimizer
. . . . . . . . . . . . . . . . . . . . . . 117
7.2.5 Value Function and Optimizer: Global Properties . . . . . . . 119
7.2.6 mp-LP Algorithm . . . . . . . . . . . . . . . . . . . . . . . . 122
7.3 Multiparametric Quadratic Programming . . . . . . . . . . . . . . . 127
7.3.1 Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
7.3.2 Critical Regions, Value Function and Optimizer: Local Prop-
erties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
7.3.3 Propagation of the Set of Active Constraints . . . . . . . . . 131
7.3.4 Value Function and Optimizer: Global Properties . . . . . . . 133
7.3.5 mp-QP Algorithm . . . . . . . . . . . . . . . . . . . . . . . . 135
7.4 Multiparametric Mixed-Integer Linear Programming . . . . . . . . . 139
7.4.1 Formulation and Properties . . . . . . . . . . . . . . . . . . . 139
7.4.2 Geometric Algorithm for mp-MILP . . . . . . . . . . . . . . . 139
Initialization . . . . . . . . . . . . . . . . . . . . . . . . . . . 140
mp-LP subproblem . . . . . . . . . . . . . . . . . . . . . . . . 140
MILP subproblem . . . . . . . . . . . . . . . . . . . . . . . . 141
Recursion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
7.4.3 Solution Properties . . . . . . . . . . . . . . . . . . . . . . . . 142
7.5 Multiparametric Mixed-Integer Quadratic Programming . . . . . . . 143
7.5.1 Formulation and Properties . . . . . . . . . . . . . . . . . . . 143
7.6 Literature Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
III Optimal Control 147
8 General Formulation and Discussion 149
8.1 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . 149
8.2 Solution via Batch Approach . . . . . . . . . . . . . . . . . . . . . . 151
x Contents
8.3 Solution via Recursive Approach . . . . . . . . . . . . . . . . . . . . 152
8.4 Optimal Control Problem with Innite Horizon . . . . . . . . . . . . 154
8.4.1 Value Function Iteration . . . . . . . . . . . . . . . . . . . . . 155
8.4.2 Receding Horizon Control . . . . . . . . . . . . . . . . . . . . 156
8.5 Lyapunov Stability . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158
8.5.1 General Stability Conditions . . . . . . . . . . . . . . . . . . 158
8.5.2 Quadratic Lyapunov Functions for Linear Systems . . . . . . 162
8.5.3 1/ Norm Lyapunov Functions for Linear Systems . . . . . . 163
9 Linear Quadratic Optimal Control 165
9.4 Comparison Of The Two Approaches . . . . . . . . . . . . . . . . . . 169
9.5 Innite Horizon Problem . . . . . . . . . . . . . . . . . . . . . . . . . 170
9.6 Stability of the Innite Horizon LQR . . . . . . . . . . . . . . . . . . 171
10 Linear 1/ Norm Optimal Control 173
10.3 Comparison Of The Two Approaches . . . . . . . . . . . . . . . . . . 178
10.4 Innite Horizon Problem . . . . . . . . . . . . . . . . . . . . . . . . . 179
IV Constrained Optimal Control of Linear Systems 183
11 Controllability, Reachability and Invariance 185
11.1 Controllable and Reachable Sets . . . . . . . . . . . . . . . . . . . . 185
11.1.1 Computation of Controllable and Reachable Sets . . . . . . . 187
11.2 Invariant Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193
11.3 Robust Controllable and Reachable Sets . . . . . . . . . . . . . . . . 199
11.3.1 Linear Systems with Additive Uncertainty and without In-
puts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200
11.3.2 Linear Systems with Additive Uncertainty and Inputs . . . . 202
11.3.3 Linear Systems with Parametric Uncertainty . . . . . . . . . 205
11.4 Robust Invariant Sets . . . . . . . . . . . . . . . . . . . . . . . . . . 208
12 Constrained Optimal Control 215
12.2 Feasible Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218
12.3 2-Norm Case Solution . . . . . . . . . . . . . . . . . . . . . . . . . . 222
12.3.1 Solution via QP . . . . . . . . . . . . . . . . . . . . . . . . . 222
12.3.2 State Feedback Solution via Batch Approach . . . . . . . . . 223
12.3.3 State Feedback Solution via Recursive Approach . . . . . . . 226
12.3.4 Innite Horizon Problem . . . . . . . . . . . . . . . . . . . . 226
12.3.5 CLQR Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . 228
12.3.6 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 230
12.4 1-Norm and -Norm Case Solution . . . . . . . . . . . . . . . . . . 232
Contents xi
12.4.1 Solution via LP . . . . . . . . . . . . . . . . . . . . . . . . . . 232
12.4.2 State Feedback Solution via Batch Approach . . . . . . . . . 234
12.4.3 State Feedback Solution via Recursive Approach . . . . . . . 236
12.4.4 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 238
12.4.5 Innite-Time Solution . . . . . . . . . . . . . . . . . . . . . . 239
12.5 State Feedback Solution, Minimum-Time Control . . . . . . . . . . . 240
12.6 Comparison of the Design Approaches and Controllers . . . . . . . . 242
13 Receding Horizon Control 249
13.1 RHC Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . 250
13.2 RHC Main Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257
13.2.1 Feasibility of RHC . . . . . . . . . . . . . . . . . . . . . . . . 258
13.2.2 Stability of RHC . . . . . . . . . . . . . . . . . . . . . . . . . 260
13.3 State Feedback Solution of RHC, 2-Norm Case . . . . . . . . . . . . 263
13.4 State Feedback Solution of RHC, 1, -Norm Case . . . . . . . . . . 266
13.5 Tuning and Practical Use . . . . . . . . . . . . . . . . . . . . . . . . 268
13.6 Oset-Free Reference Tracking . . . . . . . . . . . . . . . . . . . . . 273
14 Approximate Receding Horizon Control 283
14.1 Stability of Approximate Receding Horizon Control . . . . . . . . . . 284
14.2 Barycentric Interpolation . . . . . . . . . . . . . . . . . . . . . . . . 287
14.2.1 Feasibility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 287
14.2.2 Suboptimality . . . . . . . . . . . . . . . . . . . . . . . . . . . 288
14.2.3 Constructive Algorithm . . . . . . . . . . . . . . . . . . . . . 291
14.3 Partitioning and Interpolation Methods . . . . . . . . . . . . . . . . 292
14.3.1 Triangulation . . . . . . . . . . . . . . . . . . . . . . . . . . . 292
14.3.2 Outer Polyhedral Approximation . . . . . . . . . . . . . . . . 293
Implicit Double Description Algorithm: Computing an Outer
Polyhedral Approximation of a Convex Set . . . . . 293
Polyhedral Approximation Approximate Control Law . . . 296
Barycentric Functions for Polytopes . . . . . . . . . . . . . . 296
14.3.3 Second-Order Interpolants . . . . . . . . . . . . . . . . . . . . 297
15 On-line Control Computation 309
15.1 Storage and On-line Evaluation of the PWA Control Law . . . . . . 309
15.1.1 Ecient Implementation, 1, -Norm Case . . . . . . . . . . . 311
15.1.2 Ecient Implementation, 2-Norm Case . . . . . . . . . . . . 312
Generating a PWA descriptor function from the value function317
Generating a PWA descriptor function from the optimal inputs318
15.1.3 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 319
CFTOC based on LP . . . . . . . . . . . . . . . . . . . . . . 319
CFTOC based on QP . . . . . . . . . . . . . . . . . . . . . . 319
15.1.4 Literature Review . . . . . . . . . . . . . . . . . . . . . . . . 320
15.2 Gradient Method Applied to MPC . . . . . . . . . . . . . . . . . . . 320
15.3 Interior Point Method Applied to MPC . . . . . . . . . . . . . . . . 323
xii Contents
16 Constrained Robust Optimal Control 325
16.1.1 State Feedback Solutions Summary . . . . . . . . . . . . . . . 331
16.2 Feasible Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 332
16.3 State Feedback Solution, Nominal Cost . . . . . . . . . . . . . . . . 338
16.4 State Feedback Solution, Worst-Case Cost, 1-Norm and -Norm Case339
16.4.1 Batch Approach: Open-Loop Predictions . . . . . . . . . . . 340
16.4.2 Recursive Approach: Closed-Loop Predictions . . . . . . . . . 340
16.4.3 Solution to CROC-CL and CROC-OL via mp-MILP* . . . . 342
16.5 Parametrizations of the Control Policies . . . . . . . . . . . . . . . . 344
16.6 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 350
16.7 Robust Receding Horizon Control . . . . . . . . . . . . . . . . . . . . 351
V Constrained Optimal Control of Hybrid Systems 355
17 Models of Hybrid Systems 357
17.1 Models of Hybrid Systems . . . . . . . . . . . . . . . . . . . . . . . . 357
17.2 Piecewise Ane Systems . . . . . . . . . . . . . . . . . . . . . . . . . 358
17.2.1 Modeling Discontinuities . . . . . . . . . . . . . . . . . . . . . 360
17.2.2 Binary States, Inputs, and Outputs . . . . . . . . . . . . . . 361
17.3 Discrete Hybrid Automata . . . . . . . . . . . . . . . . . . . . . . . . 364
17.3.1 Switched Ane System (SAS) . . . . . . . . . . . . . . . . . 365
17.3.2 Event Generator (EG) . . . . . . . . . . . . . . . . . . . . . . 365
17.3.3 Boolean Algebra . . . . . . . . . . . . . . . . . . . . . . . . . 366
17.3.4 Finite State Machine (FSM) . . . . . . . . . . . . . . . . . . 367
17.3.5 Mode Selector . . . . . . . . . . . . . . . . . . . . . . . . . . 368
17.3.6 DHA Trajectories . . . . . . . . . . . . . . . . . . . . . . . . . 368
17.4 Logic and Mixed-Integer Inequalities . . . . . . . . . . . . . . . . . . 369
17.4.1 Transformation of Boolean Relations . . . . . . . . . . . . . . 369
17.4.2 Translating DHA Components into Linear Mixed-Integer Re-
lations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 370
17.5 Mixed Logical Dynamical Systems . . . . . . . . . . . . . . . . . . . 371
17.6 Model Equivalence . . . . . . . . . . . . . . . . . . . . . . . . . . . . 373
17.7 The HYSDEL Modeling Language . . . . . . . . . . . . . . . . . . . 373
18 Optimal Control of Hybrid Systems 383
18.2 Properties of the State Feedback Solution, 2-Norm Case . . . . . . . 385
18.3 Properties of the State Feedback Solution, 1, -Norm Case . . . . . 391
18.4 Computation of the Optimal Control Input via Mixed Integer Pro-
gramming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 392
18.4.1 Mixed-Integer Optimization Methods . . . . . . . . . . . . . 396
18.5 State Feedback Solution via Batch Approach . . . . . . . . . . . . . 398
18.6 State Feedback Solution via Recursive Approach . . . . . . . . . . . 398
18.6.1 Preliminaries and Basic Steps . . . . . . . . . . . . . . . . . . 398
Contents xiii
18.6.2 Multiparametric Programming with Multiple Quadratic Func-
tions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 401
18.6.3 Algorithmic Solution of the Bellman Equations . . . . . . . . 403
18.6.4 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 407
18.7 Discontinuous PWA systems . . . . . . . . . . . . . . . . . . . . . . . 408
18.8 Receding Horizon Control . . . . . . . . . . . . . . . . . . . . . . . . 409
18.8.1 Stability and Feasibility Issues . . . . . . . . . . . . . . . . . 410
18.8.2 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 410
Bibliography 421
xiv Contents
Symbols and Acronyms
Logic Operators and Functions
A B A implies B, i.e., if A is true then B must be true
A B A implies B and B implies A, i.e., A is true if and only if (i) B is true
Sets
R (R
+
) Set of (non-negative) real numbers
N (N
+
) Set of (non-negative) integers
R
n
Set of real vectors with n elements
R
nm
Set of real matrices with n rows and m columns
Algebraic Operators and Matrices
[a, b] A closed interval in R
[a b] A row vector
A
Transpose of matrix A
A
1
Inverse of matrix A
A
Generalized Inverse of A, A
= (A
A)
1
A
det(A) Determinant of matrix A

A (_)0 A symmetric positive (semi)denite matrix, x
Ax > ()0, x ,= 0
A (_)0 A symmetric negative (semi)denite matrix, x
Ax < ()0, x ,= 0.
A
i
i-th row of matrix A
x
i
i-th element of vector x
x R
n
, x > 0 (x 0) True i x
i
> 0 (x
i
0) i = 1, . . . , n
x R
n
, x < 0 (x 0) True i x
i
< 0 (x
i
0) i = 1, . . . , n
[x[, x R Absolute value of x
|x| Any vector norm of x
|x|
2
Euclidian norm of vector x R
n
, |x|
2
=
_
n
i=1
[x
i
[
2
|x|
1
Sum of absolute elements of vector x R
n
, |x|
1
=
n
i=1
[x
i
[
|x|
Largest absolute value of the vector x R

n
, |x|
= max
i{1,...,n}
[x
i
[
|S|
Matrix -norm of S C
mn
,i.e., |S|
= max
i{1,...,m}
n
j=1
[s
i,j
[
|S|
1
Matrix 1-norm of S C
mn
, i.e., |S|
1
= max
j{1,...,n}
m
i=1
[s
i,j
[
I Identity matrix
1 Vector of ones, 1 = [1 1 . . . 1]
0 Vector of zeros, 0 = [0 0 . . . 0]
Contents xv
Set Operators and Functions
The empty set
[T[ The cardinality of T, i.e., the number of elements in T
T Q Set intersection T Q = x : x T and x Q
T Q Set union T Q = x : x T or x Q
r{1,...,R}
T
r
Union of R sets T
r
, i.e.
r{1,...,R}
T
r
= x : x T
0
or . . . or x T
R
T
c
Complement of the set T, T
c
= x : x / T
T Q Set dierence T Q = x : x T and x / Q
T Q The set T is a subset of Q, x T x Q
T Q The set T is a strict subset of Q, x T x Q and x (Q T)
T Q The set T is a superset of Q
T Q The set T is a strict superset of Q
T Q Pontryagin dierence T Q = x : x +q T, q Q
T Q Minkowski sum T Q = x +q : x T, q Q
T The boundary of T
int(T) The interior of T, i.e. int(T) = T T
: Such that
f(x) With abuse of notation denotes the value of the function f at x or
the function f, f : x f(x).
We use the notation f : R
n
R
s
to mean that f is a R
s
-valued function on
some subset of R
n
, its domain, which we denote by dom f.
f(x) continuous f : R
n
R continuous for all x R
n
, f(0) = 0 and f(x) > 0 x R
n
0
and positive denite
f(x) 0 f(x) continuous and positive denite
f(x) continuous f : R
n
R continuous for all x R
n
, f(x) 0 x R
n
and positive semi-denite
f(x) _ 0 f(x) continuous and positive semi-denite
xvi Contents
Acronyms
ARE Algebraic Riccati Equation
BMI Bilinear Matrix Inequality
CLQR Constrained Linear Quadratic Regulator
CFTOC Constrained Finite Time Optimal Control
CITOC Constrained Innite Time Optimal Control
DP Dynamic Program(ming)
LMI Linear Matrix Inequality
LP Linear Program(ming)
LQR Linear Quadratic Regulator
LTI Linear Time Invariant
MILP Mixed Integer Linear Program
MIQP Mixed Integer Quadratic Program
MPC Model Predictive Control
mp-LP multi-parametric Linear Program
mp-QP multi-parametric Quadratic Program
PWA Piecewise Ane (See Denition 5.7)
PPWA Piecewise Ane on Polyhedra (See Denition 5.8)
PWP Piecewise Polynomial
PWQ Piecewise Quadratic
QP Quadratic Program(ming)
RHC Receding Horizon Control
rhs right-hand side
SDP Semi Denite Program(ming)
Dynamical Systems
x(k) Measurement/Estimation of state x at time k
x
k
Predicted value of state x at time k, given a measurement x(0)
x
+
Successor of vector x, i.e. if x = x(k) then x
+
= x(k + 1)
T
Minimal robust positively invariant set

O
Maximal robust positively invariant set

c
Maximal robust control invariant set

/
Maximal robust stabilizable set

O
LQR
Maximal positively invariant set O
for LTI system under LQR feedback

Part I
Basics of Optimization
Chapter 1
Main Concepts
In this chapter we recall the main concepts and denitions of continuous and dis-
crete optimization. Our intent is to provide only the necessary background for
the understanding of the rest of the book. The notions of feasibility, optimality,
convexity and active constraints introduced in this chapter will be widely used in
the rest of the book.
1.1 Optimization Problems
An optimization problem is generally formulated as
inf
z
f(z)
subj. to z S Z,
(1.1)
where the vector z collects the decision variables, Z is the optimization problem
domain, S Z is the set of feasible or admissible decisions. The function f : Z R
assigns to each decision z a cost f(z) R. We will often use the following shorter
form of problem (1.1)
inf
zSZ
f(z). (1.2)
Solving problem (1.2) means to compute the least possible cost f
= inf
zS
f(z).
The number f
is the optimal value of problem (1.2), i.e. the greatest lower bound
of f(z) over the set S:
f(z) f
z S and ( > 0 z S : f(z) f
+).
If f
= we say that the problem is unbounded below. If the set S is empty

then the problem is said to be infeasible and we set f
= + by convention. If
S = Z the problem is said to be unconstrained.
4 1 Main Concepts
In general, one is also interested in nding an optimal solution, that is in nding
a decision whose associated cost equals the optimal value, i.e., z
S with f(z
) =
f
. If such z
exists, then we rewrite problem (1.2) as

f
= min
zS
f(z) (1.3)
and z
is called an optimizer or global optimizer or optimal solution. Minimizer or

global minimizer are also used to refer to an optimizer of a minimization problem.
The set of all optimal solutions is denoted by
argmin
zS
f(z) = z S : f(z) = f
.
A problem of determining whether the set of feasible decisions is empty and, if not,
to nd a point which is feasible, is called a feasibility problem.
1.1.1 Continuous Problems
In continuous optimization the problemdomain Z is a subset of the nite-dimensional
Euclidian vector-space R
s
and the subset of admissible vectors is dened through
a list of equality and inequality constraints:
inf
z
f(z)
subj. to g
i
(z) 0 for i = 1, . . . , m
h
j
(z) = 0 for j = 1, . . . , p
z Z,
(1.4)
where f, g
1
, . . . , g
m
, h
1
, . . . , h
p
are real-valued functions dened over R
s
, i.e., f :
R
s
R, g
i
: R
s
R, h
i
: R
s
R. The domain Z is the intersection of the
domains of the cost and constraint functions:
Z = z R
s
: z dom f, z dom g
i
, i = 1, . . . , m, z dom h
j
, j = 1, . . . , p.
(1.5)
In the sequel we will consider the constraint z Z implicit in the optimization
problem and often omit it. Problem (1.4) is unconstrained if m = p = 0.
The inequalities g
i
(z) 0 are called inequality constraints and the equations
h
i
(z) = 0 are called equality constraints. A point z R
s
is feasible for problem (1.4)
if: (i) it belongs to Z, (ii) it satises all inequality and equality constraints, i.e.,
g
i
( z) 0, i = 1, . . . , m, h
j
( z) = 0, i = j, . . . , p. The set of feasible vectors is
S = z R
s
: z Z, g
i
(z) 0, i = 1, . . . , m, h
j
(z) = 0, j = 1, . . . , p. (1.6)
Problem (1.4) is a continuous nite-dimensional optimization problem (since
Z is a nite-dimensional Euclidian vector space). We will also refer to (1.4) as
a nonlinear mathematical program or simply nonlinear program. Let f
be the
optimal value of problem (1.4). An optimizer, if it exists, is a feasible vector z
with f(z
) = f
.
1.1 Optimization Problems 5
A feasible point z is locally optimal for problem (1.4) if there exists an R > 0
such that
f( z) = inf
z
f(z)
subj. to g
i
(z) 0 for i = 1, . . . , m
h
i
(z) = 0 for i = 1, . . . , p
|z z| R
z Z.
(1.7)
Roughly speaking, this means that z is the minimizer of f(z) in a feasible neigh-
borhood of z dened by |z z| R. The point z is called a local optimizer or
local minimizer.
Active, Inactive and Redundant Constraints
Consider a feasible point z. We say that the i-th inequality constraint g
i
(z) 0 is
active at z if g
i
( z) = 0. If g
i
( z) < 0 we say that the constraint g
i
(z) 0 is inactive
at z. Equality constraints are always active for all feasible points.
We say that a constraint is redundant if removing it from the list of constraints
does not change the feasible set S. This implies that removing a redundant con-
straint from problem (1.4) does not change its solution.
Problems in Standard Forms
Optimization problems can be cast in several forms. In this book we use the
form (1.4) where we adopt the convention to minimize the cost function and to
have the right-hand side of the inequality and equality constraints equal to zero.
Any problem in a dierent form (e.g., a maximization problem or a problem with
box constraints) can be transformed and arranged into this form. The interested
reader is referred to Chapter 4 of [64] for a detailed discussion on transformations
of optimization problems into dierent standard forms.
Eliminating Equality Constraints
Often in this book we will restrict our attention to problems without equality
constraints, i.e., p = 0
inf
z
f(z)
subj. to g
i
(z) 0 for i = 1, . . . , m.
(1.8)
The simplest way to remove equality constraints is to replace them with two in-
equalities for each equality, i.e., h
i
(z) = 0 is replaced by h
i
(z) 0 and h
i
(z) 0.
Such a method, however, can lead to poor numerical conditioning and may ruin
the eciency and accuracy of a numerical solver.
If one can explicitly parameterize the solution of the equality constraint h
i
(z) =
0, then the equality constraint can be eliminated from the problem. This process
can be described in a simple way for linear equality constraints. Assume the equal-
ity constraints to be linear, Az b = 0, with A R
ps
. If Az = b is inconsistent
then the problem is infeasible. The general solution of the equation Az = b can be
expressed as z = Fx + z
0
where F is a matrix of full rank whose spanned space
coincides with the null space of the A matrix, i.e., 1(F) = A(A), F R
sk
,
6 1 Main Concepts
where k is the dimension of the null space of A. The variable x R
k
is the new
optimization variable and the original problem becomes
inf
x
f(Fx +z
0
)
subj. to g
i
(Fx +z
0
) 0 for i = 1, . . . , m.
(1.9)
We want to point out that in some cases the elimination of equality constraints
can make the problem harder to analyze and understand and can make a solver
less ecient. In large problems it can destroy useful structural properties of the
problem such as sparsity. Some advanced numerical solvers perform elimination
automatically.
Problem Description
The functions f, g
i
and h
i
can be available in analytical form or can be described
through an oracle model (also called black box or subroutine model). In an
oracle model f, g
i
and h
i
are not known explicitly but can be evaluated by querying
the oracle. Often the oracle consists of subroutines which, called with the argument
z, return f(z), g
i
(z) and h
i
(z) and their gradients f(z), g
i
(z), h
i
(z). In
the rest of the book we assume that analytical expressions of the cost and the
constraints of the optimization problem are available.
1.1.2 Integer and Mixed-Integer Problems
If the decision set Z in the optimization problem (1.2) is nite, then the optimiza-
tion problem is called combinatorial or discrete. If Z 0, 1
s
, then the problem
is said to be integer.
If Z is a subset of the Cartesian product of an integer set and a real Euclidian
space, i.e., Z [z
c
, z
b
] : z
c
R
sc
, z
b
0, 1
s
b
, then the problem is said to be
mixed-integer. The standard formulation of a mixed-integer nonlinear program is
inf
[zc,z
b
]
f(z
c
, z
b
)
subj. to g
i
(z
c
, z
b
) 0 for i = 1, . . . , m
h
j
(z
c
, z
b
) = 0 for j = 1, . . . , p
z
c
R
sc
, z
b
0, 1
s
b
[z
c
, z
b
] Z,
(1.10)
where f, g
1
, . . . , g
m
, h
1
, . . . , h
p
are real-valued functions dened over Z.
For combinatorial, integer and mixed-integer optimization problems all deni-
tions introduced in the previous section apply.
1.2 Convexity
A set S R
s
is convex if
z
1
+ (1 )z
2
S for all z
1
S, z
2
S and [0, 1].
1.2 Convexity 7
A function f : S R is convex if S is convex and
f(z
1
+ (1 )z
2
) f(z
1
) + (1 )f(z
2
)
for all z
1
S, z
2
S and [0, 1].
A function f : S R is strictly convex if S is convex and
f(z
1
+ (1 )z
2
) < f(z
1
) + (1 )f(z
2
)
for all z
1
S, z
2
S and (0, 1).
A twice dierentiable function f : S R is strongly convex if the Hessian
2
f(z) 0 for all z S.
A function f : S R is concave if S is convex and f is convex.
Operations preserving convexity
Various operations preserve convexity of functions and sets. A detailed list can be
found in Chapter 3.2 of [64]. A few operations used in this book are mentioned
below.
1. The intersection of an arbitrary number of convex sets is a convex set:
if S
1
, S
2
, . . . , S
k
are convex, then S
1
S
2
. . . S
k
is convex.
This property extends to the intersection of an innite number of sets:
if S
n
is convex n N
+
then
nN+
S
n
is convex.
The empty set is convex because it satises the denition of convexity.
2. The sub-level sets of a convex function f on S are convex:
if f(z) is convex then S
= z S : f(z) is convex R.
3. If f
1
, . . . , f
N
are convex functions, then
N
i=1

i
f
i
is a convex function for all
i
0, i = 1, . . . , N.
4. The composition of a convex function f(z) with an ane map z = Ax + b
generates a convex function f(Ax +b) of x:
if f(z) is convex then f(Ax +b) is convex on x : Ax +b dom(f)
5. Suppose f(x) = h(g(x)) = h(g
1
(x), . . . , g
k
(x)) with h : R
k
R, g
i
: R
s
R.
Then,
(a) f is convex if h is convex, h is nondecreasing in each argument, and g
i
are convex,
8 1 Main Concepts
(b) f is convex if h is convex, h is nonincreasing in each argument, and g
i
are concave,
(c) f is concave if h is concave, h is nondecreasing in each argument, and
g
i
are concave.
6. The pointwise maximum of a set of convex functions is a convex function:
f
1
(z), . . . , f
k
(z) convex functions f(z) = maxf
1
(z), . . . , f
k
(z) is a convex function.
The property holds also when the set is innite.
Linear and quadratic convex functions
1. A linear function f(z) = c
z +r is both convex and concave.

2. A quadratic function f(z) = z
Hz + 2q
z +r is convex if and only if H _ 0.

3. A quadratic function f(z) = z
Hz + 2q
z +r is strictly convex if and only if

H 0. A strictly convex quadratic function is also strongly convex.
Convex optimization problems
The standard optimization problem (1.4) is said to be convex if the cost function f
is convex on Z and S is a convex set. A fundamental property of convex optimiza-
tion problems is that local optimizers are also global optimizers. This is proven
next.
Theorem 1.1 Consider a convex optimization problem and let z be a local opti-
mizer. Then, z is a global optimizer.
Proof: By hypothesis z is feasible and there exists R such that
f( z) = inff(z) : g
i
(z) 0 i = 1, . . . , m, h
j
(z) = 0, j = 1, . . . , p |z z| R.
(1.11)
Now suppose that z is not globally optimal. Then, there exist a feasible y such
that f(y) < f( z), which implies that |y z| > R. Now consider the point z given
by
z = (1 ) z +y, =
R
2|y z|
.
Then |z z| = R/2 < R and by convexity of the feasible set z is feasible. By
convexity of the cost function f
f(z) (1 )f( z) +f(y) < f( z),
which contradicts (1.11).
Theorem 1.1 does not make any statement about the existence of a solution to
problem (1.4). It merely states that all local minima of problem (1.4) are also global
minima. For this reason, convexity plays a central role in the solution of continuous
optimization problems. It suces to compute a local minimum to problem (1.4)
1.2 Convexity 9
to determine its global minimum. Convexity also plays a major role in most non-
convex optimization problems which are solved by iterating between the solutions
of convex sub-problems.
It is dicult to determine whether the feasible set S of the optimization prob-
lem (1.4) is convex or not except in special cases. For example, if the functions
g
1
(z), . . . , g
m
(z) are convex and all the h
i
(z) (if any) are ane in z, then the feasible
region S in (1.6) is an intersection of convex sets and therefore convex. Moreover
there are non-convex problems which can be transformed into convex problems
through a change of variables and manipulations of cost and constraints. The dis-
cussion of this topic goes beyond the scope of this overview on optimization. The
interested reader is referred to [64].
Remark 1.1 With the exception of trivial cases, integer and mixed-integer optimiza-
tion problems are always non-convex problems because |0, 1 is not a convex set.
10 1 Main Concepts
Chapter 2
Optimality Conditions
In general, an analytical solution to problem (1.4), restated below, does not exist.
inf
z
f(z)
subj. to g
i
(z) 0 for i = 1, . . . , m
h
j
(z) = 0 for j = 1, . . . , p
z Z.
(2.1)
Solutions are usually computed by iterative algorithms which start from an initial
guess z
0
and at step k generate a point z
k
such that the sequence f(z
k
)
k=0,1,2,...
converges to f
as k increases. These algorithms iteratively use and/or solve condi-

tions for optimality, i.e., analytical conditions that a point z must satisfy in order
to be an optimizer. For instance, for convex, unconstrained optimization prob-
lems with a smooth cost function the most commonly used optimality criterion
requires the gradient to vanish at the optimizer, i.e., z is an optimizer if and only
if f(z) = 0. In this chapter we summarize necessary and sucient optimality
conditions for unconstrained and constrained optimization problems.
2.1 Optimality Conditions for Unconstrained Problems
The proofs of the theorems presented next can be found in Chapter 4 and Section
8.6.1 of [26].
Necessary conditions
Theorem 2.1 Suppose that f : R
s
R is dierentiable at z. If there exists a
vector d such that f( z)
d < 0, then there exists a > 0 such that f( z+d) <

f( z) for all (0, ).
The vector d in the theorem above is called a descent direction. At a given
point z a descent direction d satises the condition f( z)
d < 0. Theorem 2.1

states that if a descent direction exists at a point z, then it is possible to move
from z towards a new point z whose associated cost f( z) is lower than f( z).
The smaller f( z)
d is the smaller will be the cost at the new point f( z). The
12 2 Optimality Conditions
direction of steepest descent d
s
at a given point z is dened as the normalized
direction where f( z)
d
s
< 0 is minimized. The direction d
s
of steepest
descent is d
s
=
f( z)
f( z)
.
Two corollaries of Theorem 2.1 are stated next.
Corollary 2.1 Suppose that f : R
s
R is dierentiable at z. If z is a local
minimizer, then f( z) = 0.
Corollary 2.2 Suppose that f : R
s
R is twice dierentiable at z. If z
is a local minimizer, then f( z) = 0 and the Hessian
2
f( z) is positive
semidenite.
Sucient condition
s
R is twice dierentiable at z. If
f( z) = 0 and the Hessian of f(z) at z is positive denite, then z is a
local minimizer.
Necessary and sucient condition
s
R is dierentiable at z. If f is convex,
then z is a global minimizer if and only if f( z) = 0.
When the optimization is constrained and the cost function is not suciently
smooth, the conditions for optimality become more complicated. The intent of this
chapter is to give an overview of some important optimality criteria for constrained
nonlinear optimization. The optimality conditions derived here will be the main
building blocks for the theory developed later in this book.
2.2 Lagrange Duality Theory
Consider the nonlinear program (2.1). Let f
be the optimal value. Denote by Z

the domain of cost and constraints (1.5). Any feasible point z provides an upper
bound to the optimal value f( z) f
. Next we will show how to generate a lower

bound on f
.
Starting from the standard nonlinear program (2.1) we construct another prob-
lem with dierent variables and constraints. The original problem (2.1) will be
called the primal problem while the new one will be called the dual problem. First,
we augment the objective function with a weighted sum of the constraints. In this
way the Lagrange dual function (or Lagrangian) L is obtained
L(z, u, v) = f(z) +u
1
g
1
(z) + +u
m
g
m
(z) +
+v
1
h
1
(z) + +v
p
h
p
(z), (2.2)
where the scalars u
1
, . . . , u
m
, v
1
, . . . , v
p
are real variables called dual variables or
Lagrange multipliers. We can write equation (2.2) in the compact form
L(z, u, v) = f(z) +u
g(z) +v
h(z), (2.3)
2.2 Lagrange Duality Theory 13
where u = [u
1
, . . . , u
m
]
, v = [v
1
, . . . , v
p
]
and L : R
s
R
m
R
p
R. The
components u
i
and v
i
are called dual variables. Note that the i-th dual variable
u
i
is associated with the i-th inequality constraint of problem (2.1), the i-th dual
variable v
i
is associated with the i-th equality constraint of problem (2.1).
Let z be a feasible point: for arbitrary vectors u 0 and v we trivially obtain
a lower bound on f(z)
L(z, u, v) f(z). (2.4)
We minimize both sides of equation (2.4)
inf
zZ, g(z)0, h(z)=0
L(z, u, v) inf
zZ, g(z)0, h(z)=0
f(z) (2.5)
in order to reconstruct the original problem on the right-hand side of the expression.
Since for arbitrary u 0 and v
inf
zZ, g(z)0, h(z)=0
L(z, u, v) inf
zZ
L(z, u, v), (2.6)
we obtain
inf
zZ
L(z, u, v) inf
zZ, g(z)0, h(z)=0
f(z). (2.7)
Equation (2.7) implies that for arbitrary u 0 and v the solution to
inf
zZ
L(z, u, v) (2.8)
provides us with a lower bound to the original problem. The best lower bound
is obtained by maximizing problem (2.8) over the dual variables
sup
(u,v), u0
inf
zZ
L(z, u, v) inf
zZ, g(z)0, h(z)=0
f(z).
Dene the dual cost d(u, v) as follows
d(u, v) = inf
zZ
L(z, u, v) [, +]. (2.9)
Then the Lagrange dual problem is dened as
sup
(u,v), u0
d(u, v), (2.10)
and its optimal solution is denoted by (u
, v
). The dual cost d(u, v) is the opti-

mal value of an unconstrained optimization problem. Problem (2.9) is called the
Lagrange dual subproblem. Only points (u, v) with d(u, v) > are interesting
for the Lagrange dual problem. A point (u, v) will be called dual feasible if u 0
and d(u, v) > . d(u, v) is always a concave function since it is the pointwise
inmum of a family of ane functions of (u, v). This implies that the dual problem
is a convex optimization problem (max of a concave function over a convex set)
even if the original problem is not convex. Therefore, it is easier in principle to
solve the dual problem than the primal (which is in general non-convex). However,
in general, the solution to the dual problem is only a lower bound of the primal
problem:
sup
(u,v), u0
d(u, v) inf
zZ, g(z)0, h(z)=0
f(z).
Such a property is called weak duality. In a simpler form, let f
and d
be the
primal and dual optimal value, respectively,
f
= inf
zZ, g(z)0, h(z)=0
f(z), (2.11a)
d
= sup
(u,v), u0
d(u, v), (2.11b)
then, we always have
f
(2.12)
and the dierence f
is called the optimal duality gap. The weak duality

inequality (2.12) holds also when d
and f
are innite. For example, if the primal

problem is unbounded below, so that f
= , we must have d
= , i.e., the
Lagrange dual problem is infeasible. Conversely, if the dual problem is unbounded
above, so that d
= +, we must have f
= +, i.e., the primal problem is

infeasible.
Remark 2.1 We have shown that the dual problem is a convex optimization problem
even if the original problem is non-convex. As stated in this section, for non-convex
optimization problems it is easier to solve the dual problem than the primal prob-
lem. However the evaluation of d( u, v) at a point ( u, v) requires the solution of the
non-convex optimization problem (2.9), which, in general, is not easy.
2.2.1 Strong Duality and Constraint Qualications
If d
= f
, then the duality gap is zero and we say that strong duality holds:
sup
(u,v), u0
d(u, v) = inf
zZ, g(z)0, h(z)=0
f(z). (2.13)
This means that the best lower bound obtained by solving the dual problem coin-
cides with the optimal cost of the primal problem. In general, strong duality does
not hold, even for convex primal problems. Constraint qualications are condi-
tions on the constraint functions which imply strong duality for convex problems.
A detailed discussion on constraint qualications can be found in Chapter 5 of [26].
A well known simple constraint qualication is Slaters condition:
Denition 2.1 (Slaters condition) Consider problem (2.1). There exists z
R
s
which belongs to the relative interior of the problem domain Z, which is feasible
(g( z) 0, h( z) = 0) and for which g
j
( z) < 0 for all j for which g
j
is not an ane
function.
Remark 2.2 Note that Slaters condition reduces to feasibility when all inequality
constraints are linear and Z = R
n
.
2.3 Complementary Slackness 15
Theorem 2.4 (Slaters theorem) Consider the primal problem (2.11a) and its
dual problem (2.11b). If the primal problem is convex, Slaters condition holds and
f
is bounded then d
= f
.
2.2.2 Certicate of Optimality
Consider the (primal) optimization problem (2.1) and its dual (2.10). Any feasible
point z gives us information about an upper bound on the cost, i.e., f
f(z).
If we can nd a dual feasible point (u, v) then we can establish a lower bound
on the optimal value of the primal problem: d(u, v) f
. In summary, without
knowing the exact value of f
we can give a bound on how suboptimal a given

feasible point is. In fact, if z is primal feasible and (u, v) is dual feasible then
d(u, v) f
f(z). Therefore z is -suboptimal, with equal to the primal-dual

gap, i.e., = f(z) d(u, v).
The optimal value of the primal (and dual) problems will lie in the same interval
f
[d(u, v), f(z)] and d
[d(u, v), f(z)].

For this reason (u, v) is also called a certicate that proves the (sub)optimality
of z. Optimization algorithms make extensive use of such criteria. Primal-dual
algorithms iteratively solve primal and dual problems and generate a sequence of
primal and dual feasible points z
k
, (u
k
, v
k
), k 0 until a certain is reached. The
condition
f(z
k
) d(u
k
, v
k
) < ,
for terminating the algorithm guarantees that when the algorithm terminates, z
k
is -suboptimal. If strong duality holds the condition can be met for arbitrarily
small tolerances .
2.3 Complementary Slackness
Consider the (primal) optimization problem (2.1) and its dual (2.10). Assume that
strong duality holds. Suppose that z
and (u
, v
) are primal and dual feasible

with zero duality gap (hence, they are primal and dual optimal):
f(z
) = d(u
, v
).
By denition of the dual problem, we have
f(z
) = inf
zZ
_
f(z) +u
g(z) +v
h(z)
_
.
Therefore
f(z
) f(z
) +u
g(z
) +v
h(z
), (2.14)
and since h(z
) = 0, u
0 and g(z
) 0 we have
f(z
) f(z
) +u
g(z
) f(z
). (2.15)
From the last equation we can conclude that u
g(z
) =
m
i=1
u
i
g
i
(z
) = 0 and
since u
i
0 and g
i
(z
) 0, we have
u
i
g
i
(z
) = 0, i = 1, . . . , m. (2.16)
Conditions (2.16) are called complementary slackness conditions. Complemen-
tary slackness conditions can be interpreted as follows. If the i-th inequality con-
straint of the primal problem is inactive at the optimum (g
i
(z
) < 0), then the

i-th dual optimizer has to be zero (u
i
= 0). Vice versa, if the i-th dual optimizer
is dierent from zero (u
i
> 0), then the i-th constraint is active at the optimum
(g
i
(z
) = 0).
Relation (2.16) implies that the inequality in (2.14) holds as equality
f(z
) +
i
u
i
g
i
(z
) +
j
v
j
h
j
(z
) = min
zZ
_
_
f(z) +
i
u
i
g
i
(z) +
j
v
j
h
j
(z)
_
_
.
(2.17)
Therefore, complementary slackness conditions implies that z
is a minimizer of
L(z, u
, v
).
2.4 Karush-Kuhn-Tucker Conditions
Consider the (primal) optimization problem (2.1) and its dual (2.10). Assume that
strong duality holds. Assume that the cost functions and constraint functions f, g
i
,
h
i
are dierentiable. Let z
and (u
, v
) be primal and dual optimal points, respec-

tively. Complementary slackness conditions implies that z
minimizes L(z, u
, v
)
under no constraints (equation (2.17)). Since f, g
i
, h
i
are dierentiable, the gra-
dient of L(z, u
, v
) must be zero at z
f(z
) +
i
u
i
g
i
(z
) +
j
v
j
h
j
(z
) = 0.
In summary, the primal and dual optimal pair z
, (u
, v
) of an optimization
problem with dierentiable cost and constraints and zero duality gap, have to
satisfy the following conditions:
f(z
) +
m
i=1
u
i
g
i
(z
) +
p
j=1
v
j
h
i
(z
) = 0, (2.18a)
u
i
g
i
(z
) = 0, i = 1, . . . , m (2.18b)
u
i
0, i = 1, . . . , m (2.18c)
g
i
(z
) 0, i = 1, . . . , m (2.18d)
h
j
(z
) = 0, j = 1, . . . , p (2.18e)
2.4 Karush-Kuhn-Tucker Conditions 17
where equations (2.18d)-(2.18e) are the primal feasibility conditions, equation (2.18c)
is the dual feasibility condition and equation (2.18b) are the complementary slack-
ness conditions.
Conditions (2.18a)-(2.18e) are called the Karush-Kuhn-Tucker (KKT) condi-
tions. We have shown that the KKT conditions are necessary conditions for any
primal-dual optimal pair if strong duality holds and the cost and constraints are
dierentiable, i.e., any primal and dual optimal points z
, (u
, v
) must satisfy
the KKT conditions (2.18). If the primal problem is also convex then the KKT
conditions are sucient, i.e., a primal dual pair z
, (u
, v
) which satises condi-

tions (2.18a)-(2.18e) is a primal dual optimal pair with zero duality gap.
There are several theorems which characterize primal and dual optimal points
z
and (u
, v
) by using KKT conditions. They mainly dier on the type of

constraint qualication chosen for characterizing strong duality. Next we report
just two examples.
If a convex optimization problem with dierentiable objective and constraint
functions satises Slaters condition, then the KKT conditions provide necessary
and sucient conditions for optimality.
Theorem 2.5 [26, p. 244] Consider problem (2.1) and let Z be a nonempty set of
R
s
. Suppose that problem (2.1) is convex and that cost and constraints f, g
i
and
h
i
are dierentiable at a feasible z
. If problem (2.1) satises Slaters condition

then z
is optimal if and only if there are (u
, v
) that, together with z
, satisfy
the KKT conditions (2.18).
If a convex optimization problem with dierentiable objective and constraint func-
tions has a linearly independent set of active constraints gradients, then the KKT
conditions provide necessary and sucient conditions for optimality.
Theorem 2.6 (Section 4.3.7 in [26]) Consider problem (2.1) and let Z be a
nonempty open set of R
s
. Let z
be a feasible solution and A = i : g

i
(z
) = 0
be the set of active constraints at z
. Suppose cost and constraints f, g

i
are dier-
entiable at z
for all i and that h

j
are continuously dierentiable at z
for all j.
Further, suppose that g
i
(z
) for i A and h
j
(z
) for j = 1, . . . , p, are linearly

independent. If z
, (u
, v
) are primal and dual optimal points, then they satisfy

the KKT conditions (2.18). In addition, if problem (2.1) is convex, then z
is
optimal if and only if there are (u
, v
) that, together with z
, satisfy the KKT

conditions (2.18).
The KKT conditions play an important role in optimization. In a few special cases
it is possible to solve the KKT conditions (and therefore, the optimization problem)
analytically. Many algorithms for convex optimization are conceived as, or can be
interpreted as, methods for solving the KKT conditions as Boyd and Vandenberghe
observe in [64].
The following example [26] shows a convex problem where the KKT conditions
are not fullled at the optimum. In particular, both the constraint qualications
of Theorem 2.6 and Slaters condition in Theorem 2.5 are violated.
z
2
z
1
(1, 1)
(1, 1)
g
2
(1, 0)
g
1
(1, 0)
f(1, 0)
Figure 2.1 Constraints, feasible set and gradients of Example 2.1.
Example 2.1 [26] Consider the convex optimization problem
min z1
subj. to (z1 1)
2
+ (z2 1)
2
1
(z1 1)
2
+ (z2 + 1)
2
1
(2.19)
From the graphical interpretation in Figure 2.1 it is immediate that the feasible set is a
single point z = [1, 0]
. The optimization problem does not satisfy Slaters conditions

and moreover z does not satisfy the constraint qualications in Theorem 2.6. At
the optimum z Equation (2.18a) cannot be satised for any pair of nonnegative real
numbers u1 and u2.
2.4.1 Geometric Interpretation of KKT Conditions
A geometric interpretation of the KKT conditions is depicted in Figure 2.2 for an
optimization problem in two dimensions with inequality constraints and no equality
constraints.
Equation (2.18a) and Equation (2.18c) can be rewritten as
f(z) =
iA
u
i
g
i
(z), u
i
0, (2.20)
where A = 1, 2 at z
1
and A = 2, 3 at z
2
. This means that the negative gradient
of the cost at the optimum f(z
) (which represents the direction of steepest

2.4 Karush-Kuhn-Tucker Conditions 19
g
1
(z
1
)
f(z
1
)
g
2
(z
1
)
f(z
2
)
g
2
(z
2
)
g
3
(z
2
)
z
1
z
2
g
1
(
z
)
0
g
2
(
z
)
0
g(z) 0
g
3
(
z
)
0
Figure 2.2 Graphical interpretation of KKT conditions [26].
descent) has to belong to the cone spanned by the gradients of the active constraints
g
i
, (since inactive constraints have the corresponding Lagrange multipliers equal
to zero). In Figure 2.2, condition (2.20) is not satised at z
2
. In fact, one can
move within the set of feasible points g(z) 0 and decrease f, which implies that
z
2
is not optimal. At point z
1
, on the other hand, the cost f can only decrease
if some constraint is violated. Any movement in a feasible direction increases the
cost. Conditions (2.20) are fullled and hence z
1
is optimal.
Chapter 3
Linear and Quadratic
Optimization
This chapter focuses on two widely known and used subclasses of convex optimiza-
tion problems: linear and quadratic programs. Their success is due to the great
capability of modeling and solving eciently a large variety of important practical
problems. Also, they are the basic building blocks of many other optimization algo-
rithms. This chapter presents their formulation together with their main properties
and some fundamental results.
3.1 Polyhedra and Polytopes
We rst introduce a few concepts needed for the geometric interpretation of linear
an quadratic optimization. They will be discussed in more detail in Section 5.2.
A polyhedron T in R
n
denotes an intersection of a nite set of closed halfspaces
in R
n
:
T = x R
n
: Ax b, (3.1)
where Ax b is the usual shorthand form for a system of inequalities, namely
a
i
x b
i
, i = 1, . . . , m, where a
1
, . . . , a
m
are the rows of A, and b
1
, . . . , b
m
are
the components of b. A polytope is a bounded polyhedron. In Figure 3.1 a two-
dimensional polytope is plotted.
A linear inequality c
z c
0
is said to be valid for T if it is satised for all points
z T. A face of T is any nonempty set of the form
T = T z R
s
: c
z = c
0
, (3.2)
where c
z c
0
is a valid inequality for T. All faces of T satisfying T T are called
proper faces and have dimension less than dim(T). The faces of dimension 0,1,
dim(T)-2 and dim(T)-1 are called vertices, edges, ridges, and facets, respectively.
22 3 Linear and Quadratic Optimization
a
1
x
b
1
a
2
x
b
2
a
3 x
b
3
a
4
x
b
4
Figure 3.1 Polytope. The planes (here lines) dening the halfspaces are
a
i
x bi = 0.
3.2 Linear Programming
When the cost and the constraints of the continuous optimization problem (1.4)
are ane, then the problem is called a linear program (LP). The most general form
of a linear program is
inf
z
c
z
subj. to Gz w,
(3.3)
where G R
ms
, w R
m
. Linear programs are convex optimization problems.
Two other common forms of linear programs include both equality and inequal-
ity constraints:
inf
z
c
z
subj. to Gz w
Az = b,
(3.4)
where A R
ps
, b R
p
, or only equality constraints and positive variables:
inf
z
c
z
subj. to Az = b
z 0.
(3.5)
By standard and simple manipulations [64, p. 146] it is always possible to convert
one of the three forms (3.3), (3.4) and (3.5) into the other.
3.2.1 Graphical Interpretation and Solution Properties
Let T be the feasible set (1.6) of problem (3.3). As Z = R
s
, this implies that T
is a polyhedron dened by the inequality constraints in (3.3). If T is empty, then
the problem is infeasible. We will assume for the following discussion that T is
3.2 Linear Programming 23
k
1
k
2
k
3
k
4
T
(a) Case 1.
T
z
k
1
k
2
k
3
k
4
(b) Case 2.
T
z
k
1
k
2
k
3
k
4
(c) Case 3.
Figure 3.2 Linear Program Graphical Interpretation, ki < ki1.
not empty. Denote by f
the optimal value and by Z
the set of optimizers of

problem (3.3)
Z
= argmin
zP
c
z.
Three cases can occur.
Case 1. The LP solution is unbounded, i.e., f
= .
Case 2. The LP solution is bounded, i.e., f
> and the optimizer is unique.

z
= Z
is a singleton.
Case 3. The LP solution is bounded and there are multiple optima. Z
is an
uncountable subset of R
s
which can be bounded or unbounded.
The two-dimensional geometric interpretation of the three cases discussed above
is depicted in Figure 3.2.
The level curves of the cost function c
z are represented by the parallel lines.

All points z belonging both to the line c
z = k
i
and to the polyhedron T are feasible
points with an associated cost k
i
, with k
i
< k
i1
. Solving (3.3) amounts to nding
a feasible z which belongs to the level curve with the smallest cost k
i
. Since the
gradient of the cost is c, the direction of steepest descent is c/|c|.
Case 1 is depicted in Figure 3.2(a). The feasible set T is unbounded. One can
move in the direction of steepest descent c and be always feasible, thus decreasing
the cost to . Case 2 is depicted in Figure 3.2(b). The optimizer is unique and
it coincides with one of the vertices of the feasible polyhedron. Case 3 is depicted
in Figure 3.2(c). The whole facet of the feasible polyhedron T is optimal, i.e., the
cost for any point z belonging to the facet equals the optimal value f
. In general,
the optimal facet will be a facet of the polyhedron T parallel to the hyperplane
c
z = 0.
From the analysis above we can conclude that the optimizers of any bounded
LP always lie on the boundary of the feasible polyhedron T.
3.2.2 Dual of LP
Consider the LP (3.3)
inf
z
c
z
subj. to Gz w,
(3.6)
with z R
s
and G R
ms
.
The Lagrange function as dened in (2.3) is
L(z, u) = c
z +u
(Gz w).
The dual cost is
d(u) = inf
z
L(z, u) = inf
z
(c
+u
G)z u
w =
_
u
w if G
u = c
if G
u ,= c.
Since we are interested only in cases where d is nite, from the relation above we
conclude that the dual problem is
sup
u
u
w
subj. to G
u = c
u 0,
(3.7)
which can be rewritten as
inf
u
w
u
subj. to G
u = c
u 0.
(3.8)
Note that for LPs feasibility implies strong duality (Remark 2.2).
3.2.3 KKT condition for LP
The KKT conditions (2.18a)-(2.18e) for the LP (3.3) become
G
u +c = 0, (3.9a)
u
i
(G
i
z w
i
) = 0, i = 1, . . . , m (3.9b)
u 0, (3.9c)
Gz w 0. (3.9d)
They are: stationarity condition (3.9a), complementary slackness conditions (3.9b),
dual feasibility (3.9c) and primal feasibility (3.9d). Often dual feasibility in linear
programs refers to both (3.9a) and (3.9c).
3.2.4 Active Constraints and Degeneracies
Consider the LP (3.3). Let I = 1, . . . , m be the set of constraint indices. For any
A I, let G
A
and w
A
be the submatrices of G and w, respectively, comprising
the rows indexed by A and denote with G
j
and w
j
the j-th row of G and w,
respectively. Let z be a feasible point and consider the set of active and inactive
constraints at z:
A(z) = i I : G
i
z = w
i
NA(z) = i I : G
i
z < w
i
.
(3.10)
From (3.10) we have
G
A(z
)
z
= w
A(z
)
G
NA(z
)
z
< w
NA(z
)
.
(3.11)
Denition 3.1 (Linear Independence Constraint Qualication (LICQ)) We
say that LICQ holds at z
if the matrix G
A(z
)
has full row rank.
Lemma 3.1 Assume that the feasible set T of problem (3.3) is bounded. If the
LICQ is violated at z
1
Z
then there exists z
2
Z
such that [A(z
2
)[ > s.
Consider z
1
Z
on an optimal facet. Lemma 3.1 states the simple fact that if

LICQ is violated at z
1
, then there is a vertex z
2
Z
on the same facet where

[A(z
2
)[ > s. Thus, violation of LICQ is equivalent to having more than s constraints
active at an optimal vertex.
Denition 3.2 The LP (3.3) is said to be primal degenerate if there exists a z
such that the LICQ does not hold at z
.
Figure 3.3 depicts a case of primal degeneracy with three constraints active at the
optimal vertex, i.e. more than the minimum number two.
1
2
3
4
5
6
z
T
Figure 3.3 Primal Degeneracy in a Linear Program.
Denition 3.3 The LP (3.3) is said to be dual degenerate if its dual problem is
primal degenerate.
The LICQ condition is invoked so that the optimization problem is well behaved
in a way we will explain next. Let us look at the equality constraints of the
dual problem (3.8) at the optimum G
= c or equivalently G
A
u
A
= c since
u
NA
= 0. If LICQ is satised then the equality constraints will allow only a unique
optimizer u
. If LICQ is not satised, then the dual may have multiple optimizers.
Thus we have the following Lemma.
Lemma 3.2 If the primal problem (3.3) is not degenerate then the dual problem
has a unique optimizer. If the dual problem (3.8) is not degenerate, then the primal
problem has a unique optimizer.
Multiple dual optimizers imply primal degeneracy and multiple primal optimizers
imply dual degeneracy but the reverse is not true as we will illustrate next. In
other words LICQ is only a sucient condition for uniqueness of the dual.
Example 3.1 Primal and dual degeneracies
Consider the following pair of primal and dual LPs
Primal
inf [1 1]x
_
_
_
_
0 1
1 1
1 0
0 1
_
_
x
_
_
_
_
1
1
0
0
_
_
(3.12)
Dual
inf [1 1 0 0]u
_
0 1 1 0
1 1 0 1
_
u =
_
1
1
_
u 0
(3.13)
Substituting for u3 and u4 from the equality constraints in (3.13) we can rewrite the
dual as
Dual
inf [1 1]
_
u1
u2
_
_
_
_
_
1 0
0 1
1 1
0 1
_
_
_
u1
u2
_
_
_
_
_
0
1
1
0
_
_
(3.14)
The situation is portrayed in Figure 3.4.
max
primal solution
1
2
x
1
x
2
(a) Case 2 - Primal LP.
min
dual solution
u
1
u
2
(b) Case 2 - Dual LP.
Figure 3.4 Example 3.1 LP with Primal and Dual Degeneracy. The vectors
max and min point in the direction for which the objective improves. The
feasible regions are shaded.
Consider the two solutions for the primal LP denoted with 1 and 2 in Fig-
ure 3.4(a) and referred to as basic solutions. Basic solution 1 is primal non-
degenerate, since it is dened by exactly as many active constraints as there
are variables. Basic solution 2 is primal degenerate, since it is dened by three
active constraints, i.e., more than two. Any convex combination of optimal solu-
tions 1 and 2 is also optimal. This continuum of optimal solutions in the primal
problem corresponds to a degenerate solution in the dual space, that is, the dual
problem is primal-degenerate. Hence the primal problem is dual-degenerate. In
conclusion, Figures 3.4(a) and 3.4(b) show an example of a primal problem with
multiple optima and the corresponding dual problem being primal degenerate.
Next we want to show that the statement if the dual problem is primal de-
generate then the primal problem has multiple optima is, in general, not true.
Switch dual and primal problems, i.e., call the dual problem primal problem
and the primal problem dual problem (this can be done since the dual of
the dual problem is the primal problem). Then, we have a dual problem which
is primal degenerate in solution 2 while the primal problem does not present
multiple optimizers.
3.2.5 Convex Piecewise Ane Optimization
Consider a continuous and convex piecewise ane function f : 1 R
s
R:
f(z) = c
i
z +d
i
if z 1
i
, i = 1, . . . , p, (3.15)
where 1
i
p
i=1
are polyhedral sets with disjoint interiors, 1 =
p
i=1
1
i
is a poly-
hedron and c
i
R
s
, d
i
R. Then, f(z) can be rewritten as [255]
f(z) = max
i=1,...,k
c
i
z +d
i
, z 1, (3.16)
or [64]:
f(z) = min
,z

c
i
z +d
i
, i = 1, . . . , k
z 1.
(3.17)
Consider the following optimization problem
f
= min
z
f(z)
subj. to Gz w
z 1,
(3.18)
where the cost function has the form (3.15). Substituting (3.17) this becomes
f
= min
z,

subj. to Gz w
c
i
z +d
i
, i = 1, . . . , k
z 1.
(3.19)
The previous result can be extended to the sum of continuous and convex
piecewise ane functions. Let f : 1 R
s
R be dened as:
f(z) =
r
j=1
f
j
(z), (3.20)
with
f
j
(z) = max
i=1,...,k
j
c
j
i
z +d
j
i
, z 1. (3.21)
3.3 Quadratic Programming 29
1
1
1
3
1
4
1
2
c
1
z +d
1
c
2
z +d
2
c
3
z +d
3
c
4
z +d
4
f
(
z
)
Figure 3.5 Convex PWA Function Described as the Max of Ane Functions.
Then the following optimization problem
f
= min
z
f(z)
subj. to Gz w
z 1,
(3.22)
where the cost function has the form (3.20), can be solved by the following linear
program:
f
= min
z,
1
,...,
r
1
+ +
r
subj. to Gz w
c
1
i
z +d
1
i

1
, i = 1, . . . , k
1
c
2
i
z +d
2
i

2
, i = 1, . . . , k
2
.
.
.
c
r
i
z +d
r
i

r
, i = 1, . . . , k
r
z 1.
(3.23)
Remark 3.1 Note that the results of this section can be immediately applied to the
minimization of one or innity norms. For any y R, [y[ = max |y, y. Therefore
for any Q R
ks
and p R
k
:
|Qz p| = max|Q1
z +p1, Q1
z p1, . . . , Q
k
z +p
k
, Q
k
z p
k
and
|Qz p|1 =
k
i=1
[Qi
z +pi[ =
k
i=1
max|Qi
z +pi, Qi
z pi
3.3 Quadratic Programming
The continuous optimization problem (1.4) is called a quadratic program (QP) if the
constraint functions are ane and the cost function is a convex quadratic function.
T
z
(a) Case 1, optimizer in interior of T.

T
z
(b) Case 2, optimizer at the boundary of T.

Figure 3.6 Graphical Interpretation of the Quadratic Program Solution
In this book we will use the form:
min
z
1
2
z
Hz +q
z +r
subj. to Gz w,
(3.24)
where z R
s
, H = H
0 R
ss
, q R
s
, G R
ms
. In (3.24) the constant
term can be omitted if one is only interested in the optimizer.
Other QP forms often include equality and inequality constraints:
min
z
1
2
z
Hz +q
z +r
subj. to Gz w
Az = b.
(3.25)
3.3.1 Graphical Interpretation and Solution Properties
Let T be the feasible set (1.6) of problem (3.24). As Z = R
s
, this implies that T is
a polyhedron dened by the inequality constraints in (3.24). The two dimensional
geometric interpretation is depicted in Figure 3.6. The level curves of the cost
function
1
2
z
Hz+q
z+r are represented by the ellipsoids. All the points z belonging

both to the ellipsoid
1
2
z
Hz + q
z + r = k
i
and to the polyhedron T are feasible
points with an associated cost k
i
. The smaller the ellipsoid, the smaller is its cost
k
i
. Solving (3.24) amounts to nding a feasible z which belongs to the level curve
with the smallest cost k
i
. Since H is strictly positive denite, the QP (3.24) cannot
have multiple optima nor unbounded solutions. If T is not empty the optimizer is
unique. Two cases can occur if T is not empty:
Case 1. The optimizer lies strictly inside the feasible polyhedron (Figure 3.6(a)).
Case 2. The optimizer lies on the boundary of the feasible polyhedron (Fig-
ure 3.6(b)).
3.3 Quadratic Programming 31
In case 1 the QP (3.24) is unconstrained and we can nd the minimizer by
setting the gradient equal to zero
Hz
+q = 0. (3.26)
Since H 0 we obtain z
= H
1
q.
3.3.2 Dual of QP
Consider the QP (3.24)
min
z
1
2
z
Hz +q
z
subj. to Gz w.
The Lagrange function as dened in (2.3) is
L(z, u) =
1
2
z
Hz +q
z +u
(Gz w).
The the dual cost is
d(u) = min
z
1
2
z
Hz +q
z +u
(Gz w) (3.27)
and the dual problem is
max
u0
min
z
1
2
z
Hz +q
z +u
(Gz w). (3.28)

For a given u the Lagrange function
1
2
z
Hz +q
z +u
(Gz w) is convex. Therefore

it is necessary and sucient for optimality that the gradient is zero
Hz +q +G
u = 0.
From the equation above we can derive z = H
1
(q +G
u) and, substituting this

in equation (3.27), we obtain:
d(u) =
1
2
u
(GH
1
G
)u u
(w +GH
1
q)
1
2
q
H
1
q. (3.29)
By using (3.29) the dual problem (3.28) can be rewritten as:
min
u
1
2
u
(GH
1
G
)u +u
(w +GH
1
q) +
1
2
q
H
1
q
subj. to u 0.
(3.30)
Note that for convex QPs feasibility implies strong duality (Remark 2.2).
3.3.3 KKT conditions for QP
Consider the QP (3.24). Then, f(z) = Hz +q, g
i
(z) = G
i
z w
i
(where G
i
is the
i-th row of G), g
i
(z) = G
i
. The KKT conditions become
Hz +q +G
u = 0 (3.31a)
u
i
(G
i
z w
i
) = 0, i = 1, . . . , m (3.31b)
u 0 (3.31c)
Gz w 0. (3.31d)
3.3.4 Active Constraints and Degeneracies
Consider the denition of active set A(z) in (3.10). Note that A(z
) may be empty
in the case of a QP. We dene primal and dual degeneracy as in the LP case.
Denition 3.4 The QP (3.24) is said to be primal degenerate if there exists a
z
such that the LICQ does not hold at z
.
Note that if the QP (3.24) is not primal degenerate, then the dual QP (3.30) has
a unique solution since u
NA
= 0 and (G
A
H
1
G
A
) is invertible.
Denition 3.5 The QP (3.24) is said to be dual degenerate if its dual problem is
primal degenerate.
We note from (3.30) that all the constraints are independent. Therefore LICQ
always holds and dual degeneracy can never occur for QPs with H 0.
3.3.5 Constrained Least-Squares Problems
The problem of minimizing the convex quadratic function
|Az b|
2
2
= z
Az 2b
Az +b
b (3.32)
is an (unconstrained) QP. It arises in many elds and has many names, e.g., linear
regression or least-squares approximation. From (3.26) we nd the minimizer
z
= (A
A)
1
A
b = A
b,
where A
is the generalized inverse of A. When linear inequality constraints are

added, the problem is called constrained linear regression or constrained least-
squares, and there is no longer a simple analytical solution. As an example we can
consider regression with lower and upper bounds on the variables, i.e.,
min
z
|Az b|
2
2
subj. to l
i
z
i
u
i
, i = 1, . . . , n,
(3.33)
which is a QP. In Chapter 7.3.1 we will show how to compute an analytical solu-
tion to the constrained least-squares problem. In particular we will show how to
compute the solution z
as a function of A, b, u
i
s and l
i
s.
3.4 Mixed-Integer Optimization
As discussed in Section 1.1.2, if the decision set Z in the optimization problem (1.2)
is the Cartesian product of a binary set and a real Euclidian space, i.e., Z
[z
c
, z
b
] : z
c
R
sc
, z
b
0, 1
s
b
, then the optimization problem is said to be
3.4 Mixed-Integer Optimization 33
mixed-integer. In this section Mixed Integer Linear Programming (MILP) and
Mixed Integer Quadratic Programming (MIQP) are introduced.
When the cost of the optimization problem (1.10) is quadratic and the con-
straints are ane, then the problem is called a mixed integer quadratic program
(MIQP). The most general form of an MIQP is
inf
[zc,z
b
]
1
2
z
Hz +q
z +r
subj. to G
c
z
c
+G
b
z
b
w
A
c
z
c
+A
b
z
b
= b
z
c
R
sc
, z
b
0, 1
s
b
z = [z
c
, z
b
],
(3.34)
where H _ 0 R
ss
, G
c
R
msc
, G
b
R
ms
b
, w R
m
, A
c
R
psc
, A
b

R
ps
b
, b R
p
and s = s
c
+ s
d
. Mixed integer quadratic programs are nonconvex
optimization problems, in general. When H = 0 the problem is called a mixed
integer linear program (MILP). Often the term r is omitted from the cost since
it does not aect the optimizer, but r has to be considered when computing the
optimal value. In this book we will often use the form of MIQP with inequality
constraints only
inf
[zc,z
b
]
1
2
z
Hz +q
z +r
subj. to G
c
z
c
+G
b
z
b
w
z
c
R
sc
, z
b
0, 1
s
b
z = [z
c
, z
b
].
(3.35)
The general form can always be translated into the form with inequality constraints
only by standard and simple manipulations.
For a xed integer value z
b
of z
b
, the MIQP (3.34) becomes a quadratic program:
inf
zc
1
2
z
c
H
c
z
c
+q
c
(z
b
)
z
c
+k(z
b
)
subj. to G
c
z
c
w G
b
z
b
A
c
z
c
= b
eq
A
b
z
b
z
c
R
sc
.
(3.36)
Therefore the most obvious way to interpret and solve an MIQP is to enumerate
all the 2
s
b
integer values of the variable z
b
and solve the corresponding QPs. By
comparing the 2
s
b
optimal costs one can derive the optimizer and the optimal cost
of the MIQP (3.34). Although this approach is not used in practice, it gives a
simple way for proving what is stated next. Let T
z
b
be the feasible set (1.6) of
problem (3.35) for a xed z
b
= z
b
. The cost is a quadratic function dened over
R
sc
and T
z
b
is a polyhedron dened by the inequality constraints
G
c
z
c
w G
b
z
b
. (3.37)
Denote by f
the optimal value and by Z
the set of optimizers of problem (3.34)

Z
= arg min
z
b
{0,1}
s
b,zcPz
b
1
2
z
c
H
c
z
c
+q
c
(z
b
)
z
c
+k(z
b
)
If T
z
b
is empty for all z
b
, then the problem (3.35) is infeasible. Five cases can occur
if T
z
b
is not empty for at last one z
b
0, 1
s
b
:
Case 1. The MIQP solution is unbounded, i.e., f
= . This cannot happen if

H
c
0.
Case 2. The MIQP solution is bounded, i.e., f
> and the optimizer is

unique. Z
is a singleton.
Case 3. The MIQP solution is bounded and there are innitely many optimizers
corresponding to the same integer value. Z
is the Cartesian product of an

innite dimensional subset of R
s
and an integer number z
b
. This cannot
happen if H
c
0.
Case 4. The MIQP solution is bounded and there are nitely many optimizers
corresponding to dierent integer values. Z
is a nite set of optimizers

(z
1,c
, z
1,b
), . . . , (z
N,c
, z
N,b
).
Case 5. The union of Case 3 and Case 4.
Chapter 4
Numerical Methods for
Optimization
Contributed by Dr. Alexander Domahidi and Dr. Stefan Richter
Swiss Federal Institute of Technology Zurich
domahidi@embotech.com, richters@alumni.ethz.ch
There is a great variety of algorithms for the solution of unconstrained and
constrained optimization problems. In this chapter, we introduce those algorithms
that are important for the problems encountered in this book and explain the
underlying concepts. In this chapter, we present numerical methods for nding a
minimizer z
to the optimization problem

min
z
f(z)
subj. to z S
(P)
where both the objective function f : R
s
R and the feasible set S are convex,
hence (P) is a convex program. We will assume that if the problem is feasible, an
optimizer z
S is attained, i.e. f(z
) = f
.
Many model predictive control problems for linear systems have the form (P)
with a linear or convex quadratic objective function and a polyhedral feasible set
S dened by equalities and inequalities. In all but the simplest cases, analytically
nding a minimizer z
is impossible. Hence, one has to resort to numerical methods

that compute an approximate minimizer that is good enough (we give a precise
denition of this term below). Any such method proceeds in an iterative manner,
i.e. starting from an initial guess z
0
, it computes a sequence z
k
k=kmax
k=1
, where
z
k+1
=
_
z
k
, f, S
_
,
with being an update rule depending on the method. For a method to be useful,
it should terminate after k
max
iterations and return an approximate minimizer
z
kmax
that satises
[f
_
z
kmax
_
f (z
) [ and dist(z
kmax
, S) ,
36 4 Numerical Methods for Optimization
where
dist(z, S) = min
yS
|y z|
is the shortest distance between a point and a set in R
s
measured by the norm | |.
The parameters , > 0 dene the required accuracy of the approximate minimizer.
There exist mainly two classes of optimization methods for solving (P): rst-
order methods that make use of rst-order information of the objective function
(subgradients or gradients) and second-order methods using in addition second-
order information (Hessians). Note that either method may also use zero-order
information, i.e. the objective value itself, for instance, to compute step sizes.
In the following list, key aspects that are important for any optimization method
are summarized. Some of these aspects are discussed in detail for the corresponding
optimization methods later in this chapter.
Convergence: Is k
max
nite for all accuracies , > 0?
Convergence rate: How do errors [f
_
z
k
_
f (z
) [ and dist(z
k
, S) depend on
the iteration counter i?
Feasibility: Does the method produce feasible iterates, i.e. dist(z
k
, S) = 0 for
all iterations i?
Numerical robustness: Do the properties above change in presence of nite
precision arithmetics when using xed or oating point number representa-
tions?
Warm-starting: Can the method take advantage of z
0
being close to the
minimizer z
?
Preconditioning: Is it possible to transform (P) into an equivalent prob-
lem (P
) that can be solved in fewer iterations?

Computational complexity: How many arithmetic operations are needed for
computing the approximate solution z
kmax
?
The speed or rate of convergence relates to the second point in the list above
and is an important distinctive feature between various methods. We present a
denition of convergence rates in the following.
Denition 4.1 (Convergence Rates (cf. [221, 2.2])) Let e
k
be a sequence
of positive reals that converge to 0. The convergence rate of this sequence is called
linear if there exists a constant q (0, 1) such that
limsup
i
e
k+1
e
k
q.
If q = 1, the convergence rate is said to be sublinear, whereas if q = 0, the sequence
has superlinear convergence rate. A superlinearly converging sequence converges
with order p if
limsup
i
e
k+1
e
k
p
C,
37
where C is a positive constant, not necessarily less than 1. In particular, superlinear
convergence with order p = 2 is called quadratic convergence.
For an optimization method, sequence e
k
in the denition above can be any
sequence of errors associated with the particular method, e.g. e
k
can be dened as
the absolute error in the objective value
e
k
= [f
_
z
k
_
f (z
) [
or any other nonnegative error measure that vanishes only at a minimizer z
.
In the previous classication of convergence rates, sublinear convergence is even-
tually slower than linear convergence which itself is eventually slower than superlin-
ear convergence. Within superlinear convergence rates, the ones with higher order
converge faster eventually. However, for a xed number of iterations, the actual
ordering of convergence rates also depends on constants q and C, for instance, a
linearly converging sequence with a small value of q might have a smaller error
after 10 steps than a quadratically converging sequence.
In the remainder of this chapter, we rst introduce methods that solve the
unconstrained problem, i.e. S = R
s
, in Section 4.1. They are prerequisite for the
understanding of constrained optimization methods, i.e. S R
s
, which are then
presented in Section 4.2.
4.1 Unconstrained Optimization
In unconstrained smooth convex optimization we are interested in solving the prob-
lem
min
z
f(z), (4.1)
where f : R
s
R is convex and continuously dierentiable. We assume that a
minimizer z
R
s
of (4.1) exists and dene the optimal value as f
= f(z
).
For the purpose of this section, we focus on so-called descent methods for un-
constrained optimization. These methods obtain the next iterate z
k+1
from the
current iterate z
k
by taking a step of size h
k
> 0 along a certain direction d
k
:
z
k+1
= z
k
+h
k
d
k
(4.2)
Particular methods dier in the way d
k
and h
k
are chosen. In general, d
k
has to
be a descent direction, i.e. for all z
k
,= z
it must satisfy
f(z
k
)
d
k
< 0. (4.3)
The step sizes h
k
have to be chosen carefully in order to ensure a suciently large
decrease in the function value, which is crucial for convergence of descent methods.
Since solving for the optimal step size,
h
k,
arg min
h0
f(z
k
+hd
k
),
is generally too expensive, inexact line search methods are used instead. They
compute h
k
cheaply and achieve a reasonably good decrement in the function value
while ensuring convergence of the algorithm. We discuss line search methods in
Section 4.1.3.
In the following, we present two of the most important instances of descent
methods: the classic gradient method in Section 4.1.1 and Newtons method in
Section 4.1.2. We include Nesterovs fast gradient method in the discussion along
with gradient methods, although, strictly speaking, it is not a descent method. We
shall see that gradient methods, under certain assumptions on the gradient of f,
allow for a constant step size, which results in an extremely simple and ecient
algorithm in practice. Newtons method, on the contrary, requires a line search
to ensure global convergence, but it has a better (local) convergence rate than
gradient methods.
4.1.1 Gradient Methods
Classic Gradient Method
One property of the gradient of f evaluated at z
k
, f(z
k
), is that it points into
the direction of steepest local ascent. The main idea of gradient methods for
minimization is therefore to use the anti-gradient direction as a descent direction,
d
k
= f(z
k
),
4.1 Unconstrained Optimization 39
z
f(y) + f(y)
T
(z y) +
L
2
|z y|
2
f(z)
f(y) +f(y)
T
(z y) +

2
|z y|
2
f(y)
-1 -0.5 0 0.25 0.5 1
0
1
2
3
Figure 4.1 Upper and lower bounds on functions: The quadratic upper
bound stems from L-smoothness of f (cf. Denition 4.2), while the quadratic
lower bound is obtained from strong convexity of f (cf. Theorem 4.1). In
this example, f(z) = z
2
+
1
2
z + 1 and we have chosen L = 3 and = 1 for
illustration.
trivially satisfying (4.3). Before turning to the question of appropriate step sizes h
k
,
we introduce an additional assumption on the gradient of f. For the remainder of
this section, we assume that f has a Lipschitz continuous gradient, i.e. that f is
so-called L-smooth. All of the upcoming results regarding Lipschitz continuity of
the gradient are taken from [218, 1.2.2].
Denition 4.2 (L-Smoothness of a Function) Let f : R
s
R be once conti-
nuously dierentiable on R
s
. The gradient f is Lipschitz continuous on R
s
with
Lipschitz constant L > 0 if for all pairs (z, y) R
s
R
s
|f(z) f(y)| L|z y| .
If function f is twice continuously dierentiable, L-smoothness can be charac-
terized using the largest eigenvalue of the Hessian of f. This alternative charac-
terization can be useful in order to compute the actual value of the Lipschitz
constant.
Lemma 4.1 (L-Smoothness: Second-Order Characterization) Let function
f : R
s
R be twice continuously dierentiable on R
s
. Then the gradient f is
Lipschitz continuous on R
s
with Lipschitz constant L > 0 if and only if for all
z R
s
_
_
2
f(z)
_
_
L.
The most important consequence of Lipschitz continuity of the gradient is that
f can be globally upper-bounded by a quadratic. This is expressed in the next
lemma and illustrated in Figure 4.1.
Algorithm 4.1 Gradient method for smooth, convex optimization
Input: Initial iterate z
0
R
s
, Lipschitz constant L of f
Output: Approximate minimizer
repeat
z
k+1
= z
k
1
L
f(z
k
)
until stopping criterion is satised
Lemma 4.2 (Descent Lemma) Let f be L-smooth on R
s
. Then for any pair
(z, y) R
s
R
s
we have
f(z) f(y) +f(y)
(z y) +
L
2
|z y|
2
. (4.4)
We now return to the choice of step sizes h
k
in (4.2). Notice that interpreting the
next iterate z
k+1
as the result of minimizing a quadratic at the previous iterate z
k
,
z
k+1
= arg min
z
f(z, z
k
), (4.5)
where we dene
f(z, z
k
) = f(z
k
) +f(z
k
)
(z z
k
) +
1
2h
k
|z z
k
|
2
, (4.6)
yields after some standard calculus the classic gradient update rule
z
k+1
= z
k
h
k
f(z
k
).
The quadratic function

f(z, z
k
) in (4.6) is parametrized by the step length h
k
.
Choosing it as
h
k
=
1
L
,
the quadratic

f(z, z
k
) becomes an upper bound on f (cf. Lemma 4.2). Convergence
of the gradient method can then be shown using the relation
f(z
k+1
)
(1)

f(z
k+1
, z
k
)
(2)

f(z
k
, z
k
) = f(z
k
), (4.7)
where the rst and second inequality are due to (4.4) and (4.5) respectively. In
fact, the second inequality is strict for z
k
,= z
.
If the Lipschitz constant L is unknown, a simple strategy is to start with an
initial guess

L > 0, solve for z
k+1
according to (4.5) with h
k
= 1/
L and then
check whether the rst inequality in (4.7) holds true. If so, the quadratic objective
in (4.5) was a (local) upper bound of f and minimization consequently leads to a
decrease in the function value, i.e. f(z
k+1
) < f(z
k
). Otherwise, the current guess
for the Lipschitz constant needs to be increased, e.g.

L 2
L, and the procedure

repeated. Termination is guaranteed because f was assumed L-smooth.
The gradient method with constant step size summarized in Algorithm 4.1 can
be shown to converge sublinearly with the number of iterations growing propor-
tionally to L/ |z
0
z
|
2
[218, Corollary 2.1.2]. The level of suboptimality > 0
is hereby dened in terms of the error in the objective value f
_
z
k
_
f
. Note
that in case of a constant step size, the gradient method is an extremely sim-
ple, division-free algorithm (given that the gradient f can be computed without
divisions).
Algorithm 4.2 Fast gradient method for smooth, convex optimization
Input: Initial iterates z
0
R
s
, y
0
= z
0
;
0
=
1
2
(
5 1), Lipschitz constant L of f

repeat
z
k+1
= y
k
1
L
f(y
k
)
k+1
=

k
2
__
k
2
+ 4
k
_
k
=

k
(1
k
)
k
2
+
k+1
y
k+1
= z
k+1
+
k
(z
k+1
z
k
)
Fast Gradient Method
An accelerated version of the gradient method, Nesterovs fast gradient method
[218, 2.2], is given in Algorithm 4.2. The basic idea behind this method is to take
the gradient at an ane combination of the previous two iterates, i.e. the gradient
is evaluated at
y
k
= z
k
+
k1
(z
k
z
k1
)
for some carefully chosen
k1
(0, 1). This two-step method can be shown
to converge sublinearly with the number of iterations growing proportionally to
_
L/ |z
0
z
| [218, Theorem 2.2.2]. This is the best convergence rate for L-

smooth problems using only rst-order information (see [218, 2.1] for details).
The mathematical reasoning behind the improved convergence rate and optimality
of the fast gradient method amongst all rst-order methods is quite involved and
beyond the scope of this book.
Example 4.1 [Gradient method vs. fast gradient method for minimizing a quadratic]
Consider the optimization problem given by
min
1
2
z
Hz +q
z,
where the Hessian and the linear cost vector are given by
H =
_
0.2 0.2
0.2 2
_
and q =
_
0.1
0.1
_
.
For quadratic functions, a tight Lipschitz constant L of the gradient can be deter-
mined by computing the largest eigenvalue of the Hessian (cf. Lemma 4.1), i.e.
L = max(H), (4.8)
here L = 2.022. The upper left plot in Figure 4.2 shows the rst 10 iterations of
the gradient method with constant step size 1/L (Algorithm 4.1). The upper right
z
1
z
2
0
123
-5 0 5
-3
-2
-1
0
1
2
3
(a) Gradient method: step size
1
L
z
1
z
2
0
1
2
3
-5 0 5
-3
-2
-1
0
1
2
3
(b) Gradient method: step size
4
3L
z
1
z
2
0
1
2
3
-5 0 5
-3
-2
-1
0
1
2
3
(c) Gradient method: step size
2
L
z
1
z
2
0
1
2
3
-5 0 5
-3
-2
-1
0
1
2
3
(d) Fast gradient method: step
size
1
L
Figure 4.2 First 10 iterations of the gradient method with constant step size
1/L using the tight, a slightly underestimated and a strongly underestimated
Lipschitz constant L for Example 4.1. Also shown are the iterates of the
fast gradient method using the tight Lipschitz constant. The contour lines
indicate the function value of each iterate. Note that the gradients are
orthogonal to the contour lines.
and lower left plot show the rst 10 iterations using larger (but constant) step sizes,
which correspond to underestimating the tight Lipschitz constant L. It can be shown
that if L is underestimated by a factor of 2 or more, convergence is lost (cf. lower left
plot). The lower right plot shows the iterates of the fast gradient method using the
tight Lipschitz constant (Algorithm 4.2). Contrary to the gradient method, the fast
gradient method generates a non-monotonically decreasing error sequence.
Strongly Convex Problems
In the case where the objective function f in (4.1) is also strongly convex, linear
convergence rates can be established for both the gradient method and the fast
gradient method. The following theorems from [150, B] characterize strong con-
vexity whenever the function is once or twice continuously dierentiable. For an
illustration of this property see Figure 4.1.
Theorem 4.1 (Strong Convexity: First-Order Characterization) Let func-
tion f : R
s
R be once continuously dierentiable on R
s
. Then f is strongly con-
vex on R
s
with convexity parameter > 0 if and only if for all pairs (z, y) R
s
R
s
f(z) f(y) +f(y)
(z y) +

2
|z y|
2
.
Theorem 4.2 (Strong Convexity: Second-Order Characterization) Let f :
R
s
R be twice continuously dierentiable on R
s
. Then f is strongly convex on
R
s
with parameter > 0 if and only if for all z R
s
2
f(z) _ I
s
.
Algorithm 4.3 Fast gradient method for smooth, strongly convex optimization
0
R
s
, y
0
= z
0
; 0 <
_
/L
0
< 1, Lipschitz constant L
of f, strong convexity parameter of f
repeat
z
k+1
= y
k
1
L
f(y
k
)
Compute
k+1
(0, 1):
k+1
2
= (1
k+1
)
k
2
+

k+1
L
k
=

k
(1
k
)
k2
+
k+1
y
k+1
= z
k+1
+
k
(z
k+1
z
k
)
From Theorem 4.1 it follows that a dierentiable strongly convex function can
be lower-bounded by a quadratic. Relating this to the quadratic upper bound in
Lemma 4.2, the relation
f
=
L
1
follows intuitively. The constant
f
is the so-called condition number of the L-
smooth and strongly convex function f.
With the additional assumption of strong convexity, it can be shown that the
gradient method in Algorithm 4.1 converges linearly with the number of iterations
growing proportionally to
f
ln(1/) [218, Theorem 2.1.15] with > 0 being the
level of suboptimality in terms of the error in the objective. In order for the fast
gradient method in Algorithm 4.2 to achieve linear convergence rate, it either needs
to be modied or restarted after a number of steps that depends on the condition
number
f
. In both cases it can be shown that the number of iterations grows
proportionally to

f
ln(1/) [218, Theorem 2.2.2]. This is the best convergence
rate for L-smooth and strongly convex problems using only rst-order information
(see [218, 2.1] for details). A modied fast gradient method that takes strong
convexity into account explicitly and thus does not need restarting is given in
Algorithm 4.3.
Example 4.2 [Strong convexity parameter of a quadratic function] Let f : R
s
R be
a quadratic function dened as
f(z) =
1
2
z
Hz +q
z
with H 0. Then, by Theorem 4.2, the (tight) parameter for strong convexity is
the smallest eigenvalue of the Hessian H, i.e.
= min(H).
For the Hessian given in Example 4.1 we therefore have = 0.178, and the condition
number is
f
= 11.36. From the convergence results for the gradient method and the
fast gradient method for the strongly convex case it follows that condition numbers
close to 1 lead to fast convergence. So, for this example, fast convergence can be
guaranteed for both methods.
Preconditioning
In the previous section we have seen that the theoretical convergence rates for
gradient methods depend on the condition number
f
. In practice, a high condition
number usually indicates that many iterations of Algorithm 4.1 and Algorithm 4.3
are needed to achieve the specied accuracy. To improve the conditioning of the
problem and thus to lower the amount of computation needed to obtain an appro-
ximate minimizer, one can transform problem (4.1) into an equivalent problem by
a variable transformation
y = Pz
with P invertible. The aim is to choose the preconditioner P such that the new
objective function h(y) = f(P
1
y) has a smaller condition number, i.e.
h
<
f
.
Intuitively, the best choice for P is one that achieves a circular shape of the level
sets of the objective function or at least comes close to it. Mathematically speak-
ing, the preconditioner matrix P should be such that
h
1. Finding optimal
preconditioners for which
h
= 1 is impossible for all but the simplest cases. We
give one such example in the following.
Example 4.3 [Optimal preconditioner for a quadratic function] Let f(z) =
1
2
z
Hz+q
z
with Hessian H 0. It is easily veried that P = H
1/2
is the optimal preconditioner
for which
h
= 1, since h(y) =
1
2
y
y + q
1
2
y. However, computing H
1/2
(and its
inverse) is as expensive as minimizing f, so nothing has been gained in this example.
Stopping criteria
Stopping criteria test whether the current iterate z
k
is optimal or, more practically,
has attained an acceptable level of suboptimality. In optimization, the most natural
stopping criterion is
f(z
k
) f
, (4.9)
where > 0 is an accuracy measure specied by the user. However, since the
optimal value f
is rarely known, we look for a computable lower bound f

k,
( f
), so that we arrive at the evaluable sucient condition for (4.9)

f(z
k
) f
k,
.
The latter stopping criterion works for any > 0 if the lower bounds f
k,
converge
to the optimal value f
for an increasing iteration counter i.

For dierentiable convex functions such lower bounds can be constructed right
from the denition of convexity, i.e.
f
f(z
k
) +f(z
k
)
(z
z
k
)
f(z
k
) |f(z
k
)||z
z
k
|,
where the last inequality is due to Cauchy-Schwarz. Observe that for any conver-
ging method, this lower bound converges to f
from below since the gradient at

the minimizer z
vanishes in unconstrained optimization.

For many problems, the distance between an iterate and the minimizer can be
upper-bounded, i.e. |z
z
k
| R for all iterations i. Consequently, we end up
with the evaluable stopping criterion
|f(z
k
)| R ,
which is sucient for (4.9).
More sophisticated stopping criteria can be derived if properties such as strong
convexity of function f are exploited. For more details, we refer the interested
reader to [248, 6.3].
Summary
In this section, we have discussed the concept of descent methods and presented
one of the most important instances, the classic gradient method for smooth con-
vex optimization. Although not a descent method in its basic form, Nesterovs fast
gradient method was included due to its simplicity and improved speed of conver-
gence. Both methods are particularly useful in the strongly convex case, where
they converge linearly, with the fast gradient method attaining the best theoretical
convergence rate possible for rst-order methods. Despite a slower theoretical (and
most of the time also practical) convergence, the classic gradient method is well-
suited for processors with xed-point arithmetic, for instance, eld-programmable
gate arrays (FPGAs) because of its numerical robustness against inaccurate com-
putations. For the fast gradient method, recent research, e.g. [94], indicates that
care has to be taken if inexact gradients are used. When using gradient methods
in practice, preconditioning should be applied to lower the condition number of the
problem in order to decrease the number of iterations needed. The latter can also
be achieved by warm-starting, i.e. starting the methods from a point z
0
close to
a minimizer z
. This follows immediately from the dependence of the number of

iterations on the distance |z
0
z
|. More information and links to the literature

can be found in [248].
4.1.2 Newtons Method
The (fast) gradient method uses only rst-order information of the function f. A
much more quickly converging algorithm, at least locally, can be obtained when
using second-order information, i.e. the Hessian
2
f. In the following, we as-
sume f to be -strongly convex; hence the Hessian
2
f is positive denite on R
s
.
Furthermore we assume that the Hessian
2
f is Lipschitz continuous on R
s
with
Lipschitz constant M, i.e.
|
2
f(z)
2
f(y)| M|z y| z, y R
s
. (4.10)
Intuitively, the constant M measures howwell f can be approximated by a quadratic
function (M = 0 for a quadratic function f).
Algorithm 4.4 Newtons method for (4.1) (strongly convex case)
input: Initial iterate z
0
R
s
output: Point close to z
repeat
Newton direction: d
k
=
_
2
f(z
k
)
_
1
f(z
k
)
Line search: nd h
k
> 0 such that f(z
k
+h
k
d
k
) < f(z
k
)
z
k+1
= z
k
+h
k
d
k
until stopping criterion
The main idea of Newtons method is to approximate the function f by a local
quadratic model based on a second-order Taylor expansion,
f(z
k
+d) f(z
k
) +f(z
k
)
d +
1
2
d
2
f(z
k
)d = (d), (4.11)
where
2
f(z
k
) is the Hessian of f evaluated at the current iterate z
k
and d is a
search direction. The latter is chosen to minimize the quadratic model . From the
optimality condition
d
(d) = 0, the so-called Newton direction or Newton step
d
N
(z
k
) =
_
2
f(z
k
)
1
f(z
k
) (4.12)
is obtained. The Newton direction is a descent direction, as it satises (4.3):
f(z
k
)
d
N
(z
k
) = f(z
k
)
[
2
f(z
k
)]
1
f(z
k
) < 0 (4.13)
for all z
k
,= z
since, by strong convexity, the inverse of the Hessian

_
2
f(z)
1
exists for all z R
s
and is positive denite. If the Lipschitz constant of the Hessian,
M, is small, this method works very well, since the quadratic model remains a good
approximation even if a full Newton step is taken. However, the quadratic model
(d) is in general neither an upper nor a lower bound on f, and consequently a line
search (cf. Section 4.1.3) must be used to ensure a sucient decrease in the function
value, which results in global convergence of the algorithm. Newtons method is
summarized in Algorithm 4.4. It can be shown to converge globally in two phases
(for technical details see [64, 9.5] and [218, 1.2.4]):
1. Damped Newton phase: When the iterates are far away from the optimal
point, the function value is decreased sublinearly, i.e. it can be shown that
there exist constants , (with 0 < <
2
/M and > 0) such that f(z
k+1
)
f(z
k
) for all z
k
for which |f(z
k
)|
2
. The term damped is used in
this context since during this phase, the line search oftentimes returns step
sizes h
k
< 1, i.e. no full Newton steps are taken.
2. Quadratic convergence phase: If an iterate is suciently close to the op-
timum, i.e. |f(z
k
)|
2
< , a full Newton step is taken, i.e. the line search
returns h
k
= 1. The set z [ |f(z)|
2
< is called the quadratic con-
vergence zone, and once an iterate z
k
is in this zone, all subsequent iterates
remain in it. The function value f(z
k
) converges quadratically (i.e. with
i
max
O(ln ln )) to f
.
The quadratic convergence property makes Newtons method one of the most
powerful algorithms for unconstrained optimization of twice continuously dieren-
tiable functions. (The method is also widely employed for solving nonlinear equa-
tions.) As a result, it is used in interior point methods for constrained optimization
to solve the (unconstrained or equality constrained) subproblems in barrier meth-
ods or to generate primal-dual search directions. We discuss these in detail in
Section 4.2.2.
A disadvantage of Newtons method is the need to form the Hessian
2
f(z
k
),
which can be numerically ill-conditioned, and to solve the linear system
2
f(z
k
)d
N
= f(z
k
) (4.14)
for the Newton direction d
N
, which costs O(n
3
) oating point operations in general.
Each iteration is therefore signicantly more expensive than one iteration of any
of the gradient methods.
Preconditioning and Ane Invariance
Newtons method is ane invariant, i.e. a linear (or ane) transformation of
variables as discussed in Section 4.1.1 will lead to the same iterates subject to
transformation, i.e.
z
k
= Py
k
k = 0, . . . , i
max
. (4.15)
Therefore, the number of steps to reach a desired accuracy of the solution does not
change. Consequently, preconditioning has no eect when using Newtons method.
Newtons Method with Equality Constraints
Equality constraints are naturally handled in Newtons method as follows. Assume
we want to solve the problem
minimize f(z)
subject to Az = b
(4.16)
with A R
ps
(assuming full rank p) and b R
p
. As discussed above, the Newton
direction is given by the minimization of the quadratic model (d) at the current
feasible iterate z
k
(i.e. Az
k
= b) with an additional equality constraint:
d
N
= arg min
d
1
2
d
2
f(z
k
)d +f(z
k
)
d +f(z
k
)
subj. to Ad = 0
(4.17)
Since d is restricted to lie in the nullspace of A, we ensure that any iterate gener-
ated by Newtons method will satisfy Az
k+1
= A(z
k
+ h
k
d
N
(z
k
)) = b. From the
optimality conditions of (4.17), it follows that d
N
is obtained from the solution to
the linear system of dimension n +p:
_
2
f(z) A
A 0
_ _
d
y
_
=
_
f(z)
0
_
, (4.18)
where y R
p
are the Lagrange multipliers associated with the equality constraint
Ad = 0.
4.1.3 Line Search Methods
A line search determines the step length h
k
that is taken from the current iterate
z
k
R
n
along a search direction z
k
R
n
to the next iterate,
z
k+1
= z
k
+h
k
z
k
. (4.19)
In the most general setting, a merit function : R
n
R is employed to measure
the quality of the step taken to determine a value of the step size h
k
that ensures
convergence of the optimization algorithm. Merit functions can for example be the
objective function f, or the duality measure (4.41) in primal-dual interior point
methods.
The general line search problem can be written as a one dimensional optimiza-
tion problem over the step size h,
h
arg min
h0
(z
k
+hz
k
) , (4.20)
minimizing the merit function along the ray z
k
+hz
k
for h 0. It is often too
expensive to solve this problem exactly, and therefore usually inexact line search
methods are employed in practice that return a step size h
k
that approximates
the optimal step size h
from (4.20) suciently well. In the following, we give a

brief overview on practical inexact line search methods; see [222] for a thorough
treatment of the subject.
Wolfe Conditions
In order to ensure convergence of descent methods for unconstrained optimization,
for example Newtons method presented in Section 4.1, certain conditions on the
selected step size h
k
must be fullled. Popular conditions for inexact line searches
are the so-called Wolfe conditions [221]
f(z
k
+h
k
z
k
) f(z
k
) +c
1
h
k
f(z
k
)
z
k
(4.21a)
f(z
k
+h
k
z
k
)
z
k
c
2
f(z
k
)
z
k
(4.21b)
4.2 Constrained Optimization 49
with 0 < c
1
< c
2
< 1. The rst condition (4.21a) is called Armijos condition [10],
and ensures that a sucient decrease in the objective is achieved. The second
condition, the curvature condition (4.21b), ensures that the step length is not too
small [221].
One of the most popular inexact line search methods that fulls the Wolfe
conditions is the backtracking line search as discussed in the following.
Backtracking Line Search for Newtons Method
The backtracking line search for Newtons method is given in Algorithm 4.5. It
starts with a unit step size, which is iteratively reduced by a factor t (0, 1) until
Armijos condition (4.21a) is satised. It can be shown that it is not necessary to
check the curvature condition because it is implied by the backtracking rule [221,
Algorithm 3.1].
Algorithm 4.5 Backtracking Line Search for Newtons Method (Aalgorithm 4.4) [221,
Algorithm 3.1]
Require: Current iterate z
k
, descent direction z
k
, constants c (0, 1) and t (0, 1)
Ensure: Step size h
k
ensuring Wolfe conditions (4.21)
Set h
k
1
repeat
f(z
k
+h
k
z
k
) f(z
k
) +ch
k
f(z
k
)
z
k
Set h
k
th
k
until Armijos condition (4.21a) holds
Backtracking Line Search for Constrained Optimization
In order to obtain a line search method that ensures in addition feasibility of the
next iterate z
k+1
, the backtracking procedure is preceeded by nding 0 <

h 1
such that z
k
+

hz
k
is feasible [64, pp. 465]. The backtracking then starts with
h
k
=

h in line 1 of Algorithm 4.5.
4.2 Constrained Optimization
4.2.1 Gradient Projection Methods
Gradient methods presented in Section 4.1.1 for unconstrained L-smooth convex
optimization (or strongly convex optimization with parameter ) have a natural
extension to the constrained problem
min
z
f(z)
subj. to z S
(4.22)
where S is a convex subset of R
s
. The treatment of constraints within rst-order
methods is best explained in terms of the classic gradient method in Algorithm 4.1;
the accelerated versions (Algorithm 4.2 and Algorithm 4.3) can be adapted accord-
ing to the same principle.
We begin with rewriting, as in the unconstrained case, the gradient update rule
as a minimization of a quadratic upper bound on f dened in (4.6) with h
k
= 1/L,
but restrict the minimization to the set S:
z
k+1
= arg min
zS
f(z
k
) +f(z
k
)
(z z
k
) +
L
2
|z z
k
|
2
(4.23)
The solution to this constrained minimization problem can be recast as
z
k+1
=
S
(z
k
1
L
f(z
k
)), (4.24)
where
S
denotes the projection operator for set S dened as
S
(z) = arg min
yS
1
2
|y z|
2
. (4.25)
Algorithm 4.6 Gradient method for constrained smooth convex optimization
Input: Initial iterate z
0
S, Lipschitz constant L of f
repeat
z
k+1
= S
_
z
k
1
L
f(z
k
)
_
Algorithm 4.7 Fast gradient method for constrained smooth strongly convex opti-
mization
0
S, y
0
= z
0
; 0 <
_
/L
0
< 1, Lipschitz constant L
of f, strong convexity parameter of f
repeat
z
k+1
= S
_
y
k
1
L
f(y
k
)
_
Compute
k+1
(0, 1):
k+1
2
= (1
k+1
)
k
2
+

k+1
L
k
=

k
(1
k
)
k2
+
k+1
y
k+1
= z
k+1
+
k
(z
k+1
z
k
)
In order to see that (4.23) and (4.24) are indeed equivalent, it suces to verify
that their optimality conditions coincide. So, for constrained optimization, Al-
gorithms 4.1, 4.2 and 4.3 remain unchanged, except that the right hand side of
the assignment in the rst line of these algorithms is projected onto set S. The
resulting algorithmic schemes are stated in Algorithm 4.6 (gradient method) and
Algorithm 4.7 (fast gradient method for the strongly convex case) and illustrated
in Figure 4.3. It can be shown that for both Algorithm 4.6 and Algorithm 4.7 the
same convergence results hold as in the unconstrained case (cf. [218, 2.2.4]).
x
k
x
k+1
x
k
1
L
f(x
k
)
Q
(a) Gradient method.
x
k
x
k+1
y
k
1
L
f(y
k
)
Q x
k1
y
k
(b) Fast gradient method.
Figure 4.3 Classic gradient method (left) and fast gradient method (right)
for the constrained case. After computing the standard update rule from
the unconstrained case, a projection onto the feasible set is applied.
Whenever the projection operator for set S can be evaluated eciently, as for
the convex sets listed in Table 4.1, gradient projection methods have been shown
to work very well in practice. On the other hand, if the projection operator is
computationally expensive, e.g. it has to be evaluated by another algorithm, one
can often resort to solving the dual problem instead. This approach is presented
next.
Solution in Dual Domain
If set S is such that the projection (4.25) cannot be carried out eciently, one can
still use rst-order methods to solve the constrained problem (4.22) by adopting a
dual approach. In the following, we assume that set S can be written as
S = z R
s
[ Az = b, z K , (4.26)
where K is a convex subset of R
s
that allows for an ecient projection and the
pair (A, b) is from R
ps
R
p
. Many practically relevant sets can be represented
in terms of (4.26), i.e. as the intersection of a simple convex set and an ane set,
e.g. polyhedra. We can now rewrite the original problem (4.22) as
min
z
f(z)
subj. to Az = b
z K.
Using a technique called partial Lagrange relaxation [48, 4.2.2] we eliminate the
complicating equality constraints and dene the (concave) dual function as
d(v) = min
zK
f(z) +v
(Az b), (4.27)

Table 4.1 Euclidean norm projection operators of selected convex sets in R
s
.
Set Denition Projection Operator
ane set
S = |z R
s
[ Az = b
with (A, b) R
ps
R
p
S (z)=
_
z +A
(AA
)
1
(b Az) if rank A = p,
z +A
A
_
A
b z
_
otherwise
nonnegative orthant
S = |z R
s
[ z 0
(S (z))
i
=
_
zi if zi 0,
0 otherwise
i = 1, . . . , s
rectangle
S = |z R
s
[ l z u
with (l, u) R
s
R
s
(S (z))
i
=
_
_
li if zi < li,
zi if li zi ui,
ui if zi > ui
i = 1, . . . , s
2-norm ball
S = |z R
s
[ |z| r , r 0
S (z) =
_
r
z
z
if |z| > r,
z otherwise
where v R
p
. The concave dual problem is then given by
d
= max
v
d(v), (4.28)
which we also call the outer problem.
In order to apply any of the gradient methods for unconstrained optimization
in Section 4.1 for solving the outer problem, the dual function d(v) needs to fulll
two requirements: (i) dierentiability, i.e. the gradient d(v) must exist for all v,
and (ii) Lipschitz continuity of the gradient. Next, we will see that if f is strongly
convex, both requirements are indeed fullled.
Let us assume a strongly convex objective function f from here on. Strong
convexity ensures uniqueness of the minimizer z
(v) in the denition of the dual

function in (4.27). It is this uniqueness of z
(v) that results in a dierentiable dual

function as shown in [48, Danskins Theorem, Proposition B.25]. Specically, it
can be proved that
d(v) = Az
(v) b,
i.e. every evaluation of the dual gradient requires the solution of the so-called inner
problem (4.27).
The Lipschitz constant L
d
of the dual gradient can be upper bounded by
L
d

|A|
2
as stated in [219, Theorem 1]. This upper bound is sucient for the step size
computation for any gradient method (note that the step size is given by 1/L
d
).
However, the less conservative the Lipschitz constant is, the larger the step size
and the better the speed of convergence. In [249, Theorem 7] it is shown that for
the case of quadratic objective functions f(z) =
1
2
z
Hz + q
z, H 0, and under
mild assumptions, the tight Lipschitz constant of the dual gradient is
L
d
= |AH
1
2
|
2
.
So, strong convexity of the objective function f is key for using the gradient
methods in Section 4.1.1 for the solution of the dual problem (4.28). If f is not
strongly convex, there are dierent approaches available in the literature, e.g. a
smoothing approach for the dual function [219].
Note that the (unique) primal minimizer z
can be recovered from the maxi-

mizer of the dual problem v
, assuming it exists, by
z
= z
(v
).
It is also important to notice that the dual function d(v) lacks strong concavity
in general, so only sublinear convergence rates (instead of linear rates for the primal
problem) can be guaranteed for the solution of the dual problem. This and the
fact that each evaluation of the dual gradient requires solving the inner problem
can make the dual approach as presented in this section slow in practice. More
advanced rst-order methods can signicantly speed up convergence of the dual
approach, e.g. the alternating direction method of multipliers (ADMM). For more
details on ADMM, the interested reader is referred to the recent survey paper
in [63].
Summary
This section has introduced the generalization of gradient methods to the con-
strained case, the so-called gradient projection methods. As the name suggests, the
generalization comes from having to compute the projection of a gradient step onto
the feasible set. Apart from this additional operation, these methods are identical
to the ones introduced for the unconstrained case.
There exist many important convex sets for which the projection operator can
be evaluated in an ecient way, some of which are stated in Table 4.1. More
sets can be found, e.g. in [248, 5.4], which also contains further literature links.
One important consequence of having a set constraint is that a full preconditioner
matrix P (cf. Section 4.1.1 and Example 4.3) is not admissible anymore since in
the new basis the set can lose its favorable projection properties. For instance,
consider a box constraint with its projection operator given in Table 4.1. Only for
a diagonal preconditioner the set remains a box in the new basis and thus easy to
project. For quadratic objective functions, nding the best preconditioner under
set-related structure constraints turns out to be a convex semi-denite program
and hence can be solved eciently [248, 8.4].
Finally, we point the reader to the recent proximity gradient methods which
can be understood as a natural generalization of gradient projection methods. An
introduction to these methods and literature links can be found in [248, 5] and
[226].
4.2.2 Interior Point Methods
Today interior point methods are among the most widely used numerical methods
for solving convex optimization problems. This is the result of 25 years of intensive
research that has been initiated by Karmarkars seminal paper in 1984 on solving
LPs [166]. In 1994, Nesterov and Nemirovskii generalized interior point methods
to nonlinear convex problems, including second-order cone and semi-denite pro-
gramming [220]. Issues like numerical ill-conditioning, ecient and stable solution
of linear systems etc. are now well understood, and numerous free and commer-
cial codes are available that can solve linear and quadratic programs with high
reliability.
We discuss barrier methods in Section 4.2.2, which were the rst polynomial-
time algorithms for linear programming of practical relevance. Modern primal-
dual methods are presented in Section 4.2.2. This powerful class of interior point
methods forms the basis of almost all implementations today.
Throughout this section, we consider the problem
minimize f(z)
subject to Az = b
g(z) 0
(P-IPM)
where z R
s
are the primal variables, the functions f : R
s
R and g : R
s
R
m
are convex and twice continuously dierentiable (for g, this holds component-wise
for each function g
i
, i = 1, . . . , m). We assume that there exists a strictly feasible
point z
0
with respect to g(z) 0, i.e. the set
S = z R
s
[ g
i
(z) 0, i = 1, . . . , m, (4.29)
has a non-empty interior. We furthermore assume that a minimizer z
exists and
that it is attained.
Primal barrier methods
The main idea of the barrier method is to convert the constrained optimization
problem (P-IPM) into an unconstrained problem (with respect to inequalities) by
means of a barrier function
g
: R
s
R:
z
() arg min f(z) +

g
(z)
subj. to Az = b
(P())
where > 0 is called barrier parameter. The purpose of the barrier function
g
is to trap an optimal solution of P(), which we denote by z
(), in the set S

(consequently,
g
(z) must take on the value + whenever g
i
(z) > 0 for some i,
and a nite value otherwise). As a result, z
() is feasible with respect to S, but it

diers from z
since we have perturbed the objective function by the barrier term.

With the above idea in mind, we are now ready to outline the (primal) barrier
method summarized in Algorithm 4.8. Starting from (a possibly large)
0
, a solu-
tion z
(
0
) to problem P(
0
) is computed, for example by any of the methods for
unconstrained optimization presented in Section 4.1. For this reason,
g
should be
twice continuously dierentiable, allowing one to apply Newtons method. After
z
(
0
) has been computed, the barrier parameter is decreased by a constant fac-
tor,
1
=
0
/ (with > 1), and z
(
1
) is computed. This procedure is repeated
until has been suciently decreased. It can be shown under mild conditions
that z
() z
from the interior of S as 0. The points z
() form the
so-called central path, which is a continuously dierentiable curve in S that ends in
the solution set (cf. Example 4.4). Solving a subproblem P() is therefore called
centering.
If the decrease factor is not too large, z
(
k
) will be a good starting point for
solving P(
k+1
), and therefore Newtons method is applied to P(
k+1
) to exploit
the local quadratic convergence rate. Note that the equality constraints in (P-IPM)
have been preserved in P(), since they do not cause substantial diculties for
unconstrained methods as shown in Section 4.1. In particular, we refer the reader
to Section 4.1.2 for the equality constrained Newton method.
Logarithmic barrier
So far, we have not specied which function to use as a barrier. For example, the
indicator function
I
g
(z) =
_
0 if g(z) 0
+ otherwise
, (4.30)
trivially achieves the purpose of a barrier, but it is not useful since methods for
smooth convex optimization as discussed in Section 4.1 require the barrier function
to be convex and continuously dierentiable. A twice continuously dierentiable
convex barrier function that approximates I
g
well is the logarithmic barrier function
g
(z) =
m
i=1
ln (g
i
(z)) , (4.31)
with domain z R
s
[ g
i
(z) < 0 i = 1, . . . , m (the logarithmic barrier connes
z to the interior of the set S), gradient
g
(z) =
m
i=1
1
g
i
(z)
g
i
(z) (4.32)
and continuous Hessian
g
(z) =
m
i=1
1
g
i
(z)
2
g
i
(z)g
i
(z)
+
1
g
i
(z)
2
g
i
(z), (4.33)
which makes it possible to use Newtons method for solving the barrier subprob-
lems. The logarithmic barrier function is used in almost all implementations of
barrier methods. (In fact, the existence of polynomial-time algorithms for convex
optimization is closely related to the existence of barrier functions for the under-
lying feasible sets [220].)
Algorithm 4.8 Barrier interior point method for (P-IPM)
input: Strictly feasible initial iterate z
0
w.r.t. g(z) 0,
0
, > 1, tolerance > 0
repeat
Centering step:
Compute z
(
k
) by minimizing f(z) +
k
g(z) subject to Az = b starting
from the previous solution z
k1
(usually by Newtons method, cf. Algo-
rithm 4.4)
Update: z
k
= z
(
k
)
Stopping criterion: Stop if m
k
<
Decrease barrier parameter:
k+1
=
k
/
Example 4.4 [Central Path of a QP] Consider the QP
min
1
2
z
[
2 0
0 1
] z + [ 4
1
2
] z
subj. to
_
1 2.0588
1 1.7527
3.9669 1
1.1997 1.3622
1.1673 1.3111
_
z
_
1.0890
1.0623
2.0187
1
1
_
(4.34)
The solutions of P() for this Quadratic Program for dierent values of are depicted
in Fig. 4.4. For large , the level sets are almost identical to the level sets of g(z),
i.e. z
(5) is very close to the analytic center z

a
= arg ming(z) of the feasible set.
As decreases, z
() approaches z
, by following the central path.

Example 4.5 [Barrier method for QPs] In the following, we derive the barrier method
for quadratic programs of the form
minimize
1
2
z
Hz +q
z
subject to Az = b,
Gz w
(4.35)
with H 0, A R
ps
, b R
p
, G R
ms
and w R
m
from the general method
described above. The logarithmic barrier function for linear inequalities takes the
form
g(z) =
m
i=1
ln(wiGiz), g(z) =
m
i=1
1
wi Giz
G
i
,
2
g(z) =
m
i=1
1
(wi Giz)
2
G
i
Gi
(4.36)
where Gi denotes the i-th row of G and wi the i-th element of w. Hence P() takes
the form
min
1
2
z
Hz +q
z
m
i=1
ln(wi Giz)
s.t. Az = b
(4.37)
which we solve by Newtons method. Using the formula for the Newton direction dN
with equality constraints (4.18), we arrive at the following linear system of size n+p,
which has to be solved in each Newton step at the current iterate z:
_
H +
m
i=1
1
(w
i
G
i
z)
2
G
i
Gi A
A 0
_ _
dN
y
_
=
_
Hz +f +
m
j=1
1
w
i
G
i
z
G
i
0
_
. (4.38)
z
1
z
2
(a) Level curves of f(z) +
g(z) for = 5
z
1
z
2
(b) Level curves of f(z) +
g(z) for = 0.25
z
1
z
2
(c) Level curves of f(z) +
g(z) for = 0.0125
z
1
z
2
(d) Central path starting at an-
alytic center and moving to-
ward solution of (4.34) as de-
creases
Figure 4.4 Illustration of interior point method for example problem (4.34)
The resulting barrier method for solving (4.35) is summarized in Algorithm 4.9. The
inner loop resembles the Newton iterations of the algorithm, computing z
(). Note
that a primal feasible line search must be employed to ensure that the iterates remain
in the interior of S, see Section 4.1.3 for details.
Algorithm 4.9 Barrier interior point method for QPs (4.35)
input: Strictly feasible initial iterate z
0
w.r.t. Gz w,
0
, > 1, tolerance > 0
repeat
Initialize Newtons method with z
k,0
= z
k1
. Set k = 0.
loop
Newton direction: solve (4.38) for z = z
k,l
to obtain d
k
N
(z
k,l
).
Feasible line search to obtain step size h
k,l
(see Section 4.1.3).
Update: z
k,l+1
= z
k,l
+h
k,l
d
k,l
N
End inner loop: break if l = lmax, otherwise set l = l +1 and continue.
endloop
Update: z
k
= z
k,lmax
Stopping criterion: Stop if m
k
<
Decrease barrier parameter:
k+1
=
k
/
We conclude this section with a few remarks on the barrier method. The main
computational burden is the computation of the Newton directions by solving linear
systems of type (4.38). This is usually done using direct methods (matrix factor-
izations), but iterative linear solvers can be used as well. In most applications the
matrices involved have a special sparsity pattern, which can be exploited to signif-
icantly speed up the computation of the Newton direction. We discuss structure
exploitation for solving (4.38) arising from MPC problems in Section 15.3.
If the barrier parameter is decreased too quickly, many inner iterations are
needed, but only a few outer iterations. The situation is reversed if is decreased
only slowly, which leads to a quick convergence of the inner problems (usually in
one or two Newton steps), but more outer iterations are needed. This tradeo is
not very prominent in practice. Usually, the total number of Newton steps is in
the range of 20 40, essentially independent of the conditioning of the problem.
The barrier method can be shown to converge linearly; more details can be found
in [64, 11.3].
Finally, we would like to point out that the barrier method is a feasible method,
which means that it starts with a feasible point and all iterates remain feasible.
This complicates the implementation in practice, as a line search must be employed
that maintains feasibility, and a strictly feasible initial iterate z
0
has to be supplied
by the user or computed rst by what is called a Phase I method. See [64] for
more details.
Computational experience has shown that modern primal-dual methods are
signicantly more eective with only little additional computational cost when
compared to the barrier method. As a result, primal-dual methods are now pre-
dominant in commercial codes. They also allow infeasible iterates, that is, the
equality and inequality constraints on the primal variables are satised only at
convergence, which alleviates the necessity for nding a feasible initial point. We
discuss this class of methods next.
Primal-dual methods
The general idea of primal-dual interior point methods is to solve the Karush-
Kuhn-Tucker (KKT) conditions by a modied version of Newtons method. The
nonlinear KKT equations represent necessary and sucient conditions for optimal-
ity for convex problems. For (P-IPM), the KKT conditions are
f(z) +A
v +G(z)
u = 0 (4.39a)
Az b = 0 (4.39b)
g(z) +s = 0 (4.39c)
s
i
u
i
= 0, i = 1, . . . , m (4.39d)
(s, u) 0 (4.39e)
where the vector s R
m
+
denotes slack variables for the inequality constraints
g(z) 0, and u R
m
+
is the vector of the associated Lagrange multipliers. The
vector v R
p
denotes the Lagrange multipliers for the equality constraints Az = b,
and the matrix G(z) R
ms
is the Jacobian of g evaluated at z. The variables z
and s are from the primal, the variables v and u from the dual space.
If a primal-dual pair of variables (z
, v
, u
, s
) is found that satises these

conditions, the corresponding primal variables (z
, s
) are optimal for (P-IPM) (and

the dual variables (v
, u
) are optimal for the dual of (P-IPM)). Since Newtons

method is a powerful tool for solving nonlinear equations, the idea of applying it to
the system of nonlinear KKT equations is apparent. However, certain modications
to a pure Newton method are necessary to obtain a useful method for constrained
optimization. The name primal-dual indicates that the algorithm operates in both
the primal and dual space.
There are many variants of primal-dual interior point methods. For a thorough
treatment of the theory we refer the reader to the book by Wright [288], which
gives a comprehensive overview of the dierent approaches. In this section, after
introducing the general framework of primal-dual methods, we restrict ourselves
to the presentation of a variant of Mehrotras predictor-corrector method [205],
because it forms the basis of most existing implementations of primal-dual interior
point methods for convex optimization and has proven particularly eective in
practice. The method we present here allows infeasible starting points.
Central Path
We start by outlining the basic idea. Most methods track the central path, which we
have encountered already in barrier methods, to the solution set. These methods
are called path-following methods. In the primal-dual space, the central path
is dened as the set of points (z, v, u, s), for which the following relaxed KKT
conditions hold:
f(z) +A
v +G(z)
u = 0 (4.40a)
Az b = 0 (4.40b)
g(z) +s = 0 (4.40c)
s
i
u
i
= , i = 1, . . . , m (4.40d)
(s, u) > 0 (4.40e)
In (4.40) as compared to (4.39), the complementarity condition (4.39d) is relaxed
by a scalar > 0, the path parameter, and slacks s and multipliers u are required
to be positive instead of nonnegative (here is where interior property comes into
play). The main idea of primal-dual methods is to solve (4.40) for successively de-
creasing values of , and thereby to generate iterates (z
k
, v
k
, u
k
, s
k
) that approach
(z
, v
, u
, s
) as 0. The similarity with the barrier method in this regard

is evident. However, the Newton step is modied in primal-dual interior point
methods, as we shall see below.
Measure of Progress
Similar to primal barrier methods, the subproblem (4.40) is usually solved only
approximately, here by applying only one Newton step. As a result, the generated
iterates are not exactly on the central path, i.e. s
k
u
k
,= for some i. Hence a more
useful measure than the path parameter for the progress of the algorithm is the
average value of the complementarity condition (4.40d),
= (s
u)/m (4.41)
evidently, 0 implies 0. In interior point nomenclature, is called the
duality measure, as it corresponds to the duality gap scaled by 1/m if all other
equalities in (4.40) are satised. To be able to reduce substantially (at least by a
constant factor) at each iteration of the method, the pure Newton step is enhanced
with a centering component, as discussed next.
Newton Directions and Centering
A search direction is obtained by linearizing (4.40) at the current iterate (z, v, u, s)
and solving
_
_
H(z, u) A
G(z)
0
A 0 0 0
G(z) 0 0 I
0 0 S Z
_
_
_
_
z
v
u
s
_
_
=
_
_
f(z) +A
v +G(z)
u
Az b
g(z) +s
Su
_
_
(4.42)
where S = diag(s
1
, . . . , s
m
) and Z = diag(u
1
, . . . , u
m
), the (1,1) block in the coef-
cient matrix is
H(z, u) =
2
f(z) +
m
i=1
u
i
2
g
i
(z) (4.43)
and the vector R
m
allows a modication of the right-hand side, thereby gen-
erating dierent search directions that depend on the particular choice of . For
brevity, we denote the solution to (4.42) by [z, v, u, s]().
s
1
z
1
current iterate
Newton direction
resulting search
centering direction
central path c
optimal point
(x, y, z, s)
(predictor)
(x, y, z, s)(0)
direction
(x, y, z, s)()
Figure 4.5 Ance-scaling (pure Newton) and centering search directions of
classic primal-dual methods. The Newton direction makes only progress
towards the solution when it is computed from a point close to the central
path, hence centering must be added to the pure Newton step to ensure that
the iterates stay suciently close to the central path.
The the vector can takes a value between the following two extremes:
If = 0, the right hand side in (4.42) corresponds to the residual of the KKT
conditions (4.39), i.e. [z, v, u, s](0) is the pure Newton direction that aims
at satisfying the KKT conditions (4.39) in one step based on a linearization.
This direction is called ane-scaling direction. However, pure Newton steps
usually decrease some products s
k
u
k
prematurely towards zero, i.e. the gen-
erated iterate is close to the boundary of the feasible set (if s
k
u
k
0, either
s
k
or u
k
must be small). This is disadvantageous since from such points only
very small steps can be taken, and convergence would be prohibitively slow.
Hence the iterates must be centered by bringing them closer to the central
path.
If = 1, the complementarity condition (4.40d) is modied such that the
resulting search direction aims at making the individual products s
k
u
k
equal
to their average value . This is called centering. A centering step does not
decrease the average value, so no progress towards the solution is made.
A tradeo between the two goals of progressing towards the solution and centering
is achieved by computing search directions [z, v, u, s]() by solving (4.42) for
= 1 (4.44)
where (0, 1) is called the centering parameter. This is the key dierence to pure
Newton steps, and the main ingredient (together with tracking the central path)
in making Newtons method work eciently for solving the nonlinear KKT sys-
tem (4.39). Figure 4.5 depicts this combination of search directions schematically,
and a principle framework for primal-dual methods is given in Algorithm (4.10).
We will instantiate it with Mehrotras method below.
In the discussion above, we have excluded almost all theoretical aspects con-
cerned with convergence of primal-dual methods. For example, the terms sucient
close to the central path and sucient reduction in have precise mathematical
counterparts that allow one to analyze the convergence properties. The book by
Wright [288] is a good starting point for the interested reader.
Algorithm 4.10 Primal-dual interior point methods for (P-IPM)
input: Initial iterates z
0
, v
0
, u
0
> 0, s
0
> 0, Centering parameter (0, 1)
repeat
Duality measure:
k
= s
k
u
k
/m
Search direction: compute [z
k
, v
k
, u
k
, s
k
](
k
1) by solving (4.42) at the
current iterate
Choose step length h
k
such that s
k+1
> 0 and u
k+1
> 0
Update: (z
k+1
, v
k+1
, u
k+1
, s
k+1
) = (z
k
, v
k
, u
k
, s
k
)+h
k
[z
k
, v
k
, u
k
, s
k
](
k
k
1)
Predictor-Corrector Methods
Modern methods use the full Newton step [z, v, u, s](0) merely as a predictor to
estimate the error made by using a linear model of the central path. If a full step
along the ane-scaling direction were taken, the complementarity condition would
evaluate to
(S + S
a
)(u + u
a
) = Su + S
a
u +Su
a
. .
=0
+S
a
u
a
(4.45)
where capital letters denote diagonal matrices constructed from the correspond-
ing vectors. The rst three terms sum up to zero by the last equality of (4.42).
Hence the full Newton step produces an error of S
a
u
a
in the complemen-
tarity condition. In order to compensate for this error, a corrector direction
[z, v, u, s](S
a
u
a
) can be added to the Newton direction. This compen-
sation is not perfect since usually > 0, that is the predictor direction is only an
approximation; nevertheless, predictor-corrector methods work usually better than
single-direction methods. The principle of predictor-corrector methods is depicted
in Fig. 4.6.
An important contribution of Mehrotra was to dene an adaptive rule for choos-
ing the centering parameter based on the information from the predictor step.
His rule
= (
a
/)
3
(4.46)
increases the centering if the progress would be poor, that is if the predicted value
of the duality measure,
a
, is not signicantly smaller than . If good progress
can be made, then 1, and the direction along which the iterate is moved is
closer to the pure Newton direction. The corrector and centering direction can be
calculated in one step because [z, v, u, s]() is linear in ; the nal search direction
is therefore given by [z, v, u, s](1 S
a
u
a
).
A variant of Mehrotras predictor-corrector method that works well in practice
is given in Algorithm 4.11. The parameter is a safeguard against numerical errors,
s
1
z
1
current iterate
Newton direction
centering
resulting search
central path c
optimal point
(x, y, z, s)
(predictor)
(x, y, z, s)(0)
+ corrector direction
predicted point
(z
a
, s
a
)
(x, y, z, s)(1 S
a
z
a
)
direction (Mehrotra)
Figure 4.6 Search direction generation in predictor-corrector methods. The
ane search direction can be used to correct for linearization errors, which
results in better performance in practice due to closer tracking of the central
path.
keeping the iterates away from the boundary. Because of the predictor-corrector
scheme, two linear systems have to be solved in each iteration. Since the coecient
matrix is the same for both solves, direct methods are preferred to solve the linear
systems in Steps 3 and 7 of the algorithm, as the computational bottleneck of the
method is the factorization of the coecient matrix. Once it has been factored in
Step 3, the computational cost of the additional solve in Step 7 is negligible, and
is more than compensated by the savings in the number of iterations.
Algorithm 4.11 Mehrotra primal-dual interior point method for (P-IPM)
input: Initial iterates z
0
, v
0
, u
0
> 0, s
0
> 0, < 1 (typically = 0.99)
repeat
Duality measure:
k
= s
k
u
k
/m
Newton direction (predictor): compute [z
k,a
, v
k,a
, u
k,a
, s
k,a
](0) by
solving (4.42) at the current iterate
Line search: h
k,a
= max|h [0, 1] [ s
k
+hs
k,a
0, u
k
+hu
k,a
0
Predicted (ane) duality measure:
k,a
= (s
k
+h
k,a
s
k,a
)
(u
k
+h
k,a
u
k,a
)/m
Centering parameter:
k
= (
k,a
/
k
)
3
Final search direction: compute [z
k
, v
k
, u
k
, s
k
](
k
k
1 S
k,a
u
k,a
)
by solving (4.42) at
current iterate
Line search: h
k
= max|h [0, 1] [ s
k
+hs
k
0, u
k
+hu
k
0
Update: (z
k+1
, v
k+1
, u
k+1
, s
k+1
) = (z
k
, v
k
, u
k
, s
k
)+h
k
[z
k
, v
k
, u
k
, s
k
](
k
k
1
S
k,a
u
k,a
)
Relation of Primal-Dual to Primal Barrier Methods
An insightful relation between primal barrier and primal-dual methods can be
established as follows. Writing the optimality conditions for (P()) gives
f(x) +
g
(x) +A
y = 0 (4.47a)
Ax = b (4.47b)
g(x) 0 (4.47c)
where g(x) 0 has been added for the barrier function
g
to be well dened. If
one now denes
z = diag(g(x))
1
1, (4.48)
the conditions (4.47) become
f(x) +
m
i=1
z
i
g
i
(x) +A
y = 0 (4.49a)
Ax = b (4.49b)
g(x) +s = 0 (4.49c)
Sz = 1 (4.49d)
(s, z) 0 (4.49e)
where we have introduced slack variables s and used the expression for the gradient
of the barrier function
g
and the relations s = g(x), S = diag(s). Using
m
i=1
z
i
g
i
(x) = G(x)
z , (4.50)
where G(x) is the Jacobian of g evaluated at the point x, yields the same relaxed
optimality conditions as (4.40).
Hence if subproblems P() and (4.40) were solved exactly assuming (4.48), the
primal iterates of the primal-barrier method and the primal-dual method would
coincide (provided that the path parameters coincide, i.e. = ). However, primal-
dual methods are in practice more ecient than primal barrier methods, since they
generate dierent search directions using also information from the dual space, and
therefore iterates generated by the two algorithms do not coincide in general.
4.2.3 Active Set Methods
In this section we will discuss active set strategies for LPs and QPs because they are
the basis for multi-parametric programming in Chapter 7. An active set method
uses a combinatorial approach to iteratively determine the set of constraints active
at the optimum.
Active Set Method for LP
Consider the LP
min
z
c
z
subj. to Gz w,
(4.51)
and the associated KKT conditions dening the optimum
G
u +c = 0, (4.52a)
u
i
(G
i
z w
i
) = 0, i = 1, . . . , m (4.52b)
u 0, (4.52c)
Gz w 0. (4.52d)
Let us assume that we start our search for the optimum from a feasible vertex z
k
in the solution space. At this vertex, a set of constraints indicated by the matrices
G
A
k,w
A
k is active (see Section 3.2.4). The non-active set of constraints is denoted
as G
NA
kz
k
< w
NA
k . Then the KKT conditions can be rewritten as
_
0 G
A
k
G
A
k 0
_ _
z
k
u
A
k
_
=
_
c
w
A
k
_
(4.53a)
G
NA
k z
k
w
NA
k < 0 (4.53b)
u
A
k 0. (4.53c)
Let z
k,
, u
A
k
denote a solution of (4.53a). If rank(G
A
k ) = s then the primal
solution z
k,
is unique. If rank(G
A
k ) < s then (4.53a) denes an edge or a higher
dimensional hyperplane. If z
k,
satises (4.53b), then it is feasible for the LP (4.51).
Moreover, if u
A
k
obtained from the solution of (4.53a) satises the dual feasibility
conditions (4.53c), then z
k,
is a solution of the LP (4.51).
An active set strategy aims to determine iteratively the set of active constraints
A
k
and the associated solution [z
k,
, u
A
k
] of (4.53a), for which the primal and
dual feasibility conditions (4.53b)-(4.53c) are satised. Dierent sets of active
constraints imply dierent vertices in the solution space. Thus changing the set of
active constraints implies moving from one vertex to another.
When one searches for the optimal set of active constraints one generally re-
moves and adds one constraint at a time while satisfying primal feasibility (4.53b).
The violation of the dual feasibility conditions (4.53c) gives an indication how to
proceed. One looks for the dual variable constraint (4.53c), which is violated the
most and removes the associated primal constraint from the set of active con-
straints. One then adds one of the inactive constraints to the set of active con-
straints, where the choice is made such that the solution of the resulting new
Lagrangian equations is primal feasible. In this way one moves along an edge to a
neighboring feasible vertex in the solution space.
We assumed that our solution procedure starts with a feasible vertex. If a
feasible point is not obvious, e.g., when z = 0 is not feasible, then in Phase 1 of
the procedure a modied LP is dened and solved to determine a feasible point
inf
z,s
1
s
subj. to Gz s w,
s 0.
(4.54)
For this Phase 1 LP a feasible starting point is z = 0 and s
i
= max0, w
i
, i =
1, . . . , m. We can now apply the active set method presented above to determine
the optimizer ( z, s) of (4.54). If s = 0, then z is a feasible starting point for the
original LP (4.51). Otherwise the original LP (4.51) is infeasible.
Many details aect the convergence and eciency of the active set strategy.
First of all, which constraint to remove and add and not to cycle back to a
constraint set already considered, is critical for convergence. Also, the algorithm
needs to be able to handle degenerate cases (see Section 3.2.4). Finally, the solution
of the system of equations (4.53a) needs to be ecient. Here one can take advantage
of the fact that at each iteration only one equation changes. All these points are
addressed by the Simplex method [215].
Active Set Method for QP
Let us consider the QP in standard form
min
z
1
2
z
Hz +q
z +r
subj. to Gz w,
(4.55)
for which the KKT conditions are
Hz +q +G
u = 0 (4.56a)
u
i
(G
i
z w
i
) = 0, i = 1, . . . , m (4.56b)
u 0 (4.56c)
Gz w 0. (4.56d)
Just like in the LP case in the previous section we want to determine the subset of
constraints, which are active at the optimum. For the QP, however, the number of
constraints which are active is not known a priori, because the solution does not
necessarily lie at a vertex in the solution space (Section 3.3).
Before presenting an active set method for solving the QP (4.55), we will rst
consider a subset A of the constraints, which are active and the corresponding QP
with these equality constraints only.
min
z
1
2
z
Hz +q
z +r
subj. to G
A
z = w
A
.
(4.57)
For this special case the KKT conditions (3.31a)-(3.31b), can be compactly written
as
_
H G
A
G
A
0
_ _
z
A
u
A
_
=
_
q
w
A
_
. (4.58)
The solution to (4.58) has the form
z
A
=Lq +Tw
A
, (4.59a)
u
A
=T
q +Sw
A
. (4.59b)
The symmetric matrix in (4.58) is referred as the Lagrangian matrix or KKT
matrix. If the Lagrangian matrix is nonsingular then the matrices L, T and S can
be extracted from its inverse
_
L T
T
S
_
=
_
H G
A
G
A
0
_
1
. (4.60)
Several alternatives for solving (4.60) can be found in [105, p. 236]. In particu-
lar, there are instances when the Lagrangian matrix in (4.58) is not invertible or
stability problems occur.
As a simple alternative method, consider the case of a full row rank matrix G
A
with m < s rows. Any feasible point z can be written as z = Y w
A
+ Zy, where
y R
sm
, and Y and Z satisfying the properties G
A
Y = I, G
A
Z = 0 and [Y Z]
non-singular.
One possibility for computing z
A
and u
A
is to use the QR factorization of the
matrix G
A
G
A
= Q
_
R
0
_
= [Q
1
Q
2
]
_
R
0
_
= Q
1
R (4.61)
where Q R
ss
is orthogonal, R R
mm
is upper triangular and Q
1
R
sm
and
Q
2
R
s(sm)
. Then the choice
Y = Q
1
R
1
, Z = Q
2
(4.62)
satises the desired properties for Y and Z. With such a choice, the matrices in
the solution (4.59) can be computed as (cf. [105, Eqn. 10.2.7])
L = Z(Z
HZ)
1
Z
,
T = Y Z(Z
HZ)
1
Z
HY,
S = Y
HZ(Z
HZ)
1
Z
HY +Y
HY.
(4.63)
Alternative approaches for computing z
A
and u
A
can be found in [105].
Let us return to the original QP (4.55) At each iteration k the following auxiliary
problem is solved
min
k
1
2
H
k
+ (q +Hz
k
)
k
subj. to G
A
k
k
= 0
(4.64)
where z
k
is a known feasible point for the QP (3.25) and A
k
= j 1 : G
j
z
k
= w
j
is the associated active constraint set. We want to determine the direction

k
to
move from the feasible z
k
, i.e. z
k+1
= z
k
+
k
.
Problem (4.64) is an equality constrained QP. Let
k,
R
s
denote the op-
timizer of (4.64) and let u
k,
R
|A|
be the corresponding vector of Lagrange
multipliers. We note that
k,
is the best correction to z
k
in the direction where
the constraints A
k
are active. Since (4.64) is just a special case of (4.57) it can be
solved as explained before. In particular, with the QR decomposition of G
A
k as in
(4.61) for A = A
k
, and Z and Y dened by (4.62) we get
k,
= Z(Z
HZ)
1
Z
(q +Hz), (4.65)
u
k,
= (Y
HZ(Z
HZ)
1
Z
)(q +Hz). (4.66)

Three cases are possible after solving (4.64):
1.
k,
= 0. If all Lagrange multipliers are nonnegative then z
k
is the opti-
mum for (4.55). Otherwise, nd the constraint with the smallest Lagrange
multiplier, remove it from A, and iterate the procedure.
2.
k,
,= 0 and z
k
+
k,
is feasible for (4.55). Set z = z
k
+
k,
, keep A
k
, and
iterate the procedure.
3.
k,
,= 0 and z
k
+
k,
is not feasible for (4.55). Then nd the best feasible
point z = z
k
+
k,
with a line search in the direction of
k,
. Add the newly
activated constraints (by z) to A
k
and iterate the procedure.
The whole procedure is summarized in Algorithm 4.12.
Algorithm 4.12 Active set method for QP (4.55)
input: H, q, G, w, initial feasible z
0
output: z
k = 0, A
0
|i 1 [ Giz
0
= wi
repeat
Compute
k,
by (4.65)
if
k,
= 0
Compute u
k,
by (4.66)
if u
k,
0 return z
z
k
endif
A A\ A(arg min
iA
u
k,
i
)
Compute
k,
by (4.65)
endif
Compute
= min
_
1, min
jI\A, G
j
k,
>0
wj Gjz
k
Gj
k,
_
(4.67)
z
k+1
z
k
+
k,
if < 1
A A |j
, where j
is the arg min of (4.67)

endif
k k + 1
Chapter 5
Polyhedra and P-collections
Polyhedra are the fundamental geometric objects used in this book. There is a
vast body of literature related to polyhedra because they are important for a wide
range of applications. In this chapter we introduce the main denitions and the
algorithms which describe some operations on polyhedra. Our intent is to provide
only the necessary elements for the readers interested in reproducing the control
algorithms reported in this book. Most denitions given here are standard. For
additional details the reader is referred to [294, 134, 108].
5.1 General Set Denitions and Operations
An n-dimensional ball B(x
c
, ) is the set B(x
c
, ) = x R
n
: |x x
c
|
2
.
The vector x
c
is the center of the ball and is the radius.
Ane sets are sets described by the solutions of a system of linear equations:
T = x R
n
: Ax = b, with A R
mn
, b R
m
. (5.1)
If T is an ane set and x T, then the translated set 1 = x x : x T is a
subspace.
The ane combination of a nite set of points x
1
, . . . , x
k
belonging to R
n
is
dened as the point
1
x
1
+. . . +
k
x
k
where
k
i=1

i
= 1.
The ane hull of / R
n
is the set of all ane combinations of points in /
and it is denoted as a(/):
a(/) =
1
x
1
+. . . +
k
x
k
: x
i
/, i = 1, . . . , k,
k
i=1
i
= 1. (5.2)
The ane hull of / is the smallest ane set that contains / in the following sense:
if o is any ane set with / o, then a(/) o.
The dimension of an ane set, ane combination or ane hull is the dimen-
sion of the largest ball of radius > 0 included in the set.
70 5 Polyhedra and P-collections
Example 5.1 The set
T = |x R
2
: x1 +x2 = 1
is an ane set in R
2
of dimension one. The points x
1
= [0, 1]
and x
2
= [1, 0]
belong
to the set T. The point x = 0.2x
1
+1.2x
2
= [1.2, 0.2]
is an ane combination of
points x
1
and x
2
. The ane hull of x
1
and x
2
, a(|x
1
, x
2
), is the set T.
Convex sets have been dened in Section 1.2.
The convex combination of a nite set of points x
1
, . . . , x
k
belonging to R
n
is
dened as the point
1
x
1
+. . . +
k
x
k
where
k
i=1

i
= 1 and
i
0, i = 1, . . . , k.
The convex hull of a set / R
n
is the set of all convex combinations of points
in / and it is denoted as conv(/):
conv(/) =
1
x
1
+. . . +
k
x
k
: x
i
/,
i
0, i = 1, . . . , k,
k
i=1
i
= 1. (5.3)
The convex hull of / is the smallest convex set that contains / in the following
sense: if o is any convex set with / o, then conv(/) o.
Example 5.2 Consider three points
1
x
1
= [1, 1]
, x
2
= [1, 0]
, x
3
= [0, 1]
in R
2
. The
point x =
1
x
1
+
2
x
2
+
3
x
3
with
1
= 0.2,
2
= 0.2,
3
= 0.6 is x = [0.4, 0.8]
and it
is a convex combination of the points |x
1
, x
2
, x
3
. The convex hull of |x
1
, x
2
, x
3
is the triangle plotted in Figure 5.1. Note that any set in R

2
strictly contained in the
triangle and containing |x
1
, x
2
, x
3
is non-convex. This illustrates that the convex
hull is the smallest set that contains these three points.
x
1
x
2 x
-0.5 0 0.5 1 1.5 2
-0.5
0
0.5
1
1.5
2
Figure 5.1 Illustration of the convex hull of three points x
1
= [1, 1]
, x
1
=
[1, 0]
, x
3
= [0, 1]
.
1
Click here to download the Matlab c code.
5.2 Polyhedra and Representations 71
A cone spanned by a nite set of points / = x
1
, . . . , x
k
is dened as
cone(/) =
k
i=1
i
x
i
,
i
0, i = 1, . . . , k. (5.4)
We dene cone(/) = 0 if / is the empty set.
Example 5.3 Consider three points
2
x
1
= [1, 1, 1]
, x
2
= [1, 2, 1]
, x
3
= [1, 1, 2]
in
R
3
. The cone spanned by of |x
1
, x
2
, x
3
is an unbounded set. Its restriction to the
set with x
3
3 is depicted in Figure 5.2.
x
1
x
2
x
3
0
2
4
0
2
4
0
2
4
Figure 5.2 Illustration of a cone spanned by the three points x
1
= [1, 1, 1]
,
x
2
= [1, 2, 1]
, x
3
= [1, 1, 2]
.
The Minkowski sum of two sets T, Q R
n
is dened as
T Q = x +y : x T, y Q. (5.5)
By denition, any point in T Q can be written as the sum of two points, one in
T and one in Q. For instance, the Minkowski sum of two balls (T and Q) centered
at the origin and with radius 1, is a ball (T Q) centered at the origin and with
radius 2.
5.2 Polyhedra and Representations
Some of the concepts here have been briey touched in Section 3.1. In the following
we give two denitions of a polyhedron. They are mathematically equivalent but
they lead to two dierent polyhedron representations. The proof of equivalence is
not trivial and can be found in [294].
An 1-polyhedron T in R
n
denotes an intersection of a nite set of closed half-
spaces in R
n
:
T = x R
n
: Ax b, (5.6)
where Ax b is the usual shorthand form for a system of inequalities, namely
a
i
x b
i
, i = 1, . . . , m, where a
1
, . . . , a
m
are the rows of A, and b
1
, . . . , b
m
are the
components of b. In Figure 5.3 a two-dimensional 1-polyhedron is plotted.
2
a
1
x
b
1
a
2
x
b
2
a
3 x
b
3
Figure 5.3 -polyhedron.
a
1
x
b
1
a
2
x
b
2
a
3 x
b
3
a
4
x
b
4
Figure 5.4 -polytope. The planes (here lines) dening the halfspaces are
a
i
x bi = 0. The arrows represent the direction of ai into the valid
halfspace.
A 1-polyhedron T in R
n
denotes the Minkowski sum (dened in (5.5)) of the
convex hull of a nite set of points V
1
, . . . , V
k
of R
n
and the cone generated by
a nite set of vectors y
1
, . . . , y
k
of R
n
:
T = conv(V ) cone(Y ), (5.7)
for some V = [V
1
, . . . , V
k
] R
nk
, Y = [y
1
, . . . , y
k
] R
nk
. The main theorem

for polyhedra states that any 1-polyhedron can be converted into a 1-polyhedron
and vice-versa [294, p.30].
An 1-polytope is a bounded 1-polyhedron (in the sense that it does not contain
any ray x +ty : t 0). In Figure 5.4 a two-dimensional 1-polytope is plotted.
5.2 Polyhedra and Representations 73
A 1-polytope is a bounded 1-polyhedron
T = conv(V ) = V [ 0, 1
= 1 (5.8)
In Section 5.4.4 we show how to convert any 1-polytope into a 1-polytope and
vice versa.
The dimension of a polytope (polyhedron) T is the dimension of its ane hull
and is denoted by dim(T). We say that a polytope T R
n
, T = x R
n
:
P
x
x P
c
, is full-dimensional if dim(T) = n or, equivalently, if it is possible to
t a nonempty n-dimensional ball in T,
x R
n
, > 0 : B(x, ) T, (5.9)
or, equivalently,
x R
n
, > 0 : ||
2
P
x
(x +) P
c
. (5.10)
Otherwise, we say that polytope T is lower-dimensional. A polytope is referred to
as empty if
x R
n
: P
x
x P
c
. (5.11)
Furthermore, if |P
x
i
|
2
= 1, where P
x
i
denotes the i-th row of a matrix P
x
, we say
that the polytope T is normalized.
Let T be a polyhedron. A linear inequality c
z c
0
is said to be valid for T if
it is satised for all points z T. A face of T is any nonempty set of the form
T = T z R
s
: c
z = c
0
, (5.12)
where c
z c
0
is a valid inequality for T. The dimension of a face is the dimension
of its ane hull. For the valid inequality 0z 0 we get that T is a face of T. All
faces of T satisfying T T are called proper faces and have dimension less than
dim(T). The faces of dimension 0,1, dim(T)-2 and dim(T)-1 are called vertices,
edges, ridges, and facets, respectively. The set V of all the vertices of a polytope
T will be denoted as
V = extreme(T).
The next theorem summarizes basic facts about faces.
Theorem 5.1 Let T be a polytope, V the set of all its vertices and T a face.
1. T is the convex hull of its vertices: T=conv(V ).
2. T is a polytope.
3. Every intersection of faces of T is a face of T.
4. The faces of T are exactly the faces of T that are contained in T.
5. The vertices of T are exactly the vertices of T that are contained in T.
A d-simplex is a polytope of R
d
with d + 1 vertices.
In this book we will work mostly with 1-polyhedra and 1-polytopes. This
choice has important consequences for algorithm implementations and their com-
plexity. As a simple example, a unit cube in R
d
can be described through 2d
equalities as an 1-polytope, but requires 2
d
points in order to be described as
a 1-polytope. We want to mention here that many ecient algorithms that
work on polyhedra require both 1 and 1 representations. In Figure 5.5 the 1-
representation and the 1-representation of the same polytope in two dimensions
are plotted.
x
1
x
2
(a) 1-representation. The ver-
tices V
P
1
, . . . , V
P
7
are depicted as
dots. Download code.
x
1
x
2
(b) -representation. The hyper-
planes P
x
i
x = P
c
i
, i = 1, . . . , 7 are
depicted as lines. Download code.
Figure 5.5 Illustration of a polytope in - and 1- representation.
Consider Figure 5.5(b) and notice that no inequality can be removed without
changing the polyhedron. Inequalities which can be removed without changing the
polyhedron described by the original set are called redundant. The representation
of an 1-polyhedron is minimal if it does not contain redundant inequalities. De-
tecting whether an inequality is redundant for an 1-polyhedron requires solving a
linear program, as described in Section 5.4.1.
By denition a polyhedron is a closed set. In this book we will also work with
sets which are not closed but whose closure is a polyhedron. For this reason the
two following non-standard denitions will be useful.
Denition 5.1 (Open Polyhedron (Polytope)) A set c R
n
is called an
open polyhedron (polytope) if it is open and its closure is a polyhedron (polytope).
Denition 5.2 (Neither Open nor Closed Polyhedron (Polytope)) A set c
R
n
is called neither an open nor a closed polyhedron (polytope) if it is neither open
nor closed and its closure is a polyhedron (polytope).
5.3 Polytopal Complexes 75
5.3 Polytopal Complexes
According to our denition every polytope represents a convex, compact (i.e.,
bounded and closed) set. In this book we will also encounter sets that are dis-
joint or non-convex but can be represented by the union of a nite number of
polytopes. Therefore, it is useful to dene the following mathematical concepts.
Denition 5.3 (P-collection) A set c R
n
is called a P-collection (in R
n
) if
it is a collection of a nite number of n-dimensional polyhedra, i.e.
c = c
i
NC
i=1
, (5.13)
where c
i
= x R
n
: C
x
i
x C
c
i
, dim(c
i
) = n, i = 1, . . . , N
C
, with N
C
< .
Denition 5.4 (Underlying Set) The underlying set of a P-collection c = c
i
NC
i=1
is the
c =
_
PC
T =
NC
_
i=1
c
i
. (5.14)
Example 5.4 A collection c = |[2, 1], [0, 2], [2, 4] is a P-collection in R
1
with the
underlying set c = [2, 1] [0, 4]. As another example, c = |[2, 0], [1, 1], [0, 2]
is a P-collection in R
1
with underlying set c = [2, 2]. Clearly, polytopes that dene
a P-collection can overlap, the underlying set can be disconnected and non-convex.
Usually it is clear from the context if we are talking about the P-collection or
referring to the underlying set of a P-collection. Therefore for simplicity, we use
the same notation for both.
Denition 5.5 (Strict Polyhedral Partition) A collection of sets c
i
NC
i=1
is a
strict partition of a set c if (i)
NC
i=1
c
i
= c and (ii) c
i
c
j
= , i ,= j. Moreover
c
i
NC
i=1
is a strict polyhedral partition of a polyhedral set c if c
i
NC
i=1
is a strict
partition of c and

c
i
is a polyhedron for all i, where

c
i
denotes the closure of the
set c
i
.
Denition 5.6 (Polyhedral Partition) A collection of sets c
i
NC
i=1
is a parti-
tion of a set c if (i)
NC
i=1
c
i
= c and (ii) (c
i
c
i
) (c
j
c
j
) = , i ,= j. Moreover
c
i
NC
i=1
is a polyhedral partition of a polyhedral set c if c
i
NC
i=1
is a partition of c
and c
i
is a polyhedron for all i. The set c
j
is the boundary of the set c
j
.
Note that in a strict polyhedral partition of a polyhedron some of the sets C
i
must be open or neither open nor closed. In a partition all the sets may be closed
and points on the closure of a particular set may also belong to one or several other
sets. Also, note that a polyhedral partition is a special class of a P-collection.
5.3.1 Functions on Polytopal Complexes
Denition 5.7 A function h() : R, where R
s
, is piecewise ane
(PWA) if there exists a strict partition R
1
,. . . ,R
N
of and h() = H
i
+ k
i
,
R
i
, i = 1, . . . , N.
s
, is piecewise ane on
polyhedra (PPWA) if there exists a strict polyhedral partition R
1
,. . . ,R
N
of and
h() = H
i
+k
i
, R
i
, i = 1, . . . , N.
s
, is piecewise quadratic
(PWQ) if there exists a strict partition R
1
,. . . ,R
N
of and h() =
H
i
+k
i
+l
i
,
R
i
, i = 1, . . . , N.
s
, is piecewise quadratic
on polyhedra (PPWQ) if there exists a strict polyhedral partition R
1
,. . . ,R
N
of
and h() =
H
i
+k
i
+l
i
, R
i
, i = 1, . . . , N.
As long as the function h() we are dening is continuous it is not important
if the partition constituting the domain of the function is strict or not. If the
function is discontinuous at points on the closure of a set, then this function can
only be dened if the partition is strict. Otherwise we may obtain several conicting
denitions of function values (or set-valued functions). Therefore for the statement
of theoretical results involving discontinuous functions we will always assume that
the partition is strict. For notational convenience, however, when working with
continuous functions we will make use of partitions rather than strict partitions.
5.4 Basic Operations on Polytopes
We will now dene some basic operations and functions on polytopes. Note that
although we focus on polytopes and polytopic objects most of the operations de-
scribed here are directly (or with minor modications) applicable to polyhedral ob-
jects. Additional details on polytope computation can be found in [294, 134, 108].
All operations and functions described in this chapter are contained in the MPT
toolbox [148].
5.4.1 Minimal Representation
We say that a polytope T R
n
, T = x R
n
: P
x
x P
c
is in a minimal
representation if the removal of any row in P
x
x P
c
would change it (i.e., if
there are no redundant constraints). The computation of the minimal representa-
tion (henceforth referred to as polytope reduction) of polytopes is discussed in [108].
The redundancy of each constraint is checked, which generally requires the solution
of one LP for each half-space dening the non-minimal representation of T. We
5.4 Basic Operations on Polytopes 77
summarize this simple implementation of the polytope reduction in Algorithm 5.1.
An improved algorithm for polytope reduction is discussed in [268] where the au-
thors combine the procedure outlined in Algorithm 5.1 with heuristic methods,
such as bounding-boxes and ray-shooting, to discard redundant constraints more
eciently.
It is straightforward to see that a normalized, full-dimensional polytope T has
a unique minimal representation. Note that unique here means that for T = x
R
n
: P
x
x P
c
the matrix [P
x
P
c
] consists of the unique set of row vectors,
the rows order is irrelevant. This fact is very useful in practice. Normalized, full-
dimensional polytopes in a minimal representation allow us to avoid any ambiguity
when comparing them and very often speed-up other polytope manipulations.
Algorithm 5.1 Polytope in minimal representation
input T = |x : P
x
x P
c
, with P
x
R
n
P
n
, P
c
R
n
P
output Q = |x : Q
x
x Q
c
= minrep(T)
let 1 |1, . . . , nP
for i = 1 to nP
let 1 1 \ |i
let f
max
x
P
x
i
x, subj. to P
x
(I)
x P
c
(I)
, P
x
i
x P
c
i
+ 1
if f
> P
c
i
then 1 1 |i
end
let Q
x
= P
x
(I)
, Q
c
= P
c
(I)
5.4.2 Convex Hull
The convex hull of a set of points V = V
i
NV
i=1
, with V
i
R
n
, is a polytope dened
as
conv(V ) = x R
n
: x =
NV
i=1
i
V
i
,
i
0,
NV
i=1
i
= 1. (5.15)
The convex hull of a union of polytopes 1
i
R
n
, i = 1, . . . , N
R
, is a polytope
conv
_
NR
_
i=1
1
i
_
= x R
n
: x =
NR
i=1
i
x
i
, x
i
1
i
,
i
0,
NR
i=1
i
= 1. (5.16)
An illustration of the convex hull operation is given in Figure 5.6. Construction
of the convex hull of a set of polytopes is an expensive operation which is expo-
nential in the number of facets of the original polytopes. An ecient software
implementation is available in [107].
5.4.3 Envelope
The envelope of two 1-polyhedra T = x R
n
: P
x
x P
c
and Q = x R
n
:
Q
x
x Q
c
is an 1-polyhedron
env(T, Q) = x R
n
:

P
x
x

P
c
,

Q
x
x

Q
c
, (5.17)
where

P
x
x

P
c
is the subsystem of P
x
x P
c
obtained by removing all the
inequalities not valid for the polyhedron Q, and

Q
x
x

Q
c
is dened in a similar
way with respect to Q
x
x Q
c
and T [35]. In a similar fashion, the denition can
be extended to the case of the envelope of a P-collection. An illustration of the
envelope operation is depicted in Figure 5.7.
The computation of the envelope is relatively cheap since it only requires the
solution of one LP for each facet of T and Q. In particular, if a facet of Q (and T)
is found to be non redundant for the polytope T (for the polytope Q ) then it is
not part of the envelope. A version of Algorithm 5.1 can be used. Note that the
envelope of two (or more) polytopes is not necessarily a bounded set (e.g. when
T Q is shaped like a star).
5.4.4 Vertex Enumeration
The operation of extracting the vertices of a polytope T given in 1-representation
is referred to as vertex enumeration. The necessary computational eort is expo-
nential in the number of facets. Two dierent approaches for vertex enumeration
are commonly used: the double description method [110] and reverse search [12].
An ecient implementation of the double description method is available in [107].
Converting a 1-polytope into an 1-polytope, the reverse operation, is also done
via vertex enumeration as we will explain next. The procedure is based upon the
polar dual.
Denition 5.11 The polar dual of a set T is
T = y [ y
x 1, x T .
Lemma 5.1 If T = x [ Ax 1
3
, then
T = A
[ 0, 1
= 1.
Proof: From the denition of the polar dual, we see that
T is exactly the set

of all inequalities that support T:
T = y [ y
x 1 x such that Ax 1
Lemma 5.2 If T = V
[ 0, 1
= 1, then
T = x [ V x 1.
3
1 denotes a column vector of ones of appropriate length.
x
1
x
2
1
1
1
2
(a) 1 =
i{1,2}
1i.
x
1
x
2
conv(1)
(b) Convex hull of 1.
Figure 5.6 Illustration of the convex hull operation. Download code.
x
1
x
2
1
1
1
2
(a) 1 =
i{1,2}
1i.
x
1
x
2
env(1)
(b) Envelope env(1).
Figure 5.7 Illustration of the envelope operation. Download code.
Proof:
T = y [ y
x 1 x = V
, 0, 1
= 1
= y [
V y 1 0, 1
= 1
= y [ V y 1
Lemma 5.3
T = T.
Procedure for converting a 1-polytope into an 1-polytope
Given a 1-polytope T = V
[ 0, 1
= 1
1. Use Lemma 5.2 to compute the polar dual, which is in 1-representation
T = x [ V x 1
2. Use vertex enumeration to compute the 1-representation of the dual
T = A
[ 0, 1
= 1
3. Use Lemma 5.2 and Lemma 5.3 to recover the 1-representation of T
T = T = x [ Ax 1
Alternatively we can convert a 1-polytope into a 1-polytope by a convex hull
computation. We use rst Lemma 5.1 to obtain a 1-representation of the dual.
Then we compute the convex hull to obtain an 1-representation. Finally we invoke
Lemma 5.3 and Lemma 5.1 to obtain the 1-representation.
Because of the discussed symmetry we say that vertex enumeration and con-
vex hull computation are dual to each other.
Above we have assumed that the 1-polytope is normalized Ax 1, which
requires the polytope to contain the origin. Otherwise we need to rst shift the
polytope so that it contains the origin.
5.4.5 Chebychev Ball
The Chebychev Ball of a polytope T = x R
n
: P
x
x P
c
, with P
x
R
nP n
,
P
c
R
nP
, corresponds to the largest radius ball B(x
c
, R) with center x
c
, such that
B(x
c
, R) T. The center and radius of the Chebychev ball can be easily found by
solving the following linear optimization problem
max
xc,R
R (5.18a)
subj. to P
x
i
x
c
+R|P
x
i
|
2
P
c
i
, i = 1, . . . , n
P
, (5.18b)
where P
x
i
denotes the i-th row of P
x
. This can be proven as follows. Any point x
of the ball can be written as x = x
c
+v where v is a vector of length less or equal to
R. Therefore the center and radius of the Chebychev ball can be found by solving
the following optimization problem
max
xc,R
R (5.19a)
subj. to P
x
i
(x
c
+v) P
c
i
, v such that |v|
2
R, i = 1, . . . , n
P
. (5.19b)
Consider the i-th constraint
P
x
i
(x
c
+v) P
c
i
, v such that |v|
2
R.
This can be written as
P
x
i
x
c
P
c
i
P
x
i
v, v such that |v|
2
R. (5.20)
Constraint (5.20) is satised v such that |v|
2
R if and only if it is satised
at v =
P
x
i
P
x
i
2
R. Therefore we can rewrite the optimization problem (5.19) as the
linear program (5.18).
If the radius obtained by solving (5.18) is R = 0, then the polytope is lower-
dimensional. If R < 0, then the polytope is empty. Therefore, an answer to the
question is polytope T full-dimensional/empty? is obtained at the expense of
only one linear program. Furthermore, for a full-dimensional polytope we also get
a point x
c
that is in the strict interior of T. However, the center of a Chebyshev Ball
x
c
in (5.18) is not necessarily a unique point (e.g. when T is a rectangle). There
are other types of unique interior points one could compute for a full-dimensional
polytope, e.g., center of the largest volume ellipsoid, analytic center, etc., but those
computations involve the solution of Semi-Denite Programs (SDPs) and therefore
they are more expensive than the Chebyshev Ball computation [64]. An illustration
of the Chebyshev Ball is given in Figure 5.8.
x
1
x
2
Figure 5.8 Illustration of the Chebychev ball contained in a polytope
T. Download code.
5.4.6 Projection
Given a polytope T = [x
R
n+m
: P
x
x+P
y
y P
c
R
n+m
the projection
onto the x-space R
n
is dened as
proj
x
(T) = x R
n
: y R
m
: P
x
x +P
y
y P
c
. (5.21)
An illustration of a projection operation is given in Figure 5.9. Current projection
methods that can operate in general dimensions can be grouped into four classes:
Fourier elimination [81, 169], block elimination [15], vertex based approaches [109]
and wrapping-based techniques [161]. For a good introduction to projection, we
refer the reader to [161] and the references therein.
5.4.7 Set-Dierence
The set-dierence of two polytopes } and 1
0
1 = } 1
0
= x R
n
: x }, x / 1
0
, (5.22)
x
1
x
2
x
3
Figure 5.9 Illustration of the projection of a 3-dimensional polytope T onto
the plane x3 = 0. Download code.
in general, can be a nonconvex and disconnected set and can be described as a P-
collection 1 =
m
i=1
1
i
, where } =
m
i=1
1
i
(1
0
}). The P-collection 1 =
m
i=1
1
i
can be computed by consecutively inverting the half-spaces dening 1
0
as described in the following Theorem 5.2.
Note that here we use the term P-collection in the dual context of both P-
collection and its underlying set (cf. Denitions 5.3 and 5.4). The precise statement
would say that 1 = } 1
0
, where 1 is the underlying set of the P-collection 1 =
1
i
m
i=1
. However, whenever it is clear from context, we will use the former, more
compact form.
Theorem 5.2 [41] Let } R
n
be a polyhedron, 1
0
= x R
n
: Ax b, and
1
0
= x } : Ax b = 1
0
}, where b R
m
, 1
0
,= and Ax b is a minimal
representation of 1
0
. Also let
1
i
=
_
x } :
A
i
x > b
i
A
j
x b
j
, j < i
_
, i = 1, . . . , m, j = 1, . . . , m1.
Let 1 =
m
i=1
1
i
. Then, 1 is a P-collection and
1
0
, 1
1
, . . . , 1
m
is a strict
polyhedral partition of }.
Proof: (i) We want to prove that given an x }, then either x belongs to

1
0
or to 1
i
for some i but not both. If x

1
0
, we are done. Otherwise, there exists
an index i such that A
i
x > b
i
. Let i
= min
im
i : A
i
x > b
i
. Then x 1
i
, as
A
i
x > b
i
and A
j
x b
j
, j < i
, by denition of i
.
(ii) Let x

1
0
. Then there does not exist any i such that A
i
x > b
i
, which
implies that x , 1
i
, i m. Let x 1
i
and take i > j. Because x 1
i
, by
denition of 1
i
(i > j) A
j
x b
j
, which implies that x , 1
j
.
As an illustration for the procedure proposed in Theorem 5.2 consider the two-
dimensional case depicted in Figure 5.10(a). Here } is dened by the inequalities
x
1
x
1
x
+
1
, x
2
x
2
x
+
2
, and 1
0
by the inequalities g
1
0, . . . , g
5
0
where g
1
, . . ., g
5
are linear in x. The procedure consists of considering one by one
the inequalities which dene 1
0
. Considering, for example, the inequality g
1
0,
the rst set of the rest of the region }1
0
is given by 1
1
= g
1
0, x
1
x
1
, x
2

x
2
x
+
2
, which is obtained by reversing the sign of the inequality g
1
0 and
removing redundant constraints in } (see Figure 5.10(b)). Thus, by considering
the rest of the inequalities we get the partition of the rest of the parameter space
}1
0
=
5
i=1
1
i
, as reported in Figure 5.10(d).
1
0
x
2
x
1
x
+
2
x
2
x
1
x
+
1
}
(a) Set of parameters ] and initial set 10.
1
1
1
0
x
2
x
1
x
+
2
x
2
x
1
x
+
1
}
g
1
0 g
1
0
(b) Partition of ]\10 - Step 1.
1
2
g
2
0
g
1
0
1
1
1
0
x
2
x
1
x
+
2
x
2
x
1
x
+
1
}
(c) Partition of ]\10 - Step 2.
1
3
1
2
1
4
1
5
1
1
1
0
x
2
x
1
x
+
2
x
2
x
1
x
+
1
}
(d) Final partition of ]\10.
Figure 5.10 Two dimensional example: partition of the rest of the space
]\10.
Remark 5.1 The set dierence of two intersecting polytopes T and Q (or any closed
sets) is not a closed set. This means that some borders of polytopes 1i from a P-
collection 1 = T \ Q are open, while other borders are closed. Even though it is
possible to keep track of the origin of particular borders of 1i, thus specifying if they
are open or closed, we are not doing so in the algorithms described in this book nor
in MPT [148]. In computations, we will henceforth only consider the closure of the
sets 1i.
The set dierence between two P-collections T and Q can be computed as
described in [22, 133, 241].
5.4.8 Pontryagin Dierence
The Pontryagin dierence (also known as Minkowski dierence) of two polytopes
T and Q is a polytope
T Q = x R
n
: x +q T, q Q. (5.23)
The Pontryagin dierence can be eciently computed for polytopes by solving a
sequence of LPs as follows. Dene the T and Q as
T = y R
n
: , P
y
y P
b
, Q = z R
n
: Q
z
z Q
b
, (5.24)
then
J = T Q (5.25a)
= x R
n
: P
y
x P
b
H(P
y
, Q), (5.25b)
where the i-th element of H(P
y
, Q) is
H
i
(P
y
, Q) = max
xQ
P
y
i
x (5.26)
and P
y
i
is the i-th row of the matrix P
y
. Note that for special cases (e.g. when Q
is a hypercube), more ecient computational methods exist [172]. An illustration
of the Pontryagin dierence is given in Figure 5.11a.
5.4.9 Minkowski Sum
The Minkowski sum of two polytopes T and Q is a polytope
T Q = y +z R
n
: y T, z Q. (5.27)
The Minkowski sum is a computationally expensive operation which requires either
vertex enumeration and convex hull computation in n-dimensions or a projection
from 2n down to n dimensions. The implementation of the Minkowski sum via
projection is described below. If the polytopes are dened by
P = y R
n
: P
y
y P
c
, Q = z R
n
: Q
z
z Q
c
,
T
T Q
Q
x
1
x
2
-6 -4 -2 2 4 6
-6
-4
-2
2
4
6
(a) Pontryagin dierence T
Q. Download code.
T Q
T
Q
x
1
x
2
-8 -6 -2 2 6 8
-8
-6
-2
2
6
8
(b) Minkowski sum
T Q. Download code.
Figure 5.11 Illustration of the Pontryagin dierence and Minkowski sum
operations.
then it holds that
W = P Q
=
_
x R
n
: x = y +z, P
y
y P
c
, Q
z
z Q
c
, y, z R
n
_
=
_
x R
n
: y R
n
, subj. to P
y
y P
c
, Q
z
(x y) Q
c
_
=
_
x R
n
: y R
n
, subj. to
_
0 P
y
Q
z
Q
z
_ _
x
y
_
_
P
c
Q
c
_
_
= proj
x
_
_
_
x
y
_
R
n+n
:
_
0 P
y
Q
z
Q
z
_ _
x
y
_
_
P
c
Q
c
_
_
_
.
Both the projection and vertex enumeration based methods are implemented in
the MPT toolbox [148]. An illustration of the Minkowski sum is given in Figure
5.11b.
Remark 5.2 The Minkowski sum is not the complement of the Pontryagin dierence.
For two polytopes T and Q, it holds that (T Q) Q T. This is illustrated in
Figure 5.12.
5.4.10 Polyhedra Union
Consider the following basic problem in polyhedral computation: given two poly-
hedra T R
n
and Q R
n
, decide whether their union is convex, and, if so,
compute it. There are three classes of algorithms for the given problem depending
on their representation: (1) T and Q are given in 1-representation, (2) T and Q
x
1
x
2
T
Q
-3 -1 1 3
-3
-1
1
3
(a) Two polytopes T and Q.
x
1
x
2
T
T Q
-3 -2 2 3
-3
-2
2
3
(b) Polytope T and Pontryagin
dierence T Q.
x
1
x
2
(T Q) Q
T Q
-3 -2 2 3
-3
-2
2
3
(c) Polytope T Q and the set
(T Q) Q.
Figure 5.12 Illustration that (T Q) Q T. Download code.
are given in 1-representation, (3) both 1- and 1-representations are available for
T and Q. Next we present an algorithm for case (1). Case (2), case (3) and the
computational complexity of all three cases are discussed in [35].
Recall the denition of envelope in Section 5.4.3. By denition, it is easy to see
that env(T, Q) is convex and that
T Q env(T, Q). (5.28)
We have the following theorem.
Theorem 5.3 T Q is convex T Q = env(T, Q).
Algorithm 5.2 checks for points x
env(T, Q) outside T Q.
Algorithm 5.2 Algorithm for recognizing the convexity of T Q, and, if convex,
determining its -representation
Construct env(T, Q) by removing non-valid constraints (see Fig. 5.7),
let

Ax ,

Bx

be the set of removed constraints, and
let env(T, Q) = |x : Cx the resulting envelope;
Remove from env(T, Q) possible duplicates (Bj, j) = (Ai, i), > 0;
for each pair

Aix i,

Bjx

j do
Determine
by solving the linear program (LP)
= max
(x,)

subj. to

Aix i +
Bjx

j +
Cx ;
/*
= if the LP is infeasible,
= if the LP is unbounded */
if
> 0, stop, return nonconvex

end
return env(T, Q). /* T Q is convex. */
Note that if
= 0 for each i, j, then the union is convex and equals env(T, Q).
On the other hand,
> 0 indicates the existence of a point x env(T, Q) outside

T Q.
For recognizing convexity and computing the union of k polyhedra, the test
can be modied by checking each k-tuple of removed constraints. Let m
1
, . . ., m
k
be the number of removed constrains from the polyhedra T
1
, . . . , T
k
, respectively.
Then similarly to the for loop in Algorithm 5.2,
k
i=1
m
i
linear programs need to
be solved in the general case.
5.4.11 Ane Mappings and Polyhedra
This section deals with the composition of ane mappings and polyhedra. Consider
a polyhedron T = x R
n
: P
x
x P
c
, with P
x
R
nP n
and an ane mapping
f(z)
f : z R
m
Az +b, A R
mAm
, b R
mA
. (5.29)
Let m
A
= n. We dene the composition of T and f as the following polyhedron
T f = z R
m
: P
x
f(z) P
c
= z R
m
: P
x
Az P
c
P
x
b. (5.30)
Let m = n. We dene the composition of f and T as the following polyhedron
f T = y R
mA
: x T and y = Ax +b . (5.31)
The polyhedron f T in (5.31) can be computed as follows. Let us write T in
1-representation
T = conv(V ), (5.32)
and let us map the set of vertices V = V
1
, . . . , V
k
through the transformation f.
Because the transformation is ane, the set f T is simply the convex hull of the
transformed vertices
f T = conv(F), F = AV
1
+b, . . . , AV
k
+b. (5.33)
The polyhedron f T in (5.31) can be computed immediately if m
A
= m = n and
A is invertible. In this case, from the denition in (5.31), x = A
1
y A
1
b and
therefore
f T = y R
mA
: P
x
A
1
y P
c
+P
x
A
1
b. (5.34)
Vertex enumeration can be avoided even if A is not invertible and m
A
m = n by
using a QR decomposition of the matrix A.
Remark 5.3 Often in the literature the symbol is omitted for linear maps f = Az.
Therefore, AT refers to the operation A T and TA refers to the operation T A.
5.5 Operations on P-collections
This section covers some results and algorithms which are specic to operations
with P-collections. P-collections are unions of polytopes (see Denition 5.3) and
therefore the set of points contained in a P-collection can be represented in an
innite number of ways, i.e. the P-collection representation is not unique. For
example, one can subdivide any polytope T into a number of smaller polytopes
whose union is a P-collection which covers T. Note that the complexity of all
subsequent computations depends strongly on the number of polytopes representing
a P-collection. The smaller the cardinality of a P-collection, the more ecient the
computations. The reader is referred to [241, 240] for proofs and comments on
computational eciency.
5.5.1 Set-Dierence
The rst two results given here show how the set dierence of a P-collection and a
P-collection (or polyhedron) may be computed.
Lemma 5.4 Let c =
j{1,...,J}
c
j
be a P-collection, where all the c
j
, j 1, . . . , J,
are nonempty polyhedra. If T is a nonempty polyhedron, then cT =
j{1,...,J}
(c
j
T) is a P-collection.
Lemma 5.5 Let the sets c =
j{1,...,J}
c
j
and T =
y=1,...,Y
T
y
be P-collections,
where all the c
j
, j 1, . . . , J, and T
y
, y 1, . . . , Y , are nonempty polyhedra.
If c
0
= c and c
y
= c
y1
T
y
, y 1, . . . , Y then c T = c
Y
is a P-collection.
The condition c T can be easily veried since c T c T = . Similarly
c = T is also easily veried since
c = T (c T = and T c = ).
Next, an algorithmfor computing the Pontryagin dierence of a P-collection and
a polytope is presented. If o and B are two subsets of R
n
then oB = [o
c
(B)]
c
where ()
c
denotes the set complement. The following algorithm implements the
computation of the Pontryagin dierence of a P-collection c =
j{1,...,J}
c
j
, where
c
j
, j 1, . . . , J are polytopes in R
n
, and a polytope B R
n
.
Algorithm 5.3 Pontryagin Dierence for P-collections, c B
input: P-collection c, polytope B
output: P-collection ( = c B
= env(c) (or = conv(c))
T = B
c = \ c
T = c (B)
( = T \ T
5.5 Operations on P-collections 89
Remark 5.4 Note that in Algorithm 5.3 can be any convex set containing the
P-collection c. The computation of is generally more ecient if the envelope
operation is used instead of convex hull.
Remark 5.5 It is important to note that (
j{1,...,J}
cj) B , =
j{1,...,J}
(cj B),
where B and cj are polyhedra; hence, the relatively high computational eort of
computing the Pontryagin dierence of a P-collection and a polytope.
Theorem 5.4 (Computation of Pontryagin Dierence, [240]) The output of
Algorithm 5.3 is ( = c B.
Proof: It holds by denition that
T = 1B = x : x +w 1, w B,
c = 1 c = x : x 1 and x / c.
By the denition of the Minkowski sum:
T = c (B) = x : x = z +w, z c, w (B)
= x : w (B), subj. to x w c.
By denition of the set dierence:
T T = x : x T and x / T
= x T : w B s.t. x +w c
= x T : x +w / c, w B.
From the denition of the set T:
T T = x : x +w 1 and x +w / c, w B
and from the denition of the set c and because c 1:
T T = x : x +w 1 and (x +w / 1 or x +w c) w B
= x : x +w c, w B
= c B.
Algorithm 5.3 is illustrated on a sample P-collection in Figures 5.13(a) to 5.13(f).

Remark 5.6 It should be noted that Algorithm 5.3 for computation of the Pontrya-
gin dierence is conceptually similar to the algorithm proposed in [262, 170]. The
envelope operation = env(c) employed in Algorithm 5.3 might reduce the number
of sets obtained when computing \ c, which in turn results in fewer Minkowski set
additions. Since the computation of a Minkowski set addition is expensive, a runtime
improvement can be expected.
x
1
x
2
c
2
c
1
B
(a)
j{1,...,J}
cj and B.
x
1
x
2
(b) H = conv(C).
x
1
x
2
1
T
(c) D = HB.
x
1
x
2
c
(d) E = H\ C.
x
1
x
2
T
(e) F = E (B).
x
1
x
2
T T
(f) G = D \ F.
Figure 5.13 Graphical illustration of Algorithm 5.3. Download code.
5.5.2 Polytope Covering
The problem of checking if some T polytope is covered with the union of other
polytopes, i.e. a P-collection Q =
i
Q
i
is discussed in this section. We consider
two related problems:
polycover: Check if T Q, and
5.5 Operations on P-collections 91
regiondi: Compute P-collection 1 = T Q.
Clearly, polycover is just a special case of regiondi, where the resulting P-
collection 1 = . Also, it is straightforward to extend the above problems to the
case where both T and Q are both P-collections.
One idea of solving the polycover problem is inspired by the following observa-
tion
T Q T =
i
(T Q
i
).
Therefore, we could create 1
i
= T Q
i
, for i = 1, . . . , N
Q
and then compute the
union of the collection of polytopes 1
i
by using the polyunion algorithm for
computing the convex union of 1-polyhedra reported discussed in Section 5.4.10.
If polyunion succeeds (i.e., the union is a convex set) and the resulting polytope is
equal to T then T is covered by Q, otherwise it is not. However, this approach is
computationally very expensive.
More details can be found in [119].
5.5.3 Union of P-collections
Consider a P-collection T = T
i
p
i=1
. We study the problem of nding a minimal
representation of T by merging one or more polyhedra belonging to the P-collection.
Clearly one could use the polyunion algorithm presented in Section 5.4.10 for all
possible subsets of the P-collection and solve the problem by comparing all solu-
tions. However this approach is not computationally ecient.
Our interest in this problem will be clear later in this book (Section 12.1) when
computing the PWA state feedback control law to optimal control problems. Once
the PWA state feedback control law has been derived, the memory requirement and
the on-line computation time are linear in the number of polyhedra of the feedback
law when using standard brute force search. Therefore, we will be interested in the
problem of nding a minimal representation of piecewise ane (PWA) systems,
or more specically, for a given PWA system, we solve the problem of deriving a
PWA system, that is both equivalent to the former and minimal in the number of
regions. This is done by associating a dierent color to a dierent feedback law,
and then collecting the polyhedra with the same color. Then, for a given color, we
try to merge the corresponding P-collection. If the number of polyhedra with the
same ane dynamic is large, the number of possible polyhedral combinations for
merging explodes. As most of these unions are not convex or even not connected
and thus cannot be merged, trying all combinations using standard techniques
based on linear programming (LP) is prohibitive. Furthermore, our objective here
is not only to reduce the number of polyhedra but rather to nd the minimal and
thus optimal number of disjoint polyhedra. This problem is known to be AT-hard.
In [119] details on the solutions of these problems can be found.
Part II
Multiparametric Programming
Chapter 6
Multiparametric Nonlinear
Programming
The operations research community has addressed parameter variations in mathe-
matical programs at two levels: sensitivity analysis, which characterizes the change
of the solution with respect to small perturbations of the parameters, and para-
metric programming, where the characterization of the solution for a full range of
parameter values is studied. In this chapter we introduce the concept of multi-
parametric programming and recall the main results of nonlinear multiparametric
programming. Later in this book we will use multiparametric programming to
characterize and compute the state feedback solution of optimal control problems.
6.1 Introduction to Multiparametric Programs
Consider the mathematical program
J
(x) = inf
z
J(z, x)
subj. to g(z, x) 0
where z is the optimization vector and x is a vector of parameters. We are interested
in studying the behavior of the value function J
(x) and the optimizer z
(x) as
we vary the parameter x. Mathematical programs where x is a scalar are referred
to as parametric programs, while programs where x is a vector are referred to as
multiparametric programs.
There are several reasons to look for ecient solvers of multiparametric pro-
grams. Typically, mathematical programs are aected by uncertainties due to
factors that are either unknown or that will be decided later. Parametric program-
ming systematically subdivides the space of parameters into characteristic regions,
which depict the feasibility and corresponding performance as a function of the
uncertain parameters, and hence provide the decision maker with a complete map
96 6 Multiparametric Nonlinear Programming
of various outcomes.
Our interest in multiparametric programming arises from the eld of system
theory and optimal control. For example, for discrete-time dynamical systems,
nite time constrained optimal control problems can be formulated as mathematical
programs where the cost function and the constraints are functions of the initial
state of the dynamical system. In particular, Zadeh and Whalen [291] appear to
have been the rst ones to express the optimal control problem for constrained
discrete-time linear systems as a linear program. We can interpret the initial state
as a parameter. By using multiparametric programming we can characterize and
compute the solution of the optimal control problem explicitly as a function of the
initial state.
We are further motivated by the model predictive control (MPC) technique.
MPC is very popular in the process industry for the automatic regulation of process
units under operating constraints, and has attracted a considerable research eort
in the last two decades. MPC requires an optimal control problem to be solved
on-line in order to compute the next command action. This mathematical program
depends on the current sensor measurements. The computation eort can be moved
o-line by solving multiparametric programs, where the control inputs are the
optimization variables and the measurements are the parameters. The solution of
the parametric program problem is a control law describing the control inputs as
function of the measurements. Model Predictive Control and its multiparametric
solution are discussed in Chapter 13.
In the following we will present several examples that illustrate the parametric
programming problem and hint at some of the issues that need to be addressed by
the solvers.
Example 6.1 Consider the parametric quadratic program
1
J
(x) = min
z
J(z, x) =
1
2
z
2
+ 2xz + 2x
2
subj. to z 1 +x,
where x R. Our goals are:
1. to nd z
(x) = arg min

z
J(z, x),
2. to nd all x for which the problem has a solution, and
3. to compute the value function J
(x).
The Lagrangian is
L(z, x, u) =
1
2
z
2
+ 2xz + 2x
2
+u(z x 1)
and the KKT conditions are (see Section 3.3.3 for KKT conditions for quadratic
programs)
z + 2x +u = 0 (6.1a)
u(z x 1) = 0 (6.1b)
u 0 (6.1c)
z x 1 0. (6.1d)
1
6.1 Introduction to Multiparametric Programs 97
Consider (6.1) and the two strictly complementary cases:
A.
z + 2x +u = 0
z x 1 = 0
u 0
_
_
_
z
= x + 1
J
=
9
2
x
2
+ 3x +
1
2
x
1
3
B.
z + 2x +u = 0
z x 1 < 0
u = 0
_
_
_
z
= 2x
J
= 0
x >
1
3
(6.2)
This solution is depicted in Figure 6.1.
x
z
(x)
J
(x)
-2 -1 0 1 2
-2
-1
0
1
2
Figure 6.1 Example 6.1: Optimizer z
(x) and value function J
(x) as a
function of the parameter x. Download code.
The above simple procedure, which required nothing but the solution of the KKT
conditions, yielded the optimizer z
(x) and the value function J
(x) for all values

of the parameter x. The set of admissible parameter values was divided into two
critical regions, dened by x
1
3
and x >
1
3
. In the region x
1
3
the
inequality constraint is active and the Lagrange multiplier is greater or equal than
zero, in the other region x >
1
3
the inequality constraint is not active and the
Lagrange multiplier is equal to zero.
In general, when there are more than one inequality constraints a critical region
is dened by the set of inequalities that are active in the region. Throughout a
critical region the conditions for optimality derived from the KKT conditions do
not change. For our example, in each critical region the optimizer z
(x) is ane and

the value function J
(x) is quadratic. Thus, considering all x, z
(x) is piecewise
ane and J
(x) is piecewise quadratic. Both z
(x) and J
(x) are continuous, but

z
(x) is not continuously dierentiable.

In much of this book we will be interested in two questions: how to nd the value
function J
(x) and the optimizer z
(x) and what are their structural properties,

e.g., continuity, dierentiability and convexity. Such questions have been addressed
for general nonlinear multiparametric programming by several authors in the past
(see [18] and references therein), by making use of quite involved mathematical
theory based on the continuity of point-to-set maps. The concept of point-to-set
maps will not be much used in this book. However, it represents a key element for a
rigorous mathematical description of the properties of a nonlinear multiparametric
program and hence a few key theoretical results for nonlinear multiparametric
programs based on the point-to-set map formalism will be recalled in this chapter.
6.2 General Results for Multiparametric Nonlinear Programs
Consider the nonlinear mathematical program dependent on a parameter x ap-
pearing in the cost function and in the constraints
J
(x) = inf
z
J(z, x)
subj. to g(z, x) 0,
(6.3)
where z : R
s
is the optimization vector, x A R
n
is the parameter vector,
J : R
s
R
n
R is the cost function and g : R
s
R
n
R
ng
are the constraints.
We denote by g
i
(z, x) the i-th component of the vector-valued function g(z, x).
A small perturbation of the parameter x in the mathematical program (6.3) can
cause a variety of results. Depending on the properties of the functions J and g the
solution z
(x) may vary smoothly or change abruptly as a function of x. Denote

by 2
Z
the set of subsets of :. We denote by R the point-to-set map which assigns
to a parameter x A the (possibly empty) set R(x) of feasible variables z :,
R : A 2
Z
R(x) = z : : g(z, x) 0, (6.4)
by /
the set of feasible parameters

/
= x A : R(x) ,= , (6.5)
by J
(x) the real-valued function that expresses the dependence of the minimum
value of the objective function over /
on x
J
(x) = inf
z
J(z, x) : z R(x), (6.6)
and by Z
(x) the point-to-set map which assigns the (possibly empty) set of opti-
mizers z
2
Z
to a parameter x A
Z
(x) = z R(x) : J(z, x) = J
(x). (6.7)
J
(x) will be referred to as optimal value function or simply value function, Z
(x)
will be referred to as the optimal set. If Z
(x) is a singleton for all x, then z
(x) =
Z
(x) will be called optimizer function. We remark that R and Z
are set-valued
functions. As discussed in the notation section, with abuse of notation J
(x) and
Z
(x) will denote both the functions and the value of the functions at the point x.
The context will make clear which notation is being used.
The book by Bank and coauthors [18] and Chapter 2 of [104] describe conditions
under which the solution of the nonlinear multiparametric program (6.3) is locally
well behaved and establish properties of the optimal value function and of the
optimal set. The description of such conditions requires the denition of continuity
6.2 General Results for Multiparametric Nonlinear Programs 99
of point-to-set maps. Before introducing this concept we will show through two
simple examples that continuity of the constraints g
i
(z, x) with respect to z and
x is not enough to imply any regularity of the value function and the optimizer
function.
Example 6.2 [ [18, p. 12]]
Consider the following problem:
J
(x) = minz x
2
z
2
2x(1 x)z
subj. to z 0
0 x 1.
(6.8)
Cost and constraints are continuous and continuously dierentiable. For 0 < x 1
the optimizer function is z
= (1 x)/x and the value function is J
(x) = (1 x)
2
.
For x = 0, the value function is J
(x) = 0 while the optimal set is Z
= |z R : z
0. Thus, the value function is discontinuous at 0 and the optimal set is single-valued
for all 0 < x 1 and set-valued for x = 0.
Example 6.3 Consider the following problem:
J
(x) = inf
z
z
subj. to zx 0
10 z 10
10 x 10,
(6.9)
where z R and x R. For each xed x the set of feasible z is a segment. The
point-to-set map R(x) is plotted in Figure 6.2a. The function g1 : (z, x) zx is
continuous. Nevertheless, the value function J
(x) = z
(x) has a discontinuity at the

origin as can be seen in Figure 6.2b.
x
R
(
x
)
-10 -5 0 5 10
-10
-5
0
5
10
(a) Point-to-set map R(x).
x
J
(
x
)
-10 -5 0 5 10
-10
-5
0
5
10
(b) Value function J
(x).
Figure 6.2 Example 6.3: Solution.
Example 6.4 Consider the following problem:
J
(x) = inf
z
1
,z
2
z1
subj. to g1(z1, z2) +x 0
g2(z1, z2) +x 0,
(6.10)
where examples of the functions g1(z1, z2) and g2(z1, z2) are plotted in Figures 6.3(a)
6.3(c). Figures 6.3(a)6.3(c) also depict the point-to-set map R(x) = |[z1, z2]
R
2
[g1(z1, z2) + x 0, g2(z1, z2) + x 0 for three xed x. Starting from x = x1,
as x increases, the domain of feasibility in the space z1, z2 shrinks; at the beginning
it is connected (Figure 6.3(a)), then it becomes disconnected (Figure 6.3(b)) and
eventually connected again (Figure 6.3(c)). No matter how smooth one chooses the
functions g1 and g2, the value function J
(x) = z
1
(x) will have a discontinuity at
x = x3.
z
1
z
2
g
1
(z
1
, z
2
) = x
1
g
2
(z
1
, z
2
) = x
1
(a) Set R( x1) shaded in gray.
z
1
z
2
g
1
(z
1
, z
2
) = x
2
g
2
(z
1
, z
2
) = x
2
(b) Set R( x2) shaded in gray.
z
1
z
2
g
1
(z
1
, z
2
) = x
3
g
2
(z
1
, z
2
) = x
3
(c) Set R( x3) shaded in gray.
x
J
(x)
x
1
x
3
(d) Value function J
(x).
Figure 6.3 Example 6.4: problem (6.10). (a)-(c) Projections of the point-to-
set map R(x) for three values of the parameter x: x1 < x2 < x3; (d) Value
function J
(x).
Examples 6.2, 6.3, 6.4 show the case of simple and smooth constraints which
lead to a discontinuous behavior of value function and in Examples 6.3, 6.4 we also
observe discontinuity of the optimizer function. The main causes are:
in Example 6.2 the feasible vector space : is unbounded (z 0),
in Examples 6.3 and 6.4 the feasible point-to-set map R(x) (dened in (6.4))
is discontinuous, as dened precisely below.
In the next sections we discuss both cases in detail.
Continuity of Point-to-Set Maps
Consider a point-to-set map R : x A R(x) 2
Z
. We give the following
denitions of open and closed maps according to Hogan [151]:
Denition 6.1 The point-to-set map R(x) is open at a point x /
if for all
sequences x
k
/
with x
k
x and for all z R( x) there exists an integer m
and a sequence z
k
: such that z
k
R(x
k
) for k m and z
k
z.
Denition 6.2 The point-to-set map R(x) is closed at a point x /
if for each
pair of sequences x
k
/
, and z
k
R(x
k
) with the properties
x
k
x, z
k
z.
it follows that z R( x).
We dene the continuity of a point-to-set map according to Hogan [151] as follows:
Denition 6.3 The point-to-set map R(x) is continuous at a point x in /
if it
is both open and closed at x. R(x) is continuous in /
if R(x) is continuous at
every point x in /
.
The denitions above are illustrated through two examples.
Example 6.5 Consider
R(x) = |z R : z [0, 1] if x < 1, z [0, 0.5] if x 1.
The point-to-set map R(x) is plotted in Figure 6.4. It is easy to see that R(x) is
not closed but open. In fact, if one considers a sequence |x
k
that converges to
x = 1 from the left and extracts the sequence |z
k
plotted in Figure 6.4 converging
to z = 0.75, then z / R( x) since R(1) = [0, 0.5].
Example 6.6 Consider
R(x) = |z R : z [0, 1] if x 1, z [0, 0.5] if x > 1
x
z
1
1
x
k
z
k
0.5
Figure 6.4 Example 6.5: Open and not closed point-to-set map R(x).
The point-to-set map R(x) is plotted in Figure 6.5. It is easy to verify that R(x) is
closed but not open. Choose z = 0.75 R( x). Then, for any sequence |x
k
that
converges to x = 1 from the right, one is not able to construct a sequence |z
k
?
such that z
k
R(x
k
) for and z
k
z. In fact, such sequence z
k
will always be
bounded between 0 and 0.5.
x
z
1
x
k
z
k
1
0.5
Figure 6.5 Example 6.6: Closed and not open point-to-set map R(x).
Remark 6.1 We remark that upper semicontinuous and lower semicontinuous
denitions of point-to-set map are sometimes preferred to open and closed deni-
tions [46, p. 109]. In [18, p. 25] nine dierent denitions for the continuity of
point-to-set maps are introduced and compared. We will not give any details on this
subject and refer the interested reader to [18, p. 25].
The examples above are only illustrative. In general, it is dicult to test if a
set is closed or open by applying the denitions. Several authors have proposed
sucient conditions on g
i
which imply the continuity of R(x). In the following we
introduce a theorem which summarizes the main results of [252, 90, 151, 46, 18].
Theorem 6.1 If : is convex, if each component g
i
(z, x) of g(z, x) is continuous
on : x and convex in z for each xed x A and if there exists a z such that
g( z, x) < 0, then R(x) is continuous at x.
The proof is given in [151, Theorems 10 and 12]. An equivalent proof can be also
derived from [18, Theorem 3.1.1 and Theorem 3.1.6].
Remark 6.2 Note that convexity in z for each x is not enough to imply the continuity
of R(x) everywhere in /
. In [18, Example 3.3.1 on p. 53] an example illustrating

this is presented. We remark that in Example 6.3 the origin does not satisfy the last
hypothesis of Theorem 6.1.
Remark 6.3 If the assumptions of Theorem 6.1 hold at each x . then one can
extract a continuous single-valued function (often called a continuous selection) r :
. R such that r(x) R(x), x ., provided that ? is nite-dimensional. Note
that convexity of R(x) is a critical assumption [18, Corollary 2.3.1]. The following
example shows a point-to-set map R(x) not convex for a xed x which is continuous
but has no continuous selection [18, p. 29]. Let be the unit disk in R
2
, dene R(x)
as
x R(x) = |z : |z x|2
1
2
. (6.11)
It can be shown that the point-to-set map R(x) in (6.11) is continuous according to
Denition 6.3. In fact, for a xed x the set R( x) is the set of points in the unit
disk outside the disk centered in x and of radius 0.5 (next called the half disk); small
perturbations of x yield small translations of the half disk inside the unit disk for
all x . However R(x) has no continuous selection. Assume that there exists a
continuous selection r : x r(x) . Then, there exists a point x
such that
x
= r(x
). Since r(x) R(x), x , there exists a point x
such that x
R(x
).
This is not possible since for all x
, x
/ R(x
) (recall that R(x
) is set of points
in the unit disk outside the disk centered in x
and of radius 0.5).

Remark 6.4 Let be the unit disk in R, dene R(x) as
x R(x) = |z : [z x[
1
2
. (6.12)
R(x) is closed and not open and it has no continuous selection.
Remark 6.5 Based on [18, Theorem 3.2.1-(I) and Theorem 3.3.3], the hypotheses of
Theorem 6.1 can be relaxed for ane gi(z, x). In fact, ane functions are weakly
analytic functions according to [18, p. 47]. Therefore, we can state that if ? is convex,
if each component gi(z, x) of g(z, x) is an ane function, then R(x) is continuous at
x for all x /
.
Properties of the Value Function and Optimal Set
Consider the following denition
Denition 6.4 A point-to-set map R(x) is said to be uniformly compact near x
if there exist a neighborhood N of x such that the closure of the set
xN
R(x) is
compact.
Now we are ready to state the two main theorems on the continuity of the value
function and of the optimizer function.
Theorem 6.2 [151, Theorem 7] Consider problem (6.3)(6.4). If R(x) is a con-
tinuous point-to-set map at x and uniformly compact near x and if J is continuous
on x R( x), then J
is continuous at x.
Theorem 6.3 [151, Corollary 8.1] Consider problem (6.3)(6.4). If R(x) is a
continuous point-to-set map at x, J is continuous on x R( x), Z
is nonempty
and uniformly compact near x, and Z
( x) is single valued, then Z
is continuous
at x.
Remark 6.6 Equivalent results of Theorems 6.2 and 6.3 can be found in [46, p. 116]
and [18, Chapter 4.2].
Example 6.7 [Example 6.2 revisited] Consider Example 6.2. The feasible map R(x)
is unbounded and therefore it does not satisfy the assumptions of Theorem 6.2 (since
it is not uniformly compact). Modify Example 6.2 as follows:
J
(x) = minz x
2
z
2
2x(1 x)z
subj. to 0 z M
0 x 1
(6.13)
with M 0. The solution can be computed immediately. For 1/(1 + M) < x 1
the optimizer function is z
= (1 x)/x and the value function is J
(x) = (1 x)
2
.
For 0 < x 1/(1 + M), the value function is J
(x) = x
2
M
2
2Mx(1 x) and the
optimizer function is z
= M. For x = 0, the value function is J
(x) = 0 while the

optimal set is Z
= |z R : 0 z M.
No matter how large we choose M, the value function and the optimal set are con-
tinuous for all x [0, 1].
Example 6.8 [Example 6.3 revisited] Consider Example 6.3. The feasible map R(x)
is not continuous at x = 0 and therefore it does not satisfy the assumptions of
Theorem 6.2. Modify Example 6.3 as follows:
J
(x) = inf
z
z
subj. to zx
10 z 10
10 x 10,
(6.14)
where > 0. The value function and the optimal set are depicted in Figure 6.6 for
= 1. No matter how small we choose , the value function and the optimal set are
continuous for all x [10, 10]
The following corollaries consider special classes of parametric problems.
Corollary 6.1 (mp-LP) Consider the special case of the multiparametric pro-
gram (6.3). where the objective and the constraints are linear
J
(x) = min
z
c
z
subj. to Gz w +Sx,
(6.15)
x
R
(
x
)
-10 -5 0 5 10
-10
-5
0
5
10
(a) Point-to-set map R(x).
x
J
(
x
)
-10 -5 0 5 10
-10
-5
0
5
10
(b) Value function J
(x).
Figure 6.6 Example 6.8: Solution.
and assume that there exists an x and z
( x) with a bounded cost J
( x). Then, /
is a nonempty polyhedron, J
(x) is a continuous and convex function on /
and
the optimal set Z
(x) is a continuous point-to-set map on /
.
Proof: See Theorem 5.5.1 in [18] and the bottom of page 138 in [18].
Corollary 6.2 (mp-QP) Consider the special case of the multiparametric pro-
gram (6.3). where the objective is quadratic and the constraints are linear
J
(x) = min
z
1
2
z
Hz +z
F
subj. to Gz w +Sx,
(6.16)
and assume that H 0 and that there exists ( z, x) such that G z w +S x. Then,
/
is a nonempty polyhedron, J
(x) is a continuous and convex function on /
and the optimizer function z
(x) is continuous in /
.
Proof: See Theorem 5.5.1 in [18] and the bottom of page 138 in [18].
Remark 6.7 We remark that Corollary 6.1 requires the existence of optimizer z
( x)
with a bounded cost. This is implicitly guaranteed in the mp-QP case since in
Corollary 6.2 the matrix H is assumed to be strictly positive denite. Moreover, the
existence of an optimizer z
( x) with a bounded cost guarantees that J
(x) is bounded
for all x in /
. This has been proven in [111, p. 178, Theorem 1] for the mp-LP case
and it is immediate to prove for the mp-QP case.
Remark 6.8 Both Corollary 6.1 (mp-LP) and Corollary 6.2 (mp-QP) could be formu-
lated stronger: J
and Z
are even Lipschitz-continuous. J
is also piecewise ane

(mp-LP) or piecewise quadratic (mp-QP), and for the mp-QP z
(x) is piecewise
ane. For the linear case, Lipschitz continuity is known from Walkup-Wets [281] as
a consequence of Homans theorem. For the quadratic case, Lipschitz continuity fol-
lows from Robinson [251], as e.g. shown by Klatte and Thiere [175]. The piecewise
properties are consequences of local stability analysis of parametric optimization,
e.g. [104, 18, 197] and are the main focus of the next chapter.
Chapter 7
Multiparametric Programming:
a Geometric Approach
In this chapter we will concentrate on multiparametric linear programs (mp-LP),
multiparametric quadratic programs (mp-QP) and multiparametric mixed-integer
linear programs (mp-MILP).
The main idea of the multiparametric algorithms presented in this chapter is to
construct a critical region in a neighborhood of a given parameter, by using nec-
essary and sucient conditions for optimality, and then to recursively explore the
parameter space outside such a region. For this reason the methods are classied as
geometric. All the algorithms are easy to implement once standard solvers are
available: linear programming, quadratic programming and mixed-integer linear
programming for solving mp-LP, mp-QP and mp-MILP, respectively. A literature
review is presented in Section 7.6.
7.1 Multiparametric Programs with Linear Constraints
7.1.1 Formulation
Consider the multiparametric program
J
(x) = min
z
J(z, x)
subj. to Gz w +Sx,
(7.1)
where z R
s
are the optimization variables, x R
n
is the vector of parameters,
J(z, x) : R
s+n
R is the objective function and G R
ms
, w R
m
, and
S R
mn
. Given a closed and bounded polyhedral set / R
n
of parameters,
/ = x R
n
: Tx N, (7.2)
we denote by /
/ the region of parameters x / such that (7.1) is feasible:

/
= x / : z satisfying Gz w +Sx. (7.3)

108 7 Multiparametric Programming: a Geometric Approach
In this book we assume that
1. the constraint x / is included in the constraints Gz w +Sx.
2. the polytope / is full-dimensional. Otherwise we can reformulate the problem
with a smaller set of parameters such that / becomes full-dimensional.
3. S has full column rank. Otherwise we can again reformulate the problem in
a smaller set of parameters.
Theorem 7.1 Consider the multiparametric problem (7.1). If the domain of J(z, x)
is R
s+n
then /
is a polytope.
Proof: /
is the projection of the set Gz Sx w on the x space intersected

with the polytope /.
For any given x /
, J
( x) denotes the minimum value of the objective func-

tion in problem (7.1) for x = x. The function J
: /
R expresses the depen-

dence of the minimum value of the objective function on x, J
(x) is called the value

function. The set-valued function Z
: /
2
R
s
, where 2
R
s
is the set of subsets of
R
s
, describes for any xed x /
the set Z
(x) of optimizers z
(x) yielding J
(x).
We aim to determine the feasible region /
/ of parameters, the expression of

(x) and the expression of one of the optimizers z
(x) Z
(x).
7.1.2 Denition of Critical Region
Consider the multiparametric program (7.1). Let I = 1, . . . , m be the set of
constraint indices. For any A I, let G
A
and S
A
be the submatrices of G and S,
respectively, comprising the rows indexed by A and denote with G
j
, S
j
and w
j
the
j-th row of G, S and w, respectively. We dene CR
A
as the set of parameters x
for which the same set A of constraints is active at the optimum. More formally
we have the following denitions.
Denition 7.1 The optimal partition of I at x is the partition (A(x), NA(x))
where
A(x) = j I : G
j
z
(x) S
j
x = w
j
for all z
(x) Z
(x)
NA(x) = j I : exists z
(x) Z
(x) satisfying G
j
z
(x) S
j
x < w
j
.
It is clear that A(x) and NA(x) are disjoint and their union is I.
Denition 7.2 Consider a set A I. The critical region associated with the set
of active constraints A is dened as
CR
A
= x /
: A(x) = A. (7.4)
The set CR
A
is the set of all parameters x such that the constraints indexed by A
are active at the optimum of problem (7.1). Our rst objective is to work with full-
dimensional critical regions. For this reason, next we discuss how the dimension of
the parameter space can be reduced in case it is not full-dimensional.
7.1 Multiparametric Programs with Linear Constraints 109
7.1.3 Reducing the Dimension of the Parameter Space
It may happen that the set of inequality constraints in (7.1) contains some hidden
or implicit equality constraints as the following example shows.
Example 7.1
minz J(z, x)
subj. to
_
_
z1 +z2 9 x1 x2
z1 z2 1 x1 x2
z1 +z2 7 +x1 +x2
z1 z2 1 +x1 +x2
z1 4
z2 4
z1 20 x2,
(7.5)
where / = |[x1, x2]
R
2
: 100 x1 100, 100 x2 100. The reader can
check that all feasible values of x1, x2, z1, z2 satisfy
z1 +z2 = 9 x1 x2
z1 z2 = 1 x1 x2
z1 +z2 = 7 +x1 +x2
z1 z2 = 1 +x1 +x2
z1 = 4
z2 = 4
z1 20 x2,
(7.6)
where we have identied many of the inequalities to be hidden equalities. This can
be simplied to
x1 +x2 = 1
x2 16.
(7.7)
Thus
/
=
_
_
_
[x1, x2]
R
2
:
x1 +x2 = 1
100 x1 100
100 x2 16
_
_
_
. (7.8)
The example shows that the polytope /
is contained in a lower dimensional subspace

of /, namely a line segment in R
2
.
Our goal is to identify the hidden equality constraints (as in (7.6)) and use
them to reduce the dimension of the parameter space (as in (7.7)) for which the
multiparametric program needs to be solved.
Denition 7.3 A hidden equality of the polyhedron c = R
s
: B v is an
inequality B
i
v
i
such that B
i
= v
i

c.
To nd hidden equalities we need to solve
v
i
= min B
i
subj. to B v,
for all constraints i = 1, . . . , m. If v
i
= v
i
, then B
i
= v
i
is a hidden equality.
We can apply this procedure with = [
x
z
] to identify the hidden equalities in
the set of inequalities in (7.1)
Gz w +Sx,
to obtain
G
nh
z w
nh
+S
nh
x (7.9a)
G
h
z = w
h
+S
h
x, (7.9b)
where we have partitioned G, w and S to reect the hidden equalities. In order to
nd the equality constraints involving only the parameter x that allow us to reduce
the dimension of x, we need to project the equalities (7.9b) onto the parameter
space. Let the singular value decomposition of G
h
be
G
h
= [U
1
U
2
]
_
V
1
V
2
_
,
where the columns of U
2
are the singular vectors associated with the zero singular
values. Then the matrix U
2
denes the projection of (7.9b) onto the parameter
space, i.e.
U
2
G
h
z = 0 = U
2
w
h
+U
2
S
h
x. (7.10)
We can use this set of equalities to replace the parameter x R
n
with a set of
n
= n rank(U
2
S
h
) new parameters in (7.9a) which simplies the parametric
program (7.1).
In the rest of this book we always assume that the multiparametric program has
been preprocessed using the ideas of this section so that the number of independent
parameters is reduced as much as possible and /
is full-dimensional.
7.2 Multiparametric Linear Programming
7.2.1 Formulation
Consider the special case of the multiparametric program (7.1) where the objective
is linear
J
(x) = min
z
c
z
subj. to Gz w +Sx.
(7.11)
All the variables were dened in Section 7.1.1. Our goal is to nd the value function
J
(x) and an optimizer function z
(x) for x /
. Note that /
can be determined
as discussed in Thorem 7.1. As suggested through Example 6.1 our search for
these functions proceeds by partitioning the set of feasible parameters into critical
regions. This is shown through a simple example next.
Example 7.2 Consider the parametric linear program
J
(x) = min
z
z + 1
subj. to z 1 +x
z 0,
7.2 Multiparametric Linear Programming 111
where z R and x R. Our goals are:
1. to nd z
(x) = arg min

z, z0, z1+x
z + 1,
(x).
The Lagrangian is
L(z, x, u1, u2) = z +u1(z +x + 1) +u2(z)
and the KKT conditions are (see Section 3.2.3 for KKT conditions for linear pro-
grams)
u1 u2 = 1 (7.12a)
u1(z +x + 1) = 0 (7.12b)
u2(z) = 0 (7.12c)
u1 0 (7.12d)
u2 0 (7.12e)
z +x + 1 0 (7.12f)
z 0. (7.12g)
Consider (7.12) and the three complementary cases:
A.
u1 +u2 = 1
z +x + 1 = 0
z < 0
u1 > 0
u2 = 0
_
z
= 1 +x
u
1
= 1, u
2
= 0
J
= 2 +x
x > 1
B.
u1 +u2 = 1
z +x + 1 < 0
z = 0
u1 = 0
u2 > 0
_
z
= 0
u
1
= 0, u
2
= 1
J
= 1
x < 1
C.
u1 +u2 = 1
z +x + 1 = 0
z = 0
u1 0
u2 0
_
z
= 0
u
1
0, u2 0, u
1
+u
2
= 1
J
= 1
x = 1
(7.13)
This solution is depicted in Figure 7.1.
The above simple procedure, which required nothing but the solution of the KKT
conditions, yielded the optimizer z
(x) for all values

of the parameter x. The set of admissible parameters values was divided into three
critical regions, dened by x < 1, x > 1 and x = 1. In the region x > 1
the rst inequality constraint is active (z = 1 + x) and the Lagrange multiplier u1
is greater than zero, in the second region x < 1 the second inequality constraint
is active (z = 0) and the Lagrange multiplier u2 is greater than zero. In the third
region x = 1 both constraints are active and the Lagrange multipliers belong to
the set u
1
0, u2 0, u
1
+ u
2
= 1. Throughout a critical region the conditions
z
(x)
J
(x)
x
-1
0
1
(x) as a
function of the parameter x.
for optimality derived from the KKT conditions do not change. For our example, in
each critical region the optimizer z
(x) is ane and the value function J
(x) is also
ane.
7.2.2 Critical Regions, Value Function and Optimizer: Local Properties
Consider Denition 7.2 of a critical region. In this section we show that critical re-
gions of mp-LP are polyhedra. We use primal feasibility to derive the 1-polyhedral
representation of the critical regions, the complementary slackness conditions to
compute an optimizer z
(x), and the dual problem of (7.11) to derive the optimal

value function J
(x) inside each critical region.

Consider the dual problem of (7.11):
min
u
(w +Sx)
u
subj. to G
u = c
u 0.
(7.14)
The dual feasibility, complementary slackness and primal feasibility conditions for
problems (7.11), (7.14) are
G
u +c = 0, u 0 (7.15a)
u
i
(G
i
z w
i
S
i
x) = 0, i = 1, . . . , m (7.15b)
Gz w Sx 0. (7.15c)
Let us assume that we have determined the optimal partition
(A, NA) = (A(x
), NA(x
))
for some x
. The primal feasibility condition (7.15c) can be rewritten as

G
A
z
S
A
x = w
A
(7.16a)
G
NA
z
S
NA
x < w
NA
. (7.16b)
Here (7.16a) assigns to a parameter x an optimizer z
(x) (in general, a set of

optimizers z
(x)) and (7.16b) denes the critical region CR

A
in the parameter
space /
for which this assignment is valid. In the simplest case assume that G
A
is square and of full rank. From (7.16a) we nd
z
= G
1
A
(S
A
x +w
A
), (7.17)
and substituting in (7.16b) we obtain
G
NA
G
1
A
(S
A
x +w
A
) S
NA
x < w
NA
,
the set of inequalities that denes the critical region CR
A
of parameters x for which
the optimizer is (7.17). The following derivation deals with the general case when
G
A
is not invertible and the optimizer z
(x) is not necessarily unique.

We know from (7.16a) that for x CR
A
G
A
z
S
A
x = w
A
.
For another x CR
A
we require
G
A
z S
A
x = w
A
,
or for the perturbation x = x x
G
A
z S
A
x = 0.
This equation will have a solution z for arbitrary values of x, i.e., a full dimensional
CR
A
if
rank[G
A
] = rank[G
A
S
A
].
Otherwise the dimension over which x can vary will be restricted to
n (rank[G
A
S
A
] rank[G
A
]).
Next we will show how to construct these restrictions on the variation of x.
Let l = rank G
A
and consider the QR decomposition of G
A
G
A
= Q
_
U
1
U
2
0
|A|ll
0
|A|l|A|l
_
,
where Q R
|A||A|
is a unitary matrix, U
1
R
ll
is a full-rank square matrix
and U
2
R
l|A|l
. Let
_
P
D
_
= Q
1
S
A
and
_
q
r
_
= Q
1
w
A
. From (7.16a) we
obtain
_
U
1
U
2
P
0
|A|ll
0
|A|l|A|l
D
_
_
_
z
1
z
2
x
_
_
=
_
q
r
_
. (7.18)
We partition (7.16b) accordingly
_
E F
_
z
1
z
2
_
S
NA
x < w
NA
. (7.19)
Calculating z
1
from (7.18) we obtain
z
1
= U
1
1
(U
2
z
2
Px +q), (7.20)
which substituted in (7.19) gives:
(F EU
1
1
U
2
)z
2
+ (S
NA
EU
1
1
P)x < w
NA
EU
1
1
q. (7.21)
The critical region CR
A
can be equivalently written as:
CR
A
=
_
x T
x
: z
2
such that
_
z
2
x
_
T
z2x
_
, (7.22)
where:
T
x
= x: Dx = r , (7.23)
T
z2x
=
__
z
2
x
_
: (F EU
1
1
U
2
)z
2
+ (S
NA
EU
1
1
P)x < w
NA
EU
1
1
q
_
.
(7.24)
In other words:
CR
A
= T
x
proj
x
(T
z2x
). (7.25)
We can now state some fundamental properties of the critical regions, value
function and optimizer inside a critical region.
Theorem 7.2 Let (A, NA) = (A(x
), NA(x
)) for some x
i) CR
A
is an open polyhedron of dimension d where d = n rank
_
G
A
S
A
+
rank G
A
. If d = 0 then CR
A
= x
.
ii) If rank G
A
= s (recall that z R
s
) then the optimizer z
(x) is unique and

given by an ane function of the state inside CR
A
, i.e., z
(x) = F
i
x+g
i
for
all x CR
A
.
iii) If the optimizer is not unique in CR
A
then Z
(x) is an open polyhedron for

all x CR
A
.
iv) J
(x) is an ane function of the state, i.e., J
(x) = c
i
x+d
i
for all x CR
A
.
Proof:
i) Polytope T
z2x
(7.24) is open and nonempty, therefore it is full-dimensional
in the (z
2
, x) space and dimproj
x
(T
z2x
) = n. Also,
dimT
x
= n rank D = n (rank
_
G
A
S
A
rank G
A
).
Since the intersection of T
x
and proj
x
(T
z2x
) is nonempty (it contains at least
the point x
) we can conclude that

dimCR
A
= n rank
_
G
A
S
A
+ rank G
A
.
Since we assumed that the set / in (7.2) is bounded, CR
A
is bounded. This
implies that CR
A
is an open polytope since it is the intersection of an open
polytope and the subspace Dx = r. In general, if we allow / in (7.2) to be
unbounded, then the critical region CR
A
can be unbounded.
ii) Consider (7.20) and recall that l = rank G
A
. If l = s, then the primal
optimizer is unique, U
2
is an empty matrix and
z
(x) = z
1
(x) = U
1
1
(Px +q). (7.26)
iii) If the primal optimizer is not unique in CR
A
then Z
(x) in CR
A
is the
following point-to-set map: Z
(x) = [z
1
, z
2
] : z
2
T
z2x
, U
1
z
1
+U
2
z
2
+Px =
q). Z
(x) is an open polyhedron since T

z2x
is open.
iv) Consider the dual problem (7.14) and one of its optimizer u
0
for x = x
.
By denition of a critical region u
0
remains optimal for all x CR
A(x
)
.
Therefore the value function in CR
A(x
)
is
J
(x) = (w +Sx)
0
, (7.27)
which is an ane function of x on CR
A
.
Remark 7.1 If the optimizer is unique then the computation of CRA in (7.25) does
not require the projection of the set Tz
2
x in (7.24). In fact, U2 and F are empty
matrices and
CRA = Tz
2
x =
_
x : Dx = r, (SNA EU
1
P)x < wNA EU
1
q
_
. (7.28)
7.2.3 Propagation of the Set of Active Constraints
The objective of this section is to briey describe the propagation of the set of active
constraints when moving from one full-dimensional critical region to a neighboring
full-dimensional critical region. We will use a simple example in order to illustrate
the main points.
Example 7.3 Consider the mp-LP problem
1
minz
1
,z
2
,z
3
,z
4
z1 +z2
subj. to
_
_
z1 x1 +x2
z1 z3 x2
z1 x1 x2
z1 +z3 x2
z2 z3 x1 + 2x2
z2 z3 z4 x2
z2 +z3 x1 2x2
z2 +z3 +z4 x2
z3 1
z3 1
z4 1
z4 1,
(7.29)
1
where / is given by
2.5 x1 2.5
2.5 x2 2.5.
(7.30)
A solution to the mp-LP problem is shown in Figure 7.2 and the constraints which
are active in each associated critical region are reported in Table 7.1. Clearly, as
z R
4
, CR1 = CR
{1,4,5,6,7,8}
and CR2 = CR
{2,3,5,6,7,8}
are primal degenerate
full-dimensional critical regions.
Critical Region Value function
CR1=CR
{1,4,5,6,7,8}
x
1
x
2
CR2=CR
{2,3,5,6,7,8}
x
1
+x
2
CR3=CR
{1,5,6,9}
2x
1
3x
2
1
CR4=CR
{4,6,7,10}
x
1
+ 3x
2
2
CR5=CR
{1,5,8,12}
1.5x
1
1.5x
2
+ 0.5
CR6=CR
{2,5,6,9}
x
1
3x
2
2
CR7=CR
{3,6,7,10}
2x
1
+ 3x
2
1
CR8=CR
{3,6,7,11}
1.5x
1
+ 1.5x
2
0.5
CR9=CR
{1,5,9,12}
2x
1
3x
2
1
CR10=CR
{4,7,10,12}
x
1
+ 3x
2
2
CR11=CR
{2,5,9,12}
x
1
3x
2
2
CR12=CR
{3,7,10,12}
2x
1
+ 3x
2
1
Table 7.1 Example 7.3: Critical regions and corresponding value function.
By observing Figure 7.2 and Table 7.1 we notice the following. Under no primal
and dual degeneracy, (i) full critical regions are described by a set of active con-
straints of dimension n, (ii) two neighboring full-dimensional critical regions CR
Ai
and CR
Aj
have A
i
and A
j
diering only in one constraint, (iii) CR
Ai
and CR
Aj
will share a facet which is a primal degenerate critical region CR
Ap
of dimension
n 1 with A
p
= A
i
A
j
. In Example 7.3, CR7 ad CR12 are two full-dimensional
and neighboring critical regions and the corresponding active set diers only in one
constraint (constraint 6 in CR7 and constraint 12 in CR12). They share a facet
which is a one dimensional critical region (an open line), CR13=CR
{3,6,7,10,12}
.
(The solution depicted in Figure 7.2 and detailed in Table 7.1 contains only full-
dimensional critical regions.)
If primal and/or dual degeneracy occur, then the situation becomes more com-
plex. In particular, in the case of primal degeneracy, it might happen that full-
dimensional critical regions are described by more than s active constraints (CR1
or CR2 in Example 7.3). In case of dual degeneracy, it might happen that full-
dimensional critical regions are described by fewer than s active constraints. Details
on this case are provided in the next section.
1
2
3
4
5
6
7
8
9
10
11
12
x
1
x
2
-2.5 2.5
-2.5
2.5
Figure 7.2 Polyhedral partition of the parameter space corresponding to the
solution of Example 7.3. Download code.
7.2.4 Nonunique Optimizer
If rank[G
A
] < s then Z
(x) is not a singleton and the projection of the set T

z2x
in (7.24) is required in order to compute the critical region CR
A(x
)
(see Sec-
tion 7.1.1 for a discussion of polyhedra projection).
If one needs to determine one possible optimizer z
() in the dual degenerate

region CR
A(x
)
the following simple method can be used. Choose a particular
optimizer which lies on a vertex of the feasible set, i.e., determine set

A(x
) A(x
)
of active constraints for which rank(G
A(x
)
) = s, and compute a subset

CR
A(x
)
of
the dual degenerate critical region (namely, the subset of parameters x such that
the constraints

A(x
) are active at the optimizer, which is not a critical region in

the sense of Denition 7.2). Within

CR
A(x
)
, the piecewise linear expression of an
optimizers z
(x) is available from (7.26).

The algorithm proceeds by exploring the space surrounding

CR
A(x
)
until
CR
A(x
)
is covered. The arbitrariness in choosing an optimizer leads to dier-
ent ways of partitioning CR
{A(x
)}
, where the partitions, in general, may overlap.
Nevertheless, in each region a unique optimizer is dened. The storing of over-
lapping regions can be avoided by intersecting each new region (inside the dual
degenerate region) with the current partition computed so far. This procedure is
illustrated in the following example.
CR
{2}
CR
{1,3}
CR
{1,5}
x
1
x
2
Figure 7.3 Example 7.4: Polyhedral partition of the parameter space corre-
sponding to the solution.
Example 7.4 Consider the following mp-LP reported in [111, p. 152]
min 2z1 z2
subj. to
_
_
z1 + 3z2 9 2x1 +x2
2z1 +z2 8 +x1 2x2
z1 4 +x1 +x2
z1 0
z2 0,
(7.31)
where / is given by:
10 x1 10
10 x2 10.
(7.32)
The solution is represented in Figure 7.3 and the critical regions are listed in Table 7.2.
Region Optimizer Value Function
CR
{2}
not single valued x
1
+ 2x
2
8
CR
{1,5}
z
1
= 2x
1
+x
2
+ 9, z
2
= 0 4x
1
2x
2
18
CR
{1,3}
z
1
= x
1
+x
2
+ 4, z
2
= x
1
+ 5/3 x
1
2x
2
29/3
Table 7.2 Example 7.4: Critical regions and corresponding optimal value.
{2}
is related to a dual degenerate solution with non-
unique optimizers. The analytical expression of CR
{2}
is obtained by projecting
the 1-polyhedron
z
1
+ 3z
2
+ 2x
1
x
2
< 9
2z
1
+z
2
x
1
+ 2x
2
= 8
z
1
x
1
x
2
< 4
z
1
< 0
z
2
< 0
(7.33)
on the parameter space to obtain:
CR
{2}
=
_
_
[x
1
, x
2
] :
2.5x
1
2x
2
5
0.5x
1
+x
2
4
12x
2
5
x
1
x
2
4
_
_
, (7.34)
which is eectively the result of (7.25). For all x CR
{2}
, only one constraint is
active at the optimum, which makes the optimizer not unique.
Figures 7.4, 7.5 and 7.6 show two possible ways of covering CR
{2}
without using
projection. The generation of overlapping regions is avoided by intersecting each
new region with the current partition computed so far, as shown in Figure 7.7 where
CR
{2,4}
and

CR
{2,1}
represent the intersected critical regions. In Figures 7.4, 7.5
the regions are overlapping, and in Figure 7.7 articial cuts are introduced at
the boundaries inside the degenerate critical region CR
{2}
. No articial cuts are
introduced in Figure 7.6 because the

CR
{2,3}
and

CR
{2,5}
happen to be non-
overlapping.
7.2.5 Value Function and Optimizer: Global Properties
In this section we discuss global properties of the value function J
(x), optimizer
z
(x), and of the set /
.
Theorem 7.3 Assume that for a xed x
0
/ there exists a nite optimal solution
z
(x
0
) of (7.11). Then, for all x /, (7.11) has either a nite optimum or no
feasible solution.
Proof: Consider the mp-LP (7.11) and assume by contradiction that there
exist x
0
/ and x / with a nite optimal solution z
(x
0
) and an unbounded
solution z
( x). Then, the dual problem (7.14) for x = x is infeasible. This implies
that the dual problem will be infeasible for all real vectors x since x enters only in
the cost function. This contradicts the hypothesis since the dual problem (7.14)
for x = x
0
has a nite optimal solution.
Theorem 7.4 The set of all parameters x such that the LP (7.11) has a nite
optimal solution z
(x) equals /
.
Proof: It follows directly from from Theorem 7.1 and Theorem 7.3.
Note that from Denition 7.3 /
is the set of feasible parameters. However the

LP (7.11) might be unbounded from some x /
. Theorem 7.4 excludes this case.

The following Theorem 7.5 summarizes the properties enjoyed by the multi-
parametric solution.
Theorem 7.5 The function J
() is convex and piecewise ane over /
. If the
optimizer z
(x) is unique for all x /
, then the optimizer function z
: /
R
s
is continuous and piecewise ane. Otherwise it is always possible to dene a
continuous and piecewise ane optimizer function z
such that z
(x) Z
(x) for
all x /
.
x
1
x
2
z
1
z
2
x
1
CR
{2,5}
2
5
4
1
c
x = const
(a) First region

CR
{2,5}
CR
{2}
, and below
the feasible set in the z-space corresponding
to x1

CR
{2,5}
.
x
1
x
2
z
1
z
2
x
2
CR
{2,4}
2
5
4
c
x = const
3
(b) Second region

CR
{2,4}
CR
{2}
, and be-
low the feasible set in the z-space correspond-
ing to x2

CR
{2,4}
.
Figure 7.4 Example 7.4: A possible sub-partitioning of the degenerate region
CR2 where the regions

CR
{2,5}
(Fig. 7.4(a)),

CR
{2,4}
(Fig. 7.4(b)) and
CR
{2,1}
(Fig. 7.5(a)) are overlapping. Note that below each picture the
feasible set and the level set of the value function in the z-space is depicted
for a particular choice of the parameter x indicated by a point marked with
.
x
1
x
2
z
1
z
2
x
3
CR
{2,1}
2
5
4
c
x = const
3
1
(a) Third region

CR
{2,1}
CR
{2}
, and below
the feasible set in the z-space corresponding
to x3

CR
{2,1}
.
x
1
x
2
CR
{2,1}
CR
{2,4}
CR
{2,5}
(b) Final partition of CR
{2}
. Note that the
region

CR
{2,5}
is hidden by region

CR
{2,4}
and region

CR
{2,1}
.
Figure 7.5 Example 7.4: A possible sub-partitioning of the degenerate region
CR2 where the regions

CR
{2,5}
(Fig. 7.4(a)),

CR
{2,4}
(Fig. 7.4(b)) and
CR
{2,1}
(Fig. 7.5(a)) are overlapping. Note that below each picture the
feasible set and the level set of the value function in the z-space is depicted
for a particular choice of the parameter x indicated by a point marked with
.
Remark 7.2 In Theorem 7.5, the piecewise ane property of optimizer and value
function follows immediately from Theorem 7.2 and from the enumeration of all
possible combinations of active constraint sets. Convexity of J
() and continuity of
Z
(x) follows from standard results on multiparametric programs (see Theorem 6.2
and Corollary 6.1). In the presence of multiple optimizers, the proof of existence of
x
1
x
2
CR
{2,5}
(a) First region

CR
{2,5}
CR
{2}
.
x
1
x
2
CR
{2,3}
(b) Second region

CR
{2,3}
CR
{2}
.
Figure 7.6 A possible solution to Example 7.4 where the regions

CR
{2,5}
and

CR
{2,3}
are non-overlapping
a continuous and piecewise ane optimizer function z
such that z
(x) Z
(x) for
all z /
is more involved and we refer the reader to [111, p. 180].

7.2.6 mp-LP Algorithm
The goal of an mp-LP algorithm is to determine the partition of /
into critical
regions CR
Ai
, and to nd the expression of the functions J
() and z
() for each
critical region. An mp-LP algorithm has two components: the active set gener-
ator and the KKT solver. The active set generator computes the set of active
constraints A
i
. The KKT solver computes CR
Ai
and the expression of J
() and
x
1
x
2
CR
{2,5}
CR
{2,4}
CR
{2,1}
Figure 7.7 A possible solution for Example 7.4 where

CR
{2,4}
is obtained
by intersecting

CR
{2,4}
with the complement of

CR
{2,5}
, and

CR
{2,1}
by
intersecting

CR
{2,1}

CR
{2,5}
and

CR
{2,4}
z
() in CR
Ai
as explained in Theorem 7.2.
The active set generator is the critical part. In principle, one could simply
generate all the possible combinations of active sets. However, in many problems
only a few active constraints sets generate full-dimensional critical regions inside
the region of interest /. Therefore, the goal is to design an active set generator
algorithm which computes only the active sets A
i
with associated full-dimensional
critical regions covering only /
. This avoids the combinatorial explosion of a

complete enumeration. Next we will describe one possible implementation of an
mp-LP algorithm.
In order to start solving the mp-LP problem, we need an initial vector x
0
inside the polyhedral set /
of feasible parameters. A possible choice for x

0
is the
Chebychev center (see Section 5.4.5) of /
, i.e., x
0
solving the following LP:
max
x,z,

subj. to T
i
x +|T
i
|
2
N
i
, i = 1, . . . , n
T
Gz Sx w,
(7.35)
where n
T
is the number of rows T
i
of the matrix T dening the set / in (7.2).
If 0, then the LP problem (7.11) is infeasible for all x in the interior of /.
Otherwise, we solve the primal and dual problems (7.11), (7.14) for x = x
0
. Let
z
0
and u
0
be the optimizers of the primal and the dual problem, respectively. The
value z
0
denes the following optimal partition
A(x
0
) = j I : G
j
z
0
S
j
x
0
w
j
= 0
NA(x
0
) = j I : G
j
z
0
S
j
x
0
w
j
< 0
(7.36)
and consequently the critical region CR
A(x0)
. Once the critical region CR
A(x0)
has been dened, the rest of the space R
rest
= /CR
A(x0)
has to be explored and
new critical regions generated. An approach for generating a polyhedral parti-
tion R
1
, . . . , R
nrest
of the rest of the space R
rest
is described in Theorem 5.2 in
Section 5.4.7. The procedure proposed in Theorem 5.2 for partitioning the set of
parameters allows one to recursively explore the parameter space. Such an iterative
procedure terminates after a nite time, as the number of possible combinations of
active constraints decreases with each iteration. The following two issues need to
be considered:
1. The partitioning in Theorem 5.2 denes new polyhedral regions R
k
to be
explored that are not related to the critical regions which still need to be
determined. This may split some of the critical regions, due to the articial
cuts induced by Theorem 5.2. Post-processing can be used to join cut critical
regions [41]. As an example, in Figure 7.8 the critical region CR
{3,7}
is
discovered twice, one part during the exploration of R
1
and the second part
during the exploration of R
2
.
Although algorithms exist for convexity recognition and computation of the
union of polyhedra, the post-processing operation is computationally expen-
sive. Therefore, it is more ecient not to intersect the critical region obtained
by (7.28) with halfspaces generated by Theorem 5.2, which is only used to
drive the exploration of the parameter space. Then, no post processing is
needed to join subpartitioned critical regions.
On the other hand, some critical regions may appear more than once. Dupli-
cates can be uniquely identied by the set of active constraints A(x) and can
be easily eliminated. To this aim, in the implementation of the algorithm we
keep a list of all the critical regions which have already been generated in or-
der to avoid duplicates. In Figure 7.8 the critical region CR
{3,7}
is discovered
twice but stored only once.
x
1
x
2
x
+
1
x
1
x
+
2
x
2
R
2
R
1
CR
{6,7}
CR
{3,7}
Figure 7.8 Example of a critical region explored twice
2. If a region is generated which is not full-dimensional we want to avoid further
recursion of the algorithm not producing any full-dimensional critical region,
and therefore lengthening the number of steps required to determine the
solution to the mp-LP. We perturb the parameter x
0
by a random vector
with length smaller than the Chebychev radius of the polyhedral region R
k
where we are looking for a new critical region. This will ensure that the
perturbed vector is still contained in R
k
.
Based on the above discussion, the mp-LP solver can be summarized in the
following recursive Algorithm 7.1. Note that the algorithm generates a partition
of the state space which is not strict. The algorithm could be modied to store
the critical regions as dened in (7.4) which are open sets, instead of storing their
closure. In this case the algorithm would have to explore and store all the crit-
ical regions that are not full-dimensional in order to generate a strict polyhedral
partition of the set of feasible parameters. From a practical point of view such a
procedure is not necessary since the value function and the optimizer are continuous
functions of x.
Algorithm 7.1 mp-LP Algorithm
input: Matrices c, G, w, S of the mp-LP (7.11) and set / in (7.2)
output: Multiparametric solution to the mp-LP (7.11)
execute partition(/);
end.
procedure partition(Y )
let x0 Y and be the solution to the LP (7.35);
if 0 then exit; (no full-dimensional CR is in Y )
solve the LP (7.11), (7.14) for x = x0;
if the optimizer is not unique, then choose z
0
with rank(A(x0)) = s, where
A(x0) is the set of active constraints dened in (7.36)
let A(x0) be the set of active constraints as in (7.36);
let U1, U2, P, D matrices as in (7.18).
determine z
(x) from (7.26) and CR

A(x
0
)
from (7.28);
choose z
0
as one of the possible optimizers;
let J
(x) as in (7.27) for x = x0;

dene and partition the rest of the region as in Theorem 5.2;
for each new sub-region Ri, partition(Ri);
end procedure
Remark 7.3 As remarked in Section 7.2.2, if rank(D) > 0 in Algorithm 7.1, the
region CR
A(x
0
)
is not full-dimensional. To avoid further recursion in the algorithm
which does not produce any full-dimensional critical region, after computing U, P, D
if D ,= 0 one should compute a random vector R
n
smaller than the Chebychev
radius of Y and such that the LP (7.11) is feasible for x0 + and then repeat step
where A(x0) is computed with x0 x0 +.
Remark 7.4 Note that in Algorithm 7.1 in order to choose z
0
with rank(A(x0)) = s
one can use an active set method for solving the LP (7.11) for x = x0.
Remark 7.5 The algorithm determines the partition of / recursively. After the rst
critical region is found, the rest of the region in / is partitioned into polyhedral sets
|Ri as in Theorem 5.2. By using the same method, each set Ri is further partitioned,
and so on.
7.3 Multiparametric Quadratic Programming 127
7.3 Multiparametric Quadratic Programming
7.3.1 Formulation
In this section we investigate multiparametric quadratic programs (mp-QP), a spe-
cial case of the multiparametric program (7.1) where the objective is a quadratic
function
J
(x) = min
z
J(z, x) =
1
2
z
Hz
subj. to Gz w +Sx.
(7.37)
All the variables were dened in Section 7.1.1. We assume H 0. Our goal is to
nd the value function J
(x) and the optimizer function z
(x) in /
. Note that
/
can be determined by projection as discussed in Theorem 7.1. As suggested

through Example 6.1 our search for these functions proceeds by partitioning the set
of feasible parameters into critical regions. Note that the more general problem with
J(z, x) =
1
2
z
Hz + x
Fz can always be transformed into an mp-QP of form (7.37)

by using the variable substitution z = z + H
1
F
x.
As in the previous sections, we denote with the subscript j the j-th row of a
matrix or j-th element of a vector. Also, J = 1, . . . , m is the set of constraint
indices and for any A J, G
A
, w
A
and S
A
are the submatrices of G, w and S,
respectively, consisting of the rows indexed by A. Without loss of generality we
will assume that /
is full-dimensional (if it is not, then the procedure described in

Section 7.1.3 can be used to obtain a full-dimensional /
in a reduced parameter
space).
Example 7.5 Consider the parametric quadratic program
J
(x) = min
z
J(z, x) =
1
2
z
2
subj. to z 1 + 3x,
where z R and x R. Our goals are:
1. to nd z
(x) = arg min

z, z1+3x
J(z, x),
(x).
The Lagrangian function is
L(z, x, u) =
1
2
z
2
+u(z 3x 1)
and the KKT conditions are (see Section 3.3.3 for KKT conditions for quadratic
programs)
z +u = 0 (7.38a)
u(z 3x 1) = 0 (7.38b)
u 0 (7.38c)
z 3x 1 0 (7.38d)
Consider (7.38) and the two strictly complementary cases:
A.
z +u = 0
z 3x 1 = 0
u 0
_
z
= 3x + 1
u
= 3x 1
J
=
9
2
x
2
+ 3x +
1
2
x
1
3
B.
z +u = 0
z 3x 1 < 0
u = 0
_
z
= 0
u
= 0
J
= 0
x >
1
3
(7.39)
This solution is depicted in Figure 7.9. The above simple procedure, which required
nothing but the solution of the KKT conditions, yielded the optimizer z
(x) and
(x) for all values of the parameter x. The set of admissible

parameters values was divided into two critical regions, dened by x
1
3
and
x >
1
3
. In the region x
1
3
the inequality constraint is active and the Lagrange
multiplier is greater or equal than zero, in the other region x >
1
3
the inequality
constraint is not active and the Lagrange multiplier is equal to zero. Note that for
x =
1
3
the inequality constraint z 3x 1 0 is active at z
and the Lagrange

multiplier is equal to zero.
Throughout a critical region the conditions for optimality derived from the KKT
conditions do not change. In each critical region the optimizer z
(x) is ane and the

value function J
(x) is quadratic. Both z
(x) and J
(x) are continuous.

x
z
(x)
J
(x)
-2 -1 0 1 2
-2
-1
0
1
2
(x) as a
function of the parameter x. Download code.
7.3.2 Critical Regions, Value Function and Optimizer: Local Properties
Consider the denition of a critical region given in Section 7.1.2. We show next that
critical regions of mp-QP are polyhedra. We use the KKT conditions (Section 2.4)
to derive the 1-polyhedral representation of the critical regions and to compute the
optimizer function z
(x) inside each critical region.

The following theorem introduces fundamental properties of critical regions, the
value function and the optimizer inside a critical region.
Theorem 7.6 Let (A, NA) = (A( x), NA( x)) for some x /
, Then
i) the closure of CR
A
is a polyhedron.
ii) z
(x) is an ane function of the state inside CR

A
, i.e., z
(x) = F
i
x+g
i
for
all x CR
A
.
iii) J
(x) is a quadratic function of the state inside CR

A
, i.e., J
(x) = x
M
i
x +
c
i
x +d
i
for all x CR
A
Proof: The rst-order Karush-Kuhn-Tucker (KKT) optimality conditions (see
Section 3.3.3) for the mp-QP are given by
Hz
+G
= 0, u R
m
(7.40a)
u
i
(G
i
z
w
i
S
i
x) = 0, i = 1, . . . , m (7.40b)
u
0 (7.40c)
Gz
w Sx 0. (7.40d)
We solve (7.40a) for z
= H
1
G
(7.41)
and substitute the result into (7.40b) to obtain the complementary slackness con-
ditions
u
i
(G
i
H
1
G
w
i
S
i
x) = 0, i = 1, . . . , m. (7.42)
Let u
NA
and u
A
denote the Lagrange multipliers corresponding to inactive and
active constraints, respectively. For inactive constraints u
NA
= 0. For active
constraints:
(G
A
H
1
G
A
)u
A
w
A
S
A
x = 0. (7.43)
If the set of active constraint A is empty, then u
= u
NA
= 0 and therefore z
= 0
which implies that the critical region CR
A
is
CR
A
= x : Sx +w > 0. (7.44)
Otherwise we distinguish two cases.
Case 1: LICQ holds, i.e. the rows of G
A
are linearly independent. This implies
that (G
A
H
1
G
A
) is a square full rank matrix and therefore

u
A
= (G
A
H
1
G
A
)
1
(w
A
+S
A
x), (7.45)
where G
A
, w
A
, S
A
correspond to the set of active constraints A. Thus u
is an
ane function of x. We can substitute u
A
from (7.45) into (7.41) to obtain
z
= H
1
G
A
(G
A
H
1
G
A
)
1
(w
A
+S
A
x) (7.46)
and note that z
is also an ane function of x. J
(x) =
1
2
z
(x)
Hz
(x) and
therefore is a quadratic function of x. The critical region CR
A
is computed by
substituting z
from (7.46) in the primal feasibility conditions (7.40d)

T
p
= x : GH
1
G
A
(G
A
H
1
G
A
)
1
(w
A
+S
A
x) < w +Sx, (7.47)
and the Lagrange multipliers from (7.45) in the dual feasibility conditions (7.40c)
T
d
= x : (G
A
H
1
G
A
)
1
(w
A
+S
A
x) 0. (7.48)
In conclusion, the critical region CR
A
is the intersection of T
p
and T
d
:
CR
A
= x : x T
p
, x T
d
(7.49)
Obviously, the closure of CR
A
is a polyhedron in the x-space.
The polyhedron T
p
is open and non empty (it contains at least the point x).
Therefore it is full-dimensional in the x space. This implies that dimCR
A
=
dimT
d
.
Case 2: LICQ does not hold, the rows of G
A
are not linearly independent. For
instance, this happens when more than s constraints are active at the optimizer
z
( x) R
s
, i.e., in a case of primal degeneracy. In this case the vector of Lagrange
multipliers u
might not be uniquely dened, as the dual problem of (7.37) is not

strictly convex. Note that dual degeneracy and nonuniqueness of z
( x) cannot
occur, as H 0.
Using the same arguments as in Section 7.2.2, equation (7.43) will allow a
full-dimensional critical region only if rank[(G
A
H
1
G
A
) S
A
] = rank[G
A
H
1
G
A
].
Indeed the dimension of the critical region will be
n (rank[(G
A
H
1
G
A
) S
A
] rank[G
A
H
1
G
A
]).
We will now show how to compute the critical region.
Let l = rank G
A
and consider the QR decomposition of G
A
H
1
G
A
G
A
H
1
G
A
= Q
_
U
1
U
2
0
|A|ll
0
|A|l|A|l
_
,
where Q R
|A||A|
is a unitary matrix, U
1
R
ll
is a full-rank square matrix
and U
2
R
l|A|l
. Let
_
P
D
_
= Q
1
S
A
and
_
q
r
_
= Q
1
w
A
. From (7.43) we
obtain
_
U
1
U
2
P
0
|A|ll
0
|A|l|A|l
D
_
_
_
u
A,1
u
A,2
x
_
_
=
_
q
r
_
. (7.50)
We compute u
A,1
from (7.50)
u
A,1
= U
1
1
(U
2
u
A,2
Px +q). (7.51)
Finally, we can substitute u
A
=
_
u
A,1
A,2
from (7.51) into (7.41) to obtain

z
= H
1
G
A
_
U
1
1
(U
2
u
A,2
+Px q)
u
A,2
_
. (7.52)
The optimizer z
is unique and therefore independent of the choice of u
A,2
0.
Setting u
A,2
= 0 we obtain
z
= H
1
G
A,1
U
1
1
(Px q), (7.53)
where we have partitioned G
A
as G
A
=
_
G
A,1
G
A,2
and G
A,1
R
ls
.
Note that z
is an ane function of x. J
(x) =
1
2
z
(x)
Hz
(x) and therefore

is a quadratic function of x.
A
is computed by substituting z
from (7.53) in the

primal feasibility conditions (7.40d)
T
p
= x : G(H
1
G
A,1
U
1
1
(Px q)) < w +Sx, (7.54)
and the Lagrange multipliers from (7.51) in the dual feasibility conditions (7.40c)
T
u
A,2
,x
= [ u
A,2
, x] : U
1
1
(U
2
u
A,2
Px +q) 0, u
A,2
0. (7.55)
A
is the intersection of the sets Dx = r, T
p
and proj
x
(T
u
A,2
,x
):
CR
A
= x : Dx = r, x T
p
, x proj
x
(T
u
A,2
,x
). (7.56)
The closure of CR
A
is a polyhedron in the x-space.
Remark 7.6 In general, the critical region polyhedron (7.47)-(7.49) is open on facets
arising from primal feasibility and closed on facets arising from dual feasibility.
Remark 7.7 If D in (7.50) is nonzero, then from (7.56), CRA is a lower dimensional
region, which, in general, corresponds to a common boundary between two or more
full-dimensional regions. Typically, the following denition is introduced
7.3.3 Propagation of the Set of Active Constraints
The objective of this section is to briey describe the propagation of the set of active
constraints when moving from one full-dimensional critical region to a neighboring
full-dimensional critical region. We will use a simple example in order to illustrate
the main points.
Example 7.6 Consider the mp-QP problem
2
J
(x) = min
z
1
2
z
Hz +x
Fz
subj. to Gz w +Sx,
(7.57)
with
H = [
1 0
0 1
] , F =
_
1 1
1 1
_
(7.58)
and
G =
_
_
1 0
1 0
0 1
0 1
1 1
1 1
_
_
, S =
_
_
0 0
0 0
0 0
0 0
1 1
2 1
_
_
, w =
_
_
1
1
1
1
0
0
_
_
, (7.59)
where / is given by
1 x1 1
1 x2 1.
(7.60)
Critical Region Active Constraints
CR1 5
CR2 3,5
CR3
CR4 3
CR5 6
CR6 3,6
CR7 4,6
Table 7.3 Example 7.6: Critical regions and corresponding set of active
constraints.
1
2
3
4
5
6
7
x
1
x
2
-1 1
-1
1
Figure 7.10 Polyhedral partition of the parameter space corresponding to
the solution of Example 7.6. Download code.
A solution to the mp-QP problem is shown in Figure 7.10 and the constraints which
are active in each associated full-dimensional critical region are reported in Table 7.3.
Since A
3
is empty in CR3 we can conclude from (7.44) that the facets of CR3
are facets of primal feasibility and therefore do not belong to CR3. In general,
as discussed in Remark 7.6 critical regions are open on facets arising from primal
feasibility and closed on facets arising from dual feasibility. Next we focus on the
closure of the critical regions.
2
By observing Figure 7.10 and Table 7.3 we notice that as we move away from
region CR3 (corresponding to no active constraints), the number of active con-
straints increases. In particular, for any two neighboring full-dimensional critical
regions CR
Ai
and CR
Aj
we have A
i
A
j
and [A
i
[ = [A
j
[ 1 or A
j
A
i
and
[A
i
[ = [A
j
[ +1. This means that as one moves from one full-dimensional region to
a neighboring full-dimensional region, one constraint is either included to the list
of active constraints or removed from it. This happens for instance when moving
from CR3 to CR4, CR5, CR1 or from CR1 to CR2, or from CR5 to CR7.
Things become more complex if LICQ does not hold everywhere in /
. In
particular, there might exist two neighboring full-dimensional critical regions CR
Ai
and CR
Aj
where A
i
and A
j
do not share any constraint and with [A
i
[ = [A
j
[. Also,
CR
Ai
might have multiple neighboring region on the same facet. In other words,
it can happen that the intersection of the closures of two adjacent full-dimensional
critical regions is a not a facet of both regions but only a subset of it. We refer the
reader to [267] for more details on such a degenerate condition.
7.3.4 Value Function and Optimizer: Global Properties
The convexity of the value function J
(x) and the continuity of the solution z
(x)
easily follow from the general results on multiparametric programming as proven in
Corollary 6.2. In the following we give a simple proof based on the linearity result
of Theorem 7.6 together with a proof of the piecewise linearity of the solution z
(x).
We remark that Theorem 7.1 proves that the set of feasible parameters /
/ of
an mp-QP is a polyhedron.
Theorem 7.7 Consider the multiparametric quadratic program (7.37) and let H
0. Then the optimizer z
(x) : /
R
s
is continuous and piecewise ane on
polyhedra, in particular it is ane in each critical region, and the optimal solution
J
(x) : /
R is continuous, convex and piecewise quadratic on polyhedra.

Proof: We rst prove convexity of J
(x). Take generic x

1
, x
2
/
, and let
J
(x
1
), J
(x
2
) and z
1
, z
2
the corresponding optimal values and minimizers. Let
x
= x
1
+ (1 )x
2
and z
= z
1
+ (1 )z
2
. By optimality of J
(x
),
J
(x
)
1
2
z
Hz
.
Hence
J
(x
)
1
2
[z
1
Hz
1
+ (1 )z
2
Hz
2
]
1
2
z
Hz
1
2
[z
1
Hz
1
+ (1 )z
2
Hz
2
].
We can upperbound the latter term as follows:
1
2
z
Hz
1
2
[z
1
Hz
1
+ (1 )z
2
Hz
2
] =
1
2
[
2
z
1
Hz
1
+ (1 )
2
z
2
Hz
2
+
+2(1 )z
2
Hz
1
z
1
Hz
1
(1 )z
2
Hz
2
] =
=
1
2
(1 )(z
1
z
2
)
H(z
1
z
2
) 0
In conclusion
J
(x
1
+ (1 )x
2
) J
(x
1
) + (1 )J
(x
2
), x
1
, x
2
/, [0, 1],
which proves the convexity of J
(x) on /
.
Within the closed polyhedral regions CR
i
in /
the solution z
(x) is ane (7.46)

by Theorem 7.6. The boundary between two regions belongs to both closed regions.
Since H 0, the optimum is unique, and hence the solution must be continuous
across the boundary. Therefore z
(x) : /
R
s
is continuous and piecewise ane
on polyhedra. The fact that J
(x) is continuous and piecewise quadratic follows

trivially.
We can easily derive a result equivalent to Theorem 7.7 for the dual function
d
(x) (see (7.37) and (3.28))

d
(x) = max
u0
_
d(u) =
1
2
u
(GH
1
G
)u u
(w +Sx)
_
. (7.61)
Theorem 7.8 Consider the multiparametric quadratic program (7.37) with H 0
and assume that LICQ holds x /
. Then the optimizer u
(x) : /
R
m
of the dual (7.61) is continuous and piecewise ane on polyhedra, in particular
it is ane in each critical region, and the optimal solution d
(x) : /
R is
continuous, convex and piecewise quadratic on polyhedra.
Proof: The proof follows along the lines of the proof of Theorem 7.7 once we
know that the optimizer u
(x) is unique because of the LICQ assumptions. This

is also a special case of Theorem 6.3.
Theorem 7.9 Assume that the mp-QP problem (7.37) is not degenerate, then the
value function J
(x) in (7.37) is C
(1)
.
Proof: The dual of (7.37) is
d
(x) = max
u0
1
2
u
(GH
1
G
)u u
(w +Sx). (7.62)
By strong duality we have
J
(x) = d
(x) =
1
2
u
(x)
(GH
1
G
)u
(x) u
(x)
(w +Sx). (7.63)
For the moment let us restrict our attention to the interior of a critical region.
From (7.45) we know that u
(x) is continuously dierentiable. Therefore
x
J(x) =
_
u
x
(x)
_
GH
1
G
(x)
_
u
x
(x)
_
(W +Sx) S
(x) =
=
_
u
x
(x)
_
(GH
1
G
(x) W Sx) S
(x).
(7.64)
Using (7.41)
z
= H
1
G
(7.65)
we get
x
J(x) = S
(x) +
_
u
x
(x)
_
(Gz
W Sx) =
= S
(x) +
iA
_
u
i
x
(x)
_
(G
i
z
W
i
S
i
x)
. .
0
+
+
iNA
_
u
i
x
(x)
_
. .
0
(G
i
z
W
i
S
i
x) =
= S
(x).
(7.66)
Continuous dierentiability of J
in the interior of a critical region follows from (7.45).

Continuous dierentiability of J
at the boundary follows from the fact that u
(x)
is continuous (Theorem 7.8) for all x in the interior of /
[20].
Note that in case of degeneracy the value function J
(x) in (7.37) may not be C

(1)
.
The following counterexample was given in [47].
Example 7.7
Consider the mp-QP (7.37) with
H =
_
_
3 3 1
3 11 23
1 23 75
_
_
G =
_
_
_
_
_
_
_
_
_
_
1 1 1
1 3 5
1 1 1
1 3 5
1 0 0
0 1 0
0 0 1
_
_
w =
_
_
_
_
_
_
_
_
_
_
1
0
1
0
0
0
0
_
_
S =
_
_
_
_
_
_
_
_
_
_
0
1
0
1
0
0
0
_
_
(7.67)
and / = |x R[1 x 5.
The problem was solved by using Algorithm 7.2 described next. The solution com-
prises 5 critical regions. The critical regions and the expression of the value function
are reported in Table 7.4. The reader can verify that the value function is not con-
tinuously dierentiable at x = 3. Indeed, at x = 3 the LICQ condition does not hold
and therefore, the hypothesis of Theorem 7.9 is not fullled.
7.3.5 mp-QP Algorithm
The goal of an mp-QP algorithm is to determine the partition of /
into critical
regions CR
i
, and nd the expression of the functions J
() and z
() for each critical

region. An mp-QP algorithm has two components: the active set generator and
the KKT solver. The active set generator computes the set of active constraints
A
i
. The KKT solver computes CR
Ai
and the expression of J
() and z
() in
CR
Ai
as explained in Theorem 7.6. The active set generator is the critical part.
Region Optimal value
CR
{1,2,3,4,6}
= x : 1 x < 1.5 2.5x
2
6x + 5
CR
{1,2,3,4}
= x : 1.5 x 2 0.5x
2
+ 0.5
CR
{1,2,3,4,7}
= x : 2 < x < 3 x
2
2x + 2.5
CR
{1,2,3,4,5,7}
= x : x = 3 x
2
2x + 2.5
CR
{1,2,3,4,5}
= x : 3 < x 5 5x
2
24x + 32.5
Table 7.4 Critical regions and value function corresponding to the solution
of Example 7.7
In principle, one could simply generate all the possible combinations of active
sets. However, in many problems only a few active constraints sets generate full-
dimensional critical regions inside the region of interest /. Therefore, the goal is
to design an active set generator algorithm which computes only the active sets A
i
with the associated full-dimensional critical regions covering only /
.
Next an implementation of an mp-QP algorithm is described. See Section 7.6
for a literature review on alternative approaches to the solution of an mp-QP.
In order to start solving the mp-QP problem, we need an initial vector x
0
inside the polyhedral set /
of feasible parameters. A possible choice for x

0
is the
Chebychev center (see Section 5.4.5) of /
obtained as a solution of
max
x, z,

subj. to T
i
x +|T
i
|
2
N
i
, i = 1, . . . , n
T
G z Sx w
(7.68)
where n
T
is the number of rows T
i
of the matrix T dening the set / in (7.2).
If 0, then the QP problem (7.37) is infeasible for all x in the interior of /.
Otherwise, we set x = x
0
and solve the QP problem (7.37), in order to obtain the
corresponding optimal solution z
0
. Such a solution is unique, because H 0. The
value of z
0
denes the following optimal partition
A(x
0
) = j J : G
j
z
0
S
j
x
0
w
j
= 0
NA(x
0
) = j J : G
j
z
0
S
j
x
0
w
j
< 0
(7.69)
and consequently the critical region CR
A(x0)
. Once the critical region CR
A(x0)
has been dened, the rest of the space R
rest
= /CR
A(x0)
has to be explored and
new critical regions generated. An approach for generating a polyhedral partition
R
1
, . . . , R
nrest
of the rest of the space R
rest
is described in Theorem 5.2. Theo-
rem 5.2 provides a way of partitioning the non-convex set / CR
0
into polyhedral
subsets R
i
. For each R
i
, a new vector x
i
is determined by solving the LP (7.68),
and, correspondingly, an optimum z
i
, a set of active constraints A
i
, and a critical
region CR
i
. The procedure proposed in Theorem 5.2 for partitioning the set of
parameters allows one to recursively explore the parameter space. Such an iterative
procedure terminates after a nite time, as the number of possible combinations
of active constraints decreases with each iteration. Two main elements need to be
considered:
1. As for the mp-LP algorithm, the partitioning in Theorem 5.2 denes new
polyhedral regions R
k
to be explored that are not related to the critical
regions which still need to be determined. This may split some of the critical
regions, due to the articial cuts induced by Theorem 5.2. Post-processing
can be used to join cut critical regions [41]. As an example, in Figure 7.8 the
critical region CR
{3,7}
is discovered twice, one part during the exploration of
R
1
and the second part during the exploration of R
2
.
Although algorithms exist for convexity recognition and computation of the
union of polyhedra, the post-processing operation is computationally expen-
sive. Therefore, it is more ecient not to intersect the critical region obtained
by (7.28) with halfspaces generated by Theorem 5.2, which is only used to
drive the exploration of the parameter space. Then, no post processing is
needed to join subpartitioned critical regions. On the other hand, some criti-
cal regions may appear more than once. Duplicates can be uniquely identied
by the set of active constraints A(x) and can be easily eliminated. To this
aim, in the implementation of the algorithm we keep a list of all the critical
regions which have already been generated in order to avoid duplicates. In
Figure 7.8 the critical region CR
{3,7}
is discovered twice but stored only once.
2. If Case 2 occurs in Section 7.3.2 and D is nonzero, CR
A
is a lower dimen-
sional critical region (see Remark 7.7). Therefore we do not need to explore
the actual combination G
A
, S
A
, w
A
. On the other hand, if D = 0 the
KKT conditions do not lead directly to (7.47)(7.48). In this case, a full-
dimensional critical region can be obtained from (7.56) by projecting the set
T
x,u
A,2
in (7.55).
Based on the above discussion and results, the main steps of the mp-QP solver
are outlined in the following algorithm.
Algorithm 7.2 mp-QP Algorithm
input: Matrices H, G, w, S of problem (7.37) and set / in (7.2)
output: Multiparametric solution to problem (7.37)
execute partition(/);
end.
procedure partition(Y )
let x0 Y and be the solution to the LP (7.68);
if 0 then exit; (no full-dimensional CR is in Y )
solve the QP (7.37) for x = x0 to obtain (z
0
, u
0
);
Determine the set of active constraints A when z = z
0
, x = x0, and build GA,
wA, SA;
if r = rank GA is equal to the number of rows of GA
Determine u
A
(x), z
(x) from (7.45) and (7.46);

Characterize the CR from (7.47) and (7.48);
elseif r = rank GA is less than the number of rows of GA
Determine z
(x) from (7.53);

Characterize the CR from (7.56);
end
Dene and partition the rest of the region as in Theorem 5.2;
for each new sub-region Ri, partition(Ri);
end procedure.
Remark 7.8 If rank(D) > 0 in Algorithm 7.2, the region CR
A(x
0
)
is not full-
dimensional. To avoid further recursion in the algorithm which does not produce
any full-dimensional critical region, one should compute a random vector R
n
smaller than the Chebychev radius of Y and such that the QP (7.37) is feasible for
x0 + and then repeat step where A(x0) is computed with x0 x0 +.
Remark 7.9 The algorithm solves the mp-QP problem by partitioning the given
parameter set / into Nr closed polyhedral regions. Note that the algorithm generates
a partition of the state space which is not strict. The algorithm could be modied
to store the critical regions as dened in Section 7.1.2. (which are neither closed nor
open as proven in Theorem 7.6) instead of storing their closure. This can be done by
keeping track of which facet belongs to a certain critical region and which not. From
a practical point of view, such a procedure is not necessary since the value function
and the optimizer are continuous functions of x.
Remark 7.10 The proposed algorithm does not apply to the case when H 0 and
when the optimizer may not be unique. In this case one may resort to regulariza-
tion, i.e. adding an appropriate small quadratic term, or follow the ideas of [229].
7.4 Multiparametric Mixed-Integer Linear Programming 139
7.4 Multiparametric Mixed-Integer Linear Programming
7.4.1 Formulation and Properties
Consider the mp-LP
J
(x) = min
z
J(z, x) = c
z
subj. to Gz w +Sx,
(7.70)
where z is the optimization vector, x R
s
is the vector of parameters, G
=
[G
1
. . . G
m
] and G
j
R
n
denotes the j-th row of G, c R
s
, w R
m
, and
S R
mn
. When we restrict some of the optimization variables to be 0 or 1,
z = z
c
, z
d
, z
c
R
sc
, z
d
0, 1
s
d
and s = s
c
+ s
d
, we refer to (7.70) as a
(right-hand side) multiparametric mixed-integer linear program (mp-MILP).
7.4.2 Geometric Algorithm for mp-MILP
Consider the mp-MILP (7.70). Given a closed and bounded polyhedral set / R
n
of parameters,
/ = x R
n
: Tx N, (7.71)
we denote by /
/ the region of parameters x / such that the MILP (7.70)

is feasible and the optimum J
(x) is nite. For any given x /
, J
( x) denotes
the minimum value of the objective function in problem (7.70) for x = x. The
function J
: /
R will denote the function which expresses the dependence on

x of the minimum value of the objective function over /
, J
will be called the

value function. The set-valued function Z
: /
2
R
sc
2
{0,1}
s
d
will describe for
any xed x /
the set of optimizers z
(x) related to J
(x).
We aim to determine the region /
/ of feasible parameters x and to nd the

expression of the value function J
(x) and the expression of an optimizer function

z
(x) Z
(x).
Two main approaches have been proposed for solving mp-MILP problems. In
[2], the authors develop an algorithm based on branch and bound (B&B) methods.
At each node of the B&B tree an mp-LP is solved. The solution at the root
node where all the binary variables are relaxed to the interval [0,1] represents a
valid lower bound, while the solution at a node where all the integer variables
have been xed represents a valid upper bound. As in standard B&B methods,
the complete enumeration of combinations of 0-1 integer variables is avoided by
comparing the multiparametric solutions, and by fathoming the nodes where there
is no improvement of the value function.
In [98] an alternative algorithm was proposed, which will be detailed in this
section. Problem (7.70) is alternatively decomposed into an mp-LP and an MILP
subproblem. In one step the values of the binary variable are xed for a region
and an mp-LP is solved. Its solution provides a parametric upper bound to the
value function J
(x) in the region. In the other step, the parameters x are treated
as additional free variables and an MILP is solved. In this way a parameter x is
found and an associated new integer vector which improves the value function at
this point.
The algorithm is composed of an initialization step, and a recursion between
the solution of an mp-LP subproblem and an MILP subproblem.
Initialization
Solve the following MILP problem
min
{z,x}
c
z
subj. to Gz Sx w
x /
(7.72)
where x is treated as an independent variable. If the MILP (7.72) is infeasible then
the mp-MILP (7.70) admits no solution, i.e. /
= ; otherwise its solution z
, x
provides an integer variable z
that is feasible at point x
.
At step j = 0, set: N
0
= 1, CR
1
= /, Z
1
= ,

J
1
= +, N
bi
= 0, z
1
d1
= z
d
.
mp-LP subproblem
For each CR
i
we solve the following mp-LP problem
J
i
(x) = min
z
c
z
subj. to Gz w +Sx
z
d
= z
Nbi+1
di
x CR
i
(7.73)
By Theorem 7.5, the solution of mp-LP (7.73) provides a partition of CR
i
into
polyhedral regions R
k
i
, k = 1, . . . , N
Ri
and a PWA value function
J
i
(x) = (

JR
k
i
(x) = c
k
i
x +p
k
i
) if x R
k
i
, k = 1, . . . , N
Ri
(7.74)
where

JR
j
i
(x) = + in R
j
i
if the integer variable z
di
is not feasible in R
j
i
and a
PWA continuous control law z
(x) (z
(x) is not dened in R

j
i
if

JR
j
i
(x) = +).
The function

J
i
(x) will be an upper bound of J
(x) for all x CR

i
. Such
a bound

J
i
(x) on the value function has to be compared with the current bound
J
i
(x) in CR
i
in order to obtain the lowest of the two parametric value functions
and to update the bound.
While updating

J
i
(x) three cases are possible:
1.

J
i
(x) =

JR
k
i
(x) x R
k
i
if (

JR
k
i
(x)

J
i
(x) x R
k
i
)
2.

J
i
(x) =

J
i
(x) x R
k
i
(if

JR
k
i
(x)

J
i
(x) x R
k
i
)
3.

J
i
(x) =
_

J
i
(x) x (R
k
i
)
1
= x R
k
i
:

JR
k
i
(x)

J
i
(x)
JR
k
i
(x) x (R
k
i
)
2
= x R
k
i
:

JR
k
i
(x)

J
i
(x)
The three cases above can be distinguished by using a simple linear program. We
add the constraint

JR
k
i
(x)

J
i
(x) to the constraints dening R
k
i
and tests its
redundancy (Section 5.4.1). If it is redundant we have case 1, if it is infeasible
7.4 Multiparametric Mixed-Integer Linear Programming 141
we have case 2. Otherwise

JR
k
i
(x) =

J
i
(x) denes the facet separating (R
k
i
)
1
and
(R
k
i
)
2
. In the third case, the region R
k
i
is partitioned into two regions (R
k
i
)
1
and
(R
k
i
)
2
which are convex polyhedra since

JR
k
i
(x) and

J
i
(x) are ane functions of
x.
After the mp-LP (7.73) has been solved for all i = 1, . . . , N
j
(the subindex j
denotes the that we are at step j of the recursion) and the value function has been
updated, each initial region CR
i
has been subdivided into at most 2N
Ri
polyhedral
regions R
k
i
and possibly (R
k
i
)
1
and (R
k
i
)
2
with a corresponding updated parametric
bound on the value function

J
i
(x). For each R
k
i
, (R
k
i
)
1
and (R
k
i
)
2
we dene the
set of integer variables already explored as Z
i
= Z
i
z
Nbi+1
di
, Nb
i
= Nb
i
+ 1. In
the sequel the polyhedra of the new partition will be referred to as CR
i
.
MILP subproblem
At step j for each critical region CR
i
(note that these CR
i
are the output of the
previous phase) we solve the following MILP problem.
min
{z,x}
c
z (7.75)
subj. to Gz Sx w (7.76)
c
z

J
i
(z) (7.77)
z
d
,= z
k
di
, k = 1, . . . , Nb
i
(7.78)
x CR
i
(7.79)
where constraints (7.78) prohibit integer solutions that have been already analyzed
in CR
i
from appearing again and constraint (7.77) excludes integer solutions with
higher values than the current upper bound. If problem (7.79) is infeasible then
the region CR
i
is excluded from further recursion and the current upper bound
represents the nal solution. If problem (7.79) is feasible, then the discrete optimal
component z
di
is stored and represents a feasible integer variable that is optimal
at least in one point of CR
i
.
Recursion
At step j we have stored
1. A list of N
j
polyhedral regions CR
i
and for each of them an associated
parametric ane upper bound

J
i
(x) (

J
i
(x) = + if no integer solution has
been found yet in CR
i
).
2. For each CR
i
a set of integer variables Z
i
= z
0
di
, . . . , z
Nbi
di
, that have already
been explored in the region CR
i
.
3. For each CR
i
an integer feasible variable z
Nbi+1
di
/ Z
i
such that there exists z
c
and x CR
i
for which Gz w+S x and c
z <

J
i
( x) where z = z
c
, z
Nbi+1
di
.
That is, z
Nbi+1
di
is an integer variable that improves the current bound for at
least one point of the current polyhedron.
For all the regions CR
i
not excluded from the MILPs subproblem (7.75)-(7.79)
the algorithm continues to iterate between the mp-LP (7.73) with z
Nbi+1
di
= z
di
and the MILP (7.75)-(7.79). The algorithm terminates when all the MILPs (7.75)-
(7.79) are infeasible.
Note that the algorithm generates a partition of the state space. Some param-
eter x could belong to the boundary of several regions. Dierently from the LP
and QP case, the value function may be discontinuous and therefore such a case
has to be treated carefully. If a point x belongs to dierent critical regions, the
expressions of the value function associated with such regions have to be compared
in order to assign to x the right optimizer. Such a procedure can be avoided by
keeping track of which facet belongs to a certain critical region and which not.
Moreover, if the value functions associated with the regions containing the same
parameter x coincide this may imply the presence of multiple optimizers.
7.4.3 Solution Properties
The following properties of J
(x) and Z
(x) easily follow from the algorithm de-

scribed above.
Theorem 7.10 Consider the mp-MILP (7.70). The set /
is the union of a nite

number of (possibly open) polyhedra and the value function J
is piecewise ane
on polyhedra. If the optimizer z
(x) is unique for all x /
, then the optimizer

functions z
c
: /
R
sc
and z
d
: /
0, 1
s
d
are piecewise ane and piecewise
constant, respectively, on polyhedra. Otherwise, it is always possible to dene a
piecewise ane optimizer function z
(x) Z
(x) for all x /
.
Note that, dierently from the mp-LP case, the set /
can be non-convex and

even disconnected.
7.5 Multiparametric Mixed-Integer Quadratic Programming 143
7.5 Multiparametric Mixed-Integer Quadratic Programming
7.5.1 Formulation and Properties
Consider the mp-QP
J
(x) = min
z
J(z, x) = z
H
1
z +c
1
z
subj. to Gz w +Sx,
(7.80)
When we restrict some of the optimization variables to be 0 or 1, z = z
c
, z
d
,
where z
c
R
sc
, z
d
0, 1
s
d
, we refer to (7.80) as a multiparametric mixed-integer
quadratic program (mp-MIQP). Given a closed and bounded polyhedral set / R
n
of parameters,
/ = x R
n
: Tx N, (7.81)
we denote by /
/ the region of parameters x / such that the MIQP (7.80) is

feasible and the optimum J
(x) is nite. For any given x /
, J
( x) denotes the
minimum value of the objective function in problem (7.80) for x = x. The value
function J
: /
R denotes the function which expresses the dependence on x

of the minimum value of the objective function over /
. The set-valued function

Z
: /
2
R
sc
2
{0,1}
s
d
describes for any xed x /
the set of optimizers z
(x)
related to J
(x).
We aim at determining the region /
/ of feasible parameters x and at

nding the expression of the value function J
(x) and the expression of an optimizer

function z
(x) Z
(x).
We show with a simple example that the geometric approach discussed in this
chapter cannot be used for solving mp-MIQPs. Suppose z
1
, z
2
, x
1
, x
2
R and
0, 1, then the following mp-MIQP
J
(x
1
, x
2
) = min
z1,z2,
z
2
1
+z
2
2
25 + 100
subj. to
_
_
1 0 10
1 0 10
0 1 10
0 1 10
1 0 10
1 0 10
0 1 10
0 1 10
0 0 0
0 0 0
0 0 0
0 0 0
_
_
_
_
z
1
z
2
_
_
_
1 0
1 0
0 1
0 1
0 0
0 0
0 0
0 0
1 0
1 0
0 1
0 1
_
_
_
x
1
x
2
_
+
_
_
10
10
10
10
0
0
0
0
10
10
10
10
_
_
(7.82)
can be simply solved by noting that for = 1 z
1
= x
1
and z
2
= x
2
while for = 0
z
1
= z
2
= 0. By comparing the value functions associated with = 0 and = 1
we obtain two critical regions
CR
1
= x
1
, x
2
R : x
2
1
+x
2
2
25
CR
2
= x
1
, x
2
R : 10 x
1
10, 10 x
2
10, x
2
1
+x
2
2
> 25
(7.83)
z
1
(x
1
, x
2
) =
_
x
1
if [x
1
, x
2
] CR
1
0 if [x
1
, x
2
] CR
2
z
2
(x
1
, x
2
) =
_
x
2
if [x
1
, x
2
] CR
1
0 if [x
1
, x
2
] CR
2
(7.84)
and the parametric value function
J
(x
1
, x
2
) =
_
x
2
1
+x
2
2
+ 75 if [x
1
, x
2
] CR
1
100 if [x
1
, x
2
] CR
2
(7.85)
The two critical regions and the value function are depicted in Figure 7.11.
CR1
CR2
x
1
x
2
(a) Critical Regions.
x
1
x
2
J
(
x
)
(b) Value function.
Figure 7.11 Solution to the mp-MIQP (7.82).
This example demonstrates that, in general, the critical regions of an mp-MIQP
cannot be decomposed into convex polyhedra. Therefore the method of partition-
ing the rest of the space presented in Theorem 5.2 cannot be applied here. In
Chapter 18 we will present an algorithm that eciently solves specic mp-MIQPs
that stem from the optimal control of discrete-time hybrid systems.
7.6 Literature Review
Many of the theoretical results on parametric programming can be found in[18,
104, 46, 112].
The rst method for solving parametric linear programs was proposed by Gass
and Saaty [116], and since then extensive research has been devoted to sensitivity
and (multi)-parametric linear analysis, as attested by the hundreds of references
in [111] (see also [112] for recent advances in the eld). One of the rst methods
for solving multiparametric linear programs (mp-LPs) was formulated by Gal and
Nedoma [113]. The method constructs the critical regions iteratively, by visiting
the graph of bases associated with the LP tableau of the original problem. Many
of the results on mp-LP presented in this book can be found in [111, p. 178-180].
7.6 Literature Review 145
Note that in [113, 111] a critical region is dened as a subset of the param-
eter space on which a certain basis of the linear program is optimal. The al-
gorithm proposed in [113] for solving multiparametric linear programs generates
non-overlapping critical regions by generating and exploring the graph of bases. In
the graph of bases the nodes represent optimal bases of the given multiparametric
problem and two nodes are connected by an edge if it is possible to pass from one
basis to another by one pivot step (in this case the bases are called neighbors).
In this book we use the denition (7.4) of critical regions which is not associated
with the bases but with the set of active constraints and it is directly related to
the denition given in [3, 204, 112].
The solution to multiparametric quadratic programs has been studied in detail
in [18, Chapter 5]. In [41] Bemporad and coauthors presented a simple method
for solving mp-QPs. The method constructs a critical region in a neighborhood
of a given parameter, by using the Karush-Kuhn-Tucker conditions for optimality,
and then recursively explores the parameter space outside such a region. Other
algorithms for solving mp-QPs have been proposed by Seron, DeDon a and Good-
win in [261, 96] in parallel with the study of Bemporad and coauthors in [41], by
Tondel, Johansen and Bemporad in [273] and by Baotic in [19]. All these algo-
rithms are based on an iterative procedure that builds up the parametric solution
by generating new polyhedral regions of the parameter space at each step. The
methods dier in the way they explore the parameter space, that is, the way they
identify active constraints corresponding to the critical regions neighboring to a
given critical region, i.e., in the active set generator component.
In [261, 96] the authors construct the unconstrained critical region and then
generate neighboring critical regions by enumerating all possible combinations of
active constraints.
In [273] the authors explore the parameter space outside a given region CR
i
by examining its set of active constraints A
i
. The critical regions neighboring to
CR
i
are constructed by elementary operations on the active constraints set A
i
that can be seen as an equivalent pivot for the quadratic program. For this
reason the method can be considered as an extension of the method of Gal [111]
to multiparametric quadratic programming.
In [19] the author uses a direct exploration of the parameter space as in [41]
but he avoids the partition of the state space described in Theorem 5.2. Given a
polyhedral critical region CR
i
, the procedure goes through all its facets and gen-
erates the Chebychev center of each facet. For each facet T
i
a new parameter x
i
is generated, by moving from the center of the facet in the direction of the normal
to the facet by a small step. If such parameter x
i
is infeasible or is contained in
a critical region already stored, then the exploration in the direction of T
i
stops.
Otherwise, the set of active constraints corresponding to the critical region sharing
the facet T
i
with the region CR
i
is found by solving a QP for the new parameter
x
i
.
In [2, 98] two approaches were proposed for solving mp-MILP problems. In both
methods the authors use an mp-LP algorithm and a branch and bound strategy
that avoids the complete enumeration of combinations of 0-1 integer variables by
comparing the available bounds on the multiparametric solutions.
Part III
Optimal Control
Chapter 8
General Formulation and
Discussion
In this chapter we introduce the optimal control problem we will be studying in
a very general form. We want to communicate the basic denitions and essential
concepts. We will sacrice mathematical precision for the sake of simplicity. In later
chapters we will study specic versions of this problem for specic cost functions
and system classes in greater detail.
8.1 Problem Formulation
We consider the nonlinear time-invariant system
x(t + 1) = g(x(t), u(t)), (8.1)
subject to the constraints
h(x(t), u(t)) 0 (8.2)
at all time instants t 0. In (8.1)(8.2), x(t) R
n
and u(t) R
m
are the state and
input vector, respectively. Inequality (8.2) with h : R
n
R
m
R
nc
expresses the
n
c
constraints imposed on the input and the states. These may be simple upper
and lower bounds or more complicated expressions. We assume that the origin is
an equilibrium point (g(0, 0) = 0) in the interior of the feasible set, i.e., h(0, 0) < 0.
We assumed the system to be specied in discrete time. One reason is that we
are looking for solutions to engineering problems. In practice, the controller will
almost always be implemented through a digital computer by sampling the variables
of the system and transmitting the control action to the system at discrete time
points. Another reason is that for the solution of the optimal control problems for
discrete-time systems we will be able to make ready use of powerful mathematical
programming software.
We want to caution the reader, however, that in many instances the discrete
time model is an approximation of the continuous time model. It is generally
150 8 General Formulation and Discussion
dicult to derive good discrete time models from nonlinear continuous time
models, and especially so when the nonlinear system has discontinuities as would
be the case for switched systems. We also note that continuous time switched
systems can exhibit behavioral characteristics not found in discrete-time systems,
for example, an ever increasing number of switches in an ever decreasing time
interval (Zeno behavior [124]).
We dene the following performance objective or cost function from time instant
0 to time instant N
J
0N
(x
0
, U
0N
) = p(x
N
) +
N1
k=0
q(x
k
, u
k
), (8.3)
where N is the time horizon and x
k
denotes the state vector at time k obtained by
starting from the measured state x
0
= x(0) and applying to the system model
x
k+1
= g(x
k
, u
k
), (8.4)
the input sequence u
0
, . . . , u
k1
. From this sequence we dene the vector of future
inputs U
0N
= [u
0
, . . . , u
N1
]
R
s
, s = mN. The terms q(x
k
, u
k
) and p(x
N
)
are referred to as stage cost and terminal cost, respectively, and are assumed to be
positive denite (q 0, p 0):
p(x, u) > 0 x ,= 0, u ,= 0, p(0, 0) = 0
q(x, u) > 0 x ,= 0, u ,= 0, q(0, 0) = 0.
The form of the cost function (8.3) is very general. If a practical control objective
can be expressed as a scalar function then this function usually takes the indicated
form. Specically, we consider the following constrained nite time optimal control
(CFTOC) problem.
J
0N
(x
0
) = min
U0N
J
0N
(x
0
, U
0N
)
subj. to x
k+1
= g(x
k
, u
k
), k = 0, . . . , N 1
h(x
k
, u
k
) 0, k = 0, . . . , N 1
x
N
A
f
x
0
= x(0).
(8.5)
Here A
f
R
n
is a terminal region that we want the system states to reach at the
end of the horizon. The terminal region could be the origin, for example. We dene
A
0N
R
n
to be the set of initial conditions x(0) for which there exists an input
vector U
0N
so that the inputs u
0
, . . . , u
N1
and the states x
0
, . . . , x
N
satisfy the
model x
k+1
= g(x
k
, u
k
) and the constraints h(x
k
, u
k
) 0 and that the state x
N
lies in the terminal set A
f
.
We can determine this set of feasible initial conditions in a recursive manner.
Let us denote with A
jN
the set of states x
j
at time j which can be steered into
A
f
at time N, i.e., for which the model x
k+1
= g(x
k
, u
k
) and the constraints
h(x
k
, u
k
) 0 are feasible for k = j, . . . , N 1 and x
N
A
f
. This set can be
dened recursively by
A
jN
= x R
n
: u such that (h(x, u) 0, and g(x, u) A
j+1N
),
j = 0, . . . , N 1 (8.6)
A
NN
= A
f
. (8.7)
8.2 Solution via Batch Approach 151
The set A
0N
is the nal result of these iterations starting with A
f
.
The optimal cost J
0N
(x
0
) is also called value function. In general, the problem
(8.3)(8.5) may not have a minimum, but only an inmum. We will assume that
there exists a minimum. This is the case, for example, when the set of feasible
input vectors U
0N
(dened by h and A
f
) is compact and when the functions g, p
and q are continuous. Also, there might be several input vectors U
0N
which yield
the minimum (J
0N
(x
0
) = J
0N
(x
0
, U
0N
)). In this case we will dene one of
them as the minimizer U
0N
.
Note that throughout the book we will distinguish between the current state
x(k) of system (8.1) at time k and the variable x
k
in the optimization problem (8.5),
that is the predicted state of system (8.1) at time k obtained by starting from the
state x
0
and applying to system (8.4) the input sequence u
0
, . . . , u
k1
. Analogously,
u(k) is the input applied to system (8.1) at time k while u
k
is the k-th optimization
variable of the optimization problem (8.5). Clearly, x(k) = x
k
for any k if u(k) = u
k
for all k (under the assumption that our model is perfect).
In the rest of this chapter we will be interested in the following questions related
to the general optimal control problem (8.3)(8.5).
Solution. We will show that the problem can be expressed and solved either
as one general nonlinear programming problem, or in a recursive manner by
invoking Bellmans Principle of Optimality.
Innite horizon. We will investigate if a solution exists as N , the prop-
erties of this solution, and how it can be obtained or at least approximated
by using a receding horizon technique.
8.2 Solution via Batch Approach
If we write the equality constraints appearing in (8.5) explicitly
x
1
= g(x(0), u
0
)
x
2
= g(x
1
, u
1
)
.
.
.
x
N
= g(x
N1
, u
N1
),
(8.8)
then the optimal control problem (8.3)(8.5), rewritten below
J
0N
(x
0
) = min
U0N
p(x
N
) +
N1
k=0
q(x
k
, u
k
)
subj. to x
1
= g(x
0
, u
0
)
x
2
= g(x
1
, u
1
)
.
.
.
x
N
= g(x
N1
, u
N1
)
h(x
k
, u
k
) 0, k = 0, . . . , N 1
x
N
A
f
x
0
= x(0)
(8.9)
is recognized more easily as a general nonlinear programming problem with vari-
ables u
0
, . . . , u
N1
and x
1
, . . . , x
N
.
As an alternative we may try to eliminate the state variables and equality
constraints (8.8) by successive substitution so that we are left with u
0
, . . . , u
N1
as the only decision variables. For example, we can express x
2
as a function of
x(0), u
0
and u
1
only, by eliminating the intermediate state x
1
x
2
= g(x
1
, u
1
)
x
2
= g(g(x(0), u
0
), u
1
).
(8.10)
Except when the state equations are linear this successive substitution may become
complex. Even when they are linear it may be bad from a numerical point of view.
Either with or without successive substitution the solution of the nonlinear pro-
gramming problem is a sequence of present and future inputs U
0N
= [u
0
, . . . , u
N1
]
determined for the particular initial state x(0).

8.3 Solution via Recursive Approach
The recursive approach, Bellmans dynamic programming technique, rests on a sim-
ple idea, the principle of optimality. It states that for a trajectory x
0
, x
1
, . . . , x
N
to
be optimal, the trajectory starting from any intermediate point x
j
, i.e. x
j
, x
j+1
, . . . , x
N
,
0 j N 1, must be optimal.
Consider the following example to provide an intuitive justication [52]. Sup-
pose that the fastest route from Los Angeles to Boston passes through Chicago.
Then the principle of optimality formalizes the obvious fact that the Chicago to
Boston portion of the route is also the fastest route for a trip that starts from
Chicago and ends in Boston.
We can utilize the principle of optimality for the optimal control problem we
are investigating. We dene the cost over the reduced horizon from j to N
J
jN
(x
j
, u
j
, u
j+1
, . . . , u
N1
) = p(x
N
) +
N1
k=j
q(x
k
, u
k
), (8.11)
also called the cost-to-go. Then the optimal cost-to-go J
jN
is
J
jN
(x
j
) = min
uj ,uj+1,...,uN1
J
jN
(x
j
, u
j
, u
j+1
, . . . , u
N1
)
subj. to x
k+1
= g(x
k
, u
k
), k = j, . . . , N 1
h(x
k
, u
k
) 0, k = j, . . . , N 1
x
N
A
f
.
(8.12)
Note that the optimal cost-to-go J
jN
(x
j
) depends only on the initial state x
j
.
The principle of optimality implies that the optimal cost-to-go J
j1N
from
time j 1 to the nal time N can be found by minimizing the sum of the stage
8.3 Solution via Recursive Approach 153
cost q(x
j1
, u
j1
) and the optimal cost-to-go J
jN
(x
j
) from time j onwards:
J
j1N
(x
j1
) = min
uj1
q(x
j1
, u
j1
) + J
jN
(x
j
)
subj. to x
j
= g(x
j1
, u
j1
)
h(x
j1
, u
j1
) 0
x
j
A
jN
.
(8.13)
Here the only decision variable left for the optimization is u
j1
, the input at time
j 1. All the other inputs u
j
, . . . , u
N1
have already been selected optimally to
yield the optimal cost-to-go J
jN
(x
j
). We can rewrite (8.13) as
J
j1N
(x
j1
) = min
uj1
q(x
j1
, u
j1
) + J
jN
(g(x
j1
, u
j1
))
subj. to h(x
j1
, u
j1
) 0
g(x
j1
, u
j1
) A
jN
,
(8.14)
making the dependence of x
j
on the initial state x
j1
explicit.
The optimization problem (8.14) suggests the following recursive algorithm
backwards in time to determine the optimal control law. We start with the terminal
cost and constraint
J
NN
(x
N
) = p(x
N
) (8.15)
A
NN
= A
f
, (8.16)
and then proceed backwards
J
N1N
(x
N1
) = min
uN1
q(x
N1
, u
N1
) + J
NN
(g(x
N1
, u
N1
))
subj. to h(x
N1
, u
N1
) 0,
g(x
N1
, u
N1
) A
NN
.
.
.
J
0N
(x
0
) = min
u0
q(x
0
, u
0
) + J
1N
(g(x
0
, u
0
))
subj. to h(x
0
, u
0
) 0,
g(x
0
, u
0
) A
1N
x
0
= x(0).
(8.17)
This algorithm, popularized by Bellman, is referred to as dynamic programming.
The dynamic programming problem is appealing because it can be stated compactly
and because at each step the optimization takes place over one element u
j
of the
optimization vector only. This optimization is rather complex, however. It is not a
standard nonlinear programming problem, since we have to construct the optimal
cost-to-go J
jN
(x
j
), a function dened over the subset A
jN
of the state space.
In a few special cases we know the type of function and we can nd it eciently.
For example, in the next chapter we will cover the case when the system is linear
and the cost is quadratic. Then the optimal cost-to-go is also quadratic and can be
constructed rather easily. Later in the book we will show that, when constraints
are added to this problem, the optimal cost-to-go becomes piecewise quadratic and
ecient algorithms for its construction are also available.
In general, however, we may have to resort to a brute force approach to
construct the cost-to-go function J
j1N
and to solve the dynamic program. Let us
assume that at time j1 the cost-to-go J
jN
is known and discuss how to construct
an approximation of J
j1N
. With J
jN
known, for a xed x
j1
the optimization
problem (8.14) becomes a standard nonlinear programming problem. Thus, we
can dene a grid in the set A
j1N
of the state space and compute the optimal
cost-to-go function on each grid point. We can then dene an approximate value
function

J
j1N
(x
j1
) at intermediate points via interpolation. The complexity
of constructing the cost-to-go function in this manner increases rapidly with the
dimension of the state space (curse of dimensionality).
The extra benet of solving the optimal control problem via dynamic pro-
gramming is that we do not only obtain the vector of optimal inputs U
0N
for a
particular initial state x(0) as with the batch approach. At each time j the optimal
cost-to-go function denes implicitly a nonlinear feedback control law.
u
j
(x
j
) = arg min
uj
q(x
j
, u
j
) + J
j+1N
(g(x
j
, u
j
))
subj. to h(x
j
, u
j
) 0,
g(x
j
, u
j
) A
j+1N
.
(8.18)
For a xed x
j
this nonlinear programming problem can be solved quite easily in
order to nd u
j
(x
j
). Because the optimal cost-to-go function J
jN
(x
j
) changes
with time j, the nonlinear feedback control law is time-varying.
8.4 Optimal Control Problem with Innite Horizon
We are interested in the optimal control problem (8.3)(8.5) as the horizon N
approaches innity.
J
0
(x
0
) = min
u0,u1,...
k=0
q(x
k
, u
k
)
subj. to x
k+1
= g(x
k
, u
k
), k = 0, . . . ,
h(x
k
, u
k
) 0, k = 0, . . . ,
x
0
= x(0).
(8.19)
We dene the set of initial conditions for which this problem has a solution.
A
0
= x(0) R
n
: Problem (8.19) is feasible and J
0
(x(0)) < +.
(8.20)
For the value function J
0
(x
0
) to be nite it must hold that
lim
k
q(x
k
, u
k
) = 0,
and because q(x
k
, u
k
) > 0 for all (x
k
, u
k
) ,= 0
lim
k
x
k
= 0
8.4 Optimal Control Problem with Innite Horizon 155
and
lim
k
u
k
= 0.
Thus the sequence of control actions generated by the solution of the innite horizon
problem drives the system to the origin. For this solution to exists the system must
be, loosely speaking, stabilizable.
Using the recursive dynamic programming approach we can seek the solution
of the innite horizon optimal control problem by increasing N until we observe
convergence. If the dynamic programming algorithm converges as N then
(8.14) becomes the Bellman equation
J
(x) = min
u
q(x, u) +J
(g(x, u))
subj. to h(x, u) 0
g(x, u) A
0
(8.21)
This procedure of simply increasing N may not be well behaved numerically
and it may also be dicult to dene a convergence criterion that is meaningful for
the control problem. We will describe a method, called Value Function Iteration,
in the next section.
An alternative is receding horizon control which can yield a time invariant con-
troller guaranteeing convergence to the origin without requiring N . We will
describe this important idea later in this chapter.
8.4.1 Value Function Iteration
Once the value function J
(x) is known, the nonlinear feedback control law u
(x)
is dened implicitly by (8.21)
u
(x) = arg min

u
q(x, u) +J
(g(x, u))
subj. to h(x, u) 0
g(x, u) A
0
(8.22)
It is time invariant and guarantees convergence to the origin for all states in A
0
.
For a given x A
0
, u
(x) can be found from (8.21) by solving a standard

nonlinear programming problem.
In order to nd the value function J
(x) we do not need to compute the sequence

of cost-to-go functions implied by the dynamic program (8.14) but we can solve
(8.21) directly. We can start with some initial guess

J
0
(x) for the value function
and an initial guess

A
0
for the region in the state space where we expect the innite
horizon problem to converge and iterate
i+1
(x) = min
u
q(x, u) +

J
i
(g(x, u))
subj. to h(x, u) 0
g(x, u)

A
i
(8.23)
A
i+1
= x R
n
: u (h(x, u) 0, and g(x, u)

A
i
) (8.24)
Note that here i is the iteration index and does not denote time. This iterative
procedure is called value function iteration. It can be executed as follows. Let us
assume that at iteration step i we gridded the set

A
i
and that

J
i
(x) is known at each
grind point from the previous iteration. We can approximate

J
i
(x) at intermediate
points via interpolation. For a xed point x the optimization problem (8.23) is a
nonlinear programming problem yielding

J
i
( x). In this manner the approximate
value function

J
i
(x) can be constructed at all grid points and we can proceed to
the next iteration step i + 1.
8.4.2 Receding Horizon Control
Receding Horizon Control will be covered in detail in Chapter 13. Here we illustrate
the main idea and discuss the fundamental properties.
Assume that at time t = 0 we determine the control action u
0
by solving
the nite horizon optimal control problem (8.3)-(8.5). If J
0N
(x
0
) converges to
J
0
(x
0
) as N then the eect of increasing N on the value of u
0
should
diminish as N . Thus, intuitively, instead of making the horizon innite we
can get a similar behavior when we use a long, but nite horizon N, and repeat
this optimization at each time step, in eect moving the horizon forward (moving
horizon or receding horizon control).
In the batch approach we would solve an optimal control problem with horizon
N yielding a sequence of optimal inputs u
0
, . . . , u
N1
, but we would implement
only the rst one of these inputs u
0
. At the next time step we would measure the
current state and then again solve the N-step problem with the current state as
new initial condition x
0
. If the horizon N is long enough then we expect that this
approximation of the innite horizon problem should not matter and the imple-
mented sequence should drive the states to the origin.
In the dynamic programming approach we would always implement the control
u
0
obtained from the optimization problem
J
0N
(x
0
) = min
u0
q(x
0
, u
0
) + J
1N
(g(x
0
, u
0
))
subj. to h(x
0
, u
0
) 0,
g(x
0
, u
0
) A
1N
,
x
0
= x(0)
(8.25)
where J
1N
(g(x
0
, u
0
)) is the optimal cost-to-go from the state x
1
= g(x
0
, u
0
) at
time 1 to the end of the horizon N.
If the dynamic programming iterations converge as N , then for a long,
but nite horizon N we expect that this receding horizon approximation of the
innite horizon problem should not matter and the resulting controller will drive
the system asymptotically to the origin.
In both the batch and the recursive approach, however, it is not obvious how
long N must be for the receding horizon controller to inherit these desirable con-
vergence characteristics. Indeed, for computational simplicity we would like to
keep N small. We will argue next that the proposed control scheme guarantees
convergence just like the innite horizon variety if we impose a specic terminal
constraint, for example, if we require the terminal region to be the origin A
f
= 0.
8.4 Optimal Control Problem with Innite Horizon 157
From the principle of optimality we know that
J
0N
(x
0
) = min
u0
q(x
0
, u
0
) +J
1N
(x
1
). (8.26)
Assume that we are at x(0) at time 0 and implement the optimal u
0
that takes us
to the next state x
1
= g(x(0), u
0
). At this state at time 1 we postulate to use over
the next N steps the sequence of optimal moves determined at the previous step
followed by zero: u
1
, . . . , u
N1
, 0. This sequence is not optimal but the associated
cost over the shifted horizon from 1 to N +1 can be easily determined. It consists
of three parts: 1) the optimal cost J
0N
(x
0
) from time 0 to N computed at time 0,
minus 2) the stage cost q(x
0
, u
0
) at time 0 plus 3) the cost at time N +1. But this
last cost is zero because we imposed the terminal constraint x
N
= 0 and assumed
u
N
= 0. Thus the cost over the shifted horizon for the assumed sequence of control
moves is
J
0N
(x
0
) q(x
0
, u
0
).
Because this postulated sequence of inputs is not optimal at time 1
J
1N+1
(x
1
) J
0N
(x
0
) q(x
0
, u
0
).
Because the system and the objective are time invariant J
1N+1
(x
1
) = J
0N
(x
1
)
so that
J
0N
(x
1
) J
0N
(x
0
) q(x
0
, u
0
).
As q 0 for all (x, u) ,= (0, 0), the sequence of optimal costs J
0N
(x
0
), J
0N
(x
1
), . . .
is strictly decreasing for all (x, u) ,= (0, 0). Because the cost J
0N
0 the sequence
J
0N
(x
0
), J
0N
(x
1
), . . . (and thus the sequence x
0
, x
1
,. . .) is converging. Thus we
have established the following important theorem.
Theorem 8.1 At time step j consider the cost function
J
jj+N
(x
j
, u
j
, u
j+1
, . . . , u
j+N1
) =
j+N
k=j
q(x
k
, u
k
), q 0 (8.27)
and the CFTOC problem
J
jj+N
(x
j
) = min
uj,uj+1,...,uj+N1
J
jj+N
(x
j
, u
j
, u
j+1
, . . . , u
j+N1
)
subj. to x
k+1
= g(x
k
, u
k
)
h(x
k
, u
k
) 0, k = j, . . . , j +N 1
x
N
= 0
(8.28)
Assume that only the optimal u
j
is implemented. At the next time step j + 1 the
CFTOC problem is solved again starting from the resulting state x
j+1
= g(x
j
, u
j
)
(Receding Horizon Control). Assume that the CFTOC problem has a solution for
every one of the sequence of states x
j
, x
j+1
, . . . resulting from the control policy.
Then the system will converge to the origin as j .
Thus we have established that a receding horizon controller with terminal con-
straint x
N
= 0 has the same desirable convergence characteristics as the innite
horizon controller. At rst sight the theorem appears very general and power-
ful. It is based on the implicit assumption, however, that at every time step the
CFTOC problem has a solution. Infeasibility would occur, for example, if the un-
derlying system is not stabilizable. It could also happen that the constraints on
the inputs which restrict the control action prevent the system from reaching the
terminal state in N steps. In Chapter 13 we will present special formulations of
problem (8.28) such that feasibility at the initial time guarantees feasibility for
all future times. Furthermore in addition to asymptotic convergence to the ori-
gin we will establish stability for the closed-loop system with the receding horizon
controller.
Remark 8.1 For the sake of simplicity in the rest of the book we will use the following
shorter notation
J
j
(xj) = J
jN
(xj), j = 0, . . . , N
J
(x0) = J
0
(x0)
.j = .jN, j = 0, . . . , N
. = .0
U0 = U0N
(8.29)
and use the original notation only if needed.
8.5 Lyapunov Stability
While asymptotic convergence lim
k
x
k
= 0 is a desirable property, it is gen-
erally not sucient in practice. We would also like a system to stay in a small
neighborhood of the origin when it is disturbed slightly. Formally this is expressed
as Lyapunov stability.
8.5.1 General Stability Conditions
Consider the autonomous system
x
k+1
= g(x
k
) (8.30)
with g(0) = 0.
Denition 8.1 (Lyapunov Stability) The equilibrium point x = 0 of system (8.30)
is
- stable (in the sense of Lyapunov) if, for each > 0, there is > 0 such that
|x
0
| < |x
k
| < , k 0 (8.31)
- unstable if not stable
8.5 Lyapunov Stability 159
- asymptotically stable in R
n
if it is stable and
lim
k
x
k
= 0, x
0
(8.32)
- globally asymptotically stable if it is asymptotically stable and = R
n
- exponentially stable if it is stable and there exist constants > 0 and
(0, 1) such that
|x
0
| < |x
k
| |x
0
|
k
, k 0 (8.33)
The - requirement for stability (8.31) takes a challenge-answer form. To
demonstrate that the origin is stable, for any value of that a challenger may
chose (however small), we must produce a value of such that a trajectory starting
in a neighborhood of the origin will never leave the neighborhood of the origin.
Remark 8.2 If in place of system (8.30), we consider the time-varying system x
k+1
=
g(x
k
, k), then in Denition 8.1 is a function of and k, i.e., = (, k) > 0. In this
case, we introduce the concept of uniform stability. The equilibrium point x = 0
is uniformly stable if, for each > 0, there is = () > 0 (independent from k) such
that
|x0| < |x
k
| < , k 0 (8.34)
The following example shows that Lyapunov stability and convergence are, in
general, dierent properties.
Example 8.1 Consider the following system with one state x R:
x
k+1
= x
k
(x
k
1 [x
k
1[). (8.35)
The state x = 0 is an equilibrium for the system. For any state x [1, 1] we have
(x 1) 0 and the system dynamics (8.35) become
x
k+1
= 2x
2
k
2x
k
. (8.36)
System (8.36) generates oscillating and diverging trajectories for any x0 (1, 1)\|0.
Any such trajectory will enter in nite time T the region with x 1. In this region
the system dynamics (8.35) become
x
k+1
= 0, k T. (8.37)
Therefore the origin is not Lyapunov stable, however the system converges to the
origin for all x0 (, +).
Usually to show Lyapunov stability of the origin for a particular system one
constructs a so called Lyapunov function, i.e., a function satisfying the conditions
of the following theorem.
Theorem 8.2 Consider the equilibrium point x = 0 of system (8.30). Let R
n
be a closed and bounded set containing the origin. Assume there exists a function
V : R
n
R continuous at the origin, nite for every x , and such that
V (0) = 0 and V (x) > 0, x 0 (8.38a)
V (x
k+1
) V (x
k
) (x
k
) x
k
0 (8.38b)
where : R
n
R is a continuous positive denite function. Then x = 0 is
asymptotically stable in .
Denition 8.2 A function V (x) satisfying conditions (8.38a)-(8.38b) is called a
Lyapunov Function.
The main idea of Theorem 8.2 can be explained as follows. We aim to nd a scalar
function V (x) that captures qualitative characteristics of the system response, and,
in particular, its stability. We can think of V as an energy function that is zero
at the origin and positive elsewhere (condition (8.38a)). Condition (8.38b) of The-
orem 8.2 requires that for any state x
k
, x
k
,= 0 the energy decreases as the
system evolves to x
k+1
.
A proof of Theorem 8.2 for continuous dynamical systems can be found in [181].
The continuity assumptions on the dynamical system g is not used in the proof
in [245][p. 609].
Theorem 8.2 states that if we nd an energy function which satises the two
conditions (8.38a)-(8.38b), then the system states starting from any initial state
x
0
will eventually settle to the origin.
Note that Theorem 8.2 is only sucient. If condition (8.38b) is not satised
for a particular choice of V nothing can be said about stability of the origin.
Condition (8.38b) of Theorem 8.2 can be relaxed to allow to be a continuous
positive semi-denite (psd) function:
V (x
k+1
) V (x
k
) (x
k
), x
k
,= 0, continuous and psd. (8.39)
Condition (8.39) along with condition (8.38a) are sucient to guarantee stability
of the origin as long as the set x
k
: V (g(x
k
)) V (x
k
) = 0 contains no trajectory
of the system x
k+1
= g(x
k
) except for x
k
= 0 for all k 0. This relaxation
of Theorem 8.2 is the so called Barbashin-Krasovski-LaSalle principle [180]. It
basically means that V (x
k
) may stay constant and non zero at one or more time
instants as long as it does not do so at an equilibrium point or periodic orbit of
the system.
A similar result as Theorem 8.2 can be derived for global asymptotic stability,
i.e., = R
n
.
Theorem 8.3 Consider the equilibrium point x = 0 of system (8.30). Assume
there exists a function V : R
n
R continuous at the origin, nite for every
x R
n
, and such that
|x| V (x) (8.40a)
V (0) = 0 and V (x) > 0, x ,= 0 (8.40b)
V (x
k+1
) V (x
k
) (x
k
) x
k
,= 0 (8.40c)
where : R
n
R is a continuous positive denite function. Then x = 0 is globally
asymptotically stable.
Denition 8.3 A function V (x) satisfying condition (8.40a) is said to be radially
unbounded.
Denition 8.4 A radially unbounded Lyapunov function is called a Global Lya-
punov Function.
Note that it was not enough just to restate Theorem 8.2 with = R
n
but we
also have to require V (x) to be radially unbounded to guarantee global asymptotic
stability. To motivate this condition consider the candidate Lyapunov function for
a system in R
2
[173]
V (x) =
x
2
1
1 +x
2
1
+x
2
2
, (8.41)
x
1
x
2
V
(
x
)
(a) Lyapunov function V (x).
x
1
x
2
-4 -2 0 2 4
-1
0
1
(b) Level curves of V (x).
Figure 8.1 Lyapunov function (8.41).
which is depicted in Fig. 8.1, where x
1
and x
2
denote the rst and second compo-
nents of the state vector x, respectively. V (x) in (8.41) is not radially unbounded
as for x
2
= 0
lim
x1
V (x) = 1.
For this Lyapunov function even if condition (8.40c) is satised, the state x may
escape to innity. Condition (8.40c) of Theorem 8.3 guarantees that the level sets
c
of V (x) (
c
= x R
n
: V (x) c) are closed.
The construction of suitable Lyapunov functions is a challenge except for linear
systems. First of all one can quite easily show that for linear systems Lyapunov
stability agrees with the notion of stability based on eigenvalue location.
Theorem 8.4 A linear system x
k+1
= Ax
k
is globally asymptotically stable in the
sense of Lyapunov if and only if all its eigenvalues are strictly inside the unit circle.
We also note that stability is always global for linear systems.
8.5.2 Quadratic Lyapunov Functions for Linear Systems
A simple eective Lyapunov function for linear systems is
V (x) = x
Px, P 0 (8.42)
which satises conditions (8.40a)-(8.40b) of Theorem 8.3. In order to test condi-
tion (8.40c) we compute
V (x
k+1
)V (x
k
) = x
k+1
Px
k+1
x
k
Px
k
= x
k
A
PAx
k
x
k
Px
k
= x
k
(A
PAP)x
k
(8.43)
Therefore condition (8.40c) is satised if P 0 can be found such that
A
PAP = Q, Q 0 (8.44)
Equation (8.44) is referred to as discrete-time Lyapunov equation. The following
Theorem shows that P satisfying (8.44) exists if and only if the linear system is
asymptotically stable.
Theorem 8.5 Consider the linear system x
k+1
= Ax
k
. Equation (8.44) has a
unique solution P 0 for any Q 0 if and only if A has all eigenvalues strictly
inside the unit circle.
Thus, a quadratic form x
Px is always a suitable Lyapunov function for linear

systems and an appropriate P can be found by solving (8.44) for a chosen Q 0 i
the systems eigenvalues lie inside the unit circle. For nonlinear systems, determin-
ing a suitable form for V (x) is generally dicult. The conditions of Theorem 8.5
can be relaxed as follows.
Theorem 8.6 Consider the linear system x
k+1
= Ax
k
. Equation (8.44) has a
unique solution P 0 for any Q = C
C _ 0 if and only if A has all eigenvalues

inside the unit circle and (C, A) is observable.
It may be surprising that in order to prove stability in Theorem 8.6 we do not
require that the Lyapunov function decreases at every time step, i.e. that Q is
allowed to be positive semidenite. To understand this, let us assume that for a
particular system state x, V does not decrease, i.e. x
Q x = (C x)
(C x) = 0. Then at
the next time steps we have the rate of decrease (CA x)
(CA x), (CA

2
x)
(CA
2
x), . . ..
If the system (C, A) is observable then for all x ,= 0
x
_
C (CA)
(CA
2
)
(CA
n1
)
,= 0 (8.45)
which implies that after at most (n 1) steps the rate of decrease will become
nonzero. This is a special case of the the Barbashin-Krasovski-LaSalle principle.
Note that for C square and nonsingular Theorem 8.6 reduces to Theorem 8.5.
From (8.44) it follows that for stable systems and for a chosen Q 0 one can
always nd

P 0 solving
A

PA

P +Q _ 0 (8.46)
This Lyapunov inequality shows that for a stable system we can always nd a

P
such that V (x) = x

Px decreases at a desired rate indicated by Q. We will need
this result later to prove stability of receding horizon control schemes.
8.5.3 1/ Norm Lyapunov Functions for Linear Systems
For p = 1, the function
V (x) = |Px|
p
with P R
ln
of full column rank satises the requirements (8.40a), (8.40b)
of a Lyapunov function. It can be shown that a matrix P can be found such
that condition (8.40c) is satised for the system x
k+1
= Ax
k
if and only if the
eigenvalues of A are inside the unit circle. The number of rows l necessary in P
depends on the system. The techniques to construct P are based on the following
theorem [174, 235].
Theorem 8.7 Let P R
ln
with rank(P) = n and p 1, . The function
V (x) = |Px|
p
(8.47)
is a Lyapunov function for the discrete-time system
x
k+1
= Ax
k
, k 0, (8.48)
if and only if there exists a matrix H R
ll
, such that
PA = HP, (8.49a)
|H|
p
< 1. (8.49b)
An eective method to nd both H and P was proposed by Christophersen in [87].
To prove the stability of receding horizon control, later in this book, we will
need to nd a

P such that
|
PAx|
Px|
+|Qx|
0, x R
n
. (8.50)
Note that the inequality (8.50) is equivalent to the Lyapunov inequality (8.46)
when the squared two-norm is replaced by the 1 or norm. Once we have
constructed a P and H to fulll the conditions of Theorem 8.7 we can easily nd
P to satisfy (8.50) according to the following lemma.

Lemma 8.1 Let P and H be matrices satisfying conditions (8.49), with P full
column rank. Let = 1 |H|
, = |QP
#
|
, where P
#
= (P
P)
1
P
is the
left pseudoinverse of P. Then, the square matrix
P =

P (8.51)
satises condition (8.50).
Proof: Since

P satises

PA = H

P, we obtain |
Px|
+|
PAx|
+|Qx|
=
|
Px|
+ |H

Px|
+ |Qx|
(|H|
1)|
Px|
+ |Qx|
(|H|

1)|
Px|
+|QP
#
|
|Px|
= 0. Therefore, (8.50) is satised.

Chapter 9
Linear Quadratic Optimal
Control
In this chapter we study the nite time and innite time optimal control problem
for unconstrained linear systems with linear and quadratic objective functions. We
derive the structure of the optimal control law by using two approaches: the Batch
approach and the Dynamic Programming approach. For problems with quadratic
objective functions we obtain the well known Algebraic Riccati Equations.
We consider a special case of the problem stated in the last chapter, where the
system is linear and time-invariant
x(t + 1) = Ax(t) +Bu(t). (9.1)
Again, x(t) R
n
and u(t) R
m
are the state and input vectors respectively.
We dene the following quadratic cost function over a nite horizon of N steps
J
0
(x
0
, U
0
) = x
N
Px
N
+
N1
k=0
x
k
Qx
k
+u
k
Ru
k
, (9.2)
where x
k
denotes the state vector at time k obtained by starting from the state
x
0
x
k+1
= Ax
k
+Bu
k
(9.3)
the input sequence U
0
= [u
0
, . . . , u
N1
]
. Consider the nite time optimal control

problem
J
0
(x(0)) = min
U0
J
0
(x(0), U
0
)
subj. to x
k+1
= Ax
k
+Bu
k
, k = 0, 1, . . . , N 1
x
0
= x(0).
(9.4)
166 9 Linear Quadratic Optimal Control
In (9.4) U
0
= [u
0
, . . . , u
N1
]
R
s
, s = mN is the decision vector containing
all future inputs. We will assume that the state penalty is positive semi-denite
Q = Q
_ 0, P = P
_ 0 and the input penalty is positive denite R = R
0.
As introduced in the previous chapter we will present two alternate approaches
to solve problem (9.4), the batch approach and the recursive approach using dy-
namic programming.
First we write the equality constraints (9.4) explicitly to express all future states
x
1
, x
2
, . . . as a function of the future inputs u
0
, u
1
, . . . and then we eliminate all
intermediate states by successive substitution to obtain
_
_
x(0)
x
1
.
.
.
.
.
.
x
N
_
_
. .
X
=
_
_
I
A
.
.
.
.
.
.
A
N
_
_
. .
S
x
x(0) +
_
_
0 . . . . . . 0
B 0 . . . 0
AB
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
A
N1
B . . . . . . B
_
_
. .
S
u
_
_
u
0
.
.
.
.
.
.
u
N1
_
_
. (9.5)
Here all future states are explicit functions of the present state x(0) and the future
inputs u
0
, u
1
, u
2
, . . . only. By dening the appropriate quantities we can rewrite
this expression compactly as
A = o
x
x(0) +o
u
.U
0
(9.6)
Using the same notation the objective function can be rewritten as
J(x(0), U
0
) = A

QA +U
0
RU
0
, (9.7)
where

Q = blockdiagQ, , Q, P,

Q _ 0, and

R = blockdiagR, , R,

R 0.
Substituting (9.6) into the objective function (9.7) yields
J
0
(x(0), U
0
) = (o
x
x(0) +o
u
U
0
)
Q(o
x
x(0) +o
u
U
0
) +U
0
RU
0
= U
0
(o
u
Qo
u
+

R)
. .
H
U
0
+ 2x
(0) (o
x
Qo
u
)
. .
F
U
0
+x
(0) (o
x
Qo
x
)
. .
Y
x(0)
= U
0
HU
0
+ 2x
(0)FU
0
+x
(0)Y x(0).
(9.8)
Because

R 0, also H 0. Thus J
0
(x(0), U
0
) is a positive denite quadratic
function of U
0
. Therefore, its minimum can be found by computing its gradient
and setting it to zero. This yields the optimal vector of future inputs
U
0
(x(0)) = H
1
F
x(0)
=
_
o
u
Qo
u
+

R
_
1
o
u
Qo
x
x(0).
(9.9)
With this choice of U
0
the optimal cost is
J
0
(x(0)) = x(0)
FH
1
F
x(0) +x(0)
Y x(0)
= x(0)
_
o
x
Qo
x
o
x
Qo
u
_
o
u
Qo
u
+

R
_
1
o
u
Qo
x
_
x(0).
(9.10)
Note that the optimal vector of future inputs U
0
(x(0)) is a linear function (9.9) of
the initial state x(0) and the optimal cost J
0
(x(0)) is a quadratic function (9.10)
of the initial state x(0).
Alternatively, we can use dynamic programming to solve the same problem in a
recursive manner. We dene the optimal cost J
j
(x
j
) for the N j step problem
starting from state x
j
by
J
j
(x
j
)

= min
uj, ,uN1
x
N
Px
N
+
N1
k=j
x
k
Qx
k
+u
k
Ru
k
.
According to the principle of optimality the optimal one step cost-to-go can be
obtained from
J
N1
(x
N1
) = min
uN1
x
N
P
N
x
N
+x
N1
Qx
N1
+u
N1
Ru
N1
(9.11)
x
N
= Ax
N1
+Bu
N1
P
N
= P.
(9.12)
Substituting (9.12) into the objective function (9.11),
J
N1
(x
N1
) = min
uN1
_
x
N1
(A
P
N
A +Q)x
N1
+2x
N1
A
P
N
Bu
N1
+u
N1
(B
P
N
B +R)u
N1
_
.
(9.13)
We note that the cost-to-go J
N1
(x
N1
) is a positive denite quadratic function
of the decision variable u
N1
. We nd the optimum by setting the gradient to zero
and obtain the optimal input
u
N1
= (B
P
N
B +R)
1
B
P
N
A
. .
FN1
x
N1
(9.14)
and the one-step optimal cost-to-go
J
N1
(x
N1
) = x
N1
P
N1
x
N1
, (9.15)
where we have dened
P
N1
= A
P
N
A +QA
P
N
B(B
P
N
B +R)
1
B
P
N
A. (9.16)
At the next stage, consider the two-step problem from time N 2 forward:
J
N2
(x
N2
) = min
uN2
x
N1
P
N1
x
N1
+x
N2
Qx
N2
+u
N2
Ru
N2
(9.17)
x
N1
= Ax
N2
+Bu
N2
. (9.18)
We recognize that (9.17), (9.18) has the same form as (9.11), (9.12). Therefore we
can state the optimal solution directly.
u
N2
= (B
P
N1
B +R)
1
B
P
N1
A
. .
FN2
x
N2
. (9.19)
The optimal two-step cost-to-go is
J
N2
(x
N2
) = x
N2
P
N2
x
N2
, (9.20)
where we dened
P
N2
= A
P
N1
A+QA
P
N1
B(B
P
N1
B +R)
1
B
P
N1
A. (9.21)
Continuing in this manner, at some arbitrary time k the optimal control action is
u
(k) = (B
P
k+1
B +R)
1
B
P
k+1
Ax(k),
= F
k
x(k), for k = 0, . . . , N 1,
(9.22)
where
P
k
= A
P
k+1
A+QA
P
k+1
B(B
P
k+1
B +R)
1
B
P
k+1
A (9.23)
and the optimal cost-to-go starting from the measured state x(k) is
J
k
(x(k)) = x
(k)P
k
x(k). (9.24)
Equation (9.23) (called Discrete Time Riccati Equation or Riccati Dierence Equa-
tion - RDE) is initialized with P
N
= P and is solved backwards, i.e., starting with
P
N
and solving for P
N1
, etc. Note from (9.22) that the optimal control action
u
(k) is obtained in the form of a feedback law as a linear function of the measured
state x(k) at time k. The optimal cost-to-go (9.24) is found to be a quadratic
function of the state at time k.
Remark 9.1 According to Section 8.4.2, the receding horizon control policy is ob-
tained by solving problem (9.4) at each time step t with x0 = x(t). Consider the
state feedback solution u
(k) in (9.22) to problem (9.4). Then, the receding horizon

control policy is:
u
(t) = F0x(t), t 0 (9.25)

9.4 Comparison Of The Two Approaches 169
9.4 Comparison Of The Two Approaches
We will compare the batch and the recursive dynamic programming approach in
terms of the results and the methods used to obtain the results.
Most importantly we observe that the results obtained by the two methods are
fundamentally dierent. The batch approach yields a formula for the sequence of
inputs as a function of the initial state.
U
0
=
_
o
u
Qo
u
+

R
_
1
o
u
Qo
x
x(0). (9.26)
The recursive dynamic programming approach yields a feedback policy, i.e., a se-
quence of feedback laws expressing at each time step the control action as a function
of the state at that time.
u
(k) = F
k
x(k), for k = 0, . . . , N 1. (9.27)
As this expression implies, we determine u(k) at each time k as a function of the
current state x(k) rather than use a u(k) precomputed at k = 0 as in the batch
method. If the state evolves exactly according to the linear model (9.3) then the
sequence of control actions u(k) obtained from the two approaches is identical. In
practice, the result of applying the sequence (9.26) in an open-loop fashion may
be rather dierent from applying the time-varying feedback law (9.27) because the
model (9.1) for predicting the system states may be inaccurate and the system may
be subject to disturbances not included in the model. We expect the application
of the feedback law to be more robust because at each time step the observed state
x(k) is used to determine the control action rather than the state x
k
predicted at
time t = 0.
We note that we can get the same feedback eect with the batch approach
if we recalculate the optimal open-loop sequence at each time step j with the
current measurement as initial condition. In this case we need to solve the following
optimization problem
J
j
(x(j)) = min
uj, ,uN1
x
N
Px
N
+
N1
k=j
x
k
Qx
k
+u
k
Ru
k
subj. to x
j
= x(j),
(9.28)
where we note that the horizon length is shrinking at each time step.
As seen from (9.26) the solution to (9.28) relates the sequence of inputs u
j
, u
j+1
, . . .
to the state x(j) through a linear expression. The rst part of this expression yields
again the optimal feedback law (9.27) at time j, u
(j) = F
j
x(j).
Here the dynamic programming approach is clearly a more ecient way to gen-
erate the feedback policy because it only uses a simple matrix recursion (9.23).
Repeated application of the batch approach, on the other hand, requires the re-
peated inversion of a potentially large matrix in (9.26). For such inversion, however,
one can take advantage of the fact that only a small part of the matrix H changes
at every time step.
What makes dynamic programming so eective here is that in this special case,
where the system is linear and the objective is quadratic, the optimal cost-to-go,
j
(x(j)) has a very simple form: it is quadratic. If we make the
problem only slightly more complicated, e.g., if we add constraints on the inputs or
states, the value function can still be constructed, but it is much more complex. In
general, the value function can only be approximated as discussed in the previous
chapter. Then a repeated application of the batch policy, where we resolve the
optimization problem at each time step is an attractive alternative.
9.5 Innite Horizon Problem
For continuous processes operating over a long time period it would be interesting
to solve the following innite horizon problem.
J
(x(0)) = min
u0,u1,...
k=0
x
k
Qx
k
+u
k
Ru
k
. (9.29)
Since the prediction must be carried out to innity, application of the batch method
becomes impossible. On the other hand, derivation of the optimal feedback law via
dynamic programming remains viable. We can initialize the RDE (9.23)
P
k
= A
P
k+1
A+QA
P
k+1
B(B
P
k+1
B +R)
1
B
P
k+1
A (9.30)
with the terminal cost matrix P
0
= Q and solve it backwards for k . Let
us assume for the moment that the iterations converge to a solution P
. Such P
would then satisfy the Algebraic Riccati Equation (ARE)

P
= A
A+QA
B(B
B +R)
1
B
A. (9.31)
Then the optimal feedback control law is
u
(k) = (B
B +R)
1
B
A
. .
F
x(k), k = 0, , (9.32)
and the optimal innite horizon cost is
J
(x(0)) = x(0)
x(0). (9.33)
Controller (9.32) is referred to as the asymptotic form of the Linear Quadratic
Regulator (LQR) or the -horizon LQR
Convergence of the RDE has been studied extensively. A nice summary of
the various results can be found in Appendix E of the book by Goodwin and
Sin [127]. Intuitively we expect that the system (9.3) must be controllable so that
all states can be aected by the control and that the cost function should capture
the behavior of all the states, e.g. that Q 0. These conditions are indeed
sucient for the RDE to converge and to yield a stabilizing feedback control law.
Less restrictive
Theorem 9.1 [187, Theorem 2.4-2] If (A, B) is a stabilizable pair and (Q
1/2
, A) is
an observable pair, then the Riccati dierence equation (9.30) with P
0
_ 0 converges
to the unique positive denite solution P
of the ARE (9.31) and all the eigenvalues

of (A + BF
) lie strictly inside the unit disk.

9.6 Stability of the Innite Horizon LQR 171
The rst condition is clearly necessary for J
(and P
) to be nite. To un-
derstand the second condition, we write the state dependent term in the objective
function as x
Qx = (x
Q
1/2
)(Q
1/2
x). Thus not the state but the output (Q
1/2
x)
is penalized in the objective. Therefore the second condition ((Q
1/2
, A) observable)
requires that this output captures all system modes. In this manner convergence
of the output (Q
1/2
x) implies convergence of the state to zero. This is trivial if Q
is nonsingular. For the singular case this is proven in the next section.
9.6 Stability of the Innite Horizon LQR
Consider the linear system (9.1) and the -horizon LQR solution (9.31)-(9.32).
We prove that the closed-loop system
x(t + 1) = (A +BF
)x(t) (9.34)
is asymptotically stable for any F
by showing that the -horizon cost

J
(x) = x
x (9.35)
is a Lyapunov function (and thus proving the last part of Theorem 9.1).
Let Q = C
C and R = D
D with det(D) ,= 0. The proof consists of two main

steps: (i) prove that if (C, A) is an observable pair then
__
C
DF
_
, [A BF
]
_
is also observable, (ii) prove that the ARE equation (9.31) can be rewritten as
P
= (A BF
(A BF
) +
_
C
DF
_
C
DF
_
. (9.36)
Once (i) and (ii) are proven, the nal result is immediate. In fact, equation (9.36) is
a Lyapunov equation for (ABF
). Since P
0 and
__
C
DF
_
, [A BF
]
_
is observable we can use Theorem 8.6 and claim the asymptotic stability of the
closed-loop system A BF
for any F
.
Proof of (i). Since det(D) ,= 0 then B = MD for some M. We have
rank
_
_
zI A
C
DF
_
_
= rank
_
_
I 0 M
0 I 0
0 0 I
_
_
_
_
zI A
C
DF
_
_
= rank
_
_
zI (A BF
)
C
DF
_
_
.
(9.37)
Since (C, A) is observable, by the Hautus condition
rank
_
zI A
C
_
= n for every z (9.38)
and therefore by (9.37),
__
C
DF
_
, [A BF
]
_
is observable for any F
.
Proof of (ii). From (9.36)
P
= A
A F
A A
BF
+F
BF
+Q+F
RF
= A
A F
A A
BF
+F
(B
B +R)F
+Q.
(9.39)
From the denition of F
in (9.32), and the previous equation we have

P
= A
A A
B(B
B +R)
1
B
A A
B(B
B +R)
1
B
A
+A
B(B
B +R)
1
B
A+Q,
(9.40)
which is equal to
P
= A
AA
B(B
B +R)
1
B
A+Q, (9.41)
which is the ARE equation (9.31).
Chapter 10
Linear 1/ Norm Optimal
Control
We consider a special case of the problem stated in Chapter 8, where the system
is linear and time-invariant
x(t + 1) = Ax(t) +Bu(t). (10.1)
Again, x(t) R
n
and u(t) R
m
are the state and input vector respectively.
We dene the following piecewise linear cost function over a nite horizon of N
steps
J
0
(x
0
, U
0
) = |Px
N
|
p
+
N1
k=0
|Qx
k
|
p
+|Ru
k
|
p
(10.2)
with p = 1 or p = and where x
k
denotes the state vector at time k obtained by
starting from the state x
0
x
k+1
= Ax
k
+Bu
k
(10.3)
0
= [u
0
, . . . , u
N1
]
. The weighting matrices in (10.2) could

have an arbitrary number of rows. For simplicity of notation we will assume Q
R
nn
, R R
mm
and P R
rn
. Consider the nite time optimal control problem
J
0
(x(0)) = min
U0
J
0
(x(0), U
0
)
subj. to x
k+1
= Ax
k
+Bu
k
, k = 0, 1, . . . , N 1
x
0
= x(0).
(10.4)
In (10.4) U
0
= [u
0
, . . . , u
N1
]
R
s
, s = mN is the decision vector containing all
future inputs.
We will present two alternate approaches to solve problem (10.2)-(10.4), the
batch approach and the recursive approach using dynamic programming. Unlike
in the 2-norm case presented in the previous chapter, there does not exist a simple
closed-form solution of problem (10.2)-(10.4). In this chapter we will show how to
use multiparametric linear programming to compute the solution to problem (10.2)-
(10.4). We will concentrate on the use of the -norm, the results can be extended
easily to cost functions based on the 1-norm or mixed 1/ norms.
174 10 Linear 1/ Norm Optimal Control
First we write the equality constraints (10.4) explicitly to express all future states
x
1
, x
2
, . . . as a function of the future inputs u
1
, u
2
, . . . and then we eliminate all
intermediate states by using
x
k
= A
k
x
0
+
k1
j=0
A
j
Bu
k1j
(10.5)
so that all future states are explicit functions of the present state x(0) and the
future inputs u
0
, u
1
, u
2
, . . . only.
The optimal control problem (10.4) with p = can be rewritten as a linear
program by using the following standard approach (see e.g. [78]). The sum of
components of any vector
x
0
, . . . ,
x
N
,
u
0
, . . . ,
u
N1
that satises
1
n
x
k
Qx
k
, k = 0, 1, . . . , N 1
1
n
x
k
Qx
k
, k = 0, 1, . . . , N 1
1
r
x
N
Px
N
,
1
r
x
N
Px
N
,
1
m
u
k
Ru
k
, k = 0, 1, . . . , N 1
1
m
u
k
Ru
k
, k = 0, 1, . . . , N 1
(10.6)
forms an upper bound on J
0
(x(0), U
0
), where 1
k
= [1 . . . 1
. .
k
]
, and the inequali-

ties (10.6) hold componentwise. It is easy to prove that the vector
z
0
=
x
0
, . . . ,
x
N
,
u
0
, . . . ,
u
N1
, u
0
, . . . , u
N1
R
s
, s = (m + 1)N + N + 1, that
satises equations (10.6) and simultaneously minimizes J(z
0
) =
x
0
+ . . . +
x
N
+
u
0
+ . . . +
u
N1
also solves the original problem (10.4), i.e., the same optimum
J
0
(x(0)) is achieved [291, 78]. Therefore, problem (10.4) can be reformulated as
the following LP problem
min
z0
x
0
+. . . +
x
N
+
u
0
+. . . +
u
N1
(10.7a)
subj. to 1
n
x
k
Q
_
_
A
k
x
0
+
k1
j=0
A
j
Bu
k1j
_
_
(10.7b)
1
r
x
N
P
_
_
A
N
x
0
+
N1
j=0
A
j
Bu
N1j
_
_
(10.7c)
1
m
u
k
Ru
k
(10.7d)
k = 0, . . . , N 1
x
0
= x(0), (10.7e)
where constraints (10.7b)(10.7d) are componentwise, and means that the con-
straint appears once with each sign, as in (10.6).
Remark 10.1 The cost function (10.2) with p = can be interpreted as a special
case of a cost function with 1-norm over time and -norm over space. For instance,
10.1 Solution via Batch Approach 175
the dual choice (-norm over time and 1-norm over space) leads to the following cost
function
J0(x(0), U0) = max
k=0,...,N
||Qx
k
|1 +|Ru
k
|1. (10.8)
We remark that any combination of 1- and -norms leads to a linear program. In
general, -norm over time could result in a poor closed-loop performance (only the
largest state deviation and the largest input would be penalized over the prediction
horizon), while 1-norm over space leads to an LP with a larger number of variables.
The results of this chapter hold for any combination of 1- and -norms over time and
space. Clearly the LP formulation will dier from the one in (10.7). For instance,
the 1norm in space requires the introduction of nN slack variables for the terms
|Qx
k
|1,
k,i
Q
i
x
k
k = 0, 2, . . . , N 1, i = 1, 2, . . . , n, plus r slack variables for
the terminal penalty |PxN|1, N,i PixN i = 1, 2, . . . , r, plus mN slack variables
for the input terms |Ru
k
|1,
u
k,i
Riu
k
k = 0, 1, . . . , N 1, i = 1, 2, . . . , m. Here
we have used the notation Mi to denote the i-th row of matrix M.
Problem (10.7) can be rewritten in the more compact form
min
z0
c
0
z
0
subj. to G
z
0
w
+S
x(0),
(10.9)
where c
0
R
s
and G
R
qs
, S
R
qn
and w
R
q
are
c0 = [
N+1
..
1 . . . 1
N
..
1 . . . 1
mN
..
0 . . . 0]
G =
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
N+1
..
1n 0 . . . 0
1n 0 . . . 0
0 1n . . . 0
0 1n . . . 0
. . . . . . . . . . . .
0 . . . 1n 0
0 . . . 1n 0
0 . . . 0 1r
0 . . . 0 1r
0 0 . . . 0
0 0 . . . 0
. . . . . . . . . . . .
0 0 . . . 0
0 0 . . . 0
N
..
0 . . . 0
0 . . . 0
0 . . . 0
0 . . . 0
. . . . . . . . .
0 . . . 0
0 . . . 0
0 . . . 0
0 . . . 0
1m . . . 0
1m . . . 0
. . . . . . . . .
0 . . . 1m
0 . . . 1m
mN
..
0 0 . . . 0
0 0 . . . 0
QB 0 . . . 0
QB 0 . . . 0
. . . . . . . . . . . .
QA
N2
B QA
N3
B . . . 0
QA
N2
B QA
N3
B . . . 0
PA
N1
B PA
N2
B . . . PB
PA
N1
B PA
N2
B . . . PB
R 0 . . . 0
R 0 . . . 0
. . . . . . . . . . . .
0 0 . . . R
0 0 . . . R
_
_
w = [
2nN+2r
..
0 . . . 0
2mN
..
0 . . . 0]
S = [
2nN
..
Q
(QA)
(QA)
(QA
2
)
. . . (QA
N1
)
(QA
N1
)
2r
..
(PA
N
)
(PA
N
)
2mN
..
0
m
. . . 0
m
]
.
(10.10)
Note that in (10.10) we include the zero vector w
to make the notation consistent

with the one used in Section 7.2.
By treating x(0) as a vector of parameters, the problem (10.9) becomes a multi-
parametric linear program (mp-LP) that can be solved as described in Section 7.2.
Once the multiparametric problem (10.7) has been solved, the explicit solution
z
0
(x(0)) of (10.9) is available as a piecewise ane function of x(0), and the op-
timal control law U
0
is also available explicitly, as the optimal input U
0
consists
simply of the last part of z
0
(x(0))
U
0
(x(0)) = [0 . . . 0 I
m
I
m
. . . I
m
]z
0
(x(0)). (10.11)
Theorem 7.5 states that there exists a continuous and PPWA solution z
0
(x)
of the mp-LP problem (10.9). Clearly the same properties are inherited by the
controller. The following Corollaries of Theorem 7.5 summarize the analytical
properties of the optimal control law and the value function.
Corollary 10.1 There exists a control law U
0
=

f
0
(x(0)),

f
0
: R
n
R
m
, obtained
as a solution of the optimal control problem (10.2)-(10.4) with p = 1 or p = ,
which is continuous and PPWA
f
0
(x) =

F
i
0
x if x CR
i
0
, i = 1, . . . , N
r
0
, (10.12)
where the polyhedral sets CR
i
0
= H
i
0
x 0, i = 1, . . . , N
r
0
, are a partition of R
n
.
Note that in Corollary 10.1 the control law is linear (not ane) and the critical
regions have a conic shape (CR
i
0
= H
i
0
x 0). This can be proven immediately
from the results in Section 7.2 by observing that the constant term w
at the
right-hand side on the mp-LP problem (10.9) is zero.
Corollary 10.2 The value function J
(x) obtained as a solution of the optimal

control problem (10.2)-(10.4) is convex and PPWA.
Remark 10.2 Note that if the optimizer of problem (10.4) is unique for all x(0) R
n
,
then Corollary 10.1 reads: The control law U
(0) =

f0(x(0)),

f0 : R
n
R
m
,
obtained as a solution of the optimal control problem (10.2)-(10.4) with p = 1 or
p = , is continuous and PPWA,. . .. From the results of Section 7.2 we know that
in case of multiple optimizers for some x(0) R
n
, a control law of the form (10.12)
can always be computed.
Alternatively we can use dynamic programming to solve the same problem in a
recursive manner. We dene the optimal cost J
j
(x
j
) for the N j step problem
starting from state x
j
by
J
j
(x
j
) = min
uj, ,uN1
|Px
N
|
+
N1
k=j
|Qx
k
|
+|Ru
k
|
.
According to the principle of optimality the optimal one step cost-to-go can be
obtained from
J
N1
(x
N1
) = min
uN1
|P
N
x
N
|
+|Qx
N1
|
+|Ru
N1
|
(10.13)
x
N
= Ax
N1
+Bu
N1
P
N
= P.
(10.14)
Substituting (10.14) into the objective function (10.13), we have
J
N1
(x
N1
) = min
uN1
|P
N
(Ax
N1
+Bu
N1
)|
+|Qx
N1
|
+|Ru
N1
|
.
(10.15)
We nd the optimum by solving the mp-LP
min
x
N1
,
x
N
,
u
N1
,uN1
x
N1
+
x
N
+
u
N1
(10.16a)
subj. to 1
n
x
N1
Qx
N1
(10.16b)
1
rN
x
N
P
N
[Ax
N1
+Bu
N1
] (10.16c)
1
m
u
N1
Ru
N1
, (10.16d)
where r
N
is the number of rows of the matrix P
N
. By Theorem 7.5, J
N1
is a
convex and piecewise ane function of x
N1
, the corresponding optimizer u
N1
is piecewise ane and continuous, and the feasible set A
N1
is R
n
. We use the
equivalence of representation between convex and PPWA functions and innity
norm (see Section 3.2.5) to write the one-step optimal cost-to-go as
J
N1
(x
N1
) = |P
N1
x
N1
|
(10.17)
with P
N1
dened appropriately. At the next stage, consider the two-step problem
from time N 2 forward:
J
N2
(x
N2
) = min
uN2
|P
N1
x
N1
|
+|Qx
N2
|
+|Ru
N2
|
(10.18)
x
N1
= Ax
N2
+ Bu
N2
. (10.19)
We recognize that (10.18), (10.19) has the same form as (10.13), (10.14). There-
fore we can compute the optimal solution again by solving the mp-LP
min
x
N2
,
x
N1
,
u
N2
,uN2
x
N2
+
x
N1
+
u
N2
(10.20a)
subj. to 1
n
x
N2
Qx
N2
(10.20b)
1
rN1
x
N1
P
N1
[Ax
N2
+Bu
N2
] (10.20c)
1
m
u
N2
Ru
N2
, (10.20d)
where r
N1
is the number of rows of the matrix P
N1
. The optimal two-step
cost-to-go is
J
N2
(x
N2
) = |P
N2
x
N2
|
, (10.21)
Continuing in this manner, at some arbitrary time k the optimal control action is
u
(k) = f
k
(x(k)), (10.22)
where f
k
(x) is continuous and PPWA
f
k
(x) = F
i
k
x if H
i
k
x 0, i = 1, . . . , N
r
k
, (10.23)
where the polyhedral sets H
i
k
x 0, i = 1, . . . , N
r
k
, are a partition of R
n
. The
optimal cost-to-go starting from the measured state x(k) is
J
k
(x(k)) = |P
k
x(k)|
. (10.24)
Here we have introduced the notation P
k
to express the optimal cost-to-go J
k
(x(k)) =
|P
k
x(k)|
from time k to the end of the horizon N. We also remark that the rows
of P
k
correspond to the dierent ane functions constituting J
k
and thus their
number varies with the time index k. Clearly, we do not have a closed form as
for the 2-norm with the Riccati Dierence Equation (9.23) linking cost and control
law at time k given their value at time k 1.
10.3 Comparison Of The Two Approaches
We will compare the batch and the recursive dynamic programming approach in
terms of the results and the methods used to obtain the results. Most importantly
we observe that the results obtained by the two methods are fundamentally dier-
ent. The batch approach yields a formula for the sequence of inputs as a function
of the initial state.
U
0
=

F
i
0
x(0) if

H
i
0
x(0) 0, i = 1, . . . ,

N
r
0
. (10.25)
The recursive dynamic programming approach yields a feedback policy, i.e., a se-
quence of feedback laws expressing at each time step the control action as a function
of the state at that time.
u
(k) = F
i
k
x(k) if H
i
k
x(k) 0, i = 1, . . . , N
r
k
for k = 0, . . . , N 1. (10.26)
As this expression implies, we determine u(k) at each time k as a function of the
current state x(k) rather than use a u(k) precomputed at k = 0 as in the batch
method. If the state evolves exactly according to the linear model (10.3) then the
sequence of control actions u(k) obtained from the two approaches is identical. In
practice, the result of applying the sequence (10.25) in an open-loop fashion may
be rather dierent from applying the time-varying feedback law (10.26) because the
model (10.3) for predicting the system states may be inaccurate and the system may
be subject to disturbances not included in the model. We expect the application
of the feedback law to be more robust because at each time step the observed state
x(k) is used to determine the control action rather than the state x
k
predicted at
time t = 0.
We note that we can get the same feedback eect with the batch approach
if we recalculate the optimal open-loop sequence at each time step j with the
current measurement as initial condition. In this case we need to solve the following
10.4 Innite Horizon Problem 179
optimization problem
J
j
(x(j)) = min
uj, ,uN1
|Px
N
|
N1
k=j
|Qx
k
|
+|Ru
k
|
subj. to x
j
= x(j),
(10.27)
where we note that the horizon length is shrinking at each time step.
As seen from (10.25) the solution to (10.27) relates the sequence of inputs
u
j
, u
j+1
, . . . to the state x(j) through a linear expression. The rst part of this
expression yields again the optimal feedback law (10.26) at time j, u
(j) = f
j
(x(j)).
Here the dynamic programming approach is clearly a more ecient way to
generate the feedback policy because it requires the solution of a small mp-LP
problem (10.7) for each time step. Repeated application of the batch approach, on
the other hand, requires repeatedly the solution of a larger mp-LP for each time
step.
10.4 Innite Horizon Problem
For continuous processes operating over a long time period it would be interesting
to solve the following innite horizon problem.
J
(x(0)) = min
u(0),u(1),...
k=0
|Qx
k
|
+|Ru
k
|
. (10.28)
Since the prediction must be carried out to innity, application of the batch method
becomes impossible. On the other hand, derivation of the optimal feedback law
via dynamic programming remains viable. We can use the dynamic programming
formulation
|P
j
x
j
|
= min
uj
|P
j+1
x
j+1
|
+|Qx
j
|
+|Ru
j
|
(10.29)
x
j+1
= Ax
j
+Bu
j
(10.30)
with the terminal cost matrix P
0
= Q and solve it backwards for k . Let
us assume for the moment that the iterations converge to a solution P
in a nite
number of iterations. Then the optimal feedback control law is time-invariant and
piecewise ane
u
(k) = F
i
x(k) if H
i
x 0, i = 1, . . . , N
r
(10.31)
and the optimal innite horizon cost is
J
(x(0)) = |P
x(0)|
. (10.32)
In general, the innite time optimal cost J
(x(0)) and the optimal feedback control

law are not necessarily piecewise ane (with a nite number of regions). Conver-
gence of the recursive scheme (10.29) has been studied in detail in [86]. If this
recursive scheme converges and Q and R are of full column rank, then the result-
ing control law (10.31) stabilizes the system (see Section 8.4).
Example 10.1 Consider the double integrator system
1
_
x(t + 1) =
_
1 1
0 1
_
x(t) +
_
0
1
_
u(t) (10.33)
The aim is to compute the innite horizon optimal controller that solves the opti-
mization problem (10.28) with Q =
_
1 0
0 1
_
and R = 20.
The dynamic programming iteration (10.29) converges after 18 iterations to the fol-
lowing optimal solution:
u =
_
_
[ 9.44 29.44 ] x if
_
0.10 1.00
0.71 0.71
_
x [
0
0
] (Region #1)
[ 9.00 25.00 ] x if
_
0.10 1.00
0.11 0.99
_
x [
0
0
] (Region #2)
[ 1.00 19.00 ] x if [
0.45 0.89
0.71 0.71
] x [
0
0
] (Region #3)
[ 8.00 16.00 ] x if
_
0.11 0.99
0.12 0.99
_
x [
0
0
] (Region #4)
[ 2.00 17.00 ] x if [
0.32 0.95
0.45 0.89
] x [
0
0
] (Region #5)
[ 7.00 8.00 ] x if
_
0.12 0.99
0.14 0.99
_
x [
0
0
] (Region #6)
[ 3.00 14.00 ] x if
_
0.32 0.95
0.24 0.97
_
x [
0
0
] (Region #7)
[ 6.00 1.00 ] x if
_
0.14 0.99
0.16 0.99
_
x [
0
0
] (Region #8)
[ 4.00 10.00 ] x if
_
0.24 0.97
0.20 0.98
_
x [
0
0
] (Region #9)
[ 5.00 5.00 ] x if
_
0.16 0.99
0.20 0.98
_
x [
0
0
] (Region #10)
[ 5.00 5.00 ] x if
_
0.20 0.98
0.16 0.99
_
x [
0
0
] (Region #11)
[ 4.00 10.00 ] x if
_
0.20 0.98
0.24 0.97
_
x [
0
0
] (Region #12)
[ 6.00 1.00 ] x if
_
0.16 0.99
0.14 0.99
_
x [
0
0
] (Region #13)
[ 3.00 14.00 ] x if
_
0.24 0.97
0.32 0.95
_
x [
0
0
] (Region #14)
[ 7.00 8.00 ] x if
_
0.14 0.99
0.12 0.99
_
x [
0
0
] (Region #15)
[ 2.00 17.00 ] x if
_
0.32 0.95
0.45 0.89
_
x [
0
0
] (Region #16)
[ 8.00 16.00 ] x if
_
0.12 0.99
0.11 0.99
_
x [
0
0
] (Region #17)
[ 1.00 19.00 ] x if
_
0.45 0.89
0.71 0.71
_
x [
0
0
] (Region #18)
[ 9.00 25.00 ] x if
_
0.11 0.99
0.10 1.00
_
x [
0
0
] (Region #19)
[ 9.44 29.44 ] x if [
0.10 1.00
0.71 0.71
] x [
0
0
] (Region #20)
1
10.4 Innite Horizon Problem 181
with P equal to
P =
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
9.44 29.44
9.00 25.00
1.00 19.00
8.00 16.00
2.00 17.00
7.00 8.00
3.00 14.00
6.00 1.00
4.00 10.00
5.00 5.00
5.00 5.00
4.00 10.00
6.00 1.00
3.00 14.00
7.00 8.00
2.00 17.00
8.00 16.00
1.00 19.00
9.00 25.00
9.44 29.44
_
_
(10.34)
Note that P in (10.34) has 20 rows corresponding to the 20 linear terms (or pieces)
of the piecewise linear value function J
(x) = |Px| for x R

2
. For instance,
J
(x) in region 1 is J
(x) = 9.44x1 + 29.44x2, where x1 and x2 denote the rst

and second component of the state vector x, respectively. Note that each linear term
appears twice, with positive and negative sign. Therefore J
(x) can be written in

minimal form as the innity norm of a matrix |
Px| with

P being a matrix with
ten rows. The value function J
(x) = |Px| is plotted in Figure 10.1.

x
1
x
2
J
(
x
)
Figure 10.1 Example 10.1, -norm objective function, -horizon controller.
Piecewise linear optimal cost (value function) and corresponding polyhedral
partition.
Part IV
Constrained Optimal Control of
Linear Systems
Chapter 11
Controllability, Reachability
and Invariance
This chapter is a self-contained introduction to controllability, reachability and
invariant set theory. N-steps reachable sets are dened for autonomous systems.
They represent the set of states which a system can evolve to after N steps. N-
steps controllable sets are dened for systems with inputs. They represent the set
of states to which a system can be steered into after N steps.
Invariant sets are the innite time counterparts of N-steps controllable and
reachable sets. Invariant sets are computed for autonomous systems. These types
of sets are useful to answer questions such as: For a given feedback controller
u = k(x), nd the set of states whose trajectory will never violate the system
constraints. Control invariant sets are dened for systems subject to external
inputs. These types of sets are useful to answer questions such as: Find the set
of states for which there exists a controller such that the system constraints are
never violated.
This chapter focuses on computational algorithms for constrained linear systems
and constrained linear systems subject to additive and parametric uncertainty. A
thorough presentation of the basic notions and algorithms presented in this chapter
can be found in the recent book [58].
11.1 Controllable and Reachable Sets
In this section we deal with two types of systems, namely, autonomous systems:
x(t + 1) = g
a
(x(t)), (11.1)
and systems subject to external inputs:
x(t + 1) = g(x(t), u(t)). (11.2)
186 11 Controllability, Reachability and Invariance
Both systems are subject to state and input constraints
x(t) A, u(t) |, t 0. (11.3)
The sets A and | are polyhedra.
Denition 11.1 For the autonomous system (11.1) we denote the one-step con-
trollable set to the set o as
Pre(o) = x R
n
: g
a
(x) o. (11.4)
Pre(o) is the set of states which evolve into the target set o in one time step.
Denition 11.2 For the system (11.2) we denote the one-step controllable set to
the set o as
Pre(o) = x R
n
: u | s.t. g(x, u) o. (11.5)
For a system with inputs, Pre(o) is the set of states which can be driven into the
target set o in one time step while satisfying input and state constraints.
Denition 11.3 For the autonomous system (11.1) we denote the one-step reach-
able set from the set o as
Reach(o) = x R
n
: x(0) o s.t. x = g
a
(x(0)).
Denition 11.4 For the system (11.2) with inputs we will denote the one-step
reachable set from the set o as
Reach(o) = x R
n
: x(0) o, u(0) | s.t. x = g(x(0), u(0)).
Therefore, all the states contained in o are mapped into the set Reach(o) under
the map g
a
or under the map g for some input u |.
Remark 11.1 Note that in our notation we use the word controllable for the Pre(S)
set even if the system is autonomous.
The sets Pre(S) and Reach(S) are also denoted as one-step backward-reachable set
and one-step forward-reachable set, respectively, in the literature.
N-step controllable and reachable sets are dened by iterating Pre() and Reach()
computations, respectively.
Denition 11.5 (N-Step Controllable Set /
N
(o)) For a given target set o
A, the N-step controllable set /
N
(o) of the system (11.1) or (11.2) subject to the
constraints (11.3) is dened recursively as:
/
j
(o) = Pre(/
j1
(o)) A, /
0
(o) = o, j 1, . . . , N (11.6)
11.1 Controllable and Reachable Sets 187
From Denition 11.5, all states x
0
of the system (11.1) belonging to the N-Step
Controllable Set /
N
(o) will evolve to the target set o in N steps, while satisfying
state constraints.
Also, all states x
0
of the system (11.2) belonging to the N-Step Controllable
Set /
N
(o) can be driven, by a suitable control sequence, to the target set o in N
steps, while satisfying input and state constraints.
Denition 11.6 (N-Step Reachable Set 1
N
(A
0
)) For a given initial set A
0

A, the N-step reachable set 1
N
(A
0
) of the system (11.1) or (11.2) subject to the
constraints (11.3) is dened as:
1
i+1
(A
0
) = Reach(1
i
(A
0
)), 1
0
(A
0
) = A
0
, i = 0, . . . , N 1 (11.7)
0
belonging to A
0
will evolve to the N-step reach-
able set 1
N
(A
0
) in N steps.
11.1.1 Computation of Controllable and Reachable Sets
Next we will show through simple examples the main steps involved in the com-
putation of controllable and reachable sets for constrained linear systems. Later in
this section we will provide compact formulas based on polyhedral operations.
Example 11.1 Consider the second order autonomous stable system
1
x(t + 1) = Ax(t) =
_
0.5 0
1 0.5
_
x(t) (11.8)
subject to the state constraints
x(t) . =
_
x[
_
10
10
_
x
_
10
10
__
, t 0. (11.9)
The set Pre(.) can be obtained as follows: Since the set . is a polytope, it can be
represented as a -polytope (Section 5.2)
. = |x : Hx h, (11.10)
where
H =
_
_
_
_
1 0
0 1
1 0
0 1
_
_
and h =
_
_
_
_
10
10
10
10
_
_
.
By using this -presentation and the system equation (11.8), the set Pre(.) can be
derived:
Pre(.) = |x : Hga(x) h (11.11)
= |x : HAx h . (11.12)
1
The set (11.12) may contain redundant inequalities which can be removed by using
Algorithm 5.1 in Section 5.4.1 to obtain its minimal representation. Note that by
using the notation in Section 5.4.11, the set Pre(.) in (11.12) is simply . A.
The set Pre(.) is
Pre(.) =
_
_
x :
_
_
_
_
1 0
1 0.5
1 0
1 0.5
_
_
x
_
_
_
_
20
10
20
10
_
_
_
_
.
The set Pre(.) ., the signicance of which we will discuss in Section 11.2, is
Pre(.) . =
_
_
x :
_
_
_
_
_
_
_
_
1 0
0 1
1 0
0 1
1 0.5
1 0.5
_
_
x
_
_
_
_
_
_
_
_
10
10
10
10
10
10
_
_
_
_
and it is depicted in Fig. 11.1.
x
1
x
2
A
Pre(A)
-10 -5 0 5 10
-10
-5
0
5
10
Figure 11.1 Example 11.1: one-step controllable set Pre(.) . for system
(11.8) under constraints (11.9). Download code.
The set Reach(.) is obtained by applying the map A to the set .. Let us write .
in 1-representation (see Section 5.1)
. = conv(V ), (11.13)
and let us map the set of vertices V through the transformation A. Because the
transformation is linear, the reach set is simply the convex hull of the transformed
vertices
Reach(.) = A . = conv(AV ). (11.14)
We refer the reader to Section 5.4.11 for a detailed discussion on linear transforma-
tions of polyhedra.
The set Reach(.) in -representation is
Reach(.) =
_
_
x :
_
_
_
_
1 0
1 0
1 0.5
1 0.5
_
_
x
_
_
_
_
5
5
2.5
2.5
_
_
_
_
x
1
x
2
A
R
e
a
c
h
(
A
)
-10 -5 0 5 10
-10
-5
0
5
10
Figure 11.2 Example 11.1: one-step reachable set for system (11.8). Down-
load code.
and is depicted in Fig. 11.2.
Example 11.2 Consider the second order unstable system
2
_
x(t + 1) = Ax +Bu =
_
1.5 0
1 1.5
_
x(t) +
_
1
0
_
u(t) (11.15)
subject to the input and state constraints
u(t) | = |u[ 5 u 5 , t 0 (11.16a)
x(t) . =
_
x [
_
10
10
_
x
_
10
10
__
, t 0. (11.16b)
For the non-autonomous system (11.15), the set Pre(.) can be computed using the
-representation of . and |,
. = |x [ Hx h, | = |u [ Huu hu, (11.17)
to obtain
Pre(.) =
_
x R
2
[ u | s.t. g(x, u) .,
_
(11.18)
=
_
x R
2
[ u R s.t.
_
HA HB
0 Hu
_ _
x
u
_
_
h
hu
__
. (11.19)
The half-spaces in (11.19) dene a polytope in the state-input space, and a projection
operation (see Section 5.4.6) is used to derive the half-spaces which dene Pre(.) in
the state space. The set Pre(.) . is depicted in Fig. 11.3 and reported below:
_
_
_
_
_
_
_
_
1 0
0 1
1 0
0 1
1 1.5
1 1.5
_
_
x
_
_
_
_
_
_
_
_
10
10
10
10
10
10
_
_
2
x
1
x
2
A
Pre(A) A
-10 -5 0 5 10
-10
-5
0
5
10
Figure 11.3 Example 11.2: one-step controllable set Pre(.) . for system
(11.15) under constraints (11.16). Download code.
Note that by using the denition of the Minkowski sum given in Section 5.4.9 and
the ane operation on polyhedra in Section 5.4.11 we can write the operations in
(11.19) compactly as follows:
Pre(.) = |x : u | s.t. Ax +Bu .
|x : y = Ax +Bu, y ., u |
|x : Ax = y + (Bu), y ., u |
|x : Ax c, c = . (B) |
|x : x c A, c = . (B) |
|x : x (. (B) |) A .
(11.20)
The set Reach(.) = |Ax + Bu R
2
: x ., u | is obtained by applying the
map A to the set . and then considering the eect of the input u |. As shown
before,
A . = conv(AV ) (11.21)
and therefore
Reach(.) = |y +Bu : y A ., u |.
We can use the denition of the Minkowski sum given in Section 5.4.9 and rewrite
the set Reach(.) as
Reach(.) = (A .) (B |).
We can compute the Minkowski sum via projection or vertex enumeration as ex-
plained in Section 5.4.9 and obtain the set Reach(.) in -representation
Reach(.) =
_
_
x :
_
_
_
_
_
_
_
_
1 0
1 0
0 1
0 1
1 1.5
1 1.5
_
_
x
_
_
_
_
_
_
_
_
20
20
25
25
27.5
27.5
_
_
_
_
,
which is depicted in Fig. 11.4.
In summary, the sets Pre(A) and Reach(A) are the results of linear operations
on the polyhedra A and | and therefore are polyhedra. By using the denition
x
1
x
2
Reach(A)
-20 -10 0 10 20
-40
-20
0
20
40
Figure 11.4 Example 11.2: one-step reachable sets for system (11.15) under
constraints (11.16). Download code.
of the Minkowski sum given in Section 5.4.9 and of ane operation on polyhedra
in Section 5.4.11 we can compactly summarize the Pre and Reach operations on
linear systems in Table 11.1.
x(t + 1) = Ax(t) x(t + 1) = Ax(t) +Bu(t)
Pre(A) A A (A (B |)) A
Reach(A) A A (A A) (B |)
Table 11.1 Pre and Reach operations for linear systems subject to polyhedral
state and input constraints x(t) ., u(t) |
The N-step controllable set /
N
(o) and the N-step reachable set 1
N
(A
0
)
can be computed by using their recursive formulas and computing the
Pre and Reach operations as in Table 11.1.
Example 11.3 Consider the second order unstable system (11.15) subject to the input
and state constraints (11.16). Consider the target set
3
S =
_
x [
_
1
1
_
x
_
1
1
__
.
The N-step controllable set /N(S) of the system (11.15) subject to the constraints (11.16)
can be computed by using the recursively formula (11.6)
/j(S) = Pre(/j1(S)) ., /0(S) = S, j |1, . . . , N
and the steps described in Example 11.2 to compute the Pre() set.
The sets /N(S) for N = 1, 2, 3, 4 are depicted in Fig. 11.5. The sets are shifted along
the x-axis for a clearer visualization.
3
o /
1
(o) /
2
(o) /
3
(o) /
4
(o)
Figure 11.5 Example 11.3: one-step controllable sets /j(S) for system
(11.15) under constraints (11.16) for N=1,2,3,4. Note that the sets are
shifted along the x-axis for a clearer visualization. Download code.
Example 11.4 Consider the second order unstable system (11.15) subject to the input
and state constraints (11.16)
4
. Consider the initial set
.0 =
_
x[
_
1
1
_
x
_
1
1
__
.
The N-step reachable set 1N(.0) of the system (11.15) subject to the constraints (11.16)
can be computed by using the recursively formula (11.7)
1i+1(.0) = Reach(1i(.0)), 10(.0) = .0, i = 0, . . . , N 1
and the steps described in Example 11.2 to compute the Reach() set.
The sets /N(.0)) for N = 1, 2, 3, 4 are depicted in Fig. 11.6. The sets are shifted
along the x-axis for a clearer visualization.
A
0
1
1
(A
0
) 1
2
(A
0
) 1
3
(A
0
) 1
4
(A
0
)
Figure 11.6 Example 11.3: N-step reachable sets for system (11.15) under
4
11.2 Invariant Sets 193
11.2 Invariant Sets
Consider the constrained autonomous system (11.1) and the constrained system
subject to external inputs (11.2) dened in Section 11.1.
Two dierent types of sets are considered in this section: invariant sets and
control invariant sets. We will rst discuss invariant sets.
Positive Invariant Sets
Invariant sets are used for characterizing the behavior of autonomous systems.
These types of sets are useful to answer questions such as: For a given feedback
controller u = f(x), nd the set of initial states whose trajectory will never violate
the system constraints. The following denitions, derived from [170, 57, 53, 49,
176, 136, 138, 139], introduce the dierent types of invariant sets.
Denition 11.7 (Positive Invariant Set) A set O A is said to be a positive
invariant set for the autonomous system (11.1) subject to the constraints in (11.3),
if
x(0) O x(t) O, t N
+
.
Denition 11.8 (Maximal Positive Invariant Set O
) The set O
A is
the maximal invariant set of the autonomous system (11.1) subject to the con-
straints in (11.3) if O
is invariant and O
contains all the invariant sets con-

tained in A.
Remark 11.2 The maximal invariant sets dened here are often referred to as maxi-
mal admissible sets or maximal output admissible sets in the literature (e.g. [121]),
depending on whether the system state or output is constrained.
Remark 11.3 Note that, in general, the nonlinear system (11.1) may have multi-
ple equilibrium points, and thus O might be the union of disconnected sets each
containing an equilibrium point.
Theorem 11.1 (Geometric condition for invariance [97]) A set O A is a
positive invariant set for the autonomous system (11.1) subject to the constraints
in (11.3), if and only if
O Pre(O). (11.22)
Proof: We prove both the necessary and sucient parts by contradiction. (:)
If O Pre(O) then x O such that x / Pre(O). From the denition of Pre(O),
g
a
( x) / O and thus O is not positive invariant. (:) If O is not a positive invariant
set then x O such that g
a
( x) / O. This implies that x O and x / Pre(O)
and thus O Pre(O)
It is immediate to prove that condition (11.22) of Theorem 11.1 is equivalent
to the following condition
Pre(O) O = O. (11.23)
Based on condition (11.23), the following algorithm provides a procedure for com-
puting the maximal positive invariant subset O
for system (11.1),(11.3) [11, 49,

170, 121].
Algorithm 11.1 Computation of O
input ga , .
output O
let 0 .
repeat
k = k + 1
k+1
Pre(
k
)
k
until
k+1
=
k
O
k
Algorithm 11.1 generates the set sequence
k
satisfying
k+1

k
, k N and
it terminates when
k+1
=
k
. If it terminates, then
k
is the maximal positive
invariant set O
for the system (11.1)-(11.3). If

k
= for some integer k then
the simple conclusion is that O
= .
In general, Algorithm 11.1 may never terminate. If the algorithm does not
terminate in a nite number of iterations, it can be proven that [176]
O
= lim
k+
k
.
Conditions for nite time termination of Algorithm 11.1 can be found in [120].
A simple sucient condition for nite time termination of Algorithm 11.1 requires
the system g
a
(x) to be linear and stable, and the constraint set A to be bounded
and to contain the origin.
Example 11.5 Consider the second order stable system in Example 11.1
5
. The max-
imal positive invariant set of system (11.8) subject to constraints (11.9) is depicted
in Fig. 11.7 and reported below:
_
_
_
_
_
_
_
_
1 0
0 1
1 0
0 1
1 0.5
1 0.5
_
_
x
_
_
_
_
_
_
_
_
10
10
10
10
10
10
_
_
.
Note from the previous discussion of the example and from Figure 11.1 that
here the maximal positive invariant set O
is obtained after a single step of Algo-

rithm 11.1, i.e.
O
=
1
= Pre(A) A.
5
x
1
x
2
A
O
-10 -5 0 5 10
-10
-5
0
5
10
Figure 11.7 Maximal Positive Invariant Set of system (11.8) under con-
straints (11.9). Download code.
Control Invariant Sets
Control invariant sets are dened for systems subject to external inputs. These
types of sets are useful to answer questions such as: Find the set of initial states for
which there exists a controller such that the system constraints are never violated.
The following denitions, adopted from [170, 57, 53, 49, 176], introduce the dierent
types of control invariant sets.
Denition 11.9 (Control Invariant Set) A set c A is said to be a control
invariant set for the system (11.2) subject to the constraints in (11.3), if
x(t) c u(t) | such that g(x(t), u(t)) c, t N
+
.
Denition 11.10 (Maximal Control Invariant Set c
) The set c
A is
said to be the maximal control invariant set for the system (11.2) subject to the
constraints in (11.3), if it is control invariant and contains all control invariant
sets contained in A.
Remark 11.4 The geometric conditions for invariance (11.22), (11.23) hold for control
invariant sets.
The following algorithm provides a procedure for computing the maximal con-
trol invariant set c
for system (11.2),(11.3) [11, 49, 170, 121].

Algorithm 11.2 Computation of c
input: g, . and |
output: c
let 0 .
repeat
k = k + 1
k+1
Pre(
k
)
k
until
k+1
=
k
c
k+1
k
satisfying
k+1

k
, k N. Al-
gorithm 11.2 terminates when
k+1
=
k
k
is the maximal
control invariant set c
for the system (11.2)-(11.3). In general, Algorithm 11.2

may never terminate [11, 49, 170, 164]. If the algorithm does not terminate in a -
nite number of iterations, in general, convergence to the maximal control invariant
set is not guaranteed
c
,= lim
k+
k
. (11.24)
The work in [50] reports examples of nonlinear systems where (11.24) can be ob-
served. A sucient condition for the convergence of
k
to c
as k + requires
the polyhedral sets A and | to be bounded and the system g(x, u) to be continu-
ous [50].
Example 11.6 Consider the second order unstable system in example 11.2
6
. Algo-
rithm 11.2 is used to compute the maximal control invariant set of system (11.15)
subject to constraints (11.16). Algorithm 11.2 terminates after 45 iterations and the
maximal control invariant set c is:
_
_
_
_
_
_
_
_
0 1
0 1
0.55 0.83
0.55 0.83
1 0
1 0
_
_
x
_
_
_
_
_
_
_
_
4
4
2.22
2.22
10
10
_
_
.
The results of the iterations and c
are depicted in Fig. 11.8.

x
1
x
2
A
c
-10 -5 0 5 10
-10
-5
0
5
10
Figure 11.8 Maximal Control Invariant Set of system (11.15) subject to
6
Denition 11.11 (Finitely determined set) Consider Algorithm 11.1 (Algo-
rithm 11.2). The set O
(c
) is nitely determined if and only if i N such

that
i+1
=
i
. The smallest element i N such that
i+1
=
i
is called the
determinedness index.
Remark 11.5 From the results in Section 11.1.1, for linear system with linear con-
straints the sets O and c are polyhedra if they are nitely determined.
For all states contained in the maximal control invariant set c
there exists
a control law such that the system constraints are never violated. This does not
imply that there exists a control law which can drive the state into a user-specied
target set. This issue is addressed in the following by introducing the concepts of
maximal controllable sets and stabilizable sets.
Denition 11.12 (Maximal Controllable Set /
(O)) For a given target set

O A, the maximal controllable set /
(O) for system (11.2) subject to the con-

straints in (11.3) is the union of all N-step controllable sets /
N
(O) contained in
A (N N).
We will often deal with controllable sets /
N
(O) where the target O is a control
invariant set. They are special sets, since in addition to guaranteeing that from
/
N
(O) we reach O in N steps, one can ensure that once it has reached O, the
system can stay there at all future time instants.
Denition 11.13 (N-step (Maximal) Stabilizable Set) For a given control
invariant set O A, the N-step (maximal) stabilizable set of the system (11.2)
subject to the constraints (11.3) is the N-step (maximal) controllable set /
N
(O)
(/
(O)).
The set /
(O) contains all states which can be steered into the control invariant
set O and hence /
(O) c
. The set /
(O) c
can be computed as follows

(cf. [57, 50]):
Algorithm 11.3 Maximal Stabilizable Set /(O)
let /0 O, where O is a control invariant set
repeat
c = c + 1
/c+1 Pre(/c)
.
until /c+1 = /c,
/(O) /c,
Since O is control invariant, it holds c N that /
c
(O) is control invariant and
/
c
/
c+1
. Note that Algorithm 11.3 is not guaranteed to terminate in nite time.
Remark 11.6 In general, the maximal stabilizable set /(O) is not equal to the
maximal control invariant set c, even for linear systems. /(O) c for all
control invariant sets O. The set c \ /(O) includes all initial states from which
it is not possible to steer the system to the stabilizable region /(O) and hence O.
Example 11.7 Consider the simple constrained one-dimensional system
x(t + 1) = 2x(t) +u(t) (11.25a)
[x(t)[ 1, and [u(t)[ 1 (11.25b)
and the state feedback control law
u(t) =
_
_
_
1 if x(t)
_
1,
1
2
_
2x(t) if x(t)
_
1
2
,
1
2
_
1 if x(t)
_
1
2
, 1
_
.
(11.26)
The closed-loop system has three equilibria at 1, 0, and 1 and system (11.25) is
always feasible for all initial states in [1, 1] and therefore c = [1, 1]. It is,
however, asymptotically stable only for the open set (1, 1). In fact, u(t) and any
other feasible control law cannot stabilize the system from x = 1 and from x = 1
and therefore when O = then /(O) = (1, 1) c. We note in this example
that the maximal stabilizable set is open. One can easily argue that, in general, if the
maximal stabilizable set is closed then it is equal to the maximal control invariant
set.
11.3 Robust Controllable and Reachable Sets 199
11.3 Robust Controllable and Reachable Sets
In this section we deal with two types of systems, namely, autonomous systems
x(k + 1) = g
a
(x(k), w(k)), (11.27)
and systems subject to external controllable inputs
x(k + 1) = g(x(k), u(k), w(k)). (11.28)
Both systems are subject to the disturbance w(k) and to the constraints
x(k) A, u(k) |, w(k) J k 0. (11.29)
The sets A and | are and J are polyhedra.
Denition 11.14 For the autonomous system (11.27) we will denote the one-step
robust controllable set to the set o as
Pre(o, J) = x R
n
: g
a
(x, w) o, w J. (11.30)
Pre(o, J) denes the set of system (11.27) states which evolve into the target set
o in one time step for all possible disturbance w J.
Denition 11.15 For the system (11.28) we will denote the one-step robust con-
trollable set to the set o as
Pre(o, J) = x R
n
: u | s.t. g(x, u, w) o, w J. (11.31)
For a system with inputs, Pre(o, J) is the set of states which can be robustly
driven into the target set o in one time step for all admissible disturbances.
Denition 11.16 For the autonomous system (11.27) we will denote the one-step
robust reachable set from the set o as
Reach(o, J) = x R
n
: x(0) o, w J such that x = g
a
(x(0), w).
Denition 11.17 For the system (11.28) with inputs we will denote the one-step
robust reachable set from the set o as
Reach(o, J) = x R
n
: x(0) o, u |, w J, such that x = g(x(0), u, w).
Thus, all the states contained in o are mapped into the set Reach(o, J) under the
map g
a
for all disturbances w J, and under the map g for all inputs u | and
for all disturbances w J.
Remark 11.7 Note that we in our notation we use the word controllable for the
Pre(S, V) set even if the system is autonomous.
The sets Pre(S, V) and Reach(S, V) are also denoted as one-step robust backward-
reachable set and one-step robust forward-reachable set, respectively, in the litera-
ture.
N-step robust controllable and robust reachable sets are dened by iterating
Pre(, ) and Reach(, ) computations, respectively.
Denition 11.18 (N-Step Robust Controllable Set /
N
(O, J)) For a given
target set O A, the N-step robust controllable set /
N
(O, J) of the system
(11.27) or (11.28) subject to the constraints (11.29) is dened recursively as:
/
j
(O, J) = Pre(/
j1
(O, J), J) A, /
0
(O, J) = O, j 1, . . . , N.
0
belonging to the N-Step Robust Controllable
Set /
N
(O, J) can be robustly driven, through a time-varying control law, to the
target set O in N steps, while satisfying input and state constraints for all possible
disturbances.
N-step robust reachable sets are dened analogously to N-step robust control-
lable set.
Denition 11.19 (N-Step Robust Reachable Set 1
N
(A
0
, J)) For a given ini-
tial set A
0
A, the N-step robust reachable set 1
N
(A
0
, J) of the system (11.27)
or (11.28) subject to the constraints (11.29) is dened recursively as:
1
i+1
(A
0
, J) = Reach(1
i
(A
0
, J), J), 1
0
(A
0
, J) = A
0
, i = 0, . . . , N 1.
0
belonging to A
0
will evolve to the N-step robust
reachable set 1
N
(A
0
, J) in N steps.
Next we will show through simple examples the main steps involved in the
computation of robust controllable and robust reachable sets for certain classes of
uncertain constrained linear systems.
11.3.1 Linear Systems with Additive Uncertainty and without Inputs
Example 11.8 Consider the second order autonomous system
7
x(t + 1) = Ax(t) +w(t) =
_
0.5 0
1 0.5
_
x(t) +w(t) (11.32)
subject to the state constraints
x(t) . =
_
x :
_
10
10
_
x
_
10
10
__
, t 0, (11.33)
and where the additive disturbance belongs to the set
w(t) V =
_
w :
_
1
1
_
w
_
1
1
__
, t 0. (11.34)
The set Pre(., V) can be obtained as described next. Since the set . is a polytope,
it can be represented as an -polytope (Section 5.2)
. = |x : Hx h, (11.35)
7
where
H =
_
_
_
_
1 0
0 1
1 0
0 1
_
_
and h =
_
_
_
_
10
10
10
10
_
_
.
By using this -presentation and the system equation (11.32), the set Pre(., V) can
be rewritten as
Pre(., V) = |x : Hga(x, w) h, w V (11.36a)
= |x : HAx h Hw, w V . (11.36b)
The set (11.36) can be represented as a the following polyhedron
Pre(., V) = |x R
n
: HAx

h (11.37)
with
hi = min
wW
(hi Hiw). (11.38)
In general, a linear program is required to solve problems (11.38). In this example Hi
and V have simple expressions and we get

h =
_
_
_
_
9
9
9
9
_
_
. The set (11.37) might contain
redundant inequalities which can be removed by using Algorithm 5.1 in Section 5.4.1
to obtain its minimal representation.
The set Pre(., V) is
Pre(., V) =
_
_
x :
_
_
_
_
1 0
1 0.5
1 0
1 0.5
_
_
x
_
_
_
_
18
9
18
9
_
_
_
_
.
The set Pre(., V) ., the signicance of which we will discuss below, is
Pre(., V) . =
_
_
x :
_
_
_
_
_
_
_
_
1 0
0 1
1 0
0 1
1 0.5
1 0.5
_
_
x
_
_
_
_
_
_
_
_
10
10
10
10
9
9
_
_
_
_
Note that by using the denition of Pontryagin dierence given in Section 5.4.8 and
ane operations on polyhedra in Section 5.4.11 we can compactly summarize the
operations in (11.36) and write the set Pre in (11.30) as
Pre(., V) = |x R
n
: Ax +w ., w V = |x R
n
: Ax . V =
= (. V) A.
The set
Reach(., V) = |y : x ., w V such that y = Ax +w (11.39)
x
1
x
2
A
Pre(A, J)
-10 -5 0 5 10
-10
-5
0
5
10
Figure 11.9 Example 11.8: one-step robust controllable set Pre(., V) .
for system (11.32) under constraints (11.33)-(11.34). Download code.
is obtained by applying the map A to the set . and then considering the eect of
the disturbance w V. Let us write . in 1-representation (see section 5.1)
. = conv(V ), (11.40)
and let us map the set of vertices V through the transformation A. Because the
transformation is linear, the composition of the map A with the set ., denoted as
A ., is simply the convex hull of the transformed vertices
A . = conv(AV ). (11.41)
We refer the reader to Section 5.4.11 for a detailed discussion on linear transforma-
tions of polyhedra. Rewrite (11.39) as
Reach(., V) = |y R
n
: z A ., w V such that y = z +w.
We can use the denition of Minkowski sum given in Section 5.4.9 and rewrite the
Reach set as
Reach(., V) = (A .) V.
We can compute the Minkowski sum via projection or vertex enumeration as ex-
plained in Section 5.4.9. The set Reach(., V) in -representation is
Reach(., V) =
_
_
x :
_
_
_
_
_
_
_
_
1 0.5
0 1
1 0
1 0.5
0 1
1 0
_
_
x
_
_
_
_
_
_
_
_
4
16
6
4
16
6
_
_
_
_
,
11.3.2 Linear Systems with Additive Uncertainty and Inputs
Example 11.9 Consider the second order unstable system
8
_
x(t + 1) = Ax +Bu =
_
1.5 0
1 1.5
_
x(t) +
_
1
0
_
u(t) +w(t) (11.42)
8
x
1
x
2
A
R
e
a
c
h
(
A
,
J
)
-10 0 10
-10
0
10
Figure 11.10 Example 11.8: one-step robust reachable set for system (11.32)
under constraints (11.34). Download code.
subject to the input and state constraints
u(t) | = |u : 5 u 5 , t 0 (11.43a)
x(t) . =
_
x :
_
10
10
_
x
_
10
10
__
, t 0, (11.43b)
where
w(t) V = |w : 1 w 1 , t 0. (11.44)
For the non-autonomous system (11.42), the set Pre(., V) can be computed using
the -presentation of . and |,
. = |x : Hx h, | = |u : Huu hu, (11.45)
to obtain
Pre(., V) =
_
x R
2
: u | s.t. Ax +Bu +w ., w V
_
(11.46a)
=
_
x R
2
: u R s.t.
_
HA HB
0 Hu
_ _
x
u
_
_
h Hw
hu
_
, w V
_
.
(11.46b)
As in example (11.8), the set Pre(., V) can be compactly written as
Pre(., V) =
_
x R
2
: u R s.t.
_
HA HB
0 Hu
_ _
x
u
_
_

h
hu
__
, (11.47)
where
hi = min
wW
(hi Hiw). (11.48)
In general, a linear program is required to solve problems (11.48). In this example
Hi and V have simple expressions and we get

h =
_
_
_
_
9
9
9
9
_
_
.
The halfspaces in (11.47) dene a polytope in the state-input space, and a projection
operation (see Section 5.4.6) is used to derive the halfspaces which dene Pre(., V)
in the state space. The set Pre(., V) . is depicted in Fig. 11.11 and reported
below:
_
_
_
_
_
_
_
_
1 0
1 0
1 1.5
1 1.5
0 1
0 1
_
_
x
_
_
_
_
_
_
_
_
9.3
9.3
9
9
10
10
_
_
.
x
1
x
2
A
Pre(A, J)
-10 -5 0 5 10
-10
-5
0
5
10
Figure 11.11 Example 11.9: one-step robust controllable set Pre(., V) .
for system (11.42) under constraints (11.43)-(11.44). Download code.
Note that by using the denition of a Minkowski sum given in Section 5.4.9
and the ane operation on polyhedra in Section 5.4.11 we can compactly write the
operations in (11.46) as follows:
Pre(A, J) = x : u | s.t. Ax +Bu +w A, w J
= x : y A, u | s.t. y = Ax +Bu +w, w J
= x : y A, u | s.t. Ax = y + (Bu) w, w J
= x : Ax c and c = A (B) | J
= x : x c A, c = A (B) | J
= x : x ((A J) (B |)) A .
(11.49)
Remark 11.8 Note that in (11.49) we have used the fact that if a set S is described
as S = |v : z ?, s.t. v = z w, w V, then S = |v : z ?, s.t. z =
v + w, w V or S = |v : v + w ?, w V = ? V. Also, to derive
the last equation of (11.49) we have used the associative property of the Pontryagin
dierence.
The set Reach(A, J) = y : x A, u |, w J s.t. y = Ax +Bu +w
is obtained by applying the map A to the set A and then considering the eect
of the input u | and of the disturbance w J. We can use the denition of
Minkowski sum given in Section 5.4.9 and rewrite Reach(A, J) as
Reach(A, J) = (A A) (B |) J.
x
1
x
2
Reach(A, J)
-40 -20 0 20 40
-40
-20
0
20
40
Figure 11.12 Example 11.9: one-step robust reachable set for system (11.42)
under constraints (11.43)-(11.44). Download code.
The set Reach(A, J) is depicted in Fig. 11.12.
In summary, for linear systems with additive disturbances the sets Pre(A, J)
and Reach(A, J) are the results of linear operations on the polytopes A, | and
J and therefore are polytopes. By using the denition of Minkowski sum given
in Section 5.4.9, Pontryagin dierence given in Section 5.4.8 and ane operation
on polyhedra in Section 5.4.11 we can compactly summarize the operations in
Table 11.2. Note that the summary in Table 11.2 applies also to the class of
systems x(k +1) = Ax(t) +Bu(t) +E

d(t) where

d

J. This can be transformed
into x(k + 1) = Ax(t) +Bu(t) +w(t) where w J = E

J.
x(t + 1) = Ax(t) +w(t) x(t + 1) = Ax(t) +Bu(t) +w(t)
Pre(A, J) (A J) A ((A J) (B |)) A
Reach(A), J (A A) J (A A) (B |) J
Table 11.2 Pre and Reach operations for uncertain linear systems subject
to polyhedral input and state constraints x(t) ., u(t) | with additive
polyhedral disturbances w(t) V
The N-step robust controllable set /
N
(o, J) and the N-step robust
reachable set 1
N
(A
0
, J) can be computed by using their recursive for-
mulas and computing the Pre and Reach operations as described in Ta-
ble 11.2.
11.3.3 Linear Systems with Parametric Uncertainty
The next Lemma 11.1 will help us computing Pre and Reach sets for linear systems
with parametric uncertainty.
Lemma 11.1 Let g : R
nz
R
n
R
nw
R
ng
be a function of (z, x, w) convex in w
for each (z, x). Assume that the variable w belongs to the polytope J with vertices
w
i
nW
i=1
. Then, the constraint
g(z, x, w) 0 w J (11.50)
is satised if and only if
g(z, x, w
i
) 0, i = 1, . . . , n
W
. (11.51)
Proof: Easily follows by the fact that the maximum of a convex function over
a compact convex set is attained at an extreme point of the set.
Lemma 11.2 shows how to reduce the number of constraints in (11.51) for a
specic class of constraint functions.
Lemma 11.2 Assume g(z, x, w) = g
1
(z, x) + g
2
(w). Then the constraint (11.50)
can be replaced by g
1
(z, x) g, where g =
_
g
1
, . . . , g
ng
is a vector whose i-th

component is
g
i
= max
wW
g
2
i
(w), (11.52)
and g
2
i
(w) denotes the i-th component of g
2
(w).
9
x(t + 1) = A(w
p
(t))x(t) +w
a
(t) =
_
0.5 +w
p
(t) 0
1 0.5
_
x(t) +w
a
(t)
(11.53)
x(t) . =
_
x :
_
10
10
_
x
_
10
10
__
, t 0
w
a
(t) V
a
=
_
w
a
:
_
1
1
_
w
a
_
1
1
__
, t 0
w
p
(t) V
p
= |w
p
: 0 w
p
0.5 , t 0.
(11.54)
Let w = [w
a
; w
p
] and V = V
a
V
p
. The set Pre(., V) can be obtained as follows.
The set . is a polytope and it can be represented as an -polytope (Section 5.2)
. = |x : Hx h, (11.55)
where
H =
_
_
_
_
1 0
0 1
1 0
0 1
_
_
and h =
_
_
_
_
10
10
10
10
_
_
.
By using this -presentation and the system equation (11.53), the set Pre(., V) can
be rewritten as
Pre(., V) =
_
x : Hga(x, w) h, w V
_
(11.56)
= |x : HA(w
p
)x h Hw
a
, w
a
V
a
, w
p
V
p
.
(11.57)
9
By using Lemmas 11.1 and 11.2, the set (11.57) can be rewritten as a the polytope
x Pre(., V) = |x R
n
:
_
HA(0)
HA(0.5)
_
x
_
h
_
(11.58)
with
hi = min
w
a
W
a
(hi Hiw
a
). (11.59)
The set Pre(., V) . is depicted in Fig. 11.13 and reported below:
_
_
_
_
_
_
_
_
1 0
0.89 0.44
1 0
0.89 0.44
0 1
0 1
_
_
x
_
_
_
_
_
_
_
_
9
8.049
9
8.049
10
10
_
_
.
x
1
x
2
A
Pre(A, J) A
-10 -5 0 5 10
-10
-5
0
5
10
Figure 11.13 Example 11.10: one-step robust controllable set Pre(., V).
for system (11.53) under constraints (11.54). Download code.
10
x(t + 1) = A(w
p
(t))x(t) =
_
0.5 +w
p
(t) 0
1 0.5
_
x(t) (11.60)
x(t) . =
_
x :
_
10
10
_
x
_
10
10
__
, t 0
w
p
(t) V
p
= |w
p
: 0 w
p
0.5 , t 0.
(11.61)
Let w = [w
p
] and V = V
p
. The set Reach(., V) can be written as innite union
of reachable sets
Reach(., V) =
_
wW
Reach(., w) (11.62)
where Reach(., w) is computed by as described in Example 11.1 for the deterministic
system A( w). In general, the union in equation (11.62) generates a non-convex set,
as can be seen in Fig. 11.14. Non convexity of one-step reachable sets for parametric
linear systems is also discussed in [58][Sec. 6.1.2].
10
x
1
x
2
R
e
a
c
h
(
A
,
J
)
-10 0 10
-10
0
10
Figure 11.14 Example 11.10: one-step robust reachable set Pre(., V) .
for system (11.60) under constraints (11.61). Download code.
11.4 Robust Invariant Sets
Two dierent types of sets are considered in this chapter: robust invariant sets and
robust control invariant sets. We will rst discuss robust invariant sets.
Robust Positive Invariant Sets
Robust Invariant sets are computed for autonomous systems. These types of sets
are useful to answer questions such as: For a given feedback controller u = f(x),
nd the set of states whose trajectory will never violate the system constraints for
all possible disturbances. The following denitions introduce the dierent types
of robust invariant sets.
Denition 11.20 (Robust Positive Invariant Set) A set O A is said to be
a robust positive invariant set for the autonomous system (11.27) subject to the
constraints (11.29), if
x(0) O x(t) O, w(t) J, t N
+
.
Denition 11.21 (Maximal Robust Positive Invariant Set O
) The set O

A is the maximal robust invariant set of the autonomous system (11.27) subject
to the constraints (11.29) if O
is a robust invariant set and O
contains all the

robust positive invariant sets contained in A.
Theorem 11.2 (Geometric condition for invariance) A set O A is a ro-
bust positive invariant set for the autonomous system (11.27) subject to the con-
straints (11.29), if and only if
O Pre(O, J). (11.63)
The proof of Theorem 11.2 follows the same lines of the proof of Theorem 11.1.
11.4 Robust Invariant Sets 209
It is immediate to prove that condition (11.63) of Theorem 11.2 is equivalent
to the following condition
Pre(O, J) O = O. (11.64)
Based on condition (11.64), the following algorithm provides a procedure for com-
puting the maximal robust positive invariant subset O
for system (11.27)-(11.29)

(for reference to proofs and literature see Chapter 11.1).
Algorithm 11.4 Computation of O
input ga , ., V
output O
let 0 .
repeat
k = k + 1
k+1
Pre(
k
, V)
k
until
k+1
=
k
O
k
k
satisfying
k+1

k
, k N and
it terminates when
k+1
=
k
k
is the maximal robust
positive invariant set O
for system (11.27)-(11.29). If

k
= for some integer k
then the simple conclusion is that O
= .
In general, Algorithm 11.4 may never terminate. If the algorithm does not
terminate in a nite number of iterations, it can be proven that [176]
O
= lim
k+
k
.
Conditions for nite time termination of Algorithm 11.4 can be found in [120]. A
simple sucient condition for nite time termination of Algorithm 11.1 requires the
system g
a
(x, w) to be linear and stable, and the constraint set A and disturbance
set J to be bounded and to contain the origin.
Example 11.12 Consider the second order stable system in Example 11.8
11
x(t + 1) = Ax(t) +w(t) =
_
0.5 0
1 0.5
_
x(t) +w(t) (11.65)
x(t) . =
_
x :
_
10
10
_
x
_
10
10
__
, t 0
w(t) V =
_
w :
_
1
1
_
w
_
1
1
__
, t 0.
(11.66)
11
The maximal robust positive invariant set of system (11.65) subject to constraints (11.66)
is depicted in Fig. 11.15 and reported below:
_
_
_
_
_
_
_
_
0.89 0.44
0.89 0.44
1 0
0 1
1 0
0 1
_
_
x
_
_
_
_
_
_
_
_
8.04
8.04
10
10
10
10
_
_
.
x
1
x
2
A
O
-10 -5 0 5 10
-10
-5
0
5
10
Figure 11.15 Maximal Robust Positive Invariant Set of system (11.65) sub-
ject to constraints (11.66). Download code.
Robust Control Invariant Sets
Robust control invariant sets are dened for systems subject to controllable inputs.
These types of sets are useful to answer questions such as: Find the set of states for
which there exists a controller such that the system constraints are never violated
for all possible disturbances. The following denitions introduce the dierent
types of robust control invariant sets.
Denition 11.22 (Robust Control Invariant Set) A set c A is said to be a
robust control invariant set for the system (11.28) subject to the constraints (11.29),
if
x(t) c u(t) | such that g(x(t), u(t), w(t)) c, w(t) J, t N
+
.
Denition 11.23 (Maximal Robust Control Invariant Set c
) The set c

A is said to be the maximal robust control invariant set for the system (11.28) sub-
ject to the constraints (11.29), if it is robust control invariant and contains all
robust control invariant sets contained in A.
Remark 11.9 The geometric conditions for invariance (11.63)-(11.64) hold for control
invariant sets.
The following algorithm provides a procedure for computing the maximal robust
control invariant set c
for system (11.28)-(11.29).

Algorithm 11.5 Computation of c
input: g, ., | and V
output: c
let 0 .
repeat
k = k + 1
k+1
Pre(
k
, V)
k
until
k+1
=
k
c
k+1
k
satisfying
k+1

k
, k N.
Algorithm 11.5 terminates when
k+1
=
k
k
is the
maximal robust control invariant set c
for the system (11.28)-(11.29).

In general, Algorithm 11.5 may never terminate [11, 49, 170, 164]. If the algo-
rithm does not terminate in a nite number of iterations, in general, convergence
to the maximal robust control invariant set is not guaranteed
c
,= lim
k+
k
. (11.67)
The work in [50] reports examples of nonlinear systems where (11.67) can be ob-
served. A sucient condition for the convergence of
k
to c
as k +requires
the polyhedral sets A, | and J to be bounded and the system g(x, u, w) to be
continuous [50].
Example 11.13 Consider the second order unstable system in example 11.9
12
. The
maximal robust control invariant set of system (11.42) subject to constraints (11.43)-
(11.44) is an empty set. If the uncertain set (11.44) is replaced with
w(t) V = |w : 0.1 w 0.1 , t 0
the maximal robust control invariant set is
_
_
_
_
0 1
0 1
0.55 0.83
0.55 0.83
_
_
x
_
_
_
_
3.72
3.72
2.0
2.0
_
_
which is depicted in Fig. 11.16.
Denition 11.24 (Finitely determined set) Consider Algorithm 11.4 (11.5).
The set O
(c
) is nitely determined if and only if i N such that

i+1
=
i
.
The smallest element i N such that
i+1
=
i
is called the determinedness index.
12
x
1
x
2
A
c
-10 -5 0 5 10
-10
-5
0
5
10
Figure 11.16 Maximal Robust Control Invariant Set of system (11.42) sub-
ject to constraints (11.43). Download code.
For all states contained in the maximal robust control invariant set c
there
exists a control law, such that the system constraints are never violated for all
feasible disturbances. This does not imply that there exists a control law which
can drive the state into a user-specied target set. This issue is addressed in the
following by introducing the concept of robust controllable and stabilizable sets.
Denition 11.25 (Maximal Robust Controllable Set /
(O, J)) For a given

target set O A, the maximal robust controllable set /
(O, J) for the system

(11.28) subject to the constraints (11.29) is the union of all N-step robust control-
lable sets contained in A (N N).
Robust controllable sets /
N
(O, J) where the target O is a robust control invariant
set are special sets, since in addition to guaranteeing that from /
N
(O, J) we
robustly reach O in N steps, one can ensure that once reached O, the system can
stay there at all future time instants and for all possible disturbance realizations.
Denition 11.26 (N-step (Maximal) Robust Stabilizable Set) For a given
robust control invariant set O A, the N-step (maximal) robust stabilizable set
of the system (11.28) subject to the constraints (11.29) is the N-step (maximal)
robust controllable set /
N
(O, J) (/
(O, J)).
The set /
(O, J) contains all states which can be robustly steered into the
robust control invariant set O and hence /
(O, J) c
. The set /
(O, J)
c
can be computed as follows:

Algorithm 11.6 The Maximal Robust Stabilizable Set /(O, V)
let /0 O, where O is a robust control invariant set
repeat
c = c + 1
/c+1 Pre(/c, V)
.
until /c+1 = /c,
/(O, V) = /c,
Since O is robust control invariant, it holds c N that /
c
is robust control
invariant and /
c
/
c+1
. Note that Algorithm 11.6 is not guaranteed to terminate
in nite time.
Chapter 12
Constrained Optimal Control
In this chapter we study the nite time and innite time optimal control problem for
linear systems with linear constraints on inputs and state variables. We establish
the structure of the optimal control law and derive algorithms for its computation.
For nite time problems with linear and quadratic objective functions we show
that the time varying feedback law is piecewise ane and continuous. The value
function is a convex piecewise linear, resp. piecewise quadratic function of the
initial state.
We describe how the optimal control action for a given initial state can be
computed by means of linear or quadratic programming. We also describe how
the optimal control law can be computed by means of multiparametric linear or
quadratic programming. Finally we show how to compute the innite time optimal
controller for quadratic and linear objective functions and prove that, when it
exists, the innite time controller inherits all the structural properties of the nite
time optimal controller.
Consider the linear time-invariant system
x(t + 1) = Ax(t) +Bu(t), (12.1)
where x(t) R
n
, u(t) R
m
are the state and input vectors, respectively, subject
to the constraints
x(t) A, u(t) |, t 0. (12.2)
The sets A R
n
and | R
m
are polyhedra.
Remark 12.1 The results of this chapter also hold for more general forms of linear
constraints such as mixed input and state constraints
[x(t)
, u(t)
] Tx,u, (12.3)
216 12 Constrained Optimal Control
where Tx,u is a polyhedron in R
n+m
or mixed input and state constraints over a nite
time
[x(0)
, . . . , x(N 1)
, u(0)
, . . . , u(N 1)
] Tx,u,N, (12.4)
where Tx,u,N is a polyhedron in R
N(n+m)
. Note that constraints of the type (12.4)
can arise, for example, from constraints on the input rate u(t) = u(t) u(t 1). In
this chapter, for the sake of simplicity, we will use the less general form (12.2).
Dene the cost function
J
0
(x(0), U
0
) = p(x
N
) +
N1
k=0
q(x
k
, u
k
), (12.5)
where x
k
x
0
x
k+1
= Ax
k
+Bu
k
(12.6)
0
= [u
0
, . . . , u
N1
]
.
If the 1-norm or -norm is used in the cost function (12.5), then we set
p(x
N
) = |Px
N
|
p
and q(x
k
, u
k
) = |Qx
k
|
p
+ |Ru
k
|
p
with p = 1 or p = and P,
Q, R full column rank matrices. Cost (12.5) is rewritten as
J
0
(x(0), U
0
) = |Px
N
|
p
+
N1
k=0
|Qx
k
|
p
+|Ru
k
|
p
. (12.7)
If the squared Euclidian norm is used in the cost function (12.5), then we set
p(x
N
) = x
N
Px
N
and q(x
k
, u
k
) = x
k
Qx
k
+ u
k
Ru
k
with P _ 0, Q _ 0 and R 0.
Cost (12.5) is rewritten as
J
0
(x(0), U
0
) = x
N
Px
N
+
N1
k=0
x
k
Qx
k
+u
k
Ru
k
. (12.8)
Consider the constrained nite time optimal control problem (CFTOC)
J
0
(x(0)) = min
U0
J
0
(x(0), U
0
)
subj. to x
k+1
= Ax
k
+Bu
k
, k = 0, . . . , N 1
x
k
A, u
k
|, k = 0, . . . , N 1
x
N
A
f
x
0
= x(0),
(12.9)
where N is the time horizon and A
f
R
n
is a terminal polyhedral region. In (12.5)
(12.9) U
0
= [u
0
, . . . , u
N1
]
R
s
, s = mN is the optimization vector. We denote
with A
0
A the set of initial states x(0) for which the optimal control prob-
lem (12.5)(12.9) is feasible, i.e.,
A
0
= x
0
R
n
: (u
0
, . . . , u
N1
) such that x
k
A, u
k
|, k = 0, . . . , N 1,
x
N
A
f
where x
k+1
= Ax
k
+ Bu
k
, k = 0, . . . , N 1.
(12.10)
12.1 Problem Formulation 217
Remark 12.2 Note that we distinguish between the current state x(k) of system (12.1)
at time k and the variable x
k
in the optimization problem (12.9), that is the predicted
state of system (12.1) at time k obtained by starting from the state x0 = x(0) and
applying to system (12.6) the input sequence u0, . . . , u
k1
. Analogously, u(k) is the
input applied to system (12.1) at time k while u
k
is the k-th optimization variable of
the optimization problem (12.9).
If we use cost (12.8) with the squared Euclidian norm and set
(x, u) R
n+m
: x A, u | = R
n+m
, A
f
= R
n
, (12.11)
problem (12.9) becomes the standard unconstrained nite time optimal control
problem (Chapter 9) whose solution (under standard assumptions on A, B, P, Q
and R) can be expressed through the time varying state feedback control law (9.27)
u
(k) = F
k
x(k) k = 0, . . . , N 1. (12.12)
The optimal cost is given by
J
0
(x(0)) = x(0)
P
0
x(0). (12.13)
If we let N as discussed in Section 9.5, then problem (12.8)(12.9)(12.11)
becomes the standard innite horizon linear quadratic regulator (LQR) problem
whose solution (under standard assumptions on A, B, P, Qand R) can be expressed
as the state feedback control law (cf. (9.32))
u
(k) = F
x(k), k = 0, 1, . . . (12.14)
In the following chapters we will show that the solution to problem (12.9) can
again be expressed in feedback form where now u
(k) is a continuous piecewise

ane function on polyhedra of the state x(k), i.e., u
(k) = f
k
(x(k)) where
f
k
(x) = F
j
k
x +g
j
k
if H
j
k
x K
j
k
, j = 1, . . . , N
r
k
. (12.15)
Matrices H
j
k
and K
j
k
in equation (12.15) describe the j-th polyhedron CR
j
k
=
x R
n
[H
j
k
x K
j
k
inside which the feedback optimal control law u
(k) at time
k has the ane form F
j
k
x + g
j
k
. The set of polyhedra CR
j
k
, j = 1, . . . , N
r
k
is a
polyhedral partition of the set of feasible states A
k
of problem (12.9) at time k. The
sets A
k
are discussed in detail in the next section. Since the functions f
k
(x(k)) are
continuous, the use of polyhedral partitions rather than strict polyhedral partitions
(Denition 5.5) will not cause any problem, indeed it will simplify the exposition.
In the rest of this chapter we will characterize the structure of the value function
and describe how the optimal control law can be eciently computed by means of
multiparametric linear and quadratic programming. We will distinguish the cases
1- or -norm and squared 2-norm.
12.2 Feasible Solutions
We denote with A
i
the set of states x
i
at time i for which (12.5)(12.9) is feasible,
for i = 0, . . . , N. The sets A
i
for i = 0, . . . , N play an important role in the
the solution of (12.9). They are independent of the cost function (as long as it
guarantees the existence of a minimum) and of the algorithm used to compute the
solution to problem (12.9). There are two ways to rigourously dene and compute
the sets A
i
: the batch approach and the recursive approach. In the batch approach
A
i
= x
i
A : (u
i
, . . . , u
N1
) such that x
k
A, u
k
|, k = i, . . . , N 1,
x
N
A
f
where x
k+1
= Ax
k
+Bu
k
, k = i, . . . , N 1.
(12.16)
The denition of A
i
in (12.16) requires that for any initial state x
i
A
i
there exists
a feasible sequence of inputs U
i
= [u
i
, . . . , u
N1
] which keeps the state evolution
in the feasible set A at future time instants k = i + 1, . . . , N 1 and forces x
N
into A
f
at time N. Clearly A
N
= A
f
. Next we show how to compute A
i
for
i = 0, . . . , N 1. Let the state and input constraint sets A, A
f
and | be the 1-
polyhedra A
x
x b
x
, A
f
x
N
b
f
, A
u
u b
u
, respectively. Dene the polyhedron
T
i
for i = 0, . . . , N 1 as follows
T
i
= (U
i
, x
i
) R
m(Ni)+n
: G
i
U
i
E
i
x
i
w
i
, (12.17)
where G
i
, E
i
and w
i
are dened as follows
G
i
=
_
_
A
u
0 . . . 0
0 A
u
. . . 0
.
.
.
.
.
.
.
.
.
.
.
.
0 0 . . . A
u
0 0 . . . 0
A
x
B 0 . . . 0
A
x
AB A
x
B . . . 0
.
.
.
.
.
.
.
.
.
.
.
.
A
f
A
Ni1
B A
f
A
Ni2
B . . . A
f
B
_
_
, E
i
=
_
_
0
0
.
.
.
0
A
x
A
x
A
A
x
A
2
.
.
.
A
f
A
Ni
_
_
, w
i
=
_
_
b
u
b
u
.
.
.
b
u
b
x
b
x
b
x
.
.
.
b
f
_
_
.
(12.18)
The set A
i
is a polyhedron as it is the projection of the polyhedron T
i
in (12.17)-
(12.18) on the x
i
space.
In the recursive approach,
A
i
= x A : u | such that Ax +Bu A
i+1
,
i = 0, . . . , N 1
A
N
= A
f
. (12.19)
The denition of A
i
in (12.19) is recursive and requires that for any feasible initial
state x
i
A
i
there exists a feasible input u
i
which keeps the next state Ax
i
+Bu
i
in the feasible set A
i+1
. It can be compactly written as
A
i
= Pre(A
i+1
) A. (12.20)
12.2 Feasible Solutions 219
Initializing A
N
to A
f
and solving (12.19) backward in time yields the same sets
A
i
as the batch approach. This recursive method, however, leads to an alternative
approach for computing the sets A
i
. Let A
i
be the 1-polyhedra A
Xi
x b
Xi
. Then
the set A
i1
is the projection of the following polyhedron
_
_
A
u
0
A
Xi
B
_
_
u
i
+
_
_
0
A
x
A
Xi
A
_
_
x
i

_
_
b
u
b
x
b
Xi
_
_
(12.21)
on the x
i
space.
Consider problem (12.9). The set A
0
is the set of all initial states x
0
for
which (12.9) is feasible. The sets A
i
with i = 1, . . . , N 1 are hidden. A
given

U
0
= [ u
0
, . . . , u
N1
] is feasible for problem (12.9) if and only if at all time
instants i, the state x
i
obtained by applying u
0
, . . . , u
i1
to the system model
x
k+1
= Ax
k
+ Bu
k
with initial state x
0
A
0
belongs to A
i
. Also, A
i
is the set of
feasible initial states for problem
J
i
(x(0)) = min
Ui
p(x
N
) +
N1
k=i
q(x
k
, u
k
)
subj. to x
k+1
= Ax
k
+Bu
k
, k = i, . . . , N 1
x
k
A, u
k
|, k = i, . . . , N 1
x
N
A
f
.
(12.22)
Next, we provide more insights into the set A
i
by using the invariant set theory
of Section 11.2. We consider two cases: (1) A
f
= A which corresponds to eectively
removing the terminal constraint set and (2) A
f
chosen to be a control invariant
set.
Theorem 12.1 [170, Theorem 5.3]. Let the terminal constraint set A
f
be equal
to A. Then,
1. The feasible set A
i
, i = 0, . . . , N 1 is equal to the (N i)-step controllable
set:
A
i
= /
Ni
(A).
i
, i = 0, . . . , N 1 contains the maximal control invariant
set:
c
A
i
.
i
is control invariant if and only if the maximal control
invariant set is nitely determined and N i is equal to or greater than its
determinedness index

N, i.e.
A
i
Pre(A
i
) c
= /
Ni
(A) for all i N

N.
4. A
i
A
j
if i < j for i = 0, . . . , N 1. The size of the feasible set A
i
stops
decreasing (with decreasing i) if and only if the maximal control invariant set
is nitely determined and N i is larger than its determinedness index, i.e.
A
i
A
j
if N

N < i < j < N.
Furthermore,
A
i
= c
if i N

N.
Theorem 12.2 [170, Theorem 5.4]. Let the terminal constraint set A
f
be a control
invariant subset of A. Then,
i
, i = 0, . . . , N 1 is equal to the (N i)-step stabilizable
set:
A
i
= /
Ni
(A
f
).
i
, i = 0, . . . , N1 is control invariant and contained within
the maximal control invariant set:
A
i
c
.
3. A
i
A
j
if i < j, i = 0, . . . , N 1. The size of the feasible A
i
set stops
increasing (with decreasing i) if and only if the maximal stabilizable set is
nitely determined and N i is larger than its determinedness index, i.e.
A
i
A
j
if N

N < i < j < N.
Furthermore,
A
i
= /
(A
f
) if i N

N.
Remark 12.3 Theorems 12.1 and 12.2 help us understand how the feasible sets .i
propagate backward in time as a function of the terminal set .
f
. In particular,
when .
f
= . the set .i shrinks as i becomes smaller and stops shrinking when it
becomes the maximal control invariant set. Also, depending on i, either it is not a
control invariant set or it is the maximal control invariant set. We have the opposite
if a control invariant set is chosen as terminal constraint .
f
. The set .i grows as
i becomes smaller and stops growing when it becomes the maximal stabilizable set.
Both cases are shown in the Example 12.1 below.
Remark 12.4 In this section we investigated the behavior of .i as i varies for a
xed horizon N. Equivalently, we could study the behavior of .0 as the horizon N
varies. Specically, the sets .0N
1
and .0N
2
with N2 > N1 are equal to the sets
.N
2
N
1
N and .0N, respectively, with N = N2.
Example 12.1 Consider the double integrator
1
_
_
_
x(t + 1) =
_
1 1
0 1
_
x(t) +
_
0
1
_
u(t)
y(t) =
_
1 0
_
x(t)
(12.23)
subject to the input constraints
1 u(k) 1 for all k 0 (12.24)
and the state constraints
_
5
5
_
x(k)
_
5
5
_
for all k 0. (12.25)
We compute the feasible sets .i and plot them in Figure 12.1 in two cases.
1
Case 1. .
f
is the control invariant set
_
_
_
_
0.32132 0.94697
0.32132 0.94697
1 0
1 0
_
_
x
_
_
_
_
0.3806
0.3806
2.5
2.5
_
_
. (12.26)
After six iterations the sets .i converge to the following /(.
f
)
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
0.44721 0.89443
0.24254 0.97014
0.31623 0.94868
0.24254 0.97014
0.31623 0.94868
0.44721 0.89443
1 0
1 0
0.70711 0.70711
0.70711 0.70711
_
_
x
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
2.6833
2.6679
2.5298
2.6679
2.5298
2.6833
5
5
3.5355
3.5355
_
_
. (12.27)
Note that in this case c = /(.
f
) and the determinedness index is six.
Case 2. .
f
= .. After six iterations the sets .i converge to /(.
f
) in (12.27).
x
1
x
2
-6 -4 -2 0 2 4 6
-6
-4
-2
0
2
4
6
(a) Case 1: .
f
= control invariant
set in (12.26).
x
1
x
2
-6 -4 -2 0 2 4 6
-6
-4
-2
0
2
4
6
(b) Case 2: .
f
= ..
Figure 12.1 Example 12.1: Propagation of the feasible sets .i to arrive at
c (shaded). Download code.
12.3 2-Norm Case Solution
Consider problem (12.9) with J
0
() dened by (12.8). In this chapter we always
assume that Q = Q
_ 0, R = R
0, P = P
_ 0.
J
0
(x(0)) = min
U0
J
0
(x(0), U
0
) = x
N
Px
N
+
N1
k=0
x
k
Qx
k
+u
k
Ru
k
subj. to x
k+1
= Ax
k
+Bu
k
, k = 0, . . . , N 1
x
k
A, u
k
|, k = 0, . . . , N 1
x
N
A
f
x
0
= x(0).
(12.28)
12.3.1 Solution via QP
As shown in Section 9.2, problem (12.28) can be rewritten as
J
0
(x(0)) = min
U0
J
0
(x(0), U
0
) = U
0
HU
0
+ 2x
(0)FU
0
+x
(0)Y x(0)
= min
U0
J
0
(x(0), U
0
) = [U
0
x
(0)]
_
H F
F Y
[U
0
x(0)
subj. to G
0
U
0
w
0
+E
0
x(0),
(12.29)
with G
0
, w
0
and E
0
dened in (12.18) for i = 0 and H, F, Y dened in (9.8). As
J
0
(x(0), U
0
) 0 by denition it follows that
_
H F
F Y
_ 0.
For a given vector x(0) the optimal input sequence U
0
solving problem (12.29)
can be computed by using a Quadratic Program (QP) solver (see Section 3.3 for
QP denition and properties and Chapter 4 for fast numerical methods for solving
QPs).
To obtain the problem (12.29) we have eliminated the state variables and equal-
ity constraints x
k+1
= Ax
k
+ Bu
k
by successive substitution so that we are left
with u
0
, . . . , u
N1
as the only decision variables and x(0) as a parameter vector.
In general, it might be more ecient to solve a QP problem with equality and
inequality constraints so that sparsity can be exploited. To this aim we can dene
the variable z as
z =
_
x
1
. . . x
N
u
0
. . . u
N1
and rewrite problem (12.28) as

J
0
(x(0)) = min
z
[ z
x(0)
]
_

H 0
0 Q
_
[ z
x(0)
subj. to G
0,eq
z = E
0,eq
x(0)
G
0,in
z w
0,in
+E
0,in
x(0)
(12.30)
0
solving problem (12.30) can
be computed by using a Quadratic Program (QP) solver.
To obtain problem (12.30) we have rewritten the equalities from system dy-
12.3 2-Norm Case Solution 223
namics x
k+1
= Ax
k
+Bu
k
as G
0,eq
z = E
0,eq
x(0) where
G
0,eq
=
_
_
I B
A I B
A I B
.
.
.
.
.
.
.
.
.
A I B
_
_
, E
0,eq
=
_
_
A
0
.
.
.
0
_
_
,
and rewritten state and input constraints as G
0,in
z w
0,in
+E
0,in
x(0) where
G
0,in
=
_
_
0 0
A
x
0
.
.
.
.
.
.
A
x
0
A
f
0
0 A
u
0 A
u
.
.
.
.
.
.
0 A
u
0 A
u
_
_
w
0,in
=
_
_
b
x
b
x
.
.
.
b
x
b
f
b
u
b
u
.
.
.
b
u
b
u
_
_
,
E
0,in
=
_
A
x
0 0
,
and constructed the cost matrix

H as
H =
_
_
Q
.
.
.
Q
P
R
.
.
.
R
_
_
.
12.3.2 State Feedback Solution via Batch Approach
As shown in Section 9.2, problem (12.28) can be rewritten as
J
0
(x(0)) = min
U0
J
0
(x(0), U
0
) = [U
0
x
(0)]
_
H F
F Y
[U
0
x(0)
subj. to G
0
U
0
w
0
+E
0
x(0),
(12.31)
with G
0
, w
0
and E
0
dened in (12.18) for i = 0 and H, F, Y dened in (9.8). As
J
0
(x(0), U
0
) 0 by denition it follows that
_
H F
F Y
_ 0. Note that the optimizer

U
0
is independent of the term involving Y in (12.31).
We view x(0) as a vector of parameters and our goal is to solve (12.31) for all
values of x(0) A
0
and to make this dependence explicit. The computation of
the set A
0
of initial states for which problem (12.31) is feasible was discussed in
Section 12.2.
Before proceeding further, it is convenient to dene
z = U
0
+H
1
F
x(0), (12.32)
z R
s
, remove x(0)
Y x(0) and to transform (12.31) to obtain the equivalent

problem
(x(0)) = min
z
z
Hz
subj. to G
0
z w
0
+S
0
x(0),
(12.33)
where S
0
= E
0
+G
0
H
1
F
, and

J
(x(0)) = J
0
(x(0))x(0)
(Y FH
1
F
)x(0). In
the transformed problem the parameter vector x(0) appears only on the right-hand
side of the constraints.
Problem (12.33) is a multiparametric quadratic program that can be solved by
using the algorithm described in Section 7.3.1. Once the multiparametric prob-
lem (12.33) has been solved, the solution U
0
= U
0
(x(0)) of CFTOC (12.28) and
therefore u
(0) = u
(x(0)) is available explicitly as a function of the initial state

x(0) for all x(0) A
0
.
Theorem 7.7 states that the solution z
(x(0)) of the mp-QP problem (12.33) is

a continuous and piecewise ane function on polyhedra of x(0). Clearly the same
properties are inherited by the controller. The following corollaries of Theorem 7.7
establish the analytical properties of the optimal control law and of the value
function.
Corollary 12.1 The control law u
(0) = f
0
(x(0)), f
0
: R
n
R
m
, obtained as a
solution of the CFTOC (12.28) is continuous and piecewise ane on polyhedra
f
0
(x) = F
j
0
x +g
j
0
if x CR
j
0
, j = 1, . . . , N
r
0
, (12.34)
j
0
= x R
n
[H
j
0
x K
j
0
, j = 1, . . . , N
r
0
are a partition
of the feasible polyhedron A
0
.
Proof: From (12.32) U
0
(x(0)) = z
(x(0)) H
1
F
x(0). From Theorem 7.7

we know that z
(x(0)), solution of (12.33), is PPWA and continuous. As U
0
(x(0))
is a linear combination of a linear function and a PPWA function, it is PPWA. As
U
0
(x(0)) is a linear combination of two continuous functions it is continuous. In
particular, these properties hold for the rst component u
(0) of U
0
.
Remark 12.5 Note that, as discussed in Remark 7.9, the critical regions dened
in (7.4) are in general sets that are neither closed nor open. In Corollary 12.1 the
polyhedron CR
i
0
describes the closure of a critical region. The function f0(x) is
continuous and therefore it is simpler to use a polyhedral partition rather than a
strict polyhedral partition.
0
(x(0)) obtained as solution of the CFTOC (12.28)
is convex and piecewise quadratic on polyhedra. Moreover, if the mp-QP prob-
lem (12.33) is not degenerate, then the value function J
0
(x(0)) is C
(1)
.
Proof: By Theorem 7.7

J
(x(0)) is a a convex function of x(0). As

_
H F
F Y
_ 0,
its Schur complement
2
Y FH
1
F
_ 0, and therefore J
0
(x(0)) =

J
(x(0)) +
x(0)
(Y FH
1
F
)x(0) is a convex function, because it is the sum of convex

functions. If the mp-QP problem (12.33) is not degenerate, then Theorem 7.9
implies that

J
(x(0)) is a C
(1)
function of x(0) and therefore J
0
(x(0)) is a C
(1)
function of x(0). The results of Corollary 12.1 imply that J
0
(x(0)) is piecewise
quadratic.
Remark 12.6 The relation between the design parameters of the optimal control
problem (12.28) and the degeneracy of the mp-QP problem (12.33) is complex, in
general.
The solution of the multiparametric problem (12.33) provides the state feedback
solution u
(k) = f
k
(x(k)) of CFTOC (12.28) for k = 0 and it also provides the
open-loop optimal control laws u
(k) as function of the initial state, i.e., u
(k) =
u
(k, x(0)). The state feedback PPWA optimal controllers u
(k) = f
k
(x(k)) with
f
k
: A
k
| for k = 1, . . . , N are computed in the following way. Consider the
same CFTOC (12.28) over the shortened time-horizon [i, N]
min
Ui
x
N
Px
N
+
N1
k=i
x
k
Qx
k
+u
k
Ru
k
subj. to x
k+1
= Ax
k
+Bu
k
, k = i, . . . , N 1
x
k
A, u
k
|, k = i, . . . , N 1
x
N
A
f
x
i
= x(i),
(12.35)
where U
i
= [u
i
, . . . , u
N1
]. As dened in (12.16) and discussed in Section 12.2,
A
i
R
n
is the set of initial states x(i) for which the optimal control problem (12.35)
is feasible. We denote by U
i
the optimizer of the optimal control problem (12.35).
Problem (12.35) can be translated into the mp-QP
min U
i
H
i
U
i
+ 2x
(i)F
i
U
i
+x
(i)Y
i
x(i)
subj. to G
i
U
i
w
i
+E
i
x(i),
(12.36)
where H
i
= H
i
0, F
i
, Y
i
are appropriately dened for each i and G
i
, w
i
, E
i
are
dened in (12.18). The rst component of the multiparametric solution of (12.36)
has the form
u
i
(x(i)) = f
i
(x(i)), x(i) A
i
, (12.37)
where the control law f
i
: R
n
R
m
, is continuous and PPWA
f
i
(x) = F
j
i
x +g
j
i
if x CR
j
i
, j = 1, . . . , N
r
i
, (12.38)
2
Let X =
A B
B C
and A 0. Then X 0 if and only if the Schur complement S =

C BA
1
B
0.
and where the polyhedral sets CR
j
i
= x R
n
[H
j
i
x K
j
i
, j = 1, . . . , N
r
i
are a
partition of the feasible polyhedron A
i
. Therefore the feedback solution u
(k) =
f
k
(x(k)), k = 0, . . . , N 1 of the CFTOC (12.28) is obtained by solving N mp-QP
problems of decreasing size. The following corollary summarizes the nal result.
Corollary 12.3 The state feedback control law u
(k) = f
k
(x(k)), f
k
: A
k
R
n
| R
m
, obtained as a solution of the CFTOC (12.28) and k = 0, . . . , N 1 is
time-varying, continuous and piecewise ane on polyhedra
f
k
(x) = F
j
k
x +g
j
k
if x CR
j
k
, j = 1, . . . , N
r
k
, (12.39)
j
k
= x R
n
: H
j
k
x K
j
k
, j = 1, . . . , N
r
k
are a
k
.
12.3.3 State Feedback Solution via Recursive Approach
Consider the dynamic programming formulation of the CFTOC (12.28)
J
j
(x
j
) = min
uj
x
j
Qx
j
+u
j
Ru
j
+J
j+1
(Ax
j
+Bu
j
)
subj. to x
j
A, u
j
|,
Ax
j
+Bu
j
A
j+1
(12.40)
for j = 0, . . . , N 1, with boundary conditions
J
N
(x
N
) = x
N
Px
N
(12.41)
A
N
= A
f
, (12.42)
where A
j
denotes the set of states x for which the CFTOC (12.28) is feasible at time
j (as dened in (12.16)). Note that according to Corollary 12.2, J
j+1
(Ax
j
+Bu
j
) is
piecewise quadratic for j < N 1. Therefore (12.40) is not simply an mp-QP and,
contrary to the unconstrained case (Section 9.2), the computational advantage of
the iterative over the batch approach is not obvious. Nevertheless an algorithm
was developed and can be found in Section 18.6.
12.3.4 Innite Horizon Problem
Assume Q 0, R 0 and that the constraint sets A and | contain the origin
in their interior
3
Consider the following innite-horizon linear quadratic regulation
problem with constraints (CLQR)
J
(x(0)) = min
u0,u1,...
k=0
x
k
Qx
k
+u
k
Ru
k
subj. to x
k+1
= Ax
k
+Bu
k
, k = 0, . . . ,
x
k
A, u
k
|, k = 0, . . . ,
x
0
= x(0)
(12.43)
3
As in the unconstrained case, the assumption Q 0 can be relaxed by requiring that (Q
1/2
, A)
is observable (Section 9.5).
and the set
A
= x(0) R
n
[ Problem (12.43) is feasible and J
(x(0)) < +. (12.44)

Because Q 0, R 0 any optimizer u
k
of problem (12.43) must converge to the
origin (u
k
0) and so must the state trajectory resulting from the application
of u
k
(x
k
0). Thus the origin x = 0, u = 0 must lie in the interior of the
constraint set (A, |) (if the origin were not contained in the constraint set then
J
(x(0)) would be innite). For this reason, the set A
in (12.44) is the maximal

stabilizable set /
(O) of system (12.1) subject to the constraints (12.2) with O

being the origin (Denition 11.13).
If the initial state x
0
= x(0) is suciently close to the origin, then the con-
straints will never become active and the solution of problem (12.43) will yield the
unconstrained LQR (9.32). More formally we can dene a corresponding invariant
set around the origin.
Denition 12.1 (Maximal LQR Invariant Set O
LQR
) Consider the system x(k+

1) = Ax(k) +Bu(k). O
LQR
R
n
denotes the maximal positively invariant set for
the autonomous constrained linear system:
x(k + 1) = (A + BF
)x(k), x(k) A, u(k) |, k 0,

where u(k) = F
x(k) is the unconstrained LQR control law (9.32) obtained from

the solution of the ARE (9.31).
Therefore, from the previous discussion, there is some nite time

N(x
0
), de-
pending on the initial state x
0
, at which the state enters O
LQR
and after which the

system evolves in an unconstrained manner (x
k
A, u
k
|, k >

N). This con-
sideration allows us to split problem (12.43) into two parts by using the dynamic
programming principle, one up to time k =

N where the constraints may be active
and one for longer times k >

N where there are no constraints.
J
(x(0)) = min
u0,u1,...
N1
k=0
x
k
Qx
k
+u
k
Ru
k
+J
N
(x
N
)
subj. to x
k
A, u
k
|, k = 0, . . . ,

N 1
x
k+1
= Ax
k
+Bu
k
, k 0
x
0
= x(0),
(12.45)
where
J
N
(x
N
) = min
u
N
,u
N+1
,...
k=

N
x
k
Qx
k
+u
k
Ru
k
subj. to x
k+1
= Ax
k
+Bu
k
, k

N
= x
N
P
x
N
.
(12.46)
This key insight due to Sznaier and Damborg [271] is formulated precisely in the
following.
Theorem 12.3 (Equality of Finite and Innite Optimal Control, [258]) For
any given initial state x(0), the solution to (12.45, 12.46) is equal to the innite
time solution of (12.43), if the terminal state x
N
of (12.45) lies in the positive
invariant set O
LQR
and no terminal set constraint is applied in (12.45), i.e. the

state voluntarily enters the set O
LQR
after

N steps.
Theorem 12.3 suggests that we can obtain the innite horizon constrained linear
quadratic regulator CLQR by solving the nite horizon problem for a horizon of

N
with a terminal weight of P = P
and no terminal constraint. The critical question

of how to determine

N(x
0
) or at least an upper bound was studied by several
researchers. Chmielewski and Manousiouthakis [84] presented an approach that
provides a conservative estimate N
est
of the nite horizon

N(x
0
) for all x
0
belonging
to a compact set of initial conditions o A
= /
(0) (N
est

N
S
(x
0
), x
0
o).
They solve a single, nite dimensional, convex program to obtain N
est
. Their
estimate can be used to compute the PWA solution of (12.45) for a particular
set o. Alternatively, the quadratic program with horizon N
est
can be solved to
determine u
0
, u
1
, . . . , u
Nest
for a particular x(0) o. For a given initial state
x(0), rather then a set o, Scokaert and Rawlings [258] presented an algorithm that
attempts to identify

N(x(0)) iteratively. In summary we can state the following
Theorem.
Theorem 12.4 (Explicit solution of CLQR) Assume that (A, B) is a stabi-
lizable pair and (Q
1/2
, A) is an observable pair, R 0. The state feedback so-
lution to the CLQR problem (12.43) in a compact set of the initial conditions
o A
= /
(0) is time-invariant, continuous and piecewise ane on polyhedra

u
(k) = f
(x(k)), f
(x) = F
j
x +g
j
if x CR
j
, j = 1, . . . , N
r
, (12.47)
j
= x R
n
: H
j
x K
j
, j = 1, . . . , N
r
are a
nite partition of the feasible compact polyhedron o A
.
As argued previously, the complexity of the solution manifested by the number
of polyhedral regions depends on the chosen horizon. As the various discussed
techniques yield an N
est
that may be too large by orders of magnitude this is not
a viable proposition. An ecient algorithm for computing the PPWA solution to
the CLQR problem is presented next.
12.3.5 CLQR Algorithm
In this section we will sketch an ecient algorithm to compute the PWA solution
to the CLQR problem in (12.43) for a given set o of initial conditions. Details
are available in [131, 132]. As a side product, the algorithm also computes

N
S
,
the shortest horizon

N for which the problem (12.45), (12.46) is equivalent to the
innite horizon problem (12.43).
The idea is as follows. For the CFTOC problem (12.28) with a horizon N with
no terminal constraint (A
f
= R
n
) and terminal cost P = P
, where P
is the
solution to the ARE (9.31), we solve an mp-QP and obtain the PWA control law.
From Theorem 12.3 we can conclude that for all states which enter the invariant set
O
LQR
introduced in Denition 12.1 with the computed control law in N steps, the
innite-horizon problem has been solved. For these states, which we can identify
via a reachability analysis, the computed feedback law is innite-horizon optimal.
In more detail, we start the procedure by computing the Maximal LQR Invari-
ant Set O
LQR
introduced in Denition 12.1, the polyhedron T

0
= O
LQR
= x
R
n
[H
0
x K
0
. Figure 12.2(a) depicts O
LQR
. Then, the algorithm nds a point

x
1
x
2
Step over facet
O
LQR
-3 -2 -1 0 1 2 3
-3
-2
-1
0
1
2
3
(a) Compute positive invariant
region O
LQR
and step over facet

with step-size .
x
1
x
2
O
LQR
T
1
-3 -2 -1 0 1 2 3
-3
-2
-1
0
1
2
3
(b) Solve QP for new point with
horizon N = 1 to create the rst
constrained region T1.
x
1
x
2
O
LQR
Reachability subset 1T T
1
1
-3 -2 -1 0 1 2 3
-3
-2
-1
0
1
2
3
(c) Compute reachability subset
of T1 to obtain 1T T
1
1
.
Figure 12.2 CLQR Algorithm: Region Exploration.
x by stepping over a facet of O
LQR
with a small step , as described in [19]. If

(12.28) is feasible for horizon N = 1 (terminal set constraint A
f
= R
n
, terminal
cost P = P
and x(0) = x), the active constraints will dene the neighboring
polyhedron T
1
= x R
n
[H
1
x K
1
( x T
1
, see Figure 12.2(b)) [41]. By Theo-
rem 12.3, the nite time optimal solution computed above equals the innite time
optimal solution if x
1
O
LQR
. Therefore we extract from T

1
the set of points that
will enter O
LQR
in N = 1 time-steps, provided that the optimal control law asso-

ciated with T
1
(i.e., U
1
= F
1
x(0) + g
1
) is applied. The Innite Time Polyhedron
(1T T
1
1
) is therefore dened by the intersection of the following two polyhedra:
x
1
O
LQR
, x
1
= Ax
0
+BU
1
, (12.48a)
x
0
T
1
. (12.48b)
Equation (12.48a) is the reachability constraint and (12.48b) denes the set of
states for which the computed feedback law is feasible and optimal over N = 1
steps (see [41] for details). The intersection is the set of points for which the
control law is innite time optimal.
A general step r of the algorithm involves stepping over a facet to a new point x
and determining the polyhedron T
r
and the associated control law (U
N
= F
r
x(0)+
g
r
) from (12.28) with horizon N. Then we extract from T
r
the set of points that
will enter O
LQR
in N time-steps, provided that the optimal control law associated

with T
r
is applied. The Innite Time Polyhedron (1T T
N
r
) is therefore dened by
the intersection of the following two polyhedra:
x
N
O
LQR
(12.49a)
x
0
T
r
. (12.49b)
This intersection is the set of points for which the control law is innite time
optimal. Note that, as for x
1
in the one-step case, x
N
in (12.49a) can be described
as a linear function of x
0
by substituting the feedback sequence U
N
= F
r
x
0
+ g
r
into the LTI system dynamics (12.1).
We continue exploring the facets increasing N when necessary. The algorithm
terminates when we have covered o or when we can no longer nd a new feasible
polyhedron T
r
. The following theorem shows that the algorithm also provides the
horizon

N
S
for compact sets. Exact knowledge of

N
S
can serve to improve the
performance of a wide array of algorithms presented in the literature.
Theorem 12.5 (Exact Computation of

N
S
, [131, 132]) If we explore any given
compact set o with the proposed algorithm, the largest resulting horizon is equal to
N
S
, i.e.,
N
S
= max
IT P
N
r
r=0,...,R
N.
Often the proposed algorithm is more ecient than standard multi-parametric
solvers, even if nite horizon optimal controllers are sought. The initial polyhedral
representation T
r
contains redundant constraints which need to be removed in order
to obtain a minimal representation of the controller region. The intersection with
the reachability constraint, as proposed here, can simplify this constraint removal.
12.3.6 Examples
Example 12.2 Consider the double integrator (12.23)
4
. We want to compute the
state feedback optimal controller that solves problem (12.28) with N = 6, Q = [
1 0
0 1
],
4
R = 0.1, P is equal to the solution of the Riccati equation (9.31), .
f
= R
2
. The
input constraints are
1 u(k) 1, k = 0, . . . , 5 (12.50)
_
10
10
_
x(k)
_
10
10
_
, k = 0, . . . , 5. (12.51)
This task is addressed as shown in Section (12.3.2). The feedback optimal solution
u
(0), . . . , u
(5) is computed by solving six mp-QP problems and the corresponding

polyhedral partitions of the state-space are depicted in Fig. 12.3. Only the last two
optimal control moves are reported below:
u
(5) =
_
_
[ 0.58 1.55 ] x(5) if
_
0.35 0.94
0.35 0.94
1.00 0.00
1.00 0.00
_
x(5)
_
0.61
0.61
10.00
10.00
_
(Region #1)
1.00 if
_
0.35 0.94
0.00 1.00
0.71 0.71
1.00 0.00
1.00 0.00
_
x(5)
_
0.61
10.00
7.07
10.00
10.00
_
(Region #2)
1.00 if
_
0.35 0.94
1.00 0.00
0.00 1.00
1.00 0.00
0.71 0.71
_
x(5)
_
0.61
10.00
10.00
10.00
7.07
_
(Region #3)
u
(4) =
_
_
[ 0.58 1.55 ] x(4) if
_
0.35 0.94
0.35 0.94
0.77 0.64
0.77 0.64
_
x(4)
_
0.61
0.61
2.43
2.43
_
(Region #1)
1.00 if
_
_
0.29 0.96
0.00 1.00
0.71 0.71
0.45 0.89
1.00 0.00
1.00 0.00
_
_
x(4)
_
_
0.98
10.00
7.07
4.92
10.00
10.00
_
_
(Region #2)
1.00 if
_
_
_
0.29 0.96
0.35 0.94
0.00 1.00
0.71 0.71
0.45 0.89
1.00 0.00
1.00 0.00
_
_x(4)
_
_
_
0.37
0.61
10.00
7.07
4.92
10.00
10.00
_
_ (Region #3)
1.00 if
_
_
0.29 0.96
1.00 0.00
0.00 1.00
1.00 0.00
0.71 0.71
0.45 0.89
_
_
x(4)
_
_
0.98
10.00
10.00
10.00
7.07
4.92
_
_
(Region #4)
1.00 if
_
_
_
0.29 0.96
0.35 0.94
1.00 0.00
0.00 1.00
1.00 0.00
0.71 0.71
0.45 0.89
_
_x(4)
_
_
_
0.37
0.61
10.00
10.00
10.00
7.07
4.92
_
_ (Region #5)
[ 0.44 1.43 ] x(4) 0.46 if
_
0.29 0.96
0.29 0.96
0.77 0.64
1.00 0.00
_
x(4)
_
0.98
0.37
2.43
10.00
_
(Region #6)
[ 0.44 1.43 ] x(4) + 0.46 if
_
0.29 0.96
0.29 0.96
0.77 0.64
1.00 0.00
_
x(4)
_
0.37
0.98
2.43
10.00
_
(Region #7)
Note that by increasing the horizon N, the control law changes only far away from the
origin, the larger N the more in the periphery. This must be expected from the results
of Section 12.3.4. The control law does not change anymore with increasing N in the
set where the CFTOC law becomes equal to the constrained innite-horizon linear
quadratic regulator (CLQR) problem. This set gets larger as N increases [84, 258].
Example 12.3 The innite time CLQR (12.43) was determined for Example 12.2
by using the approach presented in Section 12.3.4
5
. The resulting

NS is 12. The
state-space is divided into 117 polyhedral regions and is depicted in Fig. 12.4(a). In
Fig. 12.4(b) the same control law is represented where polyhedra with the same ane
control law were merged.
12.4 1-Norm and -Norm Case Solution
Next we consider problem (12.9) with J
0
() dened by (12.7) with p = 1 or p = .
In the following section we will concentrate on the -norm, the results can be
extended easily to cost functions with 1-norm or mixed 1/ norms.
J
0
(x(0)) = min
U0
J
0
(x(0), U
0
) = |Px
N
|
p
+
N1
k=0
|Qx
k
|
p
+|Ru
k
|
p
subj. to x
k+1
= Ax
k
+Bu
k
, k = 0, . . . , N 1
x
k
A, u
k
|, k = 0, . . . , N 1
x
N
A
f
x
0
= x(0).
(12.52)
12.4.1 Solution via LP
The optimal control problem (12.52) with p = can be rewritten as a linear pro-
gram by using the approach presented in Section 10.1. Therefore, problem (12.52)
can be reformulated as the following LP problem
5
12.4 1-Norm and -Norm Case Solution 233
min
z0
x
0
+. . . +
x
N
+
u
0
+. . . +
u
N1
(12.53a)
subj. to 1
n
x
k
Q
_
_
A
k
x
0
+
k1
j=0
A
j
Bu
k1j
_
_
, (12.53b)
1
r
x
N
P
_
_
A
N
x
0
+
N1
j=0
A
j
Bu
N1j
_
_
, (12.53c)
1
m
u
k
Ru
k
, (12.53d)
A
k
x
0
+
k1
j=0
A
j
Bu
k1j
A, u
k
|, (12.53e)
A
N
x
0
+
N1
j=0
A
j
Bu
N1j
A
f
, (12.53f)
k = 0, . . . , N 1
x
0
= x(0), (12.53g)
where constraints (12.53b)(12.53f) are componentwise, and means that the
constraint appears once with each sign.
Problem (12.53) can be rewritten in the more compact form
min
z0
c
0
z
0
subj. to

G
0
z
0
w
0
+

S
0
x(0),
(12.54)
where z
0
=
x
0
, . . . ,
x
N
,
u
0
, . . . ,
u
N1
, u
0
, . . . , u
N1
R
s
, s = (m + 1)N + N + 1
and
G
0
=
_
G
x
G
u
G
c
0 0 G
0
_
,

S
0
=
_
S
E
0
_
, w
0
=
_
w
w
0
_
, (12.55)
where [G
x
, G
u
, G
c
] is the block partition of the matrix G
into the three parts

corresponding to the variables
x
i
,
u
i
and u
i
, respectively. The vector c
0
and the
submatrices G
, w
, S
associated with the constraints (12.53b)-(12.53d) are dened

in (10.10). The matrices G
0
, w
0
and E
0
are dened in (12.18) for i = 0.
0
solving problem (12.54)
can be computed by using a Linear Program (LP) solver (see Section 3.2 for LP
denition and properties and Chapter 4 for fast numerical methods for solving
LPs).
To obtain the problem (12.54) we have eliminated the state variables and equal-
ity constraints x
k+1
= Ax
k
+Bu
k
by successive substitution so that we are left with
u
0
, . . . , u
N1
and the slack variables as the only decision variables, and x(0) as a
parameter vector. As in the 2-norm case, it might be more ecient to solve a LP
problem with equality and inequality constraints so that sparsity can be exploited.
We omit the details and refer the reader to the construction of the QP problem
without substitution in section 12.3.1.
12.4.2 State Feedback Solution via Batch Approach
As show in the previous section, problem (12.52) can be rewritten in the compact
form
min
z0
c
0
z
0
subj. to

G
0
z
0
w
0
+

S
0
x(0),
(12.56)
where z
0
=
x
0
, . . . ,
x
N
,
u
0
, . . . ,
u
N1
, u
0
, . . . , u
N1
R
s
, s = (m + 1)N + N + 1
and
G
0
=
_
G
x
G
u
G
c
0 0 G
0
_
,

S
0
=
_
S
E
0
_
, w
0
=
_
w
w
0
_
, (12.57)
where [G
x
, G
u
, G
c
] is the block partition of the matrix G
into the three parts

corresponding to the variables
x
i
,
u
i
and u
i
, respectively. The vector c
0
and the
submatrices G
, w
, S
associated with the constraints (12.53b)-(12.53d) are dened

in (10.10). The matrices G
0
, w
0
and E
0
are dened in (12.18) for i = 0.
As in the 2-norm case, by treating x(0) as a vector of parameters, problem (12.56)
becomes a multiparametric linear program (mp-LP) that can be solved as described
in Section 7.2. Once the multiparametric problem (12.56) has been solved, the
explicit solution z
0
(x(0)) of (12.56) is available as a piecewise ane function of
x(0), and the optimal control law u
(0) is also available explicitly, as the optimal

input u
(0) consists simply of m components of z
0
(x(0))
u
(0) = [0 . . . 0 I
m
0 . . . 0]z
0
(x(0)). (12.58)
Theorem 7.5 states that there always exists a continuous PPWA solution z
0
(x)
of the mp-LP problem (12.56). Clearly the same properties are inherited by the
controller. The following Corollaries of Theorem 7.5 summarize the analytical
properties of the optimal control law and of the value function.
Corollary 12.4 There exists a control law u
(0) = f
0
(x(0)), f
0
: R
n
R
m
,
obtained as a solution of the CFTOC (12.52) with p = 1 or p = , which is
continuous and PPWA
f
0
(x) = F
j
0
x +g
j
0
if x CR
j
0
, j = 1, . . . , N
r
0
, (12.59)
j
0
= H
j
0
x k
j
0
, j = 1, . . . , N
r
0
, are a partition of the
feasible set A
0
.
(x) obtained as a solution of the CFTOC (12.52)

is convex and PPWA.
Remark 12.7 Note that if the optimizer of problem (12.52) is unique for all x(0) .0,
then Corollary 12.4 reads: The control law u
(0) = f0(x(0)), f0 : R
n
R
m
,
obtained as a solution of the CFTOC (12.52) with p = 1 or p = , is continuous
and PPWA,. . .. From the results of Section 7.2 we know that in case of multiple
optimizers for some x(0) .0, a continuous control law of the form (12.59) can
always be computed.
The multiparametric solution of (12.56) provides the open-loop optimal se-
quence u
(0), . . . , u
(N 1) as an ane function of the initial state x(0). The

state feedback PPWA optimal controllers u
(k) = f
k
(x(k)) with f
k
: A
k
| for
k = 1, . . . , N are computed in the following way. Consider the same CFTOC (12.52)
over the shortened time horizon [i, N]
min
Ui
|Px
N
|
p
+
N1
k=i
|Qx
k
|
p
+|Ru
k
|
p
subj. to x
k+1
= Ax
k
+Bu
k
, k = i, . . . , N 1
x
k
A, u
k
|, k = i, . . . , N 1
x
N
A
f
x
i
= x(i),
(12.60)
where U
i
= [u
i
, . . . , u
N1
] and p = 1 or p = . As dened in (12.16) and discussed
in Section 12.2, A
i
R
n
is the set of initial states x(i) for which the optimal control
problem (12.60) is feasible. We denote by U
i
one of the optimizers of the optimal
control problem (12.60).
Problem (12.60) can be translated into the mp-LP
min
zi
c
i
z
i
subj. to

G
i
z
i
w
i
+

S
i
x(i),
(12.61)
where z
i
=
x
i
, . . . ,
x
N
,
u
i
, . . . ,
u
N1
, u
i
, . . . , u
N1
and c
i
,

G
i
,

S
i
, w
i
, are appro-
priately dened for each i. The component u
i
of the multiparametric solution
of (12.61) has the form
u
i
(x(i)) = f
i
(x(i)), x(i) A
i
, (12.62)
where the control law f
i
: R
n
R
m
, is continuous and PPWA
f
i
(x) = F
j
i
x +g
j
i
if x CR
j
i
, j = 1, . . . , N
r
i
(12.63)
and where the polyhedral sets CR
j
i
= x R
n
: H
j
i
x K
j
i
, j = 1, . . . , N
r
i
are a partition of the feasible polyhedron A
i
. Therefore the feedback solution
u
(k) = f
k
(x(k)), k = 0, . . . , N 1 of the CFTOC (12.52) with p = 1 or p = is
obtained by solving N mp-LP problems of decreasing size. The following corollary
summarizes the nal result.
Corollary 12.6 There exists a state feedback control law u
(k) = f
k
(x(k)), f
k
:
A
k
R
n
| R
m
, solution of the CFTOC (12.52) for p = 1 or p =
and k = 0, . . . , N 1 which is time-varying, continuous and piecewise ane on
polyhedra
f
k
(x) = F
j
k
x +g
j
k
if x CR
j
k
, j = 1, . . . , N
r
k
, (12.64)
j
k
= x R
n
: H
j
k
x K
j
k
, j = 1, . . . , N
r
k
are a
k
.
12.4.3 State Feedback Solution via Recursive Approach
Consider the dynamic programming formulation of (12.52) with J
0
() dened by (12.7)
with p = 1 or p =
J
j
(x
j
) = min
uj
|Qx
j
|
p
+|Ru
j
|
p
+ J
j+1
(Ax
j
+Bu
j
)
subj. to x
j
A, u
j
|
Ax
j
+Bu
j
A
j+1
,
(12.65)
for j = 0, . . . , N 1, with boundary conditions
J
N
(x
N
) = |Px
N
|
p
(12.66)
A
N
= A
f
, (12.67)
Unlike for the 2-norm case the dynamic program (12.65)-(12.67) can be solved as
explained in the next theorem.
Theorem 12.6 The state feedback piecewise ane solution (12.64) of the CFTOC (12.52)
for p = 1 or p = is obtained by solving the optimization problem (12.65)-(12.67)
via N mp-LPs.
Proof: Consider the rst step j = N 1 of dynamic programming (12.65)-
(12.67)
J
N1
(x
N1
) = min
uN1
|Qx
N1
|
p
+|Ru
N1
|
p
+J
N
(Ax
N1
+Bu
N1
)
subj. to x
N1
A, u
N1
|
Ax
N1
+Bu
N1
A
f
.
(12.68)
J
N1
(x
N1
), u
N1
(x
N1
) and A
N1
are computable via the mp-LP:
J
N1
(x
N1
) = min
,uN1

subj. to |Qx
N1
|
p
+|Ru
N1
|
p
+|P(Ax
N1
+Bu
N1
)|
p
x
N1
A, u
N1
|
Ax
N1
+Bu
N1
A
f
.
(12.69)
The constraint |Qx
N1
|
p
+|Ru
N1
|
p
+|P(Ax
N1
+Bu
N1
)|
p
in (12.69) is
converted into a set of linear constraints as discussed in Remark 3.1 of Section 3.2.5.
For instance, if p = we follow the approach of Section 10.2 and rewrite the
constraint as

x
N1
+
u
N1
+
x
N
1
n
x
N1
Qx
N1
1
m
u
N1
Ru
N1
1
rN
x
N
P
N
[Ax
N1
+Bu
N1
]
(12.70)
By Theorem 7.5, J
N1
is a convex and piecewise ane function of x
N1
, the
corresponding optimizer u
N1
is piecewise ane and continuous, and the feasible
set A
N1
is a polyhedron. Without any loss of generality we assume J
N1
to be
described as follows: J
N1
(x
N1
) = max
i=1,...,nN1
c
i
x
N1
+d
i
(see Section 3.2.5
for convex PPWA functions representation) where n
N1
is the number of ane
components comprising the value function J
N1
. At step j = N 2 of dynamic
programming (12.65)-(12.67) we have
J
N2
(x
N2
) = min
uN2
|Qx
N2
|
p
+|Ru
N2
|
p
+J
N1
(Ax
N2
+Bu
N2
)
subj. to x
N2
A, u
N2
|
Ax
N2
+Bu
N2
A
N1
.
(12.71)
Since J
N1
(x) is a convex and piecewise ane function of x, the problem (12.71)
can be recast as the following mp-LP (see Section 3.2.5 for details)
J
N2
(x
N2
) = min
,uN2

subj. to |Qx
N2
|
p
+|Ru
N2
|
p
+c
i
(Ax
N2
+Bu
N2
) +d
i
i = 1, . . . , n
N1
x
N2
A, u
N2
|
Ax
N2
+Bu
N2
A
N1
.
(12.72)
J
N2
(x
N2
), u
N2
(x
N2
) and A
N2
are computed by solving the mp-LP (12.72).
By Theorem 7.5, J
N2
N2
, the
corresponding optimizer u
N2
is piecewise ane and continuous, and the feasible
set A
N2
is a convex polyhedron.
The convexity and piecewise linearity of J
j
and the polyhedra representation
of A
j
still hold for j = N 3, . . . , 0 and the procedure can be iterated backwards
in time, proving the theorem.
Consider the state feedback piecewise ane solution (12.64) of the CFTOC (12.52)
for p = 1 or p = and assume we are interested only in the optimal controller at
time 0. In this case, by using duality arguments we can solve the equations (12.65)-
(12.67) by using vertex enumerations and one mp-LP. This is proven in the next
theorem.
Theorem 12.7 The state feedback piecewise ane solution (12.64) at time k = 0
of the CFTOC (12.52) for p = 1 or p = is obtained by solving the optimization
problem (12.65)-(12.67) via one mp-LP.
Proof: Consider the rst step j = N 1 of dynamic programming (12.65)-
(12.67)
J
N1
(x
N1
) = min
uN1
|Qx
N1
|
p
+|Ru
N1
|
p
+J
N
(Ax
N1
+Bu
N1
)
subj. to x
N1
A, u
N1
|
Ax
N1
+Bu
N1
A
f
(12.73)
and the corresponding mp-LP:
J
N1
(x
N1
) = min
,uN1

subj. to |Qx
N1
|
p
+|Ru
N1
|
p
+|P(Ax
N1
+Bu
N1
)|
p
x
N1
A, u
N1
|
Ax
N1
+Bu
N1
A
f
.
(12.74)
By Theorem 7.5, J
N1
N1
, and the
feasible set A
N1
is a polyhedron. J
N1
and A
N1
are computed without explicitly
solving the mp-LP (12.74). Rewrite problem (12.74) in the more compact form
min
zN1
c
N1
z
N1
subj. to

G
N1
z
N1
w
N1
+

S
N1
x
N1
,
(12.75)
where z
N1
collects the optimization variables , u
N1
and the auxiliary variables
need to transform the constraint |Qx
N1
|
p
+ |Ru
N1
|
p
+ |P(Ax
N1
+
Bu
N1
)|
p
into a set of linear constraints. Consider the Linear Program dual
of (12.75)
max
v
( w
N1
+

S
N1
x
N1
)
v
subj. to

G
N1
v = c
N1
v 0.
(12.76)
Consider the dual feasibility polyheron T
d
= v 0 :

G
N1
v = c
N1
. Let
V
1
, . . . , V
k
be the vertices of T
d
and y
1
, . . . , y
e
be the rays of T
d
. Since we have
a zero duality gap, we have that
J
N1
(x
N1
) = max
i=1,...,k
( w
N1
+

S
N1
x
N1
)
V
i
i.e.,
J
N1
(x
N1
) = max
i=1,...,k
(V
i

S
N1
)x
N1
w
N1
V
i
.
Recall that if the dual of a mp-LP is unbounded, then the primal is infeasible
(Theorem 7.3). For this reason the feasible set A
N1
is obtained by requiring that
the cost of (12.76) does not increase in the direction of the rays:
A
N1
= x
N1
: ( w
N1
+

S
N1
x
N1
)
y
i
0, i = 1, . . . , e
with J
N1
(x
N1
) and A
N1
available, we can iterate the procedure backwards in
time for steps N 2, N 3, . . . , 1. At step 0 one mp-LP will be required in order
to compute u
0
(x(0)).
12.4.4 Example
Example 12.4 Consider the double integrator system (12.23)
6
. We want to compute
the state feedback optimal controller that solves (12.52) with p = , N = 6, P =
_
1 0
0 1
_
, Q =
_
1 0
0 1
_
, R = 0.8, subject to the input constraints
| = |u R : 1 u 1 (12.77)
. = |x R
2
:
_
10
10
_
x
_
10
10
_
(12.78)
6
and .
f
= .. The optimal feedback solution u
(0), . . . , u
(5) was computed by solving

six mp-LP problems and the corresponding polyhedral partitions of the state-space
are depicted in Fig. 12.5, where polyhedra with the same control law were merged.
Only the last optimal control move is reported below:
u
(5) =
_
_
0 if
_
0.45 0.89
1.00 0.00
0.71 0.71
1.00 0.00
_
x(5)
_
0.00
0.00
7.07
10.00
_
(Region #1)
0 if
_
0.45 0.89
1.00 0.00
0.71 0.71
1.00 0.00
_
x(5)
_
0.00
0.00
7.07
10.00
_
(Region #2)
[ 1.00 2.00 ] x(5) if
_
0.45 0.89
0.45 0.89
0.71 0.71
1.00 0.00
_
x(5)
_
0.00
0.45
0.00
10.00
_
(Region #3)
[ 1.00 2.00 ] x(5) if
_
0.45 0.89
0.45 0.89
0.71 0.71
1.00 0.00
_
x(5)
_
0.45
0.00
0.00
10.00
_
(Region #4)
[ 1.00 0.00 ] x(5) if
_
0.71 0.71
1.00 0.00
1.00 0.00
0.00 1.00
_
x(5)
_
0.00
1.00
0.00
10.00
_
(Region #5)
[ 1.00 0.00 ] x(5) if
_
0.71 0.71
1.00 0.00
1.00 0.00
0.00 1.00
_
x(5)
_
0.00
0.00
1.00
10.00
_
(Region #6)
1.00 if
_
0.45 0.89
1.00 0.00
0.00 1.00
1.00 0.00
_
x(5)
_
0.45
1.00
10.00
10.00
_
(Region #7)
1.00 if
_
0.45 0.89
1.00 0.00
1.00 0.00
0.00 1.00
_
x(5)
_
0.45
1.00
10.00
10.00
_
(Region #8)
(12.79)
Note that the controller (12.79) is piecewise linear around the origin. In fact, the
origin belongs to multiple regions (1 to 6). Note that the number N
r
i
of regions is not
always increasing with decreasing i (N
r
5
= 8, N
r
4
= 12, N
r
3
= 12, N
r
2
= 26, N
r
1
= 28,
N
r
0
= 26). This is due to the merging procedure, before merging we have N
r
5
= 12,
N
r
4
= 22, N
r
3
= 40, N
r
2
= 72, N
r
1
= 108, N
r
0
= 152.
12.4.5 Innite-Time Solution
Assume that Q and R have full column rank and that the constraint sets A and |
contain the origin in their interior. Consider the following innite-horizon problem
with constraints
J
(x(0)) = min
u0,u1,...
k=0
|Qx
k
|
p
+|Ru
k
|
p
subj. to x
k+1
= Ax
k
+Bu
k
, k = 0, . . . ,
x
k
A, u
k
|, k = 0, . . . ,
x
0
= x(0)
(12.80)
and the set
A
= x(0) R
n
[ Problem (12.80) is feasible and J
(x(0)) < +. (12.81)

Because Q and R have full column rank, any optimizer u
k
of problem (12.43) must
converge to the origin (u
k
0) and so must the state trajectory resulting from the
application of u
k
(x
k
0). Thus the origin x = 0, u = 0 must lie in the interior
of the constraint set (A, |). (If the origin were not contained in the constraint
set then J
(x(0)) would be innite.) Furthermore, if the initial state x

0
= x(0)
is suciently close to the origin, then the state and input constraints will never
become active and the solution of problem (12.43) will yield the unconstrained
optimal controller (10.31).
The discussion for the solution of the innite horizon constrained linear quadratic
regulator (Section 12.3.4) by means of the batch approach can be repeated here
with one precaution. Since the unconstrained optimal controller (if it exists) is
PPWA the computation of the Maximal Invariant Set for the autonomous con-
strained piecewise linear system is more involved and requires algorithms which
will be presented later in Chapter 18.
Dierently from the 2-norm case, here the use of dynamic programming for
computing the innite horizon solution is a viable alternative to the batch approach.
Convergence conditions for the dynamic programming strategy and convergence
guarantees for the resulting possibly discontinuous closed-loop system are given
in [86]. A computationally ecient algorithm to obtain the innite time optimal
solution, based on a dynamic programming exploration strategy with a multi-
parametric linear programming solver and basic polyhedral manipulations, is also
presented in [86].
Example 12.5 We consider the double integrator system (12.23) from Example 12.4
with N =
7
.
The partition of the state space for the time invariant optimal control law is shown in
Fig. 12.6(a) and consists of 202 polyhedral regions. In Fig. 12.6(b) the same control
law is represented where polyhedra with the same ane control law were merged.
12.5 State Feedback Solution, Minimum-Time Control
In this section we consider the solution of minimum-time optimal control problems
J
0
(x(0)) = min
U0,N
N
subj. to x
k+1
= Ax
k
+Bu
k
, k = 0, . . . , N 1
x
k
A, u
k
|, k = 0, . . . , N 1
x
N
A
f
x
0
= x(0),
(12.82)
where A
f
R
n
is a terminal target set to be reached in minimum time.
7
12.5 State Feedback Solution, Minimum-Time Control 241
We can nd the controller that brings the states into A
f
in one time step by
solving the following multiparametric program
min
u0
c(x
0
, u
0
)
subj. to x
1
= Ax
0
+Bu
0
x
0
A, u
0
|
x
1
A
f
,
(12.83)
where c(x
0
, u
0
) is any convex quadratic function. Let us assume that the solution of
the multiparametric program generates R
1
regions T
1
r
R
1
r=1
with the ane control
law u
0
= F
1
r
x +g
1
r
in each region r. By construction we have
A
0
= /
1
(A
f
).
Continuing setting up simple multiparametric programs bring the states into A
f
in 2, 3, . . . steps, we have for step j
min
u0
c(x
0
, u
0
)
subj. to x
1
= Ax
0
+Bu
0
x
0
A, u
0
|
x
1
/
j1
(A
f
),
(12.84)
which yields R
j
regions T
j
r
R
j
r=1
with the ane control law u
0
= F
j
r
x +g
j
r
in each
region r. By construction we have
A
0
= /
j
(A
f
).
Thus to obtain /
1
(A
f
), . . . , /
N
(A
f
) we need to solve N multiparametric programs
with a prediction horizon of 1. Since the overall complexity of a multiparametric
program is exponential in N, this scheme can be exploited to yield controllers
of lower complexity than the optimal control scheme introduced in the previous
sections.
Since N multiparametric programs have been solved, the controller regions over-
lap in general. In order to achieve minimum time behavior, the feedback law asso-
ciated with the region computed for the smallest number of steps c, is selected for
any given state x.
Algorithm 12.1 Minimum-Time Controller: On-Line Application
repeat
Obtain state measurement x.
Find controller partition cmin = min
c{0,...,N}
c, s.t. x /c(.
f
).
Find controller region r, such that x T
c
min
r
and compute u0 = F
c
min
r
x+g
c
min
r
.
Apply input u0 to system and go to Step 1.
Note that the region identication for this type of controller partition is much
more ecient than simply checking all the regions. The two steps of nding a
controller partition and nding a controller region in Algorithm 12.1 correspond
to two levels of a search tree, where the search is rst performed over the feasible
sets /
c
(A
f
) and then over the controller partition T
c
r
R
c
r=1
. Furthermore, one
may discard all regions T
i
r
which are completely covered by previously computed
controllers (i.e. T
i
r

j{1,...,i1}
/
j
(A
f
)) since they are not time optimal.
Example 12.6 Consider again the double integrator from Example 12.2
8
. The
Minimum-Time Controller is computed that steers the system to the Maximal LQR
Invariant Set O
LQR
in the minimum number of time steps N. The Algorithm ter-

minated after 11 iterations, covering the Maximal Controllable Set /(O
LQR
). The
resulting controller is dened over 33 regions. The regions are depicted in Fig. 12.7(a).
The Maximal LQR Invariant Set O
LQR
is the central shaded region.

The control law on this partition is depicted in Fig. 12.7(b). Note that, in general,
the minimum-time control law is not continuous as can be seen in Fig. 12.7(b).
12.6 Comparison of the Design Approaches and Controllers
For the design, the storage and the on-line execution time of the control law we
are interested in the controller complexity. All the controllers we discussed in this
chapter are PPWA. Their storage and execution times are closely related to the
number of polyhedral regions, which is related to the number of constraints in the
mp-LP or mp-QP.
When the control objective is expressed in terms of the 1- or - norm, it
is translated into a set of constraints in the mp-LP. When the control objective
is expressed in terms of the 2-norm such a translation is not necessary. Thus
controllers minimizing the 1- or - norm do generally involve more regions than
those minimizing the 2-norm and therefore tend to be more complex.
In mp-LPs degeneracies are common and have to be taken care of in the re-
spective algorithms. Strictly convex mp-QPs for which the Linear Independent
Constraint Qualication holds are comparatively well behaved with unique solu-
tions and full-dimensional polyhedral regions. Only for the 1-norm and -norm
objective, however, we can use for the controller design an ecient Dynamic Pro-
gramming scheme involving a sequence of mp-LPs.
As we let the horizon N go to innity, the number of regions stays nite for the
2-norm objective, for the 1- and -norm nothing can be said in general. In the
examples we have usually observed a nite number, but cases can be constructed,
where the number can be proven to be innite.
As we will argue in the following chapter, innite horizon controllers based on
the 2-norm render the closed-loop system exponentially stable, while controllers
based on the 1- or -norm render the closed-loop system stable, only.
Among the controllers proposed in this chapter, the minimum-time controller
is usually the least complex involving the smallest number of regions. Minimum
time controllers are often considered to be too aggressive for practical use. The
8
12.6 Comparison of the Design Approaches and Controllers 243
controller here is dierent, however. It is minimum time only until it reaches the
terminal set A
f
. Inside the terminal set a dierent unconstrained control law can
be used. Overall the operation of the minimum-time controller is observed to be
very similar to that of the other innite time controllers in this chapter.
x
1
(0)
x
2
(
0
)
-10 -5 0 5 10
-10
-5
0
5
10
(a) Partition of the state space
for the ane control law u
(0)
(N
r
0
= 13).
x
1
(1)
x
2
(
1
)
-10 -5 0 5 10
-10
-5
0
5
10
(b) Partition of the state space
(1)
(N
r
1
= 13).
x
1
(2)
x
2
(
2
)
-10 -5 0 5 10
-10
-5
0
5
10
(c) Partition of the state space
(2)
(N
r
2
= 13).
x
1
(3)
x
2
(
3
)
-10 -5 0 5 10
-10
-5
0
5
10
(d) Partition of the state space
(3)
(N
r
3
= 11).
x
1
(4)
x
2
(
4
)
-10 -5 0 5 10
-10
-5
0
5
10
(e) Partition of the state space
(4)
(N
r
4
= 7).
x
1
(5)
x
2
(
5
)
-10 -5 0 5 10
-10
-5
0
5
10
(f) Partition of the state space
(5)
(N
r
5
= 3).
Figure 12.3 Example 12.2: Double Integrator, 2-norm objective function,
horizon N = 6. Partition of the state space for the time-varying optimal
control law. Polyhedra with the same control law were merged. Download
code.
x
1
x
2
-10 -5 0 5 10
-10
-5
0
5
10
(a) Partition before merging
(N
r
= 117).
x
1
x
2
-10 -5 0 5 10
-10
-5
0
5
10
(b) Partition after merging
(N
r
= 13).
Figure 12.4 Example 12.3: Double Integrator, 2-norm objective function,
horizon N = . Partition of the state space for the time invariant optimal
control law. Download code.
x
1
(0)
x
2
(
0
)
-10 -5 0 5 10
-10
-5
0
5
10
(a) Partition of the state space for
the ane control law u
(0)(N
r
0
=
26).
x
1
(1)
x
2
(
1
)
-10 -5 0 5 10
-10
-5
0
5
10
(b) Partition of the state space
(1)
(N
r
1
= 28).
x
1
(2)
x
2
(
2
)
-10 -5 0 5 10
-10
-5
0
5
10
(c) Partition of the state space
(2)
(N
r
2
= 26).
x
1
(3)
x
2
(
3
)
-10 -5 0 5 10
-10
-5
0
5
10
(d) Partition of the state space
(3)
(N
r
3
= 12)
x
1
(4)
x
2
(
4
)
-10 -5 0 5 10
-10
-5
0
5
10
(e) Partition of the state space
(4)
(N
r
4
= 12).
x
1
(5)
x
2
(
5
)
-10 -5 0 5 10
-10
-5
0
5
10
(f) Partition of the state space
(5)
(N
r
5
= 8).
Figure 12.5 Example 12.4: Double Integrator, -norm objective function,
horizon N = 6. Partition of the state space for the time-varying optimal
control law. Polyhedra with the same control law were merged. Download
code.
x
1
x
2
-10 -5 0 5 10
-10
-5
0
5
10
(a) Partition before merging
(N
r
= 202).
x
1
x
2
-10 -5 0 5 10
-10
-5
0
5
10
(b) Partition after merging
(N
r
= 26).
Figure 12.6 Example 12.5: Double Integrator, -norm objective function,
horizon N = . Partition of the state space for the time invariant optimal
control law. Download code.
x
1
x
2
-10 -5 0 5 10
-10
-5
0
5
10
(a) Minimum-Time State Space
Partition (N
r
=33).
x
1
x
2
u
-10
-5
0
5
10
-5
0
5
-1
0
1
(b) Minimum-Time Control Law
(discontinuous).
Figure 12.7 Example 12.6: Double Integrator, minimum-time objective func-
tion. Partition of the state space and control law. Maximal LQR invariant
set O
LQR
is the central shaded region in (a). Download code.

Chapter 13
Receding Horizon Control
In the previous chapter we discussed the solution of constrained nite time and
innite time optimal control problems for linear systems. An innite horizon sub-
optimal controller can be designed by repeatedly solving nite time optimal control
problems in a receding horizon fashion as described next.
past
future
reference
t t + 1 t +N
m
t +N
p
t + 1t + 2 t + 1 +N
m
t + 1 +N
p
u(t)
u(t)
predicted outputs y(t +k[t)
predicted outputs y(t + 1 +k[t + 1)
manipulated inputs u(t +k)
manipulated inputs u(t + 1 +k)
Figure 13.1 Receding Horizon Idea.
At each sampling time, starting at the current state, an open-loop optimal
control problem is solved over a nite horizon (top diagram in Figure 13.1). The
computed optimal manipulated input signal is applied to the process only during
the following sampling interval [t, t +1]. At the next time step t +1 a new optimal
control problem based on new measurements of the state is solved over a shifted
250 13 Receding Horizon Control
horizon (bottom diagram in Figure 13.1). The resulting controller is referred to
as a Receding Horizon Controller (RHC). A receding horizon controller where the
nite time optimal control law is computed by solving an optimization problem
on-line is usually referred to as Model Predictive Control (MPC).
In this chapter we review the basics of RHC. We discuss the stability and the
feasibility of RHC and we provide guidelines for choosing the terminal weight so
that closed-loop stability is achieved.
The second part of the chapter focuses on the RHC implementation. Since
RHC requires at each sampling time to solve an open-loop constrained nite time
optimal control problem as a function of the current state, the results of the previous
chapters imply two approaches for RHC implementation.
The rst approach consist of repeatedly solving a mathematical program for
the current initial state. The second approach consist of precomputing o-line the
explicit piecewise ane feedback policy (that provides the optimal control for all
states). This reduces the on-line computation of the RHC law to a function eval-
uation, thus avoiding the on-line solution of a quadratic or linear program. This
technique is attractive for a wide range of practical problems where the computa-
tional complexity of on-line optimization is prohibitive. It also provides insight into
the structure underlying optimization-based controllers, describing the behaviour
of the RHC controller in dierent regions of the state space. Moreover, for appli-
cations where safety is crucial, the correctness of a piecewise ane control law is
easier to verify than that of a mathematical program solver. Sections 13.3 and 13.4
present the piecewise ane feedback control structure of QP-based and LP-based
RHC. It is obtained as a simple corollary of the results of the previous chapters.
13.1 RHC Implementation
Consider the problem of regulating to the origin the discrete-time linear time-
invariant system
x(t + 1) = Ax(t) +Bu(t), (13.1)
where x(t) R
n
, u(t) R
m
are the state and input vectors, respectively, subject
to the constraints
x(t) A, u(t) |, t 0, (13.2)
where the sets A R
n
and | R
m
are polyhedra. Receding Horizon Control
(RHC) approaches such a constrained regulation problem in the following way.
Assume that a full measurement or estimate of the state x(t) is available at the
13.1 RHC Implementation 251
current time t. Then the nite time optimal control problem
J
t
(x(t)) = min
U
tt+N|t
J
t
(x(t), U
tt+N|t
) = p(x
t+N|t
) +
N1
k=0
q(x
t+k|t
, u
t+k|t
)
subj. to x
t+k+1|t
= Ax
t+k|t
+Bu
t+k|t
, k = 0, . . . , N 1
x
t+k|t
A, u
t+k|t
|, k = 0, . . . , N 1
x
t+N|t
A
f
x
t|t
= x(t)
(13.3)
is solved at time t, where U
tt+N|t
= u
t|t
, . . . , u
t+N1|t
and where x
t+k|t
denotes
the state vector at time t + k predicted at time t obtained by starting from the
current state x
t|t
= x(t) and applying to the system model
x
t+k+1|t
= Ax
t+k|t
+Bu
t+k|t
t|t
, . . . , u
t+N1|t
. Often the symbol x
t+k|t
is read as the state
x at time t + k predicted at time t. Similarly, u
t+k|t
is read as the input u at
time t + k computed at time t. For instance, x
3|1
represents the predicted state
at time 3 when the prediction is done at time t = 1 starting from the current state
x(1). It is dierent, in general, from x
3|2
which is the predicted state at time 3
when the prediction is done at time t = 2 starting from the current state x(2).
Let U
tt+N|t
= u
t|t
, . . . , u
t+N1|t
be the optimal solution of (13.3) at time t
and J
t
(x(t)) the corresponding value function. Then, the rst element of U
tt+N|t
is applied to system (13.1)
u(t) = u
t|t
(x(t)). (13.4)
The optimization problem (13.3) is repeated at time t +1, based on the new state
x
t+1|t+1
= x(t + 1), yielding a moving or receding horizon control strategy.
Let f
t
: R
n
R
m
denote the receding horizon control law that associates
the optimal input u
t|t
to the current state x(t), f
t
(x(t)) = u
t|t
(x(t)). Then the
closed-loop system obtained by controlling (13.1) with the RHC (13.3)-(13.4) is
x(k + 1) = Ax(k) +Bf
k
(x(k)) = f
cl
(x(k), k), k 0. (13.5)
Note that the notation used in this chapter is slightly dierent from the one
used in Chapter 12. Because of the receding horizon strategy, there is the need
to distinguish between the input u
(t + k) applied to the plant at time t + k, and

optimizer u
t+k|t
of the problem (13.3) at time t + k obtained by solving (13.3) at
time t with x
t|t
= x(t).
Consider problem (13.3). As the system, the constraints and the cost function
are time-invariant, the solution to problem (13.3) is a time-invariant function of
the initial state x(t). Therefore, in order to simplify the notation, we can set t = 0
in (13.3) and remove the term [0 since it is now redundant and rewrite (13.3) as
J
0
(x(t)) = min
U0
J
0
(x(t), U
0
) = p(x
N
) +
N1
k=0
q(x
k
, u
k
)
subj. to x
k+1
= Ax
k
+Bu
k
, k = 0, . . . , N 1
x
k
A, u
k
|, k = 0, . . . , N 1
x
N
A
f
x
0
= x(t),
(13.6)
where U
0
= u
0
, . . . , u
N1
and the notation in Remark 8.1 applies. Similarly
as in previous chapters, we will focus on two classes of cost functions. If the 1-
norm or -norm is used in (13.6), then we set p(x
N
) = |Px
N
|
p
and q(x
k
, u
k
) =
|Qx
k
|
p
+ |Ru
k
|
p
with p = 1 or p = and P, Q, R full column rank matrices.
The cost function is rewritten as
J
0
(x(0), U
0
) = |Px
N
|
p
+
N1
k=0
|Qx
k
|
p
+|Ru
k
|
p
. (13.7)
If the squared Euclidian norm is used in (13.6), then we set p(x
N
) = x
N
Px
N
and
q(x
k
, u
k
) = x
k
Qx
k
+ u
k
Ru
k
with P _ 0, Q _ 0 and R 0. The cost function is
rewritten as
J
0
(x(0), U
0
) = x
N
Px
N
+
N1
k=0
x
k
Qx
k
+u
k
Ru
k
. (13.8)
The control law (13.4)
u(t) = f
0
(x(t)) = u
0
(x(t)) (13.9)
and closed-loop system (13.5)
x(k + 1) = Ax(k) +Bf
0
(x(k)) = f
cl
(x(k)), k 0 (13.10)
are time-invariant as well.
Note that the notation in (13.6) does not allow us to distinguish at which time
step a certain state prediction or optimizer is computed and is valid for time-
invariant problems only. Nevertheless, we will prefer the RHC notation in (13.6)
to the one in (13.3) in order to simplify the exposition.
Compare problem (13.6) and the CFTOC (12.9). The only dierence is that
problem (13.6) is solved for x
0
= x(t), t 0 rather than for x
0
= x(0). For this
reason we can make use of all the results of the previous chapter. In particular, A
0
denotes the set of feasible states x(t) for problem (13.6) as dened and studied in
Section 12.2. Recall from Section 12.2 that A
0
is a polyhedron.
From the above explanations it is clear that a xed prediction horizon is shifted
or recedes over time, hence its name, receding horizon control. The procedure of
this on-line optimal control technique is summarized in the following algorithm.
Algorithm 13.1 On-line receding horizon control
repeat
measure the state x(t) at time instant t
obtain U
0
(x(t)) by solving the optimization problem (13.6)
if U
0
(x(t)) = then problem infeasible stop
apply the rst element u
0
of U
0
to the system
wait for the new sampling time t + 1
Example 13.1 Consider the double integrator system (12.23) rewritten below
1
:
x(t + 1) =
_
1 1
0 1
_
x(t) +
_
0
1
_
u(t) (13.11)
The aim is to compute the receding horizon controller that solves the optimization
problem (13.6) with p(xN) = x
N
PxN, q(x
k
, u
k
) = x
k
Qx
k
+u
k
Ru
k
N = 3, P = Q =
_
1 0
0 1
_
, R = 10, .
f
= R
2
0.5 u(k) 0.5, k = 0, . . . , 3 (13.12)
_
5
5
_
x(k)
_
5
5
_
, k = 0, . . . , 3. (13.13)
The QP problem associated with the RHC has the form (12.31) with
H =
_
13.50 10.00 0.50
10.00 22.00 10.00
0.50 10.00 31.50
_
, F =
_
10.50 10.00 0.50
20.50 10.00 9.50
_
, Y = [
14.50 23.50
23.50 54.50
] (13.14)
and
G0 =
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
0.50 1.00 0.50
0.50 1.00 0.50
0.50 0.00 0.50
0.50 0.00 0.50
0.50 0.00 0.50
0.50 0.00 0.50
1.00 0.00 0.00
0.00 1.00 0.00
1.00 0.00 0.00
0.00 1.00 0.00
0.00 0.00 1.00
0.00 0.00 1.00
0.00 0.00 0.00
0.50 0.00 0.50
0.00 0.00 0.00
0.50 0.00 0.50
0.50 0.00 0.50
0.50 0.00 0.50
0.00 0.00 0.00
0.00 0.00 0.00
0.00 0.00 0.00
0.00 0.00 0.00
_
_
, E0 =
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
0.50 0.50
0.50 0.50
0.50 0.50
0.50 0.50
0.50 0.50
0.50 0.50
0.00 0.00
0.00 0.00
0.00 0.00
0.00 0.00
0.00 0.00
0.00 0.00
1.00 1.00
0.50 0.50
1.00 1.00
0.50 0.50
0.50 1.50
0.50 1.50
1.00 0.00
0.00 1.00
1.00 0.00
0.00 1.00
_
_
, w0 =
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
0.50
0.50
5.00
5.00
5.00
5.00
5.00
5.00
5.00
5.00
0.50
0.50
5.00
5.00
5.00
5.00
0.50
0.50
5.00
5.00
5.00
5.00
_
_
(13.15)
The RHC (13.6)-(13.9) algorithm becomes Algorithm 13.2. We refer to the solution
U
0
of the QP (12.31) as [U
0
, Flag] = QP(H, 2F
x(0), G0, w0 +E0x(0)) where Flag

indicates if the QP was found to be feasible or not.
Algorithm 13.2 On-line receding horizon control QP-based
repeat
measure the state x(t) at time instant t
compute

F = 2F
x(t) and w0 = w0 +E0x(t)

obtain U
0
(x(t)) by solving the optimization problem [U
0
, Flag] = QP(H,

F, G0, w0)
if Flag=infeasible then problem infeasible stop
apply the rst element u
0
of U
0
to the system
wait for the new sampling time t + 1
x
1
x
2
-5 -4 -3 -2 -1 0 1 2 3 4 5
0
1
2
3
Figure 13.2 Example 13.1. Closed-loop trajectories for the initial state
x(0)=[-4.5,2] (boxes) and x(0)=[-4.5,3] (circles). Download code.
Fig. 13.2 shows two closed-loop trajectories starting at state x(0) = [4.5, 2] and
x(0) = [4.5, 3]. The trajectory starting from x(0) = [4.5, 2] converges to the
origin and satises input and state constraints. The trajectory starting from x(0) =
[4.5, 3] stops at x(2) = [1, 2] because of infeasibility. At each time step, the open-
loop predictions are depicted with dashed lines. This shows that the closed-loop
trajectories are dierent from the open-loop predicted trajectories because of the
receding horizon nature of the controller.
In Fig. 13.3(a) the feasible state space was gridded and each point of the grid was
marked with a square if the RHC law (13.6)-(13.9) generates feasible closed-loop tra-
jectories and with a circle if it does not. The set of all initial conditions generating
feasible closed-loop trajectories is the maximal positive invariant set O of the au-
tonomous system (13.10). We remark that this set is dierent from the set .0 of
feasible initial conditions for the QP problem (12.31) with matrices (13.15). Both
sets O and .0 are depicted in gure 13.3(b). The computation of f0 is discussed
later in this chapter. Because of the nonlinear nature of f0, the computation of O
for the system (13.10) is not an easy task. Therefore we will show how to choose a
terminal invariant set .
f
such that O = .0 is guaranteed automatically.
Note that a feasible closed-loop trajectory does not necessarily converge to the origin.
Feasibility, convergence and stability of RHC are discussed in detail in the next
sections. Before that we want to illustrate these issues through another example.
Example 13.2 Consider the unstable system
2
x(t + 1) =
_
2 1
0 0.5
_
x(t) +
_
1
0
_
u(t) (13.16)
with the input constraints
1 u(k) 1, k = 0, . . . , N 1 (13.17)
_
10
10
_
x(k)
_
10
10
_
, k = 0, . . . , N 1. (13.18)
1
2
replacemen
x
1
x
2
-5 -4 -3 -2 -1 0 1 2 3 4 5
-5
-4
-3
-2
-1
0
1
2
3
4
5
(a) Boxes (Circles) are initial
points leading (not leading) to
feasible closed-loop trajectories.
x
1
x
2
-5 -4 -3 -2 -1 0 1 2 3 4 5
-5
-4
-3
-2
-1
0
1
2
3
4
5
(b) Maximal positive invariant set
O (grey) and set of initial fea-
sible states .0 (white and gray).
Figure 13.3 Double integrator Example (13.1). Download code.
In the following, we study the receding horizon control problem (13.6) with
p(xN) = x
N
PxN, q(x
k
, u
k
) = x
k
Qx
k
+ u
k
Ru
k
for dierent horizons N and
weights R. We set Q = I and omit both the terminal set constraint and the
terminal weight, i.e. .
f
= R
2
, P = 0.
Fig. 13.4 shows closed-loop trajectories for receding horizon control loops that
were obtained with the following parameter settings
Setting 1: N = 2, R = 10
Setting 2: N = 3, R = 2
Setting 3: N = 4, R = 1
For Setting 1 (Fig. 13.4(a)) there is evidently no initial state that can be steered
to the origin. Indeed, it turns out, that all non-zero initial states x(0) R
2
diverge from the origin and eventually become infeasible. In contrast, Setting 2
leads to a receding horizon controller, that manages to get some of the initial
states converge to the origin, as seen in Fig. 13.4(b). Finally, Fig. 13.4(c) shows
that Setting 3 can expand the set of those initial states that can be brought to
the origin.
Note the behavior of particular initial states:
1. Closed-loop trajectories starting at state x(0) = [4, 7] behave dierently
depending on the chosen setting. Both Setting 1 and Setting 2 cannot bring
this state to the origin, but the controller with Setting 3 succeeds.
2. There are initial states, e.g. x(0) = [4, 8.5], that always lead to infeasible
trajectories independent of the chosen settings. It turns out, that no setting
can be found that brings those states to the origin.
These results illustrate that the choice of parameters for receding horizon con-
trol inuences the behavior of the resulting closed-loop trajectories in a complex
manner. A better understanding of the eect of parameter changes can be gained
from an inspection of maximal positive invariant sets O for the dierent set-
tings, and the maximal control invariant set c as depicted in Fig. 13.5.
The maximal positive invariant set stemming from Setting 1 only contains the
x
1
x
2
-8 -4 0 4 8
-10
-5
0
5
10
(a) Setting 1: N = 2, R = 10.
x
1
x
2
-8 -4 0 4 8
-10
-5
0
5
10
(b) Setting 2: N = 3, R = 2.
x
1
x
2
-8 -4 0 4 8
-10
-5
0
5
10
(c) Setting 3: N = 4, R = 1.
Figure 13.4 Example 13.2. Closed-loop trajectories for dierent settings of
horizon N and weight R. Boxes (Circles) are initial points leading (not
leading) to feasible closed-loop trajectories. Download code.
origin (O = |0) which explains why all non-zero initial states diverge from the
origin. For Setting 2 the maximal positive invariant set has grown considerably,
but does not contain the initial state x(0) = [4, 7], thus leading to infeasibility
eventually. Setting 3 leads to a maximal positive invariant set that contains this
state and thus keeps the closed-loop trajectory inside this set for all future time
steps.
From Fig. 13.5 we also see that a trajectory starting at x(0) = [4, 8.5] cannot
be kept inside any bounded set by any setting (indeed, by any controller) since
it is outside the maximal control invariant set c.
13.2 RHC Main Issues 257
x
1
x
2
x(0) = [4, 7]
x(0) = [4, 8.5]
-4 0 4
-10
-5
0
5
10
Figure 13.5 Example 13.2. Maximal positive invariant sets O for dierent
parameter settings: Setting 1 (origin), Setting 2 (dark-gray) and Setting 3
(gray and dark-gray). Also depicted is the maximal control invariant set c
(white and gray and dark-gray). Download code.
13.2 RHC Main Issues
If we solve the receding horizon problem for the special case of an innite hori-
zon (setting N = in (13.6) as we did for LQR in Section 9.5 and CLQR in
Section 12.3.4) then it is almost immediate that the closed-loop system with this
controller has some nice properties. Most importantly, the dierences between
the open-loop predicted and the actual closed-loop trajectories observed in Exam-
ple 13.1 disappear. As a consequence, if the optimization problem is feasible, then
the closed-loop trajectories will be feasible for all times. If the optimization prob-
lem has a nite solution, then in closed-loop the states and inputs will converge to
the origin asymptotically.
In RHC, when we solve the optimization problem over a nite horizon repeat-
edly at each time step, we hope that the controller resulting from this short-
sighted strategy will lead to a closed-loop behavior that mimics that of the in-
nite horizon controller. The examples in the last section indicated that at least two
problems may occur. First of all, the controller may lead us into a situation where
after a few steps the nite horizon optimal control problem that we need to solve
at each time step is infeasible, i.e., that there does not exist a sequence of control
inputs for which the constraints are obeyed. Second, even if the feasibility prob-
lem does not occur, the generated control inputs may not lead to trajectories that
converge to the origin, i.e., that the closed-loop system is asymptotically stable.
In general, stability and feasibility are not ensured by the RHC law (13.6)-
(13.9). In principle, we could analyze the RHC law for feasibility, stability and
convergence but this is dicult as the examples in the last section illustrated.
Therefore, conditions will be derived on how the terminal weight P and the terminal
constraint set A
f
should be chosen such that closed-loop stability and feasibility
are ensured.
13.2.1 Feasibility of RHC
The examples in the last section illustrate that feasibility at the initial time x(0)
A
0
does not necessarily imply feasibility for all future times. It is desirable to
design a RHC such that feasibility for all future times is guaranteed, a property we
refer to as persistent feasibility.
We would like to gain some insight when persistent feasibility occurs and how it
is aected by the formulation of the control problem and the choice of the controller
parameters. Let us rst recall the various sets introduced in Section 11.2 and
Section 12.2 and how they are inuenced.
c
: The maximal control invariant set c
is only aected by the sets A and |,

the constraints on states and inputs. It is the largest set over which we can
expect any controller to work.
A
0
: A control input U
0
can only be found, i.e. the control problem is feasible,
if x(0) A
0
. The set A
0
depends on A and |, on the controller horizon N
and on the controller terminal set A
f
. It does not depend on the objective
function and it has generally no relation with c
(it can be larger, smaller,

etc.).
O
: The maximal positive invariant set for the closed-loop system depends on
the controller and as such on all parameters aecting the controller, i.e., A, |,
N, A
f
and the objective function with its parameters P, Q and R. Clearly
O
A
0
because if it were not there would be points in O
for which
the control problem is not feasible. Because of invariance, the closed-loop is
persistently feasible for all states x(0) O
.
We can now state necessary and sucient conditions guaranteeing persistent fea-
sibility by means of invariant set theory.
Lemma 13.1 Let O
be the maximal positive invariant set for the closed-loop

system x(k + 1) = f
cl
(x(k)) in (13.10) with constraints (13.2). The RHC problem
is persistently feasible if and only if A
0
= O
Proof: For the RHC problem to be persistently feasible A

0
must be positive
invariant for the closed-loop system. We argued above that O
A
0
. As the
positive invariant set A
0
cannot be larger than the maximal positive invariant set
O
, it follows that A
0
= O
.
As A
0
does not depend on the controller parameters P, Q and R but O
does,
the requirement A
0
= O
for persistent feasibility shows that, in general, only

some P, Q and R are allowed. The parameters P, Q and R aect the performance.
The complex eect they have on persistent feasibility makes their choice extremely
dicult for the design engineer. In the following we will remedy this undesirable
situation. We will make use of the following important sucient condition for
persistent feasibility.
Lemma 13.2 Consider the RHC law (13.6)-(13.9) with N 1. If A
1
is a control
invariant set for system (13.1)-(13.2) then the RHC is persistently feasible. Also,
O
= A
0
is independent of P, Q and R.
Proof: If A
1
is control invariant then, by denition, A
1
Pre(A
1
). Also recall
that Pre(A
1
) = A
0
from the properties of the feasible sets in equation (12.20)
(note that Pre(A
1
) A = Pre(A
1
) from control invariance). Pick some x A
0
and some feasible control u for that x and dene x
+
= Ax + Bu A
1
. Then
x
+
A
1
Pre(A
1
) = A
0
. As u was arbitrary (as long as it is feasible) x
+
A
0
for all feasible u. As A
0
is positive invariant, A
0
= O
from Lemma 13.1. As A

0
is positive invariant for all feasible u, O
does not depend on P, Q and R.

Note that in the proof of Lemma 13.2, persistent feasibility does not depend on the
input u as long as it is feasible. For this reason, sometimes in the literature this
property is referred to persistently feasible for all feasible u.
We can use Lemma 13.2 in the following manner. For N = 1, A
1
= A
f
. If we
choose the terminal set to be control invariant then A
0
= O
and RHC will be

persistently feasible independent of chosen control objectives and parameters. Thus
the designer can choose the parameters to aect performance without aecting
persistent feasibility. A control horizon of N = 1 is often too restrictive, but we
can easily extend Lemma 13.2.
Theorem 13.1 Consider the RHC law (13.6)-(13.9) with N 1. If A
f
is a
control invariant set for system (13.1)-(13.2) then the RHC is persistently feasible.
Proof: If A
f
is control invariant, then A
N1
, A
N2
, . . . , A
1
are control invari-
ant and Lemma 13.2 establishes persistent feasibility for all feasible u.
Corollary 13.1 Consider the RHC law (13.6)-(13.9) with N 1. If there exists
i [1, N] such that A
i
is a control invariant set for system (13.1)-(13.2), then the
RHC is persistently feasible for all cost functions.
Proof: Follows directly from the proof of Theorem 13.1.
Recall that Theorem 12.2 together with Remark 12.4 dene the properties of
the set A
0
as N varies. Therefore, Theorem 13.1 and Corollary 13.1 provide also
guidelines on the choice of the horizon N for guaranteeing persistent feasibility
for all feasible u. For instance, if the RHC problem (13.6) for N =

N yields a
control invariant set A
0
, then from Theorem 12.2 the RHC law (13.6)-(13.9) with
N =

N +1 is persistently feasible for all feasible u. Moreover, from Corollary 13.1
the RHC law (13.6)-(13.9) with N

N + 1 is persistently feasible for all feasible
u.
Corollary 13.2 Consider the RHC problem (13.6)-(13.9). If N is greater than
the determinedness index

N of /
(A
f
) for system (13.1)-(13.2), then the RHC is
persistently feasible.
Proof: The feasible set A
i
for i = 1, . . . , N 1 is equal to the (N i)-step con-
trollable set A
i
= /
Ni
(A
f
). If the maximal controllable set is nitely determined
then A
i
= /
(A
f
) for i N

N. Note that /
(A
f
) is control invariant. Then
persistent feasibility for all feasible u follows from Corollary 13.1.
Persistent feasibility does not guarantee that the closed-loop trajectories con-
verge towards the desired equilibrium point. From Theorem 13.1 it is clear that
one can only guarantee that x(k) A
1
for all k > 0 if x(0) A
0
.
One of the most popular approaches to guarantee persistent feasibility and
stability of the RHC law (13.6)-(13.9) makes use of a control invariant terminal set
A
f
and a terminal cost P which drives the closed-loop optimal trajectories towards
A
f
. A detailed discussion follows in the next section.
13.2.2 Stability of RHC
In this section we will derive the main stability result for RHC. Our objective is
to nd a Lyapunov function for the closed-loop system. We will show next that if
the terminal cost and constraint are appropriately chosen, then the value function
J
0
() is a Lyapunov function.
Theorem 13.2 Consider system (13.1)-(13.2), the RHC law (13.6)-(13.9), the
cost function (13.8) or (13.7) and the closed-loop system (13.10). Assume that
(A0) Q = Q
0, R = R
0, P 0, if cost (13.8) is used, or Q, R, P full

column rank matrices if cost (13.7) is used.
(A1) The sets A, A
f
and | contain the origin in their interior and are closed.
(A2) A
f
is control invariant, A
f
A.
(A3) min
vU, Ax+BvX
f
(p(x) +q(x, v) +p(Ax +Bv)) 0, x A
f
.
Then,
(i) the state of the closed-loop system (13.10) converges to the origin, i.e., lim
k
x(k) =
0,
(ii) the origin of the closed-loop system (13.10) is asymptotically stable with domain
of attraction A
0
.
Proof: Based on Lemma 13.1, 13.1 and Theorem 13.2, from hypothesis (A2) we
conclude that A
0
= O
is a positive invariant set for the closed-loop system (13.10)

for any choice of Q, R and P. Thus persistent feasibility for any feasible input is
guaranteed.
Next we prove convergence and stability. We establish that the function J
0
()
in (13.6) is a Lyapunov function for the closed-loop system. Because the cost J
0
,
the system and the constraints are time-invariant we can study the properties of
J
0
between step k = 0 and step k + 1 = 1.
Consider problem (13.6) at time t = 0. Let x(0) A
0
and let U
0
= u
0
, . . . , u
N1
be the optimizer of problem (13.6) and x

0
= x(0), x
1
, . . . , x
N
be the corre-
sponding optimal state trajectory. After the implementation of u
0
we obtain
x(1) = x
1
= Ax(0) + Bu
0
. Consider now problem (13.6) for t = 1. We will con-
struct an upper bound on J
0
(x(1)). Consider the sequence

U
1
= u
1
, . . . , u
N1
, v
and the corresponding state trajectory resulting from the initial state x(1), x
1
=
x
1
, . . . , x
N
, Ax
N
+Bv. Because x
N
A
f
and (A2) there exists a feasible v such
that x
N+1
= Ax
N
+Bv A
f
and with this v the sequence

U
1
= u
1
, . . . , u
N1
, v
is feasible. Because

U
1
is not optimal J
0
(x(1),

U
1
) is an upper bound on J
0
(x(1)).
Since the trajectories generated by U
0
and

U
1
overlap, except for the rst and
last sampling intervals, it is immediate to show that
J
0
(x(1)) J
0
(x(1),

U
1
) = J
0
(x(0)) q(x
0
, u
0
) p(x
N
)+
(q(x
N
, v) +p(Ax
N
+Bv)).
(13.19)
Let x = x
0
= x(0) and u = u
0
. Under assumption (A3) equation (13.19) becomes
J
0
(Ax +Bu) J
0
(x) q(x, u), x A
0
. (13.20)
Equation (13.20) and the hypothesis (A0) on the matrices R and Q ensure that
J
0
(x) strictly decreases along the state trajectories of the closed-loop system (13.10)
for any x A
0
, x ,= 0. In addition to the fact that J
0
(x) decreases, J
0
(x) is
lower-bounded by zero and since the state trajectories generated by the closed-loop
system (13.10) starting from any x(0) A
0
lie in A
0
for all k 0, equation (13.20)
is sucient to ensure that the state of the closed-loop system converges to zero as
k 0 if the initial state lies in A
0
. We have proven (i).
In order to prove stability via Theorem 8.2 we have to establish that J
0
(x) is
a Lyapunov function. Positivity holds by denition, decrease follows from (13.20).
For continuity at the origin we will show that J
0
(x) p(x), x A
f
and as p(x)
is continuous at the origin J
0
(x) must be continuous as well. From assumption
(A2), A
f
is control invariant and thus for any x A
f
there exists a feasible input
sequence u
0
, . . . , u
N1
for problem (13.6) starting from the initial state x
0
= x
whose corresponding state trajectory is x
0
, x
1
, . . . , x
N
stays in A
f
, i.e., x
i

A
f
i = 0, . . . , N. Among all the aforementioned input sequences u
0
, . . . , u
N1
we focus on the one where u

i
satises assumption (A3) for all i = 0, . . . , N 1.
Such a sequence provides an upper bound on the function J
0
:
J
0
(x
0
)
_
N1
i=0
q(x
i
, u
i
)
_
+p(x
N
), x
i
A
f
, i = 0, . . . , N, (13.21)
which can be rewritten as
J
0
(x
0
)
_
N1
i=0
q(x
i
, u
i
)
_
+p(x
N
)
= p(x
0
) +
_
N1
i=0
q(x
i
, u
i
) +p(x
i+1
) p(x
i
)
_
x
i
A
f
, i = 0, . . . , N,
(13.22)
which from assumption (A3) yields
J
0
(x) p(x), x A
f
. (13.23)
In conclusion, there exist a nite time in which any x A
0
is steered to a level
set of J
0
(x) contained in A
f
after which convergence to and stability of the origin
follows.
Remark 13.1 The assumption on the positive deniteness of Q in Theorem 13.2 can
be relaxed as in standard optimal control by allowing Q 0 with (Q
1
2
, A) observable.
Remark 13.2 The procedure outlined in Theorem 13.2 is, in general, conservative
because it requires the introduction of an articial terminal set .
f
to guarantee
persistent feasibility and a terminal cost to guarantee stability. Requiring xN .
f
usually decreases the size of the region of attraction .0 = O. Also the performance
may be negatively aected.
Remark 13.3 A function p(x) satisfying assumption (A3) of Theorem 13.2 is often
called control Lyapunov function .
The hypothesis (A2) of Theorem 13.2 is required for guaranteeing persistent
feasibility as discussed in Section 13.2.1. In some part of the literature the con-
straint A
f
is not used. However, in this literature the terminal region constraint
A
f
is implicit. In fact, it is typically required that the horizon N is suciently
large to ensure feasibility of the RHC (13.6)(13.9) at all time instants t. Tech-
nically this means that N has to be greater than the determinedness index

N of
system (13.1)-(13.2) which by Corollary 13.2 guarantees persistent feasibility for
all inputs. We refer the reader to Section 13.2.1 for more details on feasibility.
Next we will show a few simple choices for P and A
f
satisfying the hypothesis
(A2) and (A3) of Theorem 13.2.
Stability, 2-Norm case
Consider system (13.1)-(13.2), the RHC law (13.6)-(13.9), the cost function(13.8)
and the closed-loop system (13.10). A simple choice A
f
is obtained by choosing A
f
as the maximal positive invariant set (see Section 11.1) for the closed-loop system
x(k +1) = (A+BF
)x(k) where F
is the associated unconstrained innite time

optimal controller (9.32). With this choice the assumption (A3) in Theorem 13.2
becomes
x
(A
(P PB(B
PB +R)
1
BP)A +QP)x 0, x A
f
, (13.24)
which is satised as an equality if P is chosen as the solution P
of the algebraic
Riccati equation (9.31) for system (13.1) (see proof in Section 9.6).
If system (13.1) is asymptotically stable, then A
f
can be alternatively chosen
as the positive invariant set of the autonomous system x(k + 1) = Ax(k) subject
to the state constraints x A. Therefore in A
f
the input 0 is feasible and the
assumption (A3) in Theorem 13.2 becomes
x
Px +x
PAx +x
Qx 0, x A
f
, (13.25)
which is satised if P solves x
(P +A
PA+Q)x = 0, i.e. the standard Lyapunov

equation (8.44). In this case stability implies exponential stability. The argument
is simple. As the system is closed-loop stable it enters the terminal region in nite
time. If A
f
is chosen as suggested, the closed-loop system is unconstrained after
entering A
f
. For an unconstrained linear system the convergence to the origin is
exponential.
13.3 State Feedback Solution of RHC, 2-Norm Case 263
Stability, 1, -Norm case
Consider system (13.1)-(13.2), the RHC law (13.6)-(13.9), the cost function (13.7)
and the closed-loop system (13.10). Let p = 1 or p = . If system (13.1) is
asymptotically stable, then A
f
can be chosen as the positively invariant set of the
autonomous system x(k + 1) = Ax(k) subject to the state constraints x A.
Therefore in A
f
the input 0 is feasible and the assumption (A3) in Theorem 13.2
becomes
|Px|
p
+|PAx|
p
+|Qx|
p
0, x A
f
, (13.26)
which is the corresponding Lyapunov inequality for the 1, -norm case (8.50)
whose solution has been discussed in Section 8.5.3.
In general, if the unconstrained optimal controller (10.31) exists it is PPWA.
In this case the computation of the maximal invariant set A
f
for the closed-loop
PWA system
x(k + 1) = (A +F
i
)x(k) if H
i
x 0, i = 1, . . . , N
r
(13.27)
is more involved. However if such A
f
can be computed it can be used as terminal
constraint in Theorem 13.2. With this choice the assumption (A3) in Theorem 13.2
is satised by the innite time unconstrained optimal cost P
in (10.32).
13.3 State Feedback Solution of RHC, 2-Norm Case
The state feedback receding horizon controller (13.9) with cost (13.8) for sys-
tem (13.1) is
u(t) = f
0
(x(t)), (13.28)
where f
0
(x
0
) : R
n
R
m
is the piecewise ane solution to the CFTOC (13.6) and
is obtained as explained in Section 12.3.
We remark that the implicit form (13.6) and the explicit form (13.28) describe
the same function, and therefore the stability, feasibility, and performance proper-
ties mentioned in the previous sections are automatically inherited by the piecewise
ane control law (13.28). Clearly, the explicit form (13.28) has the advantage of
being easier to implement, and provides insight into the type of controller action
in dierent regions CR
i
of the state space.
Example 13.3 Consider the double integrator system (13.11) subject to the input
constraints
3
1 u(k) 1 (13.29)
10 x(k) 10. (13.30)
We want to regulate the system to the origin by using the RHC problem (13.6)(13.9)
with cost (13.8), Q = [
1 0
0 1
], R = 0.01, and P = P where P solves the algebraic
3
Riccati equation (9.31). We consider three cases:
Case 1. N = 2, .
f
= 0,
Case 2. N = 2, .
f
is the positively invariant set of the closed-loop system x(k +1) =
(A+BF) where F is the innite time unconstrained optimal controller (9.32).
Case 3. No terminal state constraints: .
f
= R
2
and N = 6 = determinedness in-
dex+1.
From the results presented in this chapter, all three cases guarantee persistent fea-
sibility for all cost functions and asymptotic stability of the origin with region of
attraction .0 (with .0 dierent for each case) . Next we will detail the matrices of
the quadratic program for the on-line solution as well as the explicit solution for the
three cases.
Case 1: .
f
= 0. The mp-QP problem associated with the RHC has the form (12.31)
with
H =
19.08 8.55
8.55 5.31
, F =
10.57 5.29
10.59 5.29
, Y =
10.31 9.33
9.33 10.37
(13.31)
and
G
0
=
0.00 1.00
0.00 1.00
0.00 0.00
1.00 0.00
0.00 0.00
1.00 0.00
1.00 0.00
1.00 1.00
1.00 0.00
1.00 1.00
0.00 0.00
1.00 0.00
0.00 0.00
1.00 0.00
1.00 0.00
1.00 0.00
0.00 0.00
0.00 0.00
0.00 0.00
0.00 0.00
0.00 0.00
0.00 0.00
0.00 0.00
0.00 0.00
1.00 0.00
1.00 1.00
1.00 0.00
1.00 1.00
, E
0
=
0.00 0.00
0.00 0.00
1.00 1.00
1.00 1.00
1.00 1.00
1.00 1.00
0.00 0.00
1.00 1.00
0.00 0.00
1.00 1.00
1.00 1.00
1.00 1.00
1.00 1.00
1.00 1.00
1.00 2.00
1.00 2.00
1.00 0.00
0.00 1.00
1.00 0.00
0.00 1.00
1.00 0.00
0.00 1.00
1.00 0.00
0.00 1.00
0.00 0.00
1.00 1.00
0.00 0.00
1.00 1.00
, w
0
=
1.00
1.00
10.00
10.00
10.00
10.00
10.00
10.00
10.00
10.00
5.00
5.00
5.00
5.00
1.00
1.00
10.00
10.00
10.00
10.00
5.00
5.00
5.00
5.00
0.00
0.00
0.00
0.00
(13.32)
The corresponding polyhedral partition of the state-space is depicted in Fig. 13.6(a).
13.3 State Feedback Solution of RHC, 2-Norm Case 265
x
1
x
2
(a) Polyhedral partition of .0,
Case 1. N = 2, .
f
= 0, N
r
0
= 7.
x
1
x
2
(b) Polyhedral partition of .0,
Case 2. N = 2, .
f
= O
LQR
,
N
r
0
= 9.
x
1
x
2
(c) Polyhedral partition of .0,
Case 3. N = 6, .
f
= R
2
, N
r
0
=
13.
Figure 13.6 Example (13.3): 2-norm objective function, receding horizon
controller. Horizon N, terminal region .
f
. Download code.
The RHC law is:
u =
_
_
[ 0.61 1.61 ] x if
_
0.70 0.71
0.70 0.71
0.70 0.71
0.70 0.71
_
x
_
0.00
0.00
0.00
0.00
_
(Region #1)
[ 1.00 2.00 ] x if
_
_
0.71 0.71
0.70 0.71
0.45 0.89
0.45 0.89
0.71 0.71
0.70 0.71
_
_
x
_
_
0.00
0.00
0.45
0.45
0.71
0.00
_
_
(Region #2)
[ 1.00 2.00 ] x if
_
0.45 0.89
0.70 0.71
0.71 0.71
_
x
_
0.45
0.00
0.00
_
(Region #3)
[ 0.72 1.72 ] x if
_
0.39 0.92
0.70 0.71
0.70 0.71
0.70 0.71
_
x
_
0.54
0.00
0.00
0.00
_
(Region #4)
[ 1.00 2.00 ] x if
_
_
0.45 0.89
0.71 0.71
0.70 0.71
0.45 0.89
0.71 0.71
0.70 0.71
_
_
x
_
_
0.45
0.71
0.00
0.45
0.00
0.00
_
_
(Region #5)
[ 1.00 2.00 ] x if
_
0.45 0.89
0.71 0.71
0.70 0.71
_
x
_
0.45
0.00
0.00
_
(Region #6)
[ 0.72 1.72 ] x if
_
0.39 0.92
0.70 0.71
0.70 0.71
0.70 0.71
_
x
_
0.54
0.00
0.00
0.00
_
(Region #7)
The union of the regions depicted in Fig. 13.6(a) is .0. From Theorem 13.2, .0 is
also the domain of attraction of the RHC law.
Case 2: .
f
positively invariant set. The set .
f
is
.
f
= |x R
2
:
_
0.35617 0.93442
0.35617 0.93442
0.71286 0.70131
0.71286 0.70131
_
x
_
0.58043
0.58043
1.9049
1.9049
_
(13.33)
The corresponding polyhedral partition of the state-space is depicted in Fig. 13.6(b).
The union of the regions depicted in Fig. 13.6(b) is .0. Note that from Theorem 13.2
the set .0 is also the domain of attraction of the RHC law.
Case 3: .
f
= R
n
, N = 6. The corresponding polyhedral partition of the state-space
is depicted in Fig. 13.6(c).
Comparing the feasibility regions .0 in Figure 13.6 we notice that in Case 2 we obtain
a larger region than in Case 1 and that in Case 3 we obtain a feasibility region larger
than Case 1 and Case 2. This can be easily explained from the theory presented in
this and the previous chapter. In particular we have seen that if a control invariant
set is chosen as terminal constraint .
f
, the size of the feasibility region increases
with the number of control moves (increase from Case 2 to Case 3) (Remark 12.3).
Actually in Case 3, .0 = /(.
f
) with .
f
= R
2
, the maximal controllable set. Also,
the size of the feasibility region increases with the size of the target set (increase from
Case 1 to Case 2).
13.4 State Feedback Solution of RHC, 1, -Norm Case
The state feedback receding horizon controller (13.6)(13.9) with cost (13.7) for
system (13.1) is
u(t) = f
0
(x(t)) (13.34)
where f
0
(x
0
) : R
n
R
m
is the piecewise ane solution to the CFTOC (13.6)
and is computed as explained in Section 12.4. As in the 2-norm case the explicit
form (13.34) has the advantage of being easier to implement, and provides insight
into the type of control action in dierent regions CR
i
of the state space.
Example 13.4 Consider the double integrator system (13.11) subject to the input
constraints
4
1 u(k) 1 (13.35)
5 x(k) 5 (13.36)
We want to regulate the system to the origin by using the RHC controller (13.6)
(13.9) with cost (13.7), p = , Q = [
1 0
0 1
], R = 20. We consider two cases:
Case 1. .
f
= R
n
, N = 6 (determinedness index+1) and P = Q
Case 2. .
f
= R
n
, N = 6 and P = |P| given in (10.34) measuring the innite
time unconstrained optimal cost in (10.32).
From Corollary 13.2 in both cases persistent feasibility is guaranteed for all cost func-
tions and .0 = c. However, in Case 1 the terminal cost P does not satisfy (13.26)
4
13.4 State Feedback Solution of RHC, 1, -Norm Case 267
which is assumption (A3) in Theorem 13.2 and therefore the convergence to and the
stability of the origin cannot be guaranteed. In order to satisfy Assumption (A3) in
Theorem 13.2, in Case 2 we select the terminal cost to be equal to the innite time
unconstrained optimal cost computed in Example 10.1.
Next we will detail the explicit solutions for the two cases.
Case 1. The LP problem associated with the RHC has the form (12.56) with

G0
R
12418
,

S0 R
1242
and c
= [06 112]. The corresponding polyhedral partition of

the state-space is depicted in Fig. 13.7(a). The RHC law is:
x
1
x
2
(a) Polyhedral partition of .0,
Case 1.
x
1
x
2
(b) Polyhedral partition of .0,
Case 2.
x
1
x
2
(c) Closed-loop Trajectories, Case
1.
x
1
x
2
(d) Closed-loop Trajectories,
Case 2.
Figure 13.7 Double integrator Example (13.4). Download code.
u =
_
_
0 if
_
0.16 0.99
0.16 0.99
1.00 0.00
1.00 0.00
_
x
_
0.82
0.82
5.00
5.00
_
(Region #1)
[ 0.29 1.71 ] x + 1.43 if
_
1.00 0.00
0.16 0.99
1.00 0.00
0.16 0.99
_
x
_
5.00
0.82
5.00
1.40
_
(Region #2)
1.00 if
_
_
_
_
_
0.16 0.99
1.00 0.00
0.71 0.71
1.00 0.00
0.20 0.98
0.16 0.99
0.24 0.97
0.45 0.89
0.32 0.95
_
_
x
_
_
_
_
_
1.40
5.00
4.24
5.00
2.94
3.04
2.91
3.35
3.00
_
_
(Region #3)
[ 0.29 1.71 ] x 1.43 if
_
1.00 0.00
0.16 0.99
1.00 0.00
0.16 0.99
_
x
_
5.00
0.82
5.00
1.40
_
(Region #4)
1.00 if
_
_
_
_
_
0.32 0.95
0.24 0.97
0.20 0.98
0.16 0.99
1.00 0.00
0.16 0.99
0.71 0.71
0.45 0.89
1.00 0.00
_
_
x
_
_
_
_
_
3.00
2.91
2.94
3.04
5.00
1.40
4.24
3.35
5.00
_
_
(Region #5)
The union of the regions depicted in Fig. 13.7(a) is .0 and is shown in white in
Fig. 13.7(c). Since N is equal to the determinedness index plus one, .0 is a positive
invariant set for the closed-loop system and thus persistent feasibility is guaranteed
for all x(0) .0. However, it can be noticed from Fig. 13.7(c) that convergence to
the origin is not guaranteed. Starting from the initial conditions [-4,2], [-2, 2], [0,
0.5], [4,-2], [-1,-1] and [2,-0.5], the closed-loop system converges to either [5, 0] or
[5, 0].
Case 2. The LP problem associated with the RHC has the form (12.56) with

G0
R
17418
,

S0 R
1742
and c
= [06 112]. The RHC law is dened over 21 regions and

the corresponding polyhedral partition of the state-space is depicted in Fig. 13.7(b).
The union of the regions depicted in Fig. 13.7(b) is .0 and is shown in white in
Fig. 13.7(d). Since N is equal to the determinedness index plus one, .0 is a positive
invariant set for the closed-loop system and thus persistent feasibility is guaranteed
for all x(0) .0. Convergence to the origin is also guaranteed by the choice of P as
shown by the closed-loop trajectories in Fig. 13.7(d) starting from the same initial
conditions as in Case 1.
13.5 Tuning and Practical Use
Receding Horizon Control in the dierent variants and with the many parameter
options as introduced in this chapter is a very powerful control technique to address
complex control problems in practice. At present there is no other technique to
design controllers for general large linear multivariable systems with input and
output constraints with a stability guarantee. The fact that there is no eective
13.5 Tuning and Practical Use 269
tool to analyze the stability of large constrained multivariable systems makes this
stability guarantee by design so important. The application of RHC in practice,
however, requires the designer to make many choices. In this section we will oer
some guidance.
Objective Function
The squared 2-norm is employed more often as an indicator of control quality in
the objective function than the 1- or -norm. The former has many advantages.
The 1- or -norm formulation leads to an LP (12.56) from which the opti-
mal control action is determined. The solution lies always at the intersection of
constraints and changes discontinuously as the tuning parameters are varied. This
makes the formulation of the control problem (what constraints need to be added
for good performance?) and the choice of the weights in the objective function
dicult.
The optimizer may be non-unique in the case of dual degeneracy. This may
cause the control action to vary randomly as the solver picks up dierent optimizers
in succeeding time steps.
The construction of the control invariant target set A
f
and of a suitable ter-
minal cost p(x
N
) needed in Theorem 13.2 is more dicult for the 1- or -norm
formulation because the unconstrained controller is PPWA (Section 13.2.2).
If the optimal control problem is stated in an ad hoc manner by trial and
error rather than on the basis of Theorem 13.2 undesirable response characteristics
may result. The controller may cease to take any action and the system may
get stuck. This happens if any control action leads to a short-term increase in
the objective function, which can occur, for example, for a system with inverse
response characteristics.
Finally, the 1- or -norm formulation involves many more constraints than the
2-norm formulation. In general, this will lead to a larger number of regions of the
explicit control law. These regions do not need to be stored, however, but only
the value function and the controller, and the search for the region containing the
present state can be executed very eciently (Chapter 15).
Despite the discussed deciencies it may be advantageous to use the 1- or -
norm in cases when the control objective is not just an indirect tool to achieve a
control specication but reects the economics of the process. For example, the
costs of electrical power depend sometimes on peak power that can be captured
with the -norm. However, even in these cases, quadratic terms are often added
to the objective for regularization.
Design via Theorem 13.2
First, we need to choose the horizon length N and the control invariant target
set A
f
. Then we can vary the parameters Q and R freely to aect the control
performance in the same spirit as we do for designing an LQR. Stability is assured
as long as we adjust P according to the outlined procedures when changing Q and
R.
The longer the horizon N, the larger the maximal controllable set /
N
(A
f
) over
which the closed-loop system is guaranteed to be able to operate (this is true as
long as the horizon is smaller than the determinedness index of /
(A
f
)). On the
other hand, with the control horizon increases the on-line computational eort or
the complexity of the explicit controller determined o-line.
We need to keep in mind that the terminal set A
f
is introduced articially
for the sole purpose of leading to a sucient condition for persistent feasibility.
We want it to be large so that it does not compromise closed-loop performance.
The larger A
f
, the larger /
N
(A
f
). Though it is simplest to choose A
f
= 0, it is
undesirable unless N is chosen large. Ideally A
f
should be the maximal control
invariant set achieved with the unconstrained controller.
Specically, we rst design the unconstrained optimal controller as suggested
at the end of Section 13.2.2. From this construction we also obtain the terminal
cost satisfying condition (A3) of Theorem 13.2 to use in the RHC design. Then
we determine the maximal positive invariant set for the closed-loop system with
the unconstrained controller and use this set as A
f
. This set is usually dicult
to compute for systems of large dimension with the algorithms we introduced in
Section (11.2).
Note that for stable systems without state constraints /
(A
f
) = R
n
always,
i.e. the choice of A
f
is less critical. For unstable systems /
(A
f
) is the region
over which the system can operate stably in the presence of input constraints and
is of eminent practical importance.
State/Output Constraints
State constraints arise from practical restrictions on the allowed operating range
of the system. Thus, contrary to input constraints, they are rarely hard. They
can lead to complications in the controller implementation, however. As it can
never be excluded that the state of the real system moves outside the constraint
range chosen for the controller design, special provisions must be made (patches in
the algorithm) to move the state back into the range. This is dicult and these
types of patches are exactly what one wanted to avoid by choosing MPC in the
rst place.
Thus, typically, state constraints are softened in the MPC formulation. For
example,
x x
max
is approximated by
x x
max
+, 0,
and a term l() is added to the objective function to penalize violations of the
constraint. This formulation may, however, lead to a violation of the constraint
even when a feasible input exists that avoids it but is not optimal for the new
objective function with the penalty term added. Let us analyze how to choose l()
such that this does not occur [71].
13.5 Tuning and Practical Use 271
As an example take
J
= min
z
f(z)
subj. to g(z) 0,
(13.37)
where g(z) is assumed to be scalar for simplicity and f(z) to be strictly convex.
Let us soften the constraints and add the penalty as suggested
p() = min
z
f(z) +l()
subj. to g(z) ,
(13.38)
where 0. We want to choose the penalty l() so that by minimizing the
augmented objective we recover the solution to the original problem if it exists:
p(0) = J
and
arg min
0
p() = 0
which are equivalent to
p(0) = J
(13.39)
and
p() > p(0), > 0. (13.40)
For (13.39) we require l(0) = 0. To construct l() to satisfy (13.40) we assume that
strong duality holds and u
exists so that
J
= min
z
(f(z) +u
g(z)), (13.41)
where u
is an optimal dual variable. As the optimizer z
of (13.37) satises
g(z
) 0 we can add a redundant constraint without aecting the solution

J
= min
z
(f(z) +u
g(z))
subj. to g(z)
0. (13.42)
Then we can state the bounds
J
min
z
(f(z) +u
)
subj. to g(z) ,
<
p() = min
z
(f(z) +u)
subj. to g(z) ,
> 0, u > u
.
(13.43)
Thus l() = u with u > u
0 is a possible penalty term satisfying the require-

ments (13.39)-(13.40).
Because of smoothness
l() = u +v
2
, u > u
, v > 0 (13.44)
is preferable. On the other hand, note that l() = v
2
does not satisfy an inequality
of the form (13.43). Therefore it should not be used as it can lead to optimizers
z
in (13.38) which violate g(z) 0 even if a feasible optimizer to the original

problem exists.
These ideas can be extended to multiple constraints g
j
(z) 0, j = 1, . . . , r via
the penalty term
l() = u
r
j=1
j
+v
r
j=1
2
j
, (13.45)
where
u > max
j{1,...,r}
u
j
, v 0. (13.46)
Formulations also exist where the necessary constraint violations are following
a prescribed order so that less important constraints are violated rst [171].
Time-varying references, constraints, disturbances and system pa-
rameters
The standard RHC formulation (13.6) can be easily extended to include these
features. Known disturbances are simply included in the prediction model. If the
system states are not to return to the origin, but some output y is to follow some
trajectory r, then appropriate penalties of the error e = y r are included in the
control objective. How to do this and to achieve oset-free tracking is described in
Section 13.6. Theorem 13.2 can be used to design the controller in these cases.
If the constraints are time-varying then A and | become time-varying. For
example, the constraints may be shaped like a funnel tightening towards the end
of the horizon.
If the underlying system is nonlinear, one often uses a locally linearized model
for the prediction and update it at each time step. Note that the results of The-
orem 13.2 do not apply when the system model and/or the constraints are time-
varying.
In all these cases, the optimization problem to be solved on line, (12.31) or (12.56),
does not change in structure but some of the dening matrices will now change at
each time step, which will increase the necessary on-line computational eort some-
what.
If the controller is to be computed explicitly o-line, then all the varying pa-
rameters (disturbances, constraints, references), which we will denote by , be-
come parameters in the multi-parametric optimization problem and the resulting
controller becomes an explicit function of them: u(t) = F(x(t), ). As we have
learned, the complexity of the solution of mp-QP and mp-LP problems depends
primarily on the number of constraints. If the number of parameters aects the
number of constraints then this may only be possible for a relatively small number
of parameters. Thus, the possibilities to take into account time-varying control
problem features in explicit MPC are rather limited. Time-varying models cannot
be handled at all.
13.6 Oset-Free Reference Tracking 273
Multiple Horizons and Move-Blocking
Unfortunately, as we started to discuss in the previous paragraphs, in challenging
applications, the design procedure implied by Theorem 13.2 may not be strictly
applicable. Then the closed-loop behavior with the RHC has to be analyzed by
other means, in the worst case through extensive simulation studies. In principle,
this oers us also more freedom in the problem formulation and the choice of the
parameters, for example, in order to reduce the computational complexity. The
basic RHC formulation (13.6) may be modied as follows:
min
U0
p(x
Ny
) +
Ny1
k=0
q(x
k
, u
k
)
subj. to x
k+1
= Ax
k
+Bu
k
, k = 0, . . . , N
y
1
x
k
A, k = 0, . . . , N
c
u
k
|, k = 0, . . . , N
u
u
k
= Kx
k
, N
u
< k < N
y
(13.47)
where K is some feedback gain, N
y
, N
u
, N
c
are the prediction, input, and state
constraint horizons, respectively, with N
u
N
y
and N
c
N
y
. This formulation
reduces the number of constraints and as a consequence makes the long horizon
prediction used in the optimization less accurate as it is not forced to obey all the
constraints. As this approximation aects only the states far in the future, it is
hoped that it will not inuence signicantly the present control action.
Generalized Predictive Control (GPC) in its most common formulation [88] has
multiple horizons as an inherent feature but does not include constraints. Experi-
ence and theoretical analysis [55] have shown that it is very dicult to choose all
these horizons that aect not only performance but even stability in a non-intuitive
fashion. Thus, for problems where constraints are not important and the adaptive
features of GPC are not needed, it is much preferable to resort to the well estab-
lished LQR and LQG controllers for which a wealth of stability, performance and
robustness results have been established.
Another more eective way to reduce the computational eort is move-blocking
where the manipulated variables are assumed to be xed over time intervals in
the future thus reducing the degrees of freedom in the optimization problem. By
choosing the blocking strategies carefully, the available RHC stability results remain
applicable [72].
13.6 Oset-Free Reference Tracking
This section describes how the RHC problem has to be formulated to track con-
stant references without oset under model mismatch. We distinguish between the
number p of measured outputs, the number r of outputs which one desires to track
(called tracked outputs), and the number n
d
of disturbances. First we summa-
rize the conditions that need to be satised to obtain oset-free RHC by using the
arguments of the internal model principle. Then we provide a simple proof of zero
steady-state oset when r p = n
d
. Extensive treatment of reference tracking
for RHC can be found in in [13, 216, 223, 224, 195]. Consider the discrete-time
time-invariant system
_
_
_
x
m
(t + 1) = f(x
m
(t), u(t))
y
m
(t) = g(x
m
(t))
z(t) = Hy
m
(t).
(13.48)
In (13.48), x
m
(t) R
n
, u(t) R
m
and y
m
(t) R
p
are the state, input, measured
output vector, respectively. The controlled variables z(t) R
r
are a linear combi-
nation of the measured variables for which oset-free behaviour is sought. Without
any loss of generality we assume H to have full row rank.
The objective is to design an RHC based on the linear system model (13.1)
of (13.48) in order to have z(t) track r(t), where r(t) R
r
is the reference signal,
which we assume to converge to a constant, i.e. r(t) r
as t . We require
zero steady-state tracking error, i.e., (z(t) r(t)) 0 for t .
The Observer Design
The plant model (13.1) is augmented with a disturbance model in order to capture
the mismatch between (13.48) and (13.1) in steady state. Several disturbance
models have been presented in the literature [13, 203, 190, 224, 223, 292]. Here we
follow [224] and use the form:
_
_
_
x(t + 1) = Ax(t) + Bu(t) +B
d
d(t)
d(t + 1) = d(t)
y(t) = Cx(t) +C
d
d(t)
(13.49)
with d(t) R
n
d
. With abuse of notation we have used the same symbols for state
and outputs of system (13.1) and system (13.49). Later we will focus on specic
versions of the model (13.49).
The observer estimates both states and disturbances based on this augmented
model. Conditions for the observability of (13.49) are given in the following theo-
rem.
Theorem 13.3 [213, 214, 224, 13] The augmented system (13.49) is observable
if and only if (C, A) is observable and
_
A I B
d
C C
d
_
(13.50)
has full column rank.
Proof: From the Hautus observability condition system (13.49) is observable
i
_
A
I 0 C
d
I I C
d
_
has full row rank . (13.51)
Again from the Hautus condition, the rst set of rows is linearly independent i
(C, A) is observable. The second set of rows is linearly independent from the rst
n rows except possibly for = 1. Thus, for the augmented system the Hautus
condition needs to be checked for = 1 only, where it becomes (13.50).
Remark 13.4 Note that for condition (13.50) to be satised the number of distur-
bances in d needs to be smaller or equal to the number of available measurements in y,
n
d
p. Condition (13.50) can be nicely interpreted. It requires that the model of the
disturbance eect on the output d y must not have a zero at (1, 0). Alternatively
we can look at the steady state of system (13.49)
_
A I B
d
C C
d
_ _
x
d
_
=
_
0
y
_
, (13.52)
where we have denoted the steady state values with a subscript and have omit-
ted the forcing term u for simplicity. We note that from the observability condi-
tion (13.50) for system (13.49) equation (13.52) is required to have a unique solution,
which means, that we must be able to deduce a unique value for the disturbance d
from a measurement of y in steady state.
The following corollary follows directly from Theorem 13.3.
Corollary 13.3 The augmented system (13.49) with n
d
= p and C
d
= I is ob-
servable if and only if (C, A) is observable and
det
_
A I B
d
C I
_
= det(A I B
d
C) ,= 0. (13.53)
Remark 13.5 We note here how the observability requirement restricts the choice of
the disturbance model. If the plant has no integrators, then det (A I) ,= 0 and we
can choose B
d
= 0. If the plant has integrators then B
d
has to be chosen specically
to make det (A I B
d
C) ,= 0.
The state and disturbance estimator is designed based on the augmented model
as follows:
_
x(t + 1)
d(t + 1)
_
=
_
A B
d
0 I
_ _
x(t)
d(t)
_
+
_
B
0
_
u(t)+
_
L
x
L
d
_
(y
m
(t)+C x(t)+C
d

d(t)),
(13.54)
where L
x
and L
d
are chosen so that the estimator is stable. We remark that the
results below are independent of the choice of the method for computing L
x
and
L
d
. We then have the following property.
Lemma 13.3 Suppose the observer (13.54) is stable. Then, rank(L
d
) = n
d
.
Proof: From (13.54) it follows
_
x(t + 1)
d(t + 1)
_
=
_
A +L
x
C B
d
+L
x
C
d
L
d
C I +L
d
C
d
_ _
x(t)
d(t)
_
+
_
B
0
_
u(t)
_
L
x
L
d
_
y
m
(t).
(13.55)
By stability, the observer has no poles at (1, 0) and therefore
det
__
A I +L
x
C B
d
+L
x
C
d
L
d
C L
d
C
d
__
,= 0. (13.56)
For (13.56) to hold, the last n
d
rows of the matrix have to be of full row rank.
A necessary condition is that L
d
has full row rank.
In the rest of this section, we will focus on the case n
d
= p.
Lemma 13.4 Suppose the observer (13.54) is stable. Choose n
d
= p. The steady
state of the observer (13.54) satises:
_
A I B
C 0
_ _
x
_
=
_
B
d

d
y
m,
C
d

d
_
, (13.57)
where y
m,
and u
are the steady state measured output and input of the sys-
tem (13.48), x
and

d
are state and disturbance estimates from the observer (13.54)

at steady state, respectively.
Proof: From (13.54) we note that the disturbance estimate

d converges only
if L
d
(y
m,
+C x
+C
d

d
) = 0. As L
d
is square by assumption and nonsingular
by Lemma 13.3 this implies that at steady state, the observer estimates (13.54)
satisfy
y
m,
+C x
+C
d

d
= 0. (13.58)
Equation (13.57) follows directly from (13.58) and (13.54).
The MPC design
Denote by z
= Hy
m,
and r
the tracked measured outputs and their references

at steady state, respectively. For oset-free tracking at steady state we want z
=
r
. The observer condition (13.57) suggests that at steady state the MPC should
satisfy
_
A I B
HC 0
_ _
x
_
=
_
B
d

d
HC
d

d
_
, (13.59)
where x
is the MPC state at steady state. For x
and u
to exist for any

d
and r
the matrix
_
A I B
HC 0
_
must be of full row rank which implies m r.
The MPC is designed as follows
min
U0
(x
N
x
t
)
P(x
N
x
t
) +
N1
k=0
(x
k
x
t
)
Q(x
k
x
t
) + (u
k
u
t
)
R(u
k
u
t
)
subj. to x
k+1
= Ax
k
+Bu
k
+B
d
d
k
, k = 0, . . . , N
x
k
A, u
k
|, k = 0, . . . , N 1
x
N
A
f
d
k+1
= d
k
, k = 0, . . . , N
x
0
= x(t)
d
0
=

d(t),
(13.60)
with the targets u
t
and x
t
given by
_
A I B
HC 0
_ _
x
t
u
t
_
=
_
B
d

d(t)
r(t) HC
d

d(t)
_
(13.61)
and where Q _ 0, R 0, and P 0.
Let U
(t) = u
0
, . . . , u
N1
be the optimal solution of (13.60)-(13.61) at time
t. Then, the rst sample of U
(t) is applied to system (13.48)

u(t) = u
0
. (13.62)
Denote by c
0
( x(t),

d(t), r(t)) = u
0
( x(t),

d(t), r(t)) the control law when the esti-
mated state and disturbance are x(t) and

d(t), respectively. Then the closed-loop
system obtained by controlling (13.48) with the MPC (13.60)-(13.61)-(13.62) and
the observer (13.54) is:
x(t + 1) = f(x(t), c
0
( x(t),

d(t), r(t)))
x(t + 1) = (A +L
x
C) x(t) + (B
d
+L
x
C
d
)
d(t) + Bc
0
( x(t),

d(t), r(t)) L
x
y
m
(t)
d(t + 1) = L
d
C x(t) + (I +L
d
C
d
)
d(t) L
d
y
m
(t).
(13.63)
Often in practice, one desires to track all measured outputs with zero oset.
Choosing n
d
= p = r is thus a natural choice. Such a zero-oset property continues
to hold if only a subset of the measured outputs are to be tracked, i.e., n
d
= p > r.
Next we provide a very simple proof for oset-free control when n
d
= p.
Theorem 13.4 Consider the case n
d
= p. Assume that for r(t) r
as t ,
the MPC problem (13.60)-(13.61) is feasible for all t N
+
, unconstrained for t j
with j N
+
and the closed-loop system (13.63) converges to x
,

d
, y
m,
, i.e.,
x(t) x
,

d(t)

d
, y
m
(t) y
m,
as t . Then z(t) = Hy
m
(t) r
as
t .
Proof: Consider the MPC problem (13.60)-(13.61). At steady state u(t)
u
= c
0
( x
,

d
, r
), x
t
x
and u
t
u
. Note that the steady state con-

troller input u
(computed and implemented) might be dierent from the steady

state target input u
.
The asymptotic values x
, x
, u
and u
satisfy the observer conditions

(13.57)
_
A I B
C 0
_ _
x
_
=
_
B
d

d
y
m,
C
d

d
_
(13.64)
and the controller requirement (13.61)
_
AI B
HC 0
_ _
x
_
=
_
B
d

d
HC
d

d
_
. (13.65)
Dene x = x
, u = u
and the oset = z
. Notice that the

steady state target values x
and u
are both functions of r
and

d
as given
by (13.65). Left multiplying the second row of (13.64) by H and subtracting (13.65)
from the result, we obtain
(A I)x +Bu = 0
HCx = .
(13.66)
Next we prove that x =0 and thus = 0.
Consider the MPC problem (13.60)-(13.61) and the following change of variables
x
k
= x
k
x
t
, u
k
= u
k
u
t
. Notice that Hy
k
r(t) = HCx
k
+ HC
d
d
k

r(t) = HCx
k
+ HC x
t
+ HC
d
d
k
r(t) = HCx
k
from condition (13.61) with
d(t) = d
k
. Similarly, one can show that x
k+1
= Ax
k
+ Bu
k
. Then, the MPC
problem (13.60) becomes:
min
u0,...,uN1
x
N
Px
N
+
N1
k=0
x
k
Qx
k
+u
k
Ru
k
subj. to x
k+1
= Ax
k
+Bu
k
, 0 k N
x
k
A, u
k
|, k = 0, . . . , N 1
x
N
A
f
x
0
= x(t),
x(t) = x(t) x
t
.
(13.67)
Denote by K
MPC
the unconstrained MPC controller (13.67), i.e., u
0
= K
MPC
x(t).
At steady state u
0
u
= u and x(t) x
= x. Therefore, at
steady state, u = K
MPC
x. From (13.66)
(A I +BK
MPC
)x = 0. (13.68)
By assumption the unconstrained system with the MPC controller converges. Thus
K
MPC
is a stabilizing control law, which implies that (A I + BK
MPC
) is non-
singular and hence x = 0.
Remark 13.6 Theorem 13.4 was proven in [224] by using a dierent approach.
Remark 13.7 Theorem 13.4 can be extended to prove local Lyapunov stability of
the closed-loop system (13.63) under standard regularity assumptions on the state
update function f in (13.63) [201].
Remark 13.8 The proof of Theorem 13.4 assumes only that the models used for the
control design (13.1) and the observer design (13.49) are identical in steady state in
the sense that they give rise to the same relation z = z(u, d, r). It does not make
any assumptions about the behavior of the real plant (13.48), i.e. the model-plant
mismatch, with the exception that the closed-loop system (13.63) must converge to
a xed point. The models used in the controller and the observer could even be
dierent as long as they satisfy the same steady state relation.
Remark 13.9 If condition (13.61) does not specify xt and ut uniquely, it is customary
to determine xt and ut through an optimization problem, for example, minimizing
the magnitude of ut subject to the constraint (13.61) [224].
Remark 13.10 Note that in order to achieve no oset we augmented the model of the
plant with as many disturbances (and integrators) as we have measurements (n
d
= p)
(cf. equation (13.55)). Our design procedure requires the addition of p integrators
even if we wish to control only a subset of r < p measured variables. This is actually
not necessary as we suspect from basic system theory. The design procedure for the
case n
d
= r < p is, however, more involved.
If the squared 2-norm in the objective function of (13.60) is replaced with a
1- or - norm (|P(x
N
x
t
)|
p
+
N1
k=0
|Q(x
k
x
t
)|
p
+ |R(u
k
u
t
)|
p
, where
p = 1 or p = ), then our results continue to hold. In particular, Theorem 13.4
continues to hold. In fact, the unconstrained MPC controlled K
MPC
in (13.67) is
piecewise linear around the origin [42]. In particular, around the origin, u
(t) =
u
0
= K
MPC
(x(t)) is a continuous piecewise linear function of the state variation
x:
K
MPC
(x) = F
j
x if H
j
x K
j
, j = 1, . . . , N
r
, (13.69)
where H
j
and K
j
in equation (13.69) are the matrices describing the j-th poly-
hedron CR
j
= x R
n
[H
j
x K
j
inside which the feedback optimal control
law u
(t) has the linear form F

j
x(k). The polyhedra CR
j
, j = 1, . . . , N
r
are
a partition of the set of feasible states of problem (13.60) and they all contain
the origin. For Theorem 13.4 to hold, it is sucient to require that all the linear
feedback laws F
j
x(k) for i = j, . . . , N
r
are stabilizing.
Explicit Controller
Examining (13.60), (13.61) we note that the control law depends on x(t),

d(t) and
r(t). Thus in order to achieve oset free tracking of r outputs out of p measurements
we had to add the p +r parameters

d(t) and r(t) to the usual parameters x(t).
There are more involved RHC design techniques to obtain oset-free control
for models with n
d
< p and in particular, with minimum order disturbance models
n
d
= r. The total size of the parameter vector can thus be reduced to n + 2r.
This is signicant only if a small subset of the plant outputs are to be controlled.
A greater reduction of parameters can be achieved by the following method. By
Corollary 13.3, we are allowed to choose B
d
= 0 in the disturbance model if the
plant has no integrators. Recall the target conditions 13.61 with B
d
= 0
_
A I B
HC 0
_ _
x
t
u
t
_
=
_
0
r(t) HC
d

d(t)
_
. (13.70)
Clearly, any solution to (13.70) can be parameterized by r(t) HC
d

d(t). The
explicit control law is written as u(t) = c
0
( x(t), r(t) HC
d

d(t)), with only n + r
parameters. Since the observer is unconstrained, complexity is much less of an
issue. Hence, a full disturbance model with n
d
= p can be chosen to yield oset-
free control.
Remark 13.11 The choice of B
d
= 0 might be limiting in practice. In [14], the
authors have shown that for a wide range of systems, if B
d
= 0 and a Kalman lter is
chosen as observer, then the closed-loop system might suer a dramatic performance
deterioration.
Delta Input (u) Formulation.
In the u formulation, the MPC scheme uses the following linear time-invariant
system model of (13.48):
_
_
_
x(t + 1) = Ax(t) +Bu(t)
u(t) = u(t 1) +u(t)
y(t) = Cx(t).
(13.71)
System (13.71) is controllable if (A, B) is controllable. The u formulation often
arises naturally in practice when the actuator is subject to uncertainty, e.g. the
exact gain is unknown or is subject to drift. In these cases, it can be advantageous
to consider changes in the control value as input to the plant. The absolute control
value is estimated by the observer, which is expressed as follows
_
x(t + 1)
u(t + 1)
_
=
_
A B
0 I
_ _
x(t)
u(t)
_
+
_
B
I
_
u(t) +
_
L
x
L
u
_
(y
m
(t) +C x(t)).
(13.72)
The MPC problem is readily modied
min
u0,...,uN1
|y
k
r
k
|
2
Q
+|u
k
|
2
R
subj. to x
k+1
= Ax
k
+Bu
k
, k 0
y
k
= Cx
k
k 0
x
k
A, u
k
|, k = 0, . . . , N 1
x
N
A
f
u
k
= u
k1
+u
k
, k 0
u
1
= u(t)
x
0
= x(t).
(13.73)
The control input applied to the system is
u(t) = u
0
+u(t 1). (13.74)
The input estimate u(t) is not necessarily equal to the actual input u(t). This
scheme inherently achieves oset-free control, there is no need to add a disturbance
model. To see this, we rst note that u
0
= 0 in steady-state. Hence our analysis
applies as the u formulation is equivalent to a disturbance model in steady-state.
This is due to the fact that any plant/model mismatch is lumped into u(t). Indeed
this approach is equivalent to an input disturbance model (B
d
= B, C
d
= 0). If
in (13.73) the measured u(t) were substituted for its estimate, i.e. u
1
= u(t 1),
then the algorithm would show oset.
In this formulation the computation of a target input u
t
and state x
t
is not
required. A disadvantage of the formulation is that it is not applicable when there
is an excess of manipulated variables u compared to measured variables y, since
detectability of the augmented system (13.71) is then lost.
Minimum-Time Controller
In minimum-time control, the cost function minimizes the predicted number of
steps necessary to reach a target region, usually the invariant set associated with
the unconstrained LQR controller [167]. This scheme can reduce the on-line com-
putation time signicantly, especially for explicit controllers (Section 12.5). While
minimum-time MPC is computed and implemented dierently from standard MPC
controllers, there is no dierence between the two control schemes at steady-state.
In particular, one can choose the target region to be the unconstrained region
of (13.60)-(13.61). When the state and disturbance estimates and reference are
within this region, the control law is switched to (13.60)-(13.61). The analysis and
methods presented in this section therefore apply directly.
Although the basic idea of receding horizon control was already indicated by the
theoretical work of Propoi [237] in 1963 it did not gain much attention until the
mid-1970s, when Richalet and coauthors [246, 247] proposed the MPC technique
(they called it Model Predictive Heuristic Control (MPHC)). Shortly there-
after, Cutler and Ramaker [89] introduced the predictive control algorithm called
Dynamic Matrix Control (DMC) which has been hugely successful in the petro-
chemical industry. A vast variety of dierent names and methodologies followed,
such as Quadratic Dynamic Matrix Control (QDMC), Adaptive Predictive Control
(APC), Generalized Predictive Control (GPC), Sequential Open Loop Optimiza-
tion (SOLO), and others.
While the mentioned algorithms are seemingly dierent, they all share the same
structural features: a model of the plant, the receding horizon idea, and an opti-
mization procedure to obtain the control action by optimizing the systems pre-
dicted evolution.
For complex constrained multivariable control problems, model predictive con-
trol has become the accepted standard in the process industries [238]. If in the nite
time optimal control problem solved by MPC at each time step, system model and
constraints are linear and the performance index is expressed as weighted sum of
2-norm, 1-norm or -norm vectors, then the resulting optimization problem can be
cast as a quadratic program (QP) or linear program (LP), respectively, for which
a rich variety of ecient active-set and interior-point solvers are available.
Some of the rst industrial MPC algorithms like IDCOM [247] and DMC [89]
were developed for constrained MPC with quadratic performance indices. However,
in those algorithms input and output constraints were treated in an indirect ad-hoc
fashion. Only later, algorithms like QDMC [114] overcame this limitation by em-
ploying quadratic programming to solve constrained MPC problems with quadratic
performance indices. During the same period, the use of linear programming was
being studied by Gutman and coauthors [136, 137, 139]. Later an extensive theoret-
ical eort was devoted to analyze such schemes, provide conditions for guaranteeing
feasibility and closed-loop stability, and highlight the relations between MPC and
the linear quadratic regulator (see the recent survey [202]).
An extensive treatment of conditions for feasibility and stability of MPC can also
be found in the surveys [202, 200]. Theorem 13.2 in this book presents the main
result on feasibility and stability of MPC ant it has been extracted from [202, 200].
The main idea of Theorem 13.2 dates back to Keerthi and Gilbert [168], the rst to
propose specic choices of a terminal cost P and a terminal constraint A
f
, namely
A
f
= 0 and P = 0. Under this assumption (and a few other mild ones) Keerthi
and Gilbert prove the stability for general nonlinear performance functions and
nonlinear models. The work of Keerthi and Gilbert has been followed by a number
of stability result for RHC including the one in [168, 33, 1, 154, 83]. If properly
analyzed all these results have the main components required in Theorem 13.2.
The richness of theoretical and computational issues related to MPC have generated
in the past fty years a large number of research studies appeared in conference
proceedings, archival journals and research monographs. It is not the intent of
this short paragraph to be an exhaustive literature review on Model Predictive
Control. The interested reader is also referred to recent books on Model Predictive
Control [75, 194, 245] for further reading on the topic.
Chapter 14
Approximate Receding Horizon
Control
Contributed by Prof. Colin N. Jones
Ecole Polytechnique Federale de Lausanne
colin.jones@ep.ch
This chapter will discuss approaches to dene approximate explicit control laws
of desired complexity that provide certicates of recursive feasibility and stability.
The methods introduced all have the following properties:
1. Operations are only on the problem data, and the methods do not require
the computation of the explicit control law.
2. Any convex parametric problem can be approximated, not just those for
which the explicit solution can be computed.
3. The desired level of suboptimality or the desired complexity (in terms of
online storage and/or FLOPS) can be specied.
We begin by assuming that a controller with the desired properties has already
been dened via one of the predictive control methods introduced in Chapters 13,
but that solving the requisite optimization problem online, or evaluating the explicit
solution oine, is not possible due to computational limitations. This chapter
begins by itemizing the specic assumptions on this optimal control law. Then,
the key theorem is introduced, which provides sucient conditions under which an
approximate control law will be stabilizing and recursively invariant. Section 14.2
describes an abstract algorithm that will provide these properties, and Section 14.3
reviews a number of specic implementations of this algorithm that have been
proposed in the literature.
284 14 Approximate Receding Horizon Control
14.1 Stability of Approximate Receding Horizon Control
Recall problem (13.6)
J
0
(x(t)) = min
U0
J
0
(x(t), U
0
) = p(x
N
) +
N1
k=0
q(x
k
, u
k
)
subj. to x
k+1
= Ax
k
+Bu
k
, k = 0, . . . , N 1
x
k
A, u
k
|, k = 0, . . . , N 1
x
N
A
f
x
0
= x(t).
(14.1)
The cost function is either
J
0
(x(0), U
0
) = |Px
N
|
p
+
N1
k=0
|Qx
k
|
p
+|Ru
k
|
p
(14.2)
with p = 1 or p = , or
J
0
(x(0), U
0
) = x
N
Px
N
+
N1
k=0
x
k
Qx
k
+u
k
Ru
k
. (14.3)
Denote a suboptimal feasible solution of (14.1) by

U
0
= u
0
, . . . , u
N1
. The
corresponding suboptimal MPC law is
u(t) =

f
0
(x(t)) = u
0
(x(t)) (14.4)
and the closed-loop system
x(k + 1) = Ax(k) +B

f
0
(x(k)) =

f
cl
(x(k)), k 0. (14.5)
In the following we will prove that the stability of receding horizon control is
preserved when the suboptimal control law u
0
(x(t)) is used as long as the level of
suboptimality is less than the stage cost, i.e.
J
0
(x,

U
0
(x)) < J
0
(x) +q(x, 0).
This is stated more precisely in the following Theorem 14.1.
Theorem 14.1 Assume
(A0) Q = Q
0, R = R
0, P 0, if the quadratic cost (14.3) is used, or

Q, R, P full column rank matrices if the piecewise linear cost (14.2) is used.
(A1) The sets A, A
f
and | contain the origin in their interior and are closed.
(A2) A
f
is control invariant, A
f
A.
(A3) min
vU, Ax+BvX
f
(p(x) +q(x, v) +p(Ax +Bv)) 0, x A
f
.
14.1 Stability of Approximate Receding Horizon Control 285
(A4) The suboptimal

U
0
(x) satises J
0
(x,

U
0
(x)) J
0
(x) + (x), x A
0
,
_ 0.
(A5) (x) q(x, 0) 0, x A
0
, x ,= 0.
Then,
(i) the state of the closed-loop system (14.5) converges to the origin, i.e., lim
k
x(k) =
0,
(ii) the origin of the closed-loop system (14.5) is asymptotically stable with domain
of attraction A
0
.
Proof:
The proof follows the proof of Theorem 13.2. We focus only on the arguments
for convergence, which are dierent for the suboptimal MPC controller. We estab-
lish that the function J
0
() in (14.1) strictly decreases for the closed-loop system.
Because the cost J
0
, the system and the constraints are time-invariant we can study
the properties of J
0
between step k = 0 and step k + 1 = 1.
Consider problem(14.1) at time t = 0. Let x(0) A
0
and let

U
0
= u
0
, . . . , u
N1
be a feasible suboptimal solution to problem (14.1) and x

0
= x(0), x
1
, . . . , x
N
be
the corresponding state trajectory. We denote by

J
0
(x(0)) the cost at x(0) when
the feasible

U
0
is applied,

J
0
(x(0)) = J
0
(x(0),

U
0
).
After the implementation of u
0
we obtain x(1) = x
1
= Ax(0) +B u
0
. Consider
now problem (14.1) for t = 1. We will construct an upper bound on J
0
(x(1)).
Consider the sequence

U
1
= u
1
, . . . , u
N1
, v and the corresponding state trajec-
tory resulting from the initial state x(1), x
1
= x
1
, . . . , x
N
, Ax
N
+ Bv. Because
x
N
A
f
and (A2) there exists a feasible v such that x
N+1
= Ax
N
+ Bv A
f
and with this v the sequence

U
1
= u
1
, . . . , u
N1
, v is feasible. Because

U
1
is not
optimal J
0
(x(1),

U
1
) is an upper bound on J
0
(x(1)).
Since the trajectories generated by

U
0
and

U
1
overlap, except for the rst and
last sampling intervals, it is immediate to show that
J
0
(x(1)) J
0
(x(1),

U
1
) =

J
0
(x(0)) q(x
0
, u
0
) p(x
N
)+
(q(x
N
, v) +p(Ax
N
+Bv)).
(14.6)
Let x = x
0
= x(0) and u = u
0
. Under assumption (A3) equation (14.6) becomes
J
0
(Ax +Bu)

J
0
(x) q(x, u), x A
0
. (14.7)
Under assumption (A4) equation (14.7) becomes
J
0
(Ax +Bu) J
0
(x) (x) q(x, u) (x) q(x, 0), x A
0
. (14.8)
Equation (14.8), the hypothesis (A0) on the matrices Q and R and the hypothesis
(A5) ensure that J
0
(x) strictly decreases along the state trajectories of the closed-
loop system (14.5) for any x A
0
, x ,= 0.
Theorem 14.1 has introduced sucient conditions for stability for the approx-
imate solution

U
0
(x) based on its level of suboptimality specied by the Assump-
tions (A4) and (A5). In the following section we describe how these conditions can
be met by sampling the optimal control law for dierent values of x and then using
a technique called barycentric interpolation.
To simplify the discussion we will adopt the notation for the general multipara-
metric programming problem from Chapter 6. We consider the multiparametric
program
J
(x) = inf
z
J(z, x)
subj. to g(z, x) 0,
(14.9)
where z : R
s
is the optimization vector, x A R
n
is the parameter vector,
J : R
s
R
n
R is the cost function and g : R
s
R
n
R
ng
are the constraints.
We assume throughout this chapter that the set C = z, x : g(z, x) 0 is convex,
and that the function J(z, x) is simultaneously convex in z and x. Note that this
implies that the optimization problem (14.9) is convex for each xed value of the
parameter x. For a particular value of the parameter x, we denote the set of feasible
points as
R(x) = z : : g(z, x) 0, (14.10)
and dene the feasible set as the set of parameters for which (14.9) has a solution
/
= x A : R(x) ,= , (14.11)
We denote by J
(x) the real-valued function that expresses the dependence of the

minimum value of the objective function over /
on x
J
(x) = inf
z
J(z, x) : z R(x), (14.12)
and by Z
(x) the point-to-set map which assigns the (possibly empty) set of opti-
mizers z
2
Z
to a parameter x A
Z
(x) = z R(x) : J(z, x) = J
(x). (14.13)
We make the following standing assumptions:
Convexity The function J
(x) is continuous and convex.

Uniqueness The set of optimizers Z
(x) is a singleton for each value of the pa-

rameter. The function z
(x) = Z
(x) will be called optimizer function.

Continuity The function z
(x) is continuous.
Note that the solutions J
(x), Z
(x) and z
(x) of problem (14.1) satisfy these

conditions. We emphasize, however, that the approximate methods introduced in
the following work also for more general MPC problems as long as these conditions
of convexity, uniqueness and continuity are met.
For the approximation methods introduced below convexity of the function
(x) _ 0 (Theorem 14.1, Assumptions A4-A5) bounding the level of suboptimality
is necessary. It is not required, however, for the closed-loop stability with the
approximate control law.
14.2 Barycentric Interpolation 287
14.2 Barycentric Interpolation
Several schemes have been proposed in the literature that compute an approximate
solution to various problems of the form (14.9) based on interpolating the optimizer
z
(v
i
) at a number of samples of the parameter V = v
i
/
. The goal of
interpolated control is to dene a function z(x) over the convex hull of the sampled
points conv(V ), which has the required properties laid out in Theorem 14.1.
Section 14.2.1 will introduce the notion of barycentric interpolation, and will
demonstrate that such an approach will lead to an approximate function z(x),
which is feasible
z(x) R(x), x conv(V ). (14.14)
Then Section 14.2.2 introduces a convex optimization problem, whose optimal
value, if positive, veries a specied level of suboptimality
J( z(x), x) J
(x) +(x), x conv(V ). (14.15)

By Theorem 14.1 the stability of the resulting suboptimal interpolated control law
follows.
14.2.1 Feasibility
The class of interpolating functions z(x) satisfying (14.14) is called the class of
barycentric functions.
Denition 14.1 (Barycentric function) Let V = conv(v
1
, . . . , v
n
) R
n
be a
polytope. The set of functions w
v
(x), v extreme(V ) is called barycentric if three
conditions hold for all x V
w
v
(x) 0 positivity (14.16a)
vextreme(V )
w
v
(x) = 1 partition of unity (14.16b)
vextreme(V )
vw
v
(x) = x linear precision (14.16c)
Many methods have been proposed in the literature to generate barycentric
functions, several of which are detailed in Section 14.3, when we introduce con-
structive methods for generating approximate controllers. Figure 14.1 illustrates
several standard methods of barycentric interpolation.
Given a set of samples V , and a set of barycentric interpolating functions
w
v
(x), we can now dene an interpolated approximation z(x):
z(x) =
vextreme(V )
z
(v)w
v
(x) (14.17)
The key statement that can be made about functions dened via barycentric
interpolation is that the function values lie within the convex hull of the function
values at the sample points. This property is illustrated in Figure 14.2, and formally
dened in the following Lemma.
(a) Linear interpolation. (b) Wavelet interpolation.
(c) Barycentric interpolation.
Figure 14.1 Examples of interpolation.
Lemma 14.1 (Convex Hull Property of Barycentric Interpolation) If V =
v
0
, . . . , v
m
/
, z
(v
i
) is the optimizer of (14.9) for the parameter v
i
, and
w
v
(x) is a set of barycentric interpolating functions, then
z(x) =
vV
z
(v)w
v
(x) R(x) for all x conv(V ) .
Proof: The statement holds if for each x conv(V ) there exists a set of
positive multipliers
0
, . . . ,
m
such that
_
x
z(x)
_
=
m
i=0
i
_
v
i
z
(v
i
)
_
and
m
i=0

i
= 1. The properties of barycentric functions (14.16) clearly satisfy
this requirement.
14.2.2 Suboptimality
Next we investigate the level of suboptimality obtained for a barycentric inter-
polation. First we demonstrate that the value of the function J (z, x) for the
Figure 14.2 Illustration of the Convex-Hull Property of Barycentric Inter-
polation. The rst image shows a number of samples of a function, the
second the convex hull of these sample points and the third, a barycentric
interpolation that lies within this convex hull.
approximate solution z(x) is upper-bounded by the barycentric interpolation of its
optimal value taken at the sample points. Then we introduce a convex optimization
problem, whose solution upper-bounds the approximation error.
Lemma 14.2 If z(x) is a barycentric interpolation as dened in (14.17), then the
following condition holds:
J
(x) J ( z(x), x)
vV
w
v
(x)J
(v) for all x conv(V )

Proof: The lower bound follows from the fact that z(x) is feasible for the
optimization problem for all x V (Lemma 14.1).
The upper bound follows from convexity of the function J (z, x). Consider a
xed value of the parameter x conv(V ):
J ( z(x), x) = J
_
vV
w
v
(x)z
(v),
vV
w
v
(x)v
_
vV
w
v
(x)J (z
(v) , v)
=
vV
w
v
(x)J
(v)
The inequality in the above sequence follows because for a xed value of the pa-
rameter x, the barycentric interpolating functions w
v
are all positive, and sum
to one.
The previous lemma demonstrates that due to the convexity of the value func-
tion J (z, x), the interpolation of the optimal value function at the sample points
provides an upper bound of the value function evaluated at the interpolated ap-
proximation z(x). We can use this fact to formulate a convex optimization problem,
whose solution will provide a certicate for (14.15).
Theorem 14.2 If
V
0, then J ( z(x), x) J
(x) + (x) for all x conv(V ),

where
V
= min
x,z,v
J (z, x) +(x)
vV
v
J
(v) (14.18)
s.t. x =
vV
v
v,
v
0,
vV
v
= 1
(x, z) C
Proof: We see that the suboptimality bound is met within the convex hull of
V if and only if the following value is non-negative:
min
v,xconv(V )
J
(x) +(x) J ( z(x), x)

Lemma 14.2 provides a lower-bound
min
xconv(V )
J
(x) +(x)
vV
w
v
(x)J
(v)
and we obtain a further relaxation / lower bound by optimizing over all possible
barycentric functions
min
xconv(V )
J
(x) +(x)
vV
v
J
(v)
s.t. x =
vV
v
v,
v
0,
vV
v
= 1
Recalling that J
(x) = min
z
J (z, x) s.t. (x, z) C provides the desired result.
In this section we have demonstrated that if the approximate solution z(x) is
generated via barycentric interpolation, it will satisfy (14.14) and that the con-
dition for suboptimality (14.15) can be veried by solving a convex optimization
problem. In the next section, we will introduce several constructive techniques to
incrementally build approximate barycentric interpolating functions.
v
1
v
2
J
(v
1
)
J
(v
2
)
J
(x)
J( z(x), x)
w
v
(x)J
(v)
Figure 14.3 Illustration of the relationship between the optimal value func-
tion J
(x), the value function evaluated for the sub-optimal function z(x)
and the convex hull of the optimal value at the sample points.
14.2.3 Constructive Algorithm
In this section, we introduce a simple high-level algorithm to incrementally con-
struct a barycentric function that satises conditions (14.14)- (14.15) and therefore
can be used in the context of Theorem 14.1 to generate a stabilizing control law.
The idea is simple: begin with a set of sample points and solve the optimization
problem in Theorem 14.2 to test if the stability condition is met. If it is not,
then sample an additional set of points, sub-divide the resulting region into sub-
regions and repeat the procedure for each such generated region. In the following
we formalize this idea, and demonstrate that this procedure will terminate.
Algorithm 14.1 Approximate Parametric Programming with Stability Guarantee
input: Parametric Optimization Problem (14.9). Initial set of samples V
output: Sample points V /
and set of subsets 1 = |Rj, j = 1, . . ., Rj V

such that |conv(Rj), j = 1, . . . forms a partition of conv(V ) and R
j
0 for all j
let 1 V
repeat
R
argmin
R
j
R
R
j
(from problem (14.18))
if R
< 0 then
V V |xR
, where xR
is the optimizer of (14.18) for region R
1 partition(V, 1)
until R
0
return 1
Algorithm 14.1 outlines at a high level a method for constructing a polytopic
partition (Denition 5.6) of a set of initial samples, where each polytope in the
partition satises the suboptimality condition of Theorem 14.1, and therefore any
barycentrically interpolated control law dened over the partition will be stabiliz-
ing. One can see that in each iteration of the algorithm, the region R
with the
worst t measured in terms of
R
is computed, and the parameter x
R
at which
this maximum error occurs is added to the samples. The sample points V are then
updated to include this new point, and the set 1is re-computed. This set 1denes
a partition splitting conv(V ) into a set of polytopes conv(R
j
), j = 1, . . .. Once
this key function partition is dened, this algorithm can be directly implemented.
Several methods have been proposed in the literature that roughly follow the
scheme of Algorithm 14.1 [44, 160, 5, 156]. Many of these approaches dier in
the details by adding caching steps to minimize computation of
R
, add multiple
points at a time, or take an incremental re-partitioning approach. All of these
procedures can signicantly improve the computation speed of the algorithm, and
should therefore be incorporated into any practical approach.
14.3 Partitioning and Interpolation Methods
This section outlines three approaches to partitioning the set of samples and the
associated barycentric interpolation methods. While many proposals have been
put forward in the literature, we focus here on three techniques: inner polytopic
approximation (triangulation), outer polytopic approximation (convex hull) and
second-order interpolants (hierarchical grids). For each partitioning approach, we
present an appropriate barycentric interpolation method.
14.3.1 Triangulation
The simplest approach to generate a barycentric function is to rst triangulate
the sampled points V , and then dene a piece-wise ane control law over the
triangulation. The resulting interpolation of the optimal value function at the
sampled points will result in a convex piecewise ane inner approximation of the
optimal value function, and the approximate control law will be a piecewise ane
function.
Denition 14.2 (Triangulation) A triangulation of a nite set of points V
R
n
is a nite collection T
V
= R
0
, . . . , R
L
such that
R
i
= conv(V
i
) is an ndimensional simplex for some V
i
V
conv(V ) = R
i
and int R
i
int R
j
= for all i ,= j
If i ,= j then there is a common (possibly empty) face F of the boundaries of
S
i
and S
j
such that S
i
S
j
= F.
There are various triangulations possible; for example, the recursive triangula-
tion developed in [230, 44] has the strong property of generating a simple hierarchy
that can signicantly speed online evaluation of the resulting control law. The De-
launay triangulation [126], which has the nice property of minimizing the number
of skinny triangles, or those with small angles is a common choice for which incre-
mental update algorithms are well-studied and readily available (i.e. computation
of T
V {v}
given T
V
). A particularly suitable Delaunay triangulation can be dened
14.3 Partitioning and Interpolation Methods 293
by using the optimal cost function as a weighting term, which causes the resulting
triangulation to closely match the optimal partition [160].
Given a discrete set of states V , we can now dene an interpolated control law
z(x; T
V
) : R
n
R
m
.
Denition 14.3 If V A R
n
is a nite set and T = conv(V
1
), . . . , conv(V
L
)
is a triangulation of V , then the interpolated function z(x; T) : conv(V ) R
m
is
z(x; T) =
vVj
z
(v)
v
, if x conv(V
j
) (14.19)
where
v
0,
v
= 1 and x =
vVj
v
v
.
For each simplical region conv(V
j
) in the triangulation, the set of multipliers
is unique and can be found by solving the system of linear equations
=
_
v
1
v
n+1
1 1
_
1
_
x
1
_
where v
i
V
j
.
The function z(x; T) is piecewise ane, and given by:
z(x; T) =
_
_
z
(v
1
)
.
.
.
z
(v
n+1
)
_
_
v
1
v
n+1
1 1
_
1
_
x
1
_
if x V
j
= conv()v
1
, . . . , v
n+1
The simplical nature of each region immediately demonstrates that the inter-
polation 14.19 is barycentric.
14.3.2 Outer Polyhedral Approximation
The previous section introduced an approach that generates an inner polytopic
approximation of the optimal value function, which results in a triangulation of the
state space. This section introduces an outer polytopic approximation method that
will result in a general polytopic partition of the space, which is often signicantly
less conservative than a triangulation, i.e., for the same number of samples it
provides a tighter approximation of the value function.
Implicit Double Description Algorithm: Computing an Outer Polyhedral Ap-
proximation of a Convex Set
We begin by introducing an algorithm, called the implicit double description method [162],
which computes an outer polyhedral approximation in an incremental fashion that
is greedy-optimal in terms of the Hausdor metric dened below.
The epigraph of the optimal value function plus the specied desired approxi-
mation error is dened as:
epi J
,
= (x, J) [ J J
(x) +(x)
The epigraph of a convex function is a convex set. Our goal is to compute a
polyhedron P which is an inner approximation of the epigraph of the optimal
value function and an outer approximation of the optimal value function plus the
specied desired approximation error (Figure 14.4):
P J
(x) +(x)
J
(x)
Figure 14.4 Polyhedron T constructed to be an outer approximation of
epi J
,
and an inner approximation of epi J
.
epi J
P epi J
,
Once such an approximation is in hand, we will then use it to compute a polytopic
partition of /
over which the approximate control law will be dened.

The quality of approximation is measured in terms of the Hausdor distance,
which measures the distance between two sets X and Y as:
d
H
(X, Y ) = max
_
sup
xX
inf
yY
|x y|
2
, sup
yY
inf
xX
|x y|
2
_
Our goal is to compute the Hausdor distance between our polytopic approxi-
mation, and epi J
,
by solving a convex optimization problem. We rst recall the
denition of the projection of a point v onto a convex set S
proj
S
v = arg min
xS
|v x|
2
2
The following lemma states that we can compute the projection of a point
onto the epigraph of the optimal value function by solving a convex optimization
problem.
Lemma 14.3 The projection of (v, J
v
) onto the epigraph epi J
,
is the optimizer
of the following problem, (x
, J
):
min
z,x
|x v|
2
2
+|J (z, x) +(x) J
v
|
2
2
s.t. (x, z) C
We can now use the fact that the maximizer of a convex function over a convex
set will always occur at an extreme point of the set to write the Hausdor distance
as the maximum of the projection of the vertices of the polyhedron P onto the
epigraph, which can be done by testing a nite number of points, since a polyhedron
has a nite number of vertices.
Lemma 14.4 If P is a polyhedron, and P epi J
,
is a convex set, then the
Hausdor distance is:
d
H
(P, epi J
,
) = max
(v,Jv)extreme(P)
|x
(v) v|
2
2
+|J
(v) +(x) J
v
|
2
2
where (x
(v), J
(v)) is the projection of (v, J

v
) onto the epigraph.
Algorithm 14.2 below will generate a sequence of outer polyhedral approxima-
tions of the epigraph that are monotonically decreasing in terms of the Hausdor
distance. In each iteration of the algorithm, the Hausdor distance is computed,
and the extreme point v of P at which the worst-case error occurs is removed from
P by adding a seperating hyperplane between the point v and the epigraph. The
addition of a seperating hyperplane ensures that the updated polyhedron remains
an outer approximation, and since the worst-case point is removed from the set,
we have that the approximation error is monotonically decreasing in the Hausdor
distance. The algorithm terminates when all of the vertices of the P are contained
in the epigraph of J
. Figure 14.5 illustrates the process of a single cycle of the

implicit double description algorithm. Figure 14.6 shows a progression of steps
until the termination criterion extreme(P) epi J
(x) is satised.
x
1
x
2
Figure 14.5 Example of one iteration of the implicit double-description al-
gorithm. Square: Point vmax at which the worst-case error occurs. Circle:
Point z
(vmax); Projection of vmax onto the set. Red line: Separating hy-
perplane.
Algorithm 14.2 Approximate Parametric Programming with Stability Guarantee
input: Parametric Optimization Problem (14.9). Initial polyhedron P epi J
,
output: Polyhedron P such that epi J
P epi J
,
repeat
ForEach v extreme(P)
z
(v) = minzepi J
, |v z|
2
2
vmax argmax
vextreme(P)
|v z
(v)|
P P
_
z
[z
(vmax) v]
(z z
(vmax)) 0
_
until extreme(P) epi J
(x)
Polyhedral Approximation Approximate Control Law
Once an outer approximation P of the epigraph has been computed, then an ap-
proximate control law can be dened as follows. For each facet F of P dene the
polytope R
F
as the projection of the facet F onto the state-space
R
F
= x [ J, (x, J) F
The set of all such polytopes, R
F
, forms a partition of the feasible set of the
controller and we dene our approximate control law over this set:
z(x) =
vextreme(RF )
u
(v)w
v,F
(x), if v R
F
(14.20)
where w
v,F
(x) is a barycentric set of functions for the polytope R
F
. A method to
construct such a function will be described in the next sub-section.
The following theorem demonstrates that this approximate z(x) satises con-
dition (14.15).
Theorem 14.3 Let z(x) be dened as in (14.20) for a polyhedral approximation
P that satises the condition extreme(P) epi J
(x), then condition (14.15) is

satised.
Proof:
J
(x) J( z(x), x) Feasibility of barycentric interpolation

from Lemma 14.1
vextreme(RF )
J
(v)w
v,F
(x), if v R
F
Convexity of J
(v,Jv)extreme(F)
J
v
w
v,F
(x), if v R
F
P epi J
min
(x,J)P
J (v, J
v
) are all on a hyperplane
J
(x) +(x) P epi J

,
Barycentric Functions for Polytopes

Our goal is now to dene an easily computable barycentric function for each poly-
tope R
F
. If the polytope R
F
is a simplex, then the barycentric function is unique,
linear and trivially computed and so we focus on the non-simplical case. In [283]
a very elegant method of computing a barycentric function for arbitrary polytopes
was proposed that can be put to use here.
Lemma 14.5 (Barycentric coordinates for polytopes [283]) Let S = conv(V )
R
d
be a polytope and for each simple vertex v of S, let b
v
(x) be the function
b
v
(x) =

v
|v x|
2
where
v
is the area of the polytope y [ [V 1x
]y 0, (v x)
y = 1; i.e. the
area of the facet of the polar dual of S x corresponding to the vertex v x. The
function w
v
(x) = b
v
(x)/
v
b
v
(x) is barycentric over the polytope S.
The areas of the facets of the polar duals
v
can be pre-computed oine and
stored. If there are d + 1 facets incident with the vertex v (i.e. v is simplical),
then the area of the polar facet is det
__
a
0
a
d+1
_
, where a
0
, . . . , a
d+1
are
the normals of the incident facets. If the vertex is not simplical, then the area can
be easily computed by perturbing the incident facets [283]. Such computation is
straightforward because both the vertices and halfspaces of each region are available
due to the double-description representation.
14.3.3 Second-Order Interpolants
This section summarizes the hierarchical grid-based approach proposed in [269].
The main idea is to partition the space into a grid, which is then extremely fast
to evaluate online. We rst introduce an interpolation method based on second-
order interpolants and demonstrate that it is barycentric. We then discuss a sparse
hierarchical gridding structure, which allows for extremely sparse storage of the
data and rapid evaluation of the function.
Our goal is to dene a set of basis functions, one for each sampled vertex of
the grid, whose sum is barycentric within each hypercube of the grid. We begin
by dening the one-dimensional scaling function (hat function), with support on
[1, 1] as:
(x) =
_
1 [x[ , if x [1, 1]
0 , otherwise
For a particular grid point v, with grid width of w, we can extend the hat function
by scaling and shifting:
v,w
(x) =
_
x v
w
_
We extend the function to d-dimensions by simply taking the product:
v,w
(x) =
d
i=1
vi,w
(x
i
) ,
where v
i
and x
i
are the i
th
components of the vectors v and x respectively.
The resulting basis function
0,1
(x) is illustrated in two dimensions in Fig-
ure 14.7.
We now dene our interpolated approximation as:
z(x) =
vG
z
(v)
v,w
(x)
where w is the width of the grid, and G is the set of grid vertices.
Lemma 14.6 [269] Let R be the hypercube dened by
R =
w
2
(B
1) v
where B
= x [ |x|
1 is the unit innity-norm ball. The set of functions
v,w
(x) is barycentric within the set R.
Lemma 14.6 states that each hypercube in the grid is in fact barycentric. In
order to apply Algorithm 14.1 we need a method of re-partitioning the current
sample points. We do this via a hierarchical approach, in which each hypercube
of the grid is tested for compliance with condition (14.15) and, if it fails, is sub-
divided into 2
d
more hypercubes. The result is a hierarchical grid, that is very easy
to evaluate online.
What has been discussed so far is termed a nodal representation of the approxi-
mate function. It is also possible to dene what is termed a hierarchical multi-scale
representation, which, at each level of the re-gridding procedure, approximates only
the residual, rather than the original function. This approach generally results in
an extremely sparse representation of the data, and signicantly ameliorates the
downsides of sampling on a grid. See [269] for more details.
Example 14.1 We demonstrate and compare the three approximation algorithms
introduced in this Chapter on a simple example. This comparison should not be
generalized to other problems, since the performance of any given approximation
method will depend strongly on the problem being considered.
Consider the following system:
x(k + 1) =
_
1 1
0 1
_
x(k) +
_
1
0.5
_
u(k)
with state and input constraints:
x . = |x [ |x| 5 u | = |u [ |u| 1
We approximate the control law resulting for a standard quadratic-cost MPC problem
with a horizon of 10 and weighting matrices taken to be
Q =
_
0.1 0
0 2
_
R = 1
The MPC problem formulation includes a terminal set and terminal weight, which
are computed as discussed in Chapter 13 as the maximum invariant set for the system
when controlled with the optimal LQR control law.
The optimal value function J
for this problem can be seen in Figure 14.8, along with

the approximation target J
(x) +(x) for (x) = x
Qx.
Figure 14.9 shows the evolution of the approximation algorithm using a triangulation
approach, and Figure 14.10 shows the nal approximate value function and resulting
approximate control law
1
. Figure 14.11 shows the same evolution, but for the double
description/outer approximation approach
2
. One can see that far fewer sample points
are required to reach a stability certicate.
1
2
The example has been approximated using the second-order interpolant method un-
til the stability certication was achieved
3
. Figure 14.12 shows the resulting nodal
partition of the state space. Note that since it would take an innite number of hy-
percubes to approximate a non-orthogonal polytope, a soft-constrained version of the
problem is taken in the domain outside the feasible set (although the stability certi-
cate is valid only within the feasible region). One can see the resulting approximated
value function, the approximate control law, and the resulting error in Figures 14.14
and 14.13 respectively.
Table 14.1 shows the required online data storage and computation time for the
various methods introduced in this chapter for the studied example, as well as the
average and worst-case approximation errors (based on samples). One can see that
for this small example, the second-order interpolants are clearly the best solution.
However, this should not be taken as a clear indicator, or a recommendation for
the method beyond the others, since this would require a thorough computational
comparison of the various methods (or their variants in the literature) on a wide
range of practical problems.
Table 14.1 Example 14.1: Comparison of the approximation methods intro-
duced in Section 14.3. The indicated FLOPS for triangulation are based on
linear search.
Data FLOPS Approx error | u(x) u
(x)|
(numbers) (avg) (max)
Triangulation 120 1,227 3.3% 24.6%
Polyhedral approx 720 752
Second-order interpolants 187 26 1.1% 8.3%
3
P
J
(x) +(x)
J
(x)
(a) Iteration 1.
P
J
(x) +(x)
J
(x)
(b) Iteration 2.
P J
(x) +(x)
J
(x)
(c) Last iteration.
Figure 14.6 Several steps of the Double Description algorithm with the ter-
mination criterion extreme(P) epi J
(x) satised in the last step.

x
1
x
2
0
,
1
(
x
)
-1
-0.5
0
0.5
1
-1
-0.5
0
0.5
1
0
0.2
0.4
0.6
0.8
1
Figure 14.7 Plot of the second order interpolant in 2D centered at zero, with
a width of one.
x
1
x
2
-5
0
5
-5
0
5
0
10
20
30
40
Figure 14.8 Example 14.1: Optimal cost J
(x) (light gray), and stability

function J
(x) +(x) (gray).

x
1
x
2
-5
0
5
-4
-2
0
2
4
-5
0
5
10
15
x
1
x
2
-5
0
5
-4
-2
0
2
4
-5
-5
-5
0
0
5
5
x
1
x
2
-5 0 5
-5
-4
-3
-2
-1
0
1
2
3
4
5
x
1
x
2
-5 0 5
-5
-4
-3
-2
-1
0
1
2
3
4
5
x
1
x
2
-5
0
5
-5
0
5
0
10
20
30
40
(a) Sub-optimal solution with 15
vertices. Error: V = 17.8.
x
1
x
2
-5
0
5
-5
0
5
0
10
20
30
40
(b) Sub-optimal solution with 25
Figure 14.9 Example 14.1: Error function V (top). Points marked are
the vertices with worst-case t. Partition (triangulation) of the sampled
points (middle). Upper-bound on approximate value function (convex hull
of sample points) (bottom). Download code.
x
1
x
2
-5
0
5
-4
-2
0
2
4
-8
-6
-4
-2
0
2
x
1
x
2
-5
0
5
-4
-2
0
2
4
-8
-6
-4
-2
0
x
1
x
2
-5 0 5
-5
-4
-3
-2
-1
0
1
2
3
4
5
x
1
x
2
-5 0 5
-5
-4
-3
-2
-1
0
1
2
3
4
5
x
1
x
2
-5
0
5
-5
0
5
0
10
20
30
40
x
1
x
2
-5
0
5
-5
0
5
0
10
20
30
40
(b) Example 14.1: Sub-optimal
solution with 45 vertices. Error:
V = 0.82
x
1
x
2
-5
0
5
-4
-2
0
2
4
-8
-6
-4
-2
0
x
1
x
2
-5
0
5
-4
-2
0
2
4
-8
-6
-4
-2
0
x
1
x
2
-5 0 5
-5
-4
-3
-2
-1
0
1
2
3
4
5
x
1
x
2
-5 0 5
-5
-4
-3
-2
-1
0
1
2
3
4
5
x
1
x
2
-5
0
5
-5
0
5
0
10
20
30
40
(c) Sub-optimal solution with 55
x
1
x
2
-5
0
5
-5
0
5
0
10
20
30
40
(d) Example 14.1: Sub-optimal
solution with 61 vertices. Error:
V = 0, system stable.
x
1
x
2
u
(
x
)
-5
0
5
-5
0
5
-1
-0.5
0
0.5
1
(e) Approximated control law
u(x).
x
1
x
2
u
(
x
)
(
x
)
-4
-2
0
2
4
-2
0
2
-1
-0.5
0
0.5
1
(f) Approximation error u(x)
u
(x).
Figure 14.10 Example 14.1: Approximate control law using triangulation.
x
1
x
2
-5
0
5
-4
-2
0
2
4
0
5
10
15
20
x
1
x
2
-5
0
5
-4
-2
0
2
4
-5
-5
-5
0
0
5
5
x
1
x
2
-5 0 5
-5
-4
-3
-2
-1
0
1
2
3
4
5
x
1
x
2
-5 0 5
-5
-4
-3
-2
-1
0
1
2
3
4
5
x
1
x
2
-5
0
5
-5
0
5
0
10
20
30
40
facets. Error: V =.
x
1
x
2
-5
0
5
-5
0
5
0
10
20
30
40
(b) Sub-optimal solution with 13
facets. Error: V =.
x
1
x
2
-5
0
5
-4
-2
0
2
4
-6
-4
-2
0
2
4
x
1
x
2
-5
0
5
-4
-2
0
2
4
-6
-4
-2
0
2
x
1
x
2
-5 0 5
-5
-4
-3
-2
-1
0
1
2
3
4
5
x
1
x
2
-5 0 5
-5
-4
-3
-2
-1
0
1
2
3
4
5
x
1
x
2
-5
0
5
-5
0
5
0
10
20
30
40
(c) Sub-optimal solution with 21
facets. Error: V =.
x
1
x
2
-5
0
5
-5
0
5
0
10
20
30
40
(d) Example 14.1: Sub-optimal
solution with 31 facets. Error:
V = 0, system stable.
Figure 14.11 Example 14.1: Error function V (top). Points marked are
the vertices with worst-case t. Partition (triangulation) of the sampled
points (middle). Upper-bound on approximate value function (convex hull
of sample points) (bottom). Download code.
x
1
x
2
-5 0 5
-5
-4
-3
-2
-1
0
1
2
3
4
5
Figure 14.12 Example 14.1: Partition generated by the second-order inter-
polant method. Download code.
x
1
x
2
u
(
x
)
-5
0
5
-5
0
5
-1
-0.5
0
0.5
1
(a) Approximated control law
u(x).
x
1
x
2
u
(
x
)
(
x
)
-5
0
5
-5
0
5
-1
-0.5
0
0.5
1
(b) Approximation error u(x)
u
(x).
Figure 14.13 Example 14.1: Approximate control law using second-order
interpolants.
x
1
x
2
J
(
x
)
-5
0
5
-5
0
5
0
0
10
10
20
20
30
Figure 14.14 Example 14.1: Approximate value function

J(x) using second-
order interpolants.
Chapter 15
On-line Control Computation
In previous chapters we have shown how to compute the solution to constrained
nite time optimal control problems. Two approaches can be used. The rst ap-
proach consists of solving a mathematical program for a given initial state. The
second approach employs multiparametric programming to compute the solution
as an explicit piecewise ane function of the initial state. This implies that for
constrained linear systems, a Receding Horizon Control (RHC) policy requires ei-
ther the on-line solution of a quadratic or linear program, or the evaluation of a
piecewise-ane-on polyhedra function.
In this chapter we focus on ecient on-line methods for the computation of RHC
control laws. In particular, the main drawback of explicit optimal control laws is
that the number of polyhedral regions could grow exponentially with the number
of constraints in the optimal control problem. Section 15.1 presents ecient on-line
methods for the evaluation of explicit piecewise ane control laws.
If the on-line solution of a quadratic or linear program is preferred, Section 15.3
and 15.2 briey discuss how to improve the eciency of a mathematical program-
ming solver by exploiting the structure of the RHC control problem.
15.1 Storage and On-line Evaluation of the PWA Control
Law
In Chapter 12 we have shown how to compute the solution to the constrained nite
time optimal control (CFTOC) problem as an explicit piecewise ane function of
the initial state. Such a function is computed o-line by using a multiparametric
program solver, which divides the state space into polyhedral regions, and for each
region determines the linear gain and oset which produces the optimal control
action.
This method reveals its eectiveness when applied to Receding Horizon Control
(RHC). Having a precomputed solution as an explicit piecewise ane on polyhedra
(PPWA) function of the state vector reduces the on-line computation of the RHC
310 15 On-line Control Computation
control law to a function evaluation, therefore avoiding the on-line solution of a
quadratic or linear program.
The main drawback of such an explicit optimal control law is that the number
of polyhedral regions could grow dramatically with the number of constraints in
the optimal control problem. In this chapter we focus on ecient on-line methods
for the evaluation of such a piecewise ane control law.
The simplest way to implement the piecewise ane feedback laws is to store the
polyhedral cells H
i
x K
i
, perform on-line a search through them to locate the
one which contains x(t) and then look up the corresponding feedback gain (F
i
, g
i
)
(note that this procedure can be easily parallelized). This chapter presents imple-
mentation techniques which avoid the storage and the evaluation of the polyhedra
and can signicantly reduce the on-line storage demands and computational com-
plexity of RHC. They exploit the properties of the value function and the piecewise
ane optimal control law of the constrained nite time optimal control problem.
Let the explicit optimal control law be:
u
(x) = F
i
x +g
i
, x T
i
, i = 1, . . . , N
r
, (15.1)
where F
i
R
mn
, g
i
R
m
, and T
i
=
_
x R
n
: H
i
x K
i
, H
i
R
N
i
c
n
, K
i
R
N
i
c
_
,
i = 1, . . . , N
r
is a polyhedral partition of A. In the following H
i
j
denotes the j-th
row of the matrix H
i
and N
i
c
is the numbers of constraints dening the i-th poly-
hedron. The on-line implementation of the control law (15.1) is simply executed
according to the following steps:
Algorithm 15.1
1.measure the current state x(t)
2.search for the j-th polyhedron that contains x(t), (H
j
x(t) K
j
)
3.implement the j-th control law (u(t) = F
j
x(t) +g
j
)
In Algorithm 15.1, step (2) is critical and it is the only step whose eciency can
be improved. A simple implementation of step (2) would require searching for the
polyhedral region that contains the state x(t) as in the following algorithm:
Algorithm 15.2
input: Current state x(t) and polyhedral partion |Ti
N
r
i=1
of the control law (15.1)
output: Index j of the polyhedron Tj in the control law (15.1) containing the current
state x(t)
i = 0, notfound=1;
while i N
r
and notfound
j = 0, stillfeasible=1
while j N
i
c
and stillfeasible=1
if H
i
j
x(t) > K
i
j
then stillfeasible=0
else j = j + 1
end
15.1 Storage and On-line Evaluation of the PWA Control Law 311
if stillfeasible=1 then notfound=0
end
In Algorithm 15.2 H
i
j
denotes the j-th row of the matrix H
i
, K
i
j
denotes the
j-th element of the vector K
i
and N
i
c
is the number of constraints dening the
i-th polyhedron T
i
. Algorithm 15.2 requires the storage of all polyhedra T
i
, i.e.,
(n+1)N
C
real numbers (n numbers for each row of the matrix H
i
plus one number
for the corresponding element in the matrix K
i
), N
C
=
N
r
i=1
N
i
c
, and in the worst
case (the state is contained in the last region of the list) it will give a solution after
nN
C
multiplications, (n 1)N
C
sums and N
C
comparisons.
By using the properties of the value function, in this section we show how
Algorithm 15.2 can be replaced by more ecient algorithms that avoid storing the
polyhedral regions T
i
, i = 1, . . . , N
r
, therefore reducing signicantly the storage
demand, and that have a smaller computational complexity.
In the following we will distinguish between optimal control based on LP and
optimal control based on QP.
15.1.1 Ecient Implementation, 1, -Norm Case
From Corollary 12.5, the value function J
(x) corresponding to the solution of the

CFTOC problem (13.47) with 1, -norm is convex and PWA:
J
(x) = T
i
x +V
i
, x T
i
, i = 1, . . . , N
r
. (15.2)
By exploiting the convexity of the value function the storage of the polyhedral
regions T
i
can be avoided. From the equivalence of the representations of PWA
convex functions (see Section 3.2.5), the function J
(x) in equation (15.2) can be

represented alternatively as
J
(x) = max
_
T
i
x +V
i
, i = 1, . . . , N
r
_
for x A =
N
r
i=1
T
i
. (15.3)
Thus, the polyhedral region P
j
containing x can be identied simply by search-
ing for the maximum number in the list T
i
x +V
i
N
r
i=1
:
x T
j
T
j
x +V
j
= max
_
T
i
x +V
i
, i = 1, . . . , N
r
_
. (15.4)
Therefore, instead of searching for the polyhedron j that contains the point x
via Algorithm 15.2, we can just store the value function and identify region j by
searching for the maximum in the list of numbers composed of the single ane
function T
i
x +V
i
evaluated at x (see Figure 15.1):
T
1
T
3
T
4
T
2
T
1
x +V
1
T
2
x +V
2
T
3
x +V
3
T
4
x +V
4
J
(
x
)
Figure 15.1 Example for Algorithm 15.3 in one dimension: For a given point
x T3 (x = 5) we have J
(x) = max(T
1
x +V
1
, . . . , T
4
x +V
4
).
Algorithm 15.3
input: Current state x and value function 15.2, T
i
x +V
i
, i = 1, . . . , N
r
output: Index j of the polyhedron Tj containing the current state x(t) in the control
law (15.1)
compute the list / = |ni = T
i
x +V
i
, i = 1, . . . , N
r
nd j such that nj = maxn

i
L ni
The search Algorithm 15.3 requires the storage of (n + 1)N
r
real numbers
and will give a solution after nN
r
multiplications, (n 1)N
r
sums, and N
r
1
comparisons. In Table 15.1 we compare the complexity of Algorithm 15.3 against
Algorithm 15.2 in terms of storage demand and number of ops. Algorithm 15.3
will outperform Algorithm 15.2 since typically N
C
N
r
.
Table 15.1 Complexity comparison of Algorithm 15.2 and Algorithm 15.3
Algorithm 15.2 Algorithm 15.3
Storage demand (real numbers) (n + 1)N
C
(n + 1)N
r
Number of ops (worst case) 2nN
C
2nN
r
15.1.2 Ecient Implementation, 2-Norm Case
Consider the state feedback solution (12.15) of the CFTOC problem (12.9) with
p = 2. Theorem 12.2 states that the value function J
(x) is convex and piecewise

quadratic on polyhedra and the simple Algorithm 15.3 described in the previous
subsection cannot be used here. Instead, a dierent approach is described below.
It uses a surrogate of the value function to uniquely characterize the polyhedral
partition of the optimal control law.
We will rst establish the following general result: given a general polyhedral
partition of the state space, we can locate where the state lies (i.e., in which poly-
hedron) by using a search procedure based on the information provided by an
appropriate PWA continuous function dened over the same polyhedral parti-
tion. We will refer to such appropriate PWA function as a PWA descriptor
function. First we outline the properties of the PWA descriptor function and then
we describe the search procedure itself.
Let T
i
N
r
i=1
be the polyhedral partition obtained by solving the mp-QP (12.31)
and denote by C
i
= j : T
j
is a neighbor of T
i
, j = 1, . . . , N
r
, j ,= i the list of
all neighboring polyhedra of T
i
. The list C
i
has N
i
c
elements and we denote by
C
i
(k) its k-th element.
Denition 15.1 (PWA descriptor function) A continuous real-valued PWA
function
f(x) = f
i
(x) = A
i
x +B
i
, if x T
i
(15.5)
is called a descriptor function if
A
i
,= A
j
, j C
i
, i = 1, . . . , N
r
. (15.6)
Theorem 15.1 Let f(x) be a PWA descriptor function on the polyhedral partition
T
i
Nr
i=1
.
Let O
i
(x) R
N
i
c
be a vector associated with region T
i
, and let the j-th element
of O
i
(x) be dened as
O
i
j
(x) =
_
+1 f
i
(x) f
Ci(j)
(x)
1 f
i
(x) < f
Ci(j)
(x).
(15.7)
Then O
i
(x) has the following properties:
(i) O
i
(x) = S
i
= const, x T
i
, i = 1, . . . , N
r
.
(ii) O
i
(x) ,= S
i
, x / T
i
, i = 1, . . . , N
r
.
Proof: Let T = T
i
T
Ci(j)
be the common facet of T
i
and T
Ci(j)
. Dene the
linear function
g
i
j
(x) = f
i
(x) f
Ci(j)
(x). (15.8)
From the continuity of f(x) it follows that g
i
j
(x) = 0, x T. As T
i
and T
Ci(j)
are
disjoint convex polyhedra and A
i
,= A
Ci(j)
it follows that g
i
j
(
i
) > 0 (or g
i
j
(
i
) < 0,
but not both) for any interior point
i
of T
i
. Similarly for any interior point
Ci(j)
of T
Ci(j)
we have g
i
j
(
Ci(j)
) < 0 (or g
j
i
(
i
) > 0, but not both). Consequently,
g
i
j
(x) = 0 is the separating hyperplane between T
i
and T
Ci(j)
.
(i) Because g
i
j
(x) = 0 is a separating hyperplane, the function g
i
j
(x) does not
change its sign for all x T
i
, i.e., O
i
j
(x) = s
i
j
, x T
i
with s
i
j
= +1 or s
i
j
= 1.
The same reasoning can be applied to all neighbors of T
i
to get the vector S
i
=
s
i
j
R
N
i
c
.
T
1
T
3
T
4
T
2
f
(
x
)
f
1
(x)
f
2
(x)
f
3
(x)
f
4
(x)
Figure 15.2 Example for Algorithm 15.4 in one dimension: For a given point
x T2 (x = 4) we have O2(x) = [1 1]
= S2, while O1(x) = 1 ,= S1 = 1,

O3(x) = [1 1]
,= S3 = [1 1]
, O4(x) = 1 ,= S4 = 1.
(ii) x / T
i
, j C
i
such that H
i
j
x > K
i
j
. Since g
i
j
(x) = 0 is a separating
hyperplane O
i
j
(x) = s
i
j
.
Equivalently, Theorem 15.1 states that
x T
i
O
i
(x) = S
i
, (15.9)
which means that the function O
i
(x) and the vector S
i
uniquely characterize T
i
.
Therefore, to check on-line if the polyhedral region i contains the state x it is
sucient to compute the binary vector O
i
(x) and compare it with S
i
. Vectors
S
i
are calculated o-line for all i = 1, . . . , N
r
, by comparing the values of f
i
(x)
and f
Ci(j)
(x) for j = 1, . . . , N
i
c
, for a point x belonging to T
i
, for instance, the
Chebychev center of T
i
.
In Figure 15.2 a one dimensional example illustrates the procedure with N
r
= 4
regions. The list of neighboring regions C
i
and the vector S
i
can be constructed
by simply looking at the gure: C
1
= 2, C
2
= 1, 3, C
3
= 2, 4, C
4
= 3,
S
1
= 1, S
2
= [1 1]
, S
3
= [1 1]
, S
4
= 1. The point x = 4 is in region 2 and
we have O
2
(x) = [1 1]
= S
2
, while O
3
(x) = [1 1]
,= S
3
, O
1
(x) = 1 ,= S
1
,
O
4
(x) = 1 ,= S
4
. The failure of a match O
i
(x) = S
i
provides information on a
good search direction(s). The solution can be found by searching in the direction
where a constraint is violated, i.e., one should check the neighboring region T
j
for
which O
i
j
(x) ,= s
i
j
.
The overall procedure is composed of two parts:
1. (o-line) Construction of the PWA function f(x) in (15.5) satisfying (15.6)
and computation of the list of neighbors C
i
and the vector S
i
,
2. (on-line) Execution of the following algorithm
Algorithm 15.4
input: Current state x, the list of neighboring regions Ci and the vectors S
i
output: Index i of the polyhedron Tj containing the current state x(t) in the control
law (15.1)
let i = 1, notfound=1;
while notfound
compute O
i
(x)
if O
i
(x) = S
i
then notfound=0
else i = Ci(q), where O
i
q
(x) ,= s
i
q
.
end
Algorithm 15.4 does not require the storage of the polyhedra T
i
, but only
the storage of one linear function f
i
(x) per polyhedron, i.e., N
r
(n + 1) real num-
bers and the list of neighbors C
i
which requires N
C
integers. In the worst case,
Algorithm 15.4 terminates after N
r
n multiplications, N
r
(n 1) sums and N
C
comparisons.
In Table 15.2 we compare the complexity of Algorithm 15.4 against the standard
Algorithm 15.2 in terms of storage demand and number of ops.
Remark 15.1 Note that the computation of O
i
(x) in Algorithm 15.4 requires the
evaluation of N
i
c
linear functions, but the overall computation never exceeds N
r
linear
function evaluations. Consequently, Algorithm 15.4 will outperform Algorithm 15.2,
since typically NC N
r
.
Storage demand (real numbers) (n + 1)N
C
(n + 1)N
r
Number of ops (worst case) 2nN
C
(2n 1)N
r
+N
C
Now that we have shown how to locate polyhedron in which the state lies by
using a PWA descriptor function, we need a procedure for the construction of such
a function.
The image of the descriptor function is the set of real numbers R. In the
following we will show how the descriptor function can be generated from a vector-
valued function m : R
n
R
s
. This general result will be used in the following
subsections.
Denition 15.2 (Vector valued PWA descriptor function) A continuous vector-
valued PWA function
m(x) =

A
i
x +

B
i
, if x T
i
, (15.10)
is called a vector-valued PWA descriptor function if
A
i
,=

A
j
, j C
i
, i = 1, . . . , N
r
, (15.11)
where

A
i
R
sn
,

B
i
R
s
.
Theorem 15.2 Given a vector-valued PWA descriptor function m(x) dened over
a polyhedral partition T
i
Nr
i=1
it is possible to construct a PWA descriptor function
f(x) over the same polyhedral partition.
Proof: Let A
i,j
be the null-space of (

A
i
A
j
)
. Since by the denition

A
i
A
j
,=
0 it follows that A
i,j
is not full-dimensional, i.e., A
i,j
R
s1
. Consequently, it
is always possible to nd a vector w R
s
such that w(

A
i

A
j
) ,= 0 holds for all
i = 1, . . . , N
r
and j C
i
. Clearly, f(x) = w
m(x) is then a valid PWA descriptor

function.
As shown in the proof of Theorem 15.2, once we have a vector-valued PWA
descriptor function, practically any randomly chosen vector w R
s
is likely to be
satisfactory for the construction of PWA descriptor function. But, from a numerical
point of view, we would like to obtain a w that is as far away as possible from the
null-spaces A
i,j
. We show one algorithm for nding such a vector w.
For a given vector-valued PWA descriptor function we form a set of vectors
a
k
R
s
, |a
k
| = 1, k = 1, . . . , N
C
/2, by choosing and normalizing one (and only
one) nonzero column from each matrix (

A
i

A
j
), j C
i
, i = 1, . . . , N
r
. The
vector w R
s
satisfying the set of equations w
a
k
,= 0, k = 1, . . . , N
C
/2, can then
be constructed by using the following algorithm. Note that the index k goes to
N
C
/2 since the term (

A
j

A
i
) is the same as (

A
i

A
j
) and thus there is no need
to consider it twice.
Algorithm 15.5
input: Vectors ai R
s
, i = 1, . . . , N
output: The vector w R
s
satisfying the set of equations w
ai ,= 0, i = 1, . . . , N
let w [1, . . . , 1]
, R 1
while k NC/2
d w
a
k
if 0 d R then w w +
1
2
(Rd)a
k
, R
1
2
(R +d)
if R d < 0 then w w
1
2
(R+d)a
k
, R
1
2
(Rd)
end
Algorithm 15.5 is based on a construction of a sequence of balls B = x : x = w +r, |r|
2
R.
As depicted in Figure 15.3, Algorithm 15.5 starts with the initial ball of radius
R = 1, centered at w = [1, . . . , 1]
. Iteratively one hyperplane a
k
x = 0 at the
time is introduced and the largest ball B
B that does not intersect this hyper-

plane is designed. The center w of the nal ball is the vector w we wanted to
construct, while R gives some information about the degree of non-orthogonality:
[w
a
k
[ R, k.
In the following subsections we will show that the gradient of the value function,
and the optimizer, are vector-valued PWA descriptor functions and therefore we
can use Algorithm 15.5 for the construction of the PWA descriptor function.
B
R
R
d
R d
B
a
T
i
x 0
a
T
i
x 0
Figure 15.3 Illustration for Algorithm 15.5 in two dimensions.
Generating a PWA descriptor function from the value function
Let J
(x) be the convex and piecewise quadratic (CPWQ) value function obtained
as a solution of the CFTOC (12.9) problem for p = 2:
J
(x) = q
i
(x) = x
Q
i
x +T
i
x +V
i
, if x T
i
, i = 1, . . . , N
r
. (15.12)
In Section 7.3.4 we have proven that for non-degenerate problems the value
function J
(x) is a C
(1)
function. We can obtain a vector-valued PWA descriptor
function by dierentiating J
(x). We rst need to introduce the following theorem.

Theorem 15.3 ([21]) Assume that the CFTOC (12.9) problem leads to a non-
degenerate mp-QP (12.33). Consider the value function J
(x) in (15.12) and let

CR
i
, CR
j
be the closure of two neighboring critical regions corresponding to the
set of active constraints A
i
and A
j
, respectively, then
Q
i
Q
j
_ 0 or Q
i
Q
j
_ 0 and Q
i
,= Q
j
(15.13)
and
Q
i
Q
j
_ 0 i A
i
A
j
. (15.14)
Theorem 15.4 Consider the value function J
(x) in (15.12) and assume that

the CFTOC (12.9) problem leads to a non-degenerate mp-QP (12.33). Then the
gradient m(x) = J
(x), is a vector-valued PWA descriptor function.

Proof: From Theorem 7.9 we see that m(x) is continuous vector-valued PWA
function, while from equation (15.12) we get
m(x) = J
(x) = 2Q
i
x +T
i
(15.15)
Since from Theorem 15.3 we know that Q
i
,= Q
j
for all neighboring polyhedra,
it follows that m(x) satises all conditions for a vector-valued PWA descriptor
function.
Combining results of Theorem 15.4 and Theorem 15.2 it follows that by using
Algorithm 15.5 we can construct a PWA descriptor function from the gradient of
(x).
Generating a PWA descriptor function from the optimal inputs
Another way to construct descriptor function f(x) emerges naturally if we look at
the properties of the optimizer U
0
(x) corresponding to the state feedback solution
of the CFTOC problem (12.9). From Theorem 7.7 it follows that the optimizer
U
0
(x) is continuous in x and piecewise ane on polyhedra:
U
0
(x) = l
i
(x) = F
i
x +g
i
, if x T
i
, i = 1, . . . , N
r
, (15.16)
where F
i
R
sn
and g
i
R
s
. We will assume that T
i
are critical regions (as
dened in Section 7.1.2). Before going further we need the following lemma.
Lemma 15.1 Consider the state feedback solution (15.16) of the CFTOC prob-
lem (12.9) and assume that the CFTOC (12.9) leads to a non-degenerate mp-
QP (12.33). Let T
i
, T
j
be two neighboring polyhedra, then F
i
,= F
j
.
Proof: The proof is a simple consequence of Theorem15.3. As in Theorem 15.3,
without loss of generality we can assume that the set of active constraints /
i
associated with the critical region T
i
is empty, i.e., /
i
= . Suppose that the
optimizer is the same for both polyhedra, i.e., [F
i
g
i
] = [F
j
g
j
]. Then, the cost
functions q
i
(x) and q
j
(x) are also equal. From the proof of Theorem 15.3 this
implies that T
i
= T
j
, which is a contradiction. Thus we have [F
i
g
i
] ,= [F
j
g
j
].
Note that F
i
= F
j
cannot happen since, from the continuity of U
0
(x), this would
imply g
i
= g
j
. Consequently we have F
i
,= F
j
.
From Lemma 15.1 and Theorem 15.2 it follows that an appropriate PWA de-
scriptor function f(x) can be calculated from the gradient of the optimizer U
(x)
by using Algorithm 15.5.
Remark 15.2 Note that even if we are implementing a receding horizon control strat-
egy, the construction of the PWA descriptor function is based on the full optimization
vector U
(x) and the corresponding matrices F

i
and g
i
.
Remark 15.3 In some cases the use of the optimal control prole U
(x) for the

construction of descriptor function f(x) can be extremely simple. If there is a row
r, r m (m is the dimension of u) for which (F
i
)r ,= (F
j
)r, i = 1 . . . , N
r
, j Ci,
it is enough to set A
i
= (F
i
)r and B
i
= (g
i
)r, where (F
i
)r and (g
i
)r denote the i-th
row of the matrices F
i
and g
i
, respectively. The row r has to be the same for all
i = 1 . . . , N
r
. In this way we avoid the storage of the descriptor function altogether,
since it is equal to one component of the control law, which is stored anyway.
15.1.3 Example
As an example, we compare the performance of Algorithm 15.2, 15.3 and 15.4 on
CFTOC problem for the discrete-time system
_
_
x(t + 1) =
_
_
4 1.5 0.5 0.25
4 0 0 0
0 2 0 0
0 0 0.5 0
_
_
x(t) +
_
_
0.5
0
0
0
_
_
u(t)
y(t) =
_
0.083 0.22 0.11 0.02
x(t)
(15.17)
resulting from the linear system
y =
1
s
4
u (15.18)
sampled at T
s
= 1, subject to the input constraint
1 u(t) 1 (15.19)
and the output constraint
10 y(t) 10. (15.20)
CFTOC based on LP
To regulate (15.17), we design a receding horizon controller based on the optimiza-
tion problem (13.6), (13.7)) where p = , N = 2, Q = diag5, 10, 10, 10, R = 0.8,
P = 0, A
f
= R
4
. The PPWA solution of the mp-LP problem comprises 136 regions.
In Table 15.3 we report the comparison between the complexity of Algorithm 15.2
and Algorithm 15.3 for this example.
The average on-line evaluation eort of the PPWA solution for a set of 1000
random points in the state space is 2259 ops (Algorithm 15.2), and 1088 ops
(Algorithm 15.3). We note that the solution using Matlabs LP solver (function
linprog.m with interior point algorithm and LargeScale set to o) takes 25459
ops on average.
for the example in Section 15.1.3
Storage demand (real numbers) 5690 680
Number of ops (worst case) 9104 1088
Number of ops (average for 1000 random points) 2259 1088
CFTOC based on QP
To regulate (15.17), we design a receding horizon controller based on the optimiza-
tion problem (13.6), (13.7)) where N = 7, Q = I, R = 0.01, P = 0, A
f
= R
4
.
The PPWA solution of the mp-QP problem comprises 213 regions. We obtained
a descriptor function from the value function and for this example the choice of
w = [1 0 0 0]
is satisfactory. In Table 15.4 we report the comparison between the

complexity of Algorithm 15.2 and Algorithm 15.4 for this example.
The average computation eort of the PPWA solution for a set of 1000 ran-
dom points in the state space is 2114 ops (Algorithm 15.2), and 175 ops (Algo-
rithm 15.4). The solution of the corresponding quadratic program with Matlabs
QP solver (function quadprog.m and LargeScale set to o) takes 25221 ops on
average.
for the example in Section 15.1.3
Storage demand (real numbers) 9740 1065
Number of ops (worst case) 15584 3439
Number of ops (average for 1000 random points) 2114 175
15.1.4 Literature Review
The problem considered in this Chapter has been approached by several other
researchers. For instance in [274] the authors propose to organize the controller
gains of the PWA control law on a balanced search tree. By doing so, the search for
the region containing the current state has on average a logarithmic computation
complexity although it is less ecient in terms of memory requirements. At the
expense of the optimality a similar computational complexity can be achieved with
the approximative point location algorithm described in [159].
The comparison of the proposed algorithms with other ecient solvers in the
literature [21, 54, 254, 102, 207, 210, 282, 60, 25] requires the simultaneous analysis
of several issues such as speed of computation, storage demand and real time code
veriability.
15.2 Gradient Method Applied to MPC
In this section, we investigate the application of gradient methods to solving MPC
problems of type
J
0
(x(0)) = min
U0
x
N
Px
N
+
N1
k=0
x
k
Qx
k
+u
k
Ru
k
subj. to x
k+1
= Ax
k
+Bu
k
, k = 0, . . . , N 1
x
k
A, u
k
|, k = 0, . . . , N 1
x
N
A
f
x
0
= x(0),
(15.21)
15.2 Gradient Method Applied to MPC 321
where N is the time horizon and A
f
R
n
is a terminal polyhedral region. In (15.21)
U
0
= [u
0
, . . . , u
N1
]
R
s
, s = mN is the optimization vector.
We start by investigating a special case of the general setup in (15.21) with
A = A
f
= R
n
, i.e. we only impose constraints on the control inputs and show in
detail how gradient methods can be applied to solve the MPC problem (15.21) in
the input-constrained case. In order to obtain a similar setup as the one in (4.22),
we eliminate all states from the MPC problem (15.21) and rewrite it in condensed
form (cf. (9.8))
J
0
(x(0)) = min
U0
U
0
HU
0
+ 2x
(0)FU
0
+x
(0)Y x(0)
subj. to u
k
|, k = 0, . . . , N 1
x
0
= x(0).
(15.22)
where U
0
|
N
= | . . . |
Computation of Constants L and for MPC
As the objective function of the MPC problem (15.22) is twice continuously dif-
ferentiable, Lipschitz continuity of the gradient and strong convexity of the MPC
objective (15.22) can be established based on (4.1) and (4.2). Both the Lipschitz
constant L of the gradient and the convexity parameter of the MPC objec-
tive (15.22) are independent of the initial state x and are given by the maximum
(minimum) eigenvalue of the Hessian H, i.e.
L =
max
(H), =
min
(H) .
The MPC objective is strongly convex on R
Nm
if the weighting matrices in (15.21)
are chosen as Q _ 0, P _ 0 and R 0.
Projection on the Feasible Set
The projection operator in (4.25) has to be computed in every iteration of Al-
gorithm 4.6 and Algorithm 4.7. In the MPC context, this operation translates
to
U
0U
N(Y
i
) =
U
N
_
Y
i
2
L
(HY
i
+F
x(0))
_
, (15.23)
where Y
i
R
Nm
. As the feasible set is the direct product of N m-dimensional sets,
it suces to consider N projections
u
U,k
=
U
(u
f,k
) , k = 0, . . . , N 1 , (15.24)
where the original solution U
U
N(Y
i
) is obtained fromU
U
N(Y
i
) = (u
U,0
, u
U,1
, . . . , u
U,N1
).
The vectors u
f,k
in (15.24) result from splitting up vector U
f
(Y
i
) into N m-
dimensional vectors u
f,k
, i.e. (u
f,0
, u
f,1
, . . . , u
f,N1
) = U
f
(Y
i
). Hence if the input
set | allows for an ecient projection (cf. (4.1)), which is very often the case for
actuator constraints, (4.6) or (4.7) can be applied to solve (15.21). Even if the
input set | is an arbitrary polytope, there is the possibility of pre-computing an
explicit solution to (15.24) by means of multi-parametric programming with u
f,k
as the parameter.
For some control problems, the feasible set of inputs | originates from a sim-
ple set that is being translated, uniformly scaled and/or rotated. In this case,
projection is as easy as on the original simple set according to the next lemma.
Lemma 15.2 (Projection on Sets after Translation, Rotation and Scaling)
Let K R
n
be a closed convex set and
K
its associated projection operator. Pro-
jection of point z R
n
on the set
K = z R
n
[ z = Wy +c, y K
that is obtained from set K by translation with oset c R
n
, positive scaling by
> 0 and a linear transformation with an orthonormal matrix W R
ns
, can be
accomplished by
K
( z) = W
K
_
1
W
( z c)
_
+ c .
The separability property of the feasible set |
N
was used for simplifying the
projection. In case of state constraints, this benecial property is lost, rendering
the projection a hard problem. So, the gradient methods in the presented form
cannot be used to solve MPC problems with state constraints in the primal domain,
but need to be applied in the dual domain. We study this case in the following.
Application to Input and State Constrained MPC
In this section, we investigate the application of gradient methods for solving the
input- and state-constrained linear-quadratic MPC problem (15.21). Dierent from
Section 15.2, gradient methods cannot be applied in the primal domain here, since
the evaluation of the projection operator of the (primal) feasible set
_
u
k
|, x
k
A, x
N
A
f
x
k+1
= Ax
k
+Bu
k
, x
0
= x(0), k = 0 . . . N 1
_
is non-trivial in general, even if the convex sets |, A and A
f
allow ecient projec-
tions individually. The inherent complication comes from the intersection of these
sets with the ane set determined by the state update equation.
For the application of gradient methods to the input- and state-constrained
MPC problem (15.21) in the dual domain (cf. Section 4.2.1), we rewrite it as the
convex multi-parametric quadratic program
f
(x(0)) = min f(z) = z
Hz
s.t. Gz = wx(0)
z A . . . A A
f
| . . . | ,
with decision variable
z = (x
0
, x
1
, . . . , x
N
, u
0
, . . . , u
N1
) R
(N+1)n+Nm
, (15.25)
15.3 Interior Point Method Applied to MPC 323
where x R
n
is the initial state of the system, and matrices
H = blkdiag
_
Q, . . . , Q, P, R, . . . , R
_
, w =
_
I
n
, 0, . . . , 0
_
R
(N+1)nn
, (15.26)
G =
_
_
I
n
0 0 0 0
A I
n
0 0 B 0 0
0
.
.
.
.
.
.
.
.
.
.
.
. 0 B
.
.
.
.
.
.
.
.
.
.
.
. A I
n
0
.
.
.
.
.
.
.
.
. 0
0 0 A I
n
0 0 B
_
_
(15.27)
Unlike in the input constrained case, we do not eliminate the states from the opti-
mization vector. As above, by a separability argument, the projections necessary
can be carried out on the individual sets A, | and A
f
. The Lipschitz constant of
the dual function can be calculated as L
d
= |AH
1/2
|
2
, as given in (??). With
these observations, Algorithms 4.1 or 4.2 can be applied to solve (15.21) with input
and state constraints in the dual domain.
15.3 Interior Point Method Applied to MPC
All presented interior-point methods can be applied to linear model predictive con-
trol problems with convex constraints. To obtain an ecient method with low
solve times, it is crucial to exploit the sparsity structure of the resulting quadratic
program (or any other form of (P-IPM)) when computing the search directions.
Therefore, when using interior point solvers, the MPC problem is formulated as a
sparse QP in the form of problem (15.25) (sometimes with a dierent ordering of
states and inputs in the optimization vector). As a result, general sparse routines
for solving linear systems with KKT-structure can be applied to the Newton sys-
tems (4.38) and (4.42), or specic linear system solvers can be used that exploit
the MPC-specic problem structure. A good starting point for details on this topic
are the references [282, 244, 95, 198].
Chapter 16
Constrained Robust Optimal
Control
In the previous chapters we assumed perfect model knowledge for the plant to be
controlled. This chapter focuses on the case when the model is uncertain. We
focus on discrete-time uncertain linear systems with linear constraints on inputs
and states. Uncertainty is modeled as additive disturbance and/or parametric
uncertainty in the state-space matrices. We rst introduce the concepts of robust
control design with open-loop prediction and closed-loop prediction. Then
we discuss how to robustly enforce constraints satisfaction, and present several
objective functions one might be interested in minimizing. For each robust control
design we discuss the conservatism and show how to obtain the optimal control
action by using convex and non-convex programming.
For robust controllers which can be implemented by using linear and quadratic
programming, we show that the robust optimal control law is a continuous and
piecewise ane function of the state vector. The robust state feedback controller
can be computed by means of multiparametric linear or quadratic programming.
Thus, when the optimal control law is implemented in a moving horizon scheme,
the on-line computation of the resulting controller requires the evaluation of a
piecewise ane function.
Consider the linear uncertain system
x(t + 1) = A(w
p
(t))x(t) +B(w
p
(t))u(t) +Ew
a
(t) (16.1)
where x(t) R
n
and u(t) R
m
are the state and input vectors, respectively,
x(t) A, u(t) |, t 0. (16.2)
326 16 Constrained Robust Optimal Control
The sets A R
n
and | R
m
are polytopes. Vectors w
a
(t) R
na
and w
p
(t) R
np
are unknown additive disturbances and parametric uncertainties, respectively. The
disturbance vector is
w(t) = [w
a
(t)
, w
p
(t)
J R
nw
(16.3)
with n
w
= n
a
+ n
p
. We assume that bounds on w
a
(t) and w
p
(t) are known. In
particular J = J
a
J
p
with w
a
(t) J
a
and w
p
(t) J
p
, where J
a
R
na
and
J
p
R
np
are given polytopes. We also assume that A(), B() are ane functions
of w
p
A(w
p
) = A
0
+
np
i=1
A
i
w
p,i
c
, B(w
p
) = B
0
+
np
i=1
B
i
w
p,i
c
(16.4)
where A
i
R
nn
and B
i
R
nm
are given matrices for i = 0 . . . , n
p
and w
p,i
c
is
the i-th component of the vector w
p
, i.e., w
p
= [w
p,1
c
, . . . , w
p,np
c
].
Robust nite time optimal control problems can be formulated by optimizing
over Open-Loop policies or Closed-Loop policies. We will discuss the two cases
next.
Open-Loop Predictions
Dene the cost function as
J
0
(x(0), U
0
) =
W
_
p(x
N
) +
N1
k=0
q(x
k
, u
k
)
_
(16.5)
where the operation
W
evaluates the cost p(x
N
) +
N1
k=0
q(x
k
, u
k
) for the set of
uncertainties w
0
, . . . , w
N1
J J and the input sequence U
0
. The
following options are typically considered.
Nominal Cost.
In the simplest case, the objective function is evaluated for a single distur-
bance prole w
a
0
, . . . , w
a
N1
, w
p
0
, . . . , w
p
N1
:
W
=
_
p(x
N
) +
N1
k=0
q(x
k
, u
k
)
_
where
_
_
x
k+1
= A(w
p
k
)x
k
+B(w
p
k
)u
k
+Ew
a
k
x
0
= x(0)
w
a
k
= w
a
k
, w
p
k
= w
p
k
,
k = 0, . . . , N 1.
(16.6)
Expected Cost.
Another option is to consider the probability density function f(w) of the
disturbance w:
Probability[w(t) J] = 1 =
_
wW
f(w)dw.
Then the expected value of a function g(w) of the disturbance w is dened
as:
E
w
[g(w)] =
_
wW
g(w)f(w)dw.
In this case we consider the expected cost over the admissible disturbance
set.
W
= E
w0,...,wN1
_
p(x
N
) +
N1
k=0
q(x
k
, u
k
)
_
where
_
_
x
k+1
= A(w
p
k
)x
k
+B(w
p
k
)u
k
+Ew
a
k
x
0
= x(0)
w
a
k
J
a
, w
p
k
J
p
,
k = 0, . . . , N 1.
(16.7)
Worst Case Cost.
Finally the the worst case cost may be of interest
W
= max
w0,...,wN1
_
p(x
N
) +
N1
k=0
q(x
k
, u
k
)
_
where
_
_
x
k+1
= A(w
p
k
)x
k
+B(w
p
k
)u
k
+Ew
a
k
x
0
= x(0)
w
a
k
J
a
, w
p
k
J
p
,
k = 0, . . . , N 1.
(16.8)
If the 1-norm or -norm is used in the cost function of problems (16.6), (16.7)
and (16.8), then we set
p(x
N
) = |Px
N
|
p
, q(x
k
, u
k
) = |Qx
k
|
p
+|Ru
k
|
p
(16.9)
with p = 1 or p = . If the squared Euclidian norm is used in the cost function of
problems (16.6), (16.7) and (16.8), then we set
p(x
N
) = x
N
Px
N
, q(x
k
, u
k
) = x
k
Qx
k
+u
k
Ru
k
. (16.10)
Note that in (16.6), (16.7) and (16.8) x
k
denotes the state vector at time k obtained
by starting from the state x
0
x
k+1
= A(w
p
k
)x
k
+B(w
p
k
)u
k
+Ew
a
k
0
, . . . , u
k1
and the disturbance sequences w
a
= w
a
0
, . . . , w
a
N1
,
w
p
= w
p
0
, . . . , w
p
N1
.
Consider the optimal control problem
J
0
(x(0)) = min
U0
J
0
(x(0), U
0
)
subj. to
_
_
x
k+1
= A(w
p
k
)x
k
+B(w
p
k
)u
k
+Ew
a
k
x
k
A, u
k
|
k = 0, . . . , N 1
x
N
A
f
x
0
= x(0)
_
_
w
a
k
J
a
, w
p
k
J
p
k = 0, . . . , N 1
(16.11)
where A
f
R
n
is a terminal polyhedral region. In (16.11) N is the time horizon
and U
0
= [u
0
, . . . , u
N1
]
R
s
, s = mN the vector of the input sequence. We
denote by U
0
= u
0
, . . . , u
N1
the optimal solution to (16.11).
We denote with A
OL
i
A the set of states x
i
for which the robust optimal
control problem (16.8)-(16.11) is feasible, i.e.,
A
OL
i
= x
i
A : (u
i
, . . . , u
N1
) such that x
k
A, u
k
|, k = i, . . . , N 1,
x
N
A
f
w
a
k
J
a
, w
p
k
J
p
k = i, . . . , N 1,
where x
k+1
= A(w
p
k
)x
k
+B(w
p
k
)u
k
+Ew
a
k
.
(16.12)
Problem (16.11) minimizes the performance index
W
subject to the constraint
that the input sequence must be feasible for all possible disturbance realizations.
In other words, the nominal, expected or worst-case performance is minimized with
the requirement of constraint fulllment for all possible realizations of w
a
, w
p
.
Remark 16.1 Note that we distinguish between the current state x(k) of system (16.1)
at time k and the variable x
k
in the optimization problem (16.11), that is the predicted
state of system (16.1) at time k obtained by starting from the state x0 = x(0) and
applying to system x
k+1
= A(w
p
k
)x
k
+B(w
p
k
)u
k
+Ew
a
k
the input sequence u0, . . . , u
k1
and the disturbance sequences w
p
0
, . . . , w
p
k1
, w
a
0
, . . . , w
a
k1
. Analogously, u(k) is the
input applied to system (16.1) at time k while u
k
is the k-th optimization variable of
the optimization problem (16.11).
The formulation (16.11) is based on an open-loop prediction and thus referred to
as Constrained Robust Optimal Control with open-loop predictions (CROC-OL).
The optimal control problem (16.11) can be viewed as a game played between two
players: the controller U and the disturbance W [24, p. 266-272]. Regardless of the
cost function in (16.11) the player U tries to counteract any feasible disturbance
realization with just one single sequence u
0
, . . . , u
N1
. This prediction model
does not consider that at the next time step, the player can measure the state
x(1) and adjust his input u(1) based on the current measured state. By not
considering this fact, the eect of the uncertainty may grow over the prediction
horizon and may easily lead to infeasibility of the min problem (16.11). On the
contrary, in the closed-loop prediction scheme presented next, the optimization
scheme takes into account that the disturbance and the controller play one move
at a time.
The game is played dierently depending on the cost function
W
:
In problem (16.6), (16.11), among all control actions which robustly satisfy
the constraints, the player U chooses the one which minimizes a cost for a
guessed sequence played by W.
In problem (16.7), (16.11), among all control actions which robustly satisfy
the constraints, the player U chooses the one which minimizes an expected
cost over all the sequences, which can be played by W.
The optimal control problem (16.8), (16.11) is somehow more involved and
can be viewed as the solution of a zero-sum dynamic game. The player
U plays rst. Given the initial state x(0), U chooses his action over the
whole horizon u
0
, . . . , u
N1
, reveals his plan to the opponent W, who de-
cides on his actions next w
a
0
, w
p
0
, . . . , w
a
N1
, w
p
N1
by solving (16.8). By
solving (16.8), (16.11), the player U selects the action corresponding to the
smallest worst-case cost.
Closed-Loop Predictions
The constrained robust optimal control problem based on closed-loop predictions
(CROC-CL) is dened as follows:
J
0
(x(0)) = min
0(),...,N1()
J
0
(x(0), U
0
)
subj. to
_
_
x
k+1
= A(w
p
k
)x
k
+B(w
p
k
)u
k
+Ew
a
k
x
k
A, u
k
|
u
k
=
k
(x
k
)
k = 0, . . . , N 1
x
N
A
f
x
0
= x(0)
_
_
w
a
k
J
a
, w
p
k
J
p
k = 0, . . . , N 1
(16.13)
In (16.13) we allow the same cost functions as in (16.6), (16.7) and (16.8).
In problem (16.13) we look for a set of time-varying feedback policies
0
(),. . .,
N1
() which minimizes the performance index
W
and generates input sequences
0
(x
0
),. . .,
N1
(x
N1
) satisfying state and input constraints for all possible dis-
turbance realizations.
We can formulate the search for the optimal policies
0
(), . . . ,
N1
() as a
dynamic program.
J
j
(x
j
) = min
uj
J
j
(x
j
, u
j
)
subj. to
_
x
j
A, u
j
|
A(w
p
j
)x
j
+B(w
p
j
)u
j
+Ew
a
j
A
j+1
_
w
a
j
J
a
, w
p
j
J
p
(16.14)
J
j
(x
j
, u
j
) =
W
_
q(x
j
, u
j
) +J
j+1
(A(w
p
j
)x
j
+B(w
p
j
)u
j
+Ew
a
j
)
, (16.15)
for j = 0, . . . , N 1 and with boundary conditions
J
N
(x
N
) = p(x
N
) (16.16a)
A
N
= A
f
, (16.16b)
where A
j
denotes the set of states x for which (16.14)(16.16) is feasible
A
j
= x A : u | s.t. A(w
p
)x +B(w
p
)u +Ew
a
A
j+1
w
a
J
a
, w
p
J
p
.
(16.17)
and the cost operator
W
belongs to one of the following classes:
Nominal Cost.
W
=
_
q(x
j
, u
j
) +J
j+1
(A( w
p
j
)x
j
+B( w
p
j
)u
j
+E w
a
j
)
(16.18)
Expected Cost.
W
= E
w
a
j
W
a
,w
p
j
W
p
_
q(x
j
, u
j
) +J
j+1
(A(w
p
j
)x
j
+B(w
p
j
)u
j
+Ew
a
j
)
(16.19)
Worst Case Cost.
W
= max
w
a
j
W
a
,w
p
j
W
p
_
q(x
j
, u
j
) +J
j+1
(A(w
p
j
)x
j
+B(w
p
j
)u
j
+Ew
a
j
)
(16.20)
If the 1-norm or -norm is used in the cost function of problems (16.18), (16.19)
and (16.20), then we set
p(x
N
) = |Px
N
|
p
, q(x
k
, u
k
) = |Qx
k
|
p
+|Ru
k
|
p
(16.21)
with p = 1 or p = . If the squared Euclidian norm is used then we set
p(x
N
) = x
N
Px
N
, q(x
k
, u
k
) = x
k
Qx
k
+u
k
Ru
k
. (16.22)
The reason for including constraints (16.14) in the minimization problem and
not in the inner problem (16.15) is that in (16.15) w
a
j
and w
p
j
are free to act
regardless of the state constraints. On the other hand in (16.14), the input u
j
has
the duty to keep the state within the constraints (16.14) for all possible disturbance
realization. As a result of solving problem (16.14) at step j of the dynamic program,
we obtain the function u
j
(x
j
), i.e. the policy
j
().
Again, the optimal control problem (16.14)(16.15) can be viewed as a game
between two players: the controller U and the disturbance W. This time the payer
U predicts that the game is going to be played as follows. At the generic time j
player U observes x
j
and responds with u
j
=
j
(x
j
). Player W observes (x
j
, u
j
)
and responds with w
a
j
and w
p
j
. Therefore, regardless of the cost function in (16.15)
the player U counteracts any feasible disturbance realization w
j
with a feedback
controller
j
(x
j
). This prediction model takes into account that the disturbance
and the controller play one move at a time.
The game is played dierently depending on the cost function
W
.
In problem (16.14) and (16.18), among all the feedback policies u
j
=
j
(x
j
)
which robustly satisfy the constraints, the player U chooses the one which
minimizes a cost for a guessed action w
j
played by W.
In problem (16.14) and (16.19), among all the feedback policies u
j
=
j
(x
j
)
which robustly satisfy the constraints, the player U chooses the one which
minimizes the expected cost at time j.
The optimal control problem (16.14)(16.20) is somehow more involved and
can be viewed as the solution of a zero-sum dynamic game. The player U
plays rst. At the generic time j player U observes x
j
and responds with
u
j
=
j
(x
j
). Player W observes (x
j
, u
j
) and responds with the w
a
j
and w
p
j
which lead to the worst case cost. The player W will always play the worst
case action only if it has knowledge of both x
j
and u
j
=
j
(x
j
). In fact,
w
a
j
and w
p
j
in (16.20) are a function of x
j
and u
j
. If U does not reveal his
action to player W, then we can only claim that the player W might play
the worst case action. Problem (16.14)(16.20) is meaningful in both cases.
Robust constraint satisfaction and worst case minimization will always be
guaranteed.
Example 16.1 Consider the system
x
k+1
= x
k
+u
k
+w
k
(16.23)
where x, u and w are state, input and disturbance, respectively. Let u
k
|1, 0, 1
and w
k
|1, 0, 1 be the feasible input and disturbance. Here |1, 0, 1 denotes
the set with three elements: -1, 0 and 1. Let x(0) = 0 be the initial state. The
objective for player U is to play two moves in order to keep the state x2 at time 2
in the set [1, 1]. If U is able to do so for any possible disturbance, then he will win
the game.
The open-loop formulation (16.11) is infeasible. In fact, in open-loop U can choose
from nine possible sequences: (0,0), (1,1), (-1,-1), (-1,1) (1,-1), (-1,0), (1,0), (0,1)
and (0,-1). For any of those sequences there will always exist a disturbance sequence
w0, w1 which will bring x2 outside the feasible set [-1,1].
The closed-loop formulation (16.14)(16.20) is feasible and has a simple feasible so-
lution: u
k
= x
k
. In this case system (16.23) becomes x
k+1
= w
k
and x2 = w1 lies
in the feasible set [1, 1] for all admissible disturbances w1. The optimal feedback
policy might be dierent from u
k
= x
k
and depends on the choice of the objective
function.
16.1.1 State Feedback Solutions Summary
In the following sections we will describe how to compute the solution to CROC-OL
and CROC-CL problems. In particular we will show that the solution to CROC-
OL and CROC-CL problem can be expressed in feedback form where u
(k) is a
continuous piecewise ane function on polyhedra of the state x(k), i.e., u
(k) =
f
k
(x(k)) where
f
k
(x) = F
i
k
x +g
i
k
if H
i
k
x K
i
k
, i = 1, . . . , N
r
k
. (16.24)
H
i
k
and K
i
k
in equation (16.24) are the matrices describing the i-th polyhedron
CR
i
k
= x R
n
: H
i
k
x K
i
k
inside which the feedback optimal control law
u
(k) at time k has the ane form F

i
k
x +g
i
k
.
The optimal solution has the form (16.24) in the following cases.
Nominal or worst-case performance index based on 1-, -norm,
CROC-OL with no uncertainty in the dynamics matrix A (i.e., A() = A)
CROC-CL
Nominal performance index based on 2-norm,
CROC-OL with no uncertainty in the dynamics matrix A (i.e., A() = A)
If CROC-OL problems are considered, the set of polyhedra CR
i
k
, i = 1, . . . , N
r
k
is a polyhedral partition of the set of feasible states A
OL
k
(16.12) at time k.
If CROC-CL problems are considered, the set of polyhedra CR
i
k
, i = 1, . . . , N
r
k
is a polyhedral partition of the set of feasible states A
k
of problem (16.17) at time
k.
The dierence between the feasible sets A
OL
k
and A
k
associated with open-loop
prediction and closed-loop prediction, respectively, are discussed in the next section.
Remark 16.2 CROC-OL and CROC-CL problems with expected performance in-
dex are not treated in this book. In general, the calculation of expected costs over
polyhedral domains requires approximations via sampling or bounding even for dis-
turbances characterized by simple probability distribution functions (e.g., uniform
or Gaussian). Once the cost has been approximated, one can easily modify the ap-
proaches described in this chapter to solve CROC-OL and CROC-CL problems with
expected performance index.
16.2 Feasible Solutions
As in the nominal case (Section 12.2), there are two ways to rigorously dene and
compute the robust feasible sets: the batch approach and the recursive approach.
While for systems without disturbances, both approaches yield the same result, in
the robust case the batch approach provides the feasible set A
OL
i
of CROC with
open-loop predictions and the recursive approach provides the feasible set A
i
of
CROC with closed-loop predictions. From the discussion in the previous sections,
clearly we have A
OL
i
A
i
. We will detail the batch approach and the recursive
approach next and at the end of the section we will show how to modify the batch
approach in order to compute A
i
.
Batch Approach: Open-Loop Prediction
Consider the set A
OL
i
(16.12) of feasible states x
i
at time i for which (16.11) is
feasible, for i = 0, . . . , N, rewritten below
A
OL
i
= x
i
A : (u
i
, . . . , u
N1
) such that x
k
A, u
k
|, k = i, . . . , N 1,
x
N
A
f
w
a
k
J
a
, w
p
k
J
p
k = i, . . . , N 1,
where x
k+1
= A(w
p
k
)x
k
+B(w
p
k
)u
k
+Ew
a
k
Thus for any initial state x

i
A
OL
i
there exists a feasible sequence of inputs
U
i
= [u
i
, . . . , u
N1
] which keeps the state evolution in the feasible set A at future
time instants k = i +1, . . . , N 1 and forces x
N
into A
f
at time N for all feasible
disturbance sequences w
a
k
J
a
, w
p
k
J
p
, k = i, . . . , N 1. Clearly A
OL
N
= A
f
.
Next we show how to compute A
OL
i
for i = 0, . . . , N1. Let the state and input
constraint sets A, A
f
and | be the 1-polyhedra A
x
x b
x
, A
f
x b
f
, A
u
u b
u
,
respectively. Assume that the disturbance sets are represented in terms of their
vertices: J
a
= convw
a,1
, . . . , w
a,n
W
a
and J
p
= convw
p,1
, . . . , w
p,n
W
p
. Dene
U
i
= [u
i
, . . . , u
N1
] and the polyhedron T
i
of robustly feasible states and input
sequences at time i,
T
i
= (U
i
, x
i
) R
m(Ni)+n
: G
i
U
i
E
i
x
i
W
i
. (16.25)
In (16.25) G
i
, E
i
and W
i
are obtained by collecting all the following inequalities:
Input Constraints
A
u
u
k
b
u
, k = i, . . . , N 1
State Constraints
A
x
x
k
b
x
, k = i, . . . , N 1 for all w
a
l
J
a
, w
p
l
J
p
, l = i, . . . , k 1.
(16.26)
Terminal State Constraints
A
f
x
N
b
f
, for all w
a
l
J
a
, w
p
l
J
p
, l = i, . . . , N 1. (16.27)
Constraints (16.26)-(16.27) are enforced for all feasible disturbance sequences.
In order to do so, we rewrite constraints (16.26)-(16.27) at time k as a function of
x
i
and the input sequence U
i
.
A
x
_
k1
j=i
A(w
p
j
)x
i
+
k1
l=i

l1
j=i
A(w
p
j
)(B(w
p
k1l
)u
k1l
+Ew
a
k1l
)
_
b
x
,
(16.28)
In general, the constraint (16.28) is non-convex in the disturbance sequences
w
a
i
, . . . , w
a
k1
, w
p
i
, . . . , w
p
k1
because of the product
k1
j=i
A(w
p
j
).
Assume that there is no uncertainty in the dynamics matrix A, i.e. A() = A.
In this case, since B() is an ane function of w
p
(cf. equation (16.4)) and since the
composition of a convex constraint with an ane map generates a convex constraint
(Section 1.2), we can use Lemma 11.1 to rewrite constraints (16.26) at time k as
A
x
_
A
ki1
x
i
+
k1
l=i
A
li1
(B(w
p
k1l
)u
k1l
+Ew
a
k1l
)
_
b
x
,
for all w
a
j
w
a,i
n
W
a
i=1
, w
p
j
w
p,i
n
W
p
i=1
, j = i, . . . , k 1,
k = i, . . . , N 1.
(16.29)
and imposing the constraints at all the vertices of the sets J
a
J
a
. . . J
a
. .
i,...,N1
and J
p
J
p
. . . J
p
. .
i,...,N1
. Note that the constraints (16.29) are now linear in x
i
and U
i
. The same procedure can be repeated for constraint (16.27).
Remark 16.3 When only additive disturbances are present (i.e., np = 0), vertex
enumeration is not required and a set of linear programs can be used to transform
the constraints (16.26) into a smaller number of constraints than (16.29) as explained
in Lemma 11.2.
Once the matrices G
i
, E
i
and W
i
have been computed, the set A
OL
i
is a poly-
hedron and can be computed by projecting the polyhedron T
i
in (16.25) on the x
i
space.
Recursive Approach: Closed-Loop Prediction
In the recursive approach we have
A
i
= x A : u | such that A(w
p
i
)x +B(w
p
i
)u +Ew
a
i
A
i+1
,
w
a
i
J
a
, w
p
i
J
p
,
i = 0, . . . , N 1
A
N
= A
f
. (16.30)
The denition of A
i
in (16.30) is recursive and it requires that for any feasible
initial state x
i
A
i
there exists a feasible input u
i
which keeps the next state
A(w
p
i
)x +B(w
p
i
)u +Ew
a
i
in the feasible set A
i+1
for all feasible disturbances w
a
i

J
a
, w
p
i
J
p
.
Initializing A
N
to A
f
and solving (16.30) backward in time yields the feasible set
A
0
for the CROC-CL (16.14)(16.16) which, as shown in example 16.1, is dierent
from A
OL
0
.
Let A
i
be the 1-polyhedron A
Xi
x b
Xi
. Then the set A
i1
is the projection
of the following polyhedron
_
_
A
u
0
A
Xi
B(w
p
i
)
_
_
u
i1
+
_
_
0
A
x
A
Xi
A(w
p
i
)
_
_
x
i1

_
_
b
u
b
x
b
Xi
Ew
a
i
_
_
for all w
a
i
w
a,i
n
W
a
i=1
,
for all w
p
i
w
p,i
n
W
p
i=1
(16.31)
on the x
i1
space. (Note that we have used Lemma 11.1 and imposed state con-
straints at all the vertices of the sets J
a
J
p
.)
Remark 16.4 When only additive disturbances are present (i.e., np = 0), vertex
enumeration is not required and a set of linear programs can be used to transform the
constraints in (16.30) into a smaller number of constraints than (16.31) as explained
in Lemma 11.2.
The backward evolution in time of the feasible sets A
i
enjoys the properties
described by Theorems 12.1 and 12.2 for the nominal case. In particular if a
robust controlled invariant set is chosen as terminal constraint A
f
, then set A
i
grows as i becomes smaller and stops growing when it becomes the maximal robust
stabilizable set.
Theorem 16.1 Let the terminal constraint set A
f
be a robust control invariant
subset of A. Then,
i
, i = 0, . . . , N 1 is equal to the (N i)-step robust
stabilizable set:
A
i
= /
Ni
(A
f
, J)
i
, i = 0, . . . , N 1 is robust control invariant and contained
within the maximal robust control invariant set:
A
i
c
3. A
i
A
j
if i < j, i = 0, . . . , N 1. The size of the feasible A
i
set stops
increasing (with decreasing i) if and only if the maximal robust stabilizable
set is nitely determined and N i is larger than its determinedness index
N, i.e.
A
i
A
j
if N

N < i < j < N
Furthermore,
A
i
= /
(A
f
, J) if i N

N
Batch Approach: Closed-Loop Prediction
The batch approach can be modied in order to obtain A

i
instead of A
OL
i
. The
main idea is to augment the number of inputs by allowing one input sequence for
each vertex l of the set containing all disturbance sequences over the horizon [0,N-
1]. We will explain the approach under the assumption that there is no parametric
uncertainty (w = w
a
). From causality, the input u
j
is a function of x
0
and the
sequence w
0
, . . . , w
j1
. Therefore, u
0
will be unique, u
1
function of the vertices of
the disturbance set J
a
, u
2
function of the vertices of the disturbance set J
a
J
a
,
and u
N1
function of J
a
J
a
. . . J
a
. .
N1
. The generic l-th input sequence can
be written as
U
l
= [u
0
, u
j1
1
, u
j2
2
, . . . u
jN1
N1
]
where the lower index denotes time, and the upper index is used to associate a
dierent input for every disturbance realization. For instance, at time 0 the ex-
treme realizations of additive and parametric uncertainty are n
W
a. At time 1
there is one input for each realization and thus j
1
1, . . . , n
W
a. At time 2
the extreme realizations of the disturbance sequence w
0
, w
1
are n
2
W
a and there-
fore j
2
1, . . . , n
2
W
a. By repeating this argument we have j
k
1, . . . , n
k
W
a,
k = 1, . . . , N 1. In total we will have n
N1
W
a dierent control sequences U
l
and
m
_
1 +n
W
a +n
2
W
a + +n
N1
W
a
_
control variables to optimize over.
Figure 16.1 depicts an example where the disturbance set has two vertices w
1
and w
2
(n
W
a = 2) and the horizon is N = 4. The continuous line represents the
occurrence of w
1
, while the dashed line represents the occurrence of w
2
. Over the
horizon there are eight possible scenarios associated to eight extreme disturbance
realizations. The state x
j
3
denotes the predicted state at time 3 associated to the
j-th disturbance scenario.
Since the approach is computationally demanding, we prefer to present the main
idea through a very simple example rather than including all the tedious details.
x
k+1
= x
k
+u
k
+w
k
(16.32)
(x
0
, u
0
)
( x
1
1
, u
1
1
)
( x
2
1
, u
2
1
)
( x
1
2
, u
1
2
)
( x
2
2
, u
2
2
)
( x
3
2
, u
3
2
)
( x
4
2
, u
4
2
)
( x
1
3
, u
1
3
)
( x
2
3
, u
2
3
)
( x
3
3
, u
3
3
)
( x
4
3
, u
4
3
)
( x
5
3
, u
5
3
)
( x
6
3
, u
6
3
)
( x
7
3
, u
7
3
)
( x
8
3
, u
8
3
)
Figure 16.1 Batch Approach with Closed-Loop Prediction. State propaga-
tion and associated input variables. The disturbance set has two vertices w
1
and w
2
(nW
a = 2) and the horizon is N = 4. The continuous line represents
the occurrence of w
1
, while the dashed line represents the occurrence of w
2
.
Over the horizon there are eight possible scenarios associated to eight ex-
treme disturbance realizations. The state x
j
3
denotes the predicted state at
time 3 associated to the j-th disturbance scenario.
k
[1, 1]
and w
k
[1, 1] be the feasible input and disturbance. The objective for player U
is to play two moves in order to keep the state at time three x3 in the set .
f
= [1, 1].
Batch approach
We rewrite the terminal constraint as
x3 = x0 +u0 +u1 +u2 +w0 +w1 +w2 [1, 1]
for all w0 [1, 1], w1 [1, 1], w2 [1, 1]
(16.33)
which by Lemma 11.1 becomes
1 x0 +u0 +u1 +u2 + 3 1
1 x0 +u0 +u1 +u2 + 1 1
1 x0 +u0 +u1 +u2 1 1
1 x0 +u0 +u1 +u2 3 1
(16.34)
which by removing redundant constraints becomes the constraint
2 x0 +u0 +u1 +u2 2 (16.35)
which is infeasible and thus .0 = . For the sake of clarity we continue with
the procedure as if we did not notice that constraint (16.35) was infeasible.
The set .
OL
0
is the projection on the x0 space of the polyhedron T0 of combined
feasible input and state constraints
T0 = |(u0, u1, u2, x0) R
4
:
_
_
_
_
_
_
_
_
_
_
_
_
1 0 0
1 0 0
0 1 0
0 1 0
0 0 1
0 0 1
1 1 1
1 1 1
_
_
_
_
u0
u1
u2
_
_
+
_
_
_
_
_
_
_
_
_
_
_
_
0
0
0
0
0
0
1
1
_
_
x0
_
_
_
_
_
_
_
_
_
_
_
_
1
1
1
1
1
1
2
2
_
_
.
The projection will provide an empty set since the last two constraints corre-
spond to the infeasible terminal state constraint (16.35).
Recursive approach
For the recursive approach we have .3 = .
f
= [1, 1]. We rewrite the terminal
constraint as
x3 = x2 +u2 +w2 [1, 1] for all w2 [1, 1] (16.36)
which by Lemma 11.1 becomes
1 x2 +u2 + 1 1
1 x2 +u2 1 1
(16.37)
which by removing redundant constraints becomes
0 x2 +u2 0.
The set .2 is the projection on the x2 space of the polyhedron
_
_
_
_
1 0
1 0
1 1
1 1
_
_
_
u2
x2
_
_
_
_
_
1
1
0
0
_
_
(16.38)
which yields .2 = [1, 1]. Since .2 = .3 one can conclude that .2 is the
maximal controllable robust invariant set and .0 = .1 = .2 = [1, 1].
Batch approach with closed-loop predictions
The horizon N is three and therefore we have 2

(N1)
= 4 dierent control
sequences: |u0, u
1
1
, u
1
2
, |u0, u
1
1
, u
2
2
, |u0, u
2
1
, u
3
2
, and |u0, u
2
1
, u
4
2
, cor-
responding to the extreme disturbance realizations |w0 = 1, w1 = 1, w2,
|w0 = 1, w1 = 1, w2, |w0 = 1, w1 = 1, w2 and |w0 = 1, w1 = 1, w2,
respectively.
The terminal constraint is thus rewritten as
1 x0 +u0 + u
1
1
+ u
1
2
1 1 +w2 1, w2 [1, 1]
1 x0 +u0 + u
1
1
+ u
2
2
1 + 1 +w2 1, w2 [1, 1]
1 x0 +u0 + u
2
1
+ u
3
2
+ 1 1 +w2 1, w2 [1, 1]
1 x0 +u0 + u
2
1
+ u
4
2
+ 1 + 1 +w2 1, w2 [1, 1]
1 u0 1
1 u
1
1
1
1 u
2
1
1
1 u
1
2
1
1 u
2
2
1
1 u
3
2
1
1 u
4
2
1
(16.39)
which becomes by using Lemma 11.1
1 x0 +u0 + u
1
1
+ u
1
2
1 1 + 1 1
1 x0 +u0 + u
1
1
+ u
1
2
1 1 1 1
1 x0 +u0 + u
1
1
+ u
2
2
1 + 1 + 1 1
1 x0 +u0 + u
1
1
+ u
2
2
1 + 1 1 1
1 x0 +u0 + u
2
1
+ u
3
2
+ 1 1 + 1 1
1 x0 +u0 + u
2
1
+ u
3
2
+ 1 1 1 1
1 x0 +u0 + u
2
1
+ u
4
2
+ 1 + 1 + 1 1
1 x0 +u0 + u
2
1
+ u
4
2
+ 1 + 1 1 1
1 u0 1
1 u
1
1
1
1 u
2
1
1
1 u
1
2
1
1 u
2
2
1
1 u
3
2
1
1 u
4
2
1
(16.40)
The set .0 can be obtained by projecting the polyhedron (16.40) in the
(x0, u0, u
1
1
, u
2
1
, u
1
2
, u
2
2
, u
3
2
, u
4
2
)-space on the x0 space. This yields .0 = [1, 1],
the same as in the recursive approach.
16.3 State Feedback Solution, Nominal Cost
Theorem 16.2 Consider the CROC-OL (16.6), (16.11) with cost (16.9) or (16.10).
Assume that the parametric uncertainties are in the B matrix only (A(w
p
) = A).
Then, there exists a solution u
(0) = f
OL
0
(x(0)), f
0
: R
n
R
m
, which is continu-
ous and PPWA
f
OL
0
(x) = F
i
0
x +g
i
0
if x CR
i
0
, i = 1, . . . , N
r
0
(16.41)
i
0
= H
i
0
x k
i
0
, i = 1, . . . , N
r
0
feasible set A
OL
0
. Moreover f
0
can be found by solving an mp-LP for cost (16.9)
and an mp-QP for cost (16.10). The same result holds for u
(k) = f
OL
k
(x(0)), k =
1, . . . , N 1.
Proof: The proof can be easily derived from the results in Chapter 12. The
only dierence is that the constraints have to be rst robustied as explained in
Section 16.2.
Theorem 16.3 There exists a state feedback control law u
(k) = f
k
(x(k)), f
k
:
A
k
R
n
| R
m
, solution of the CROC-CL (16.14)(16.16), (16.18) with
cost (16.21) and k = 0, . . . , N 1 which is time-varying, continuous and piecewise
ane on polyhedra
f
k
(x) = F
i
k
x +g
i
k
if x CR
i
k
, i = 1, . . . , N
r
k
(16.42)
16.4 State Feedback Solution, Worst-Case Cost, 1-Norm and -Norm Case 339
i
k
= x R
n
: H
i
k
x K
i
k
, i = 1, . . . , N
r
k
are a
k
. Moreover f
i
, i = 0, . . . , N 1 can be found
by solving N mp-LPs.
Proof: The proof can be easily derived from the results in Chapter 12. The
only dierence is that the constraints have to be rst robustied as explained in
Section 16.2.
16.4 State Feedback Solution, Worst-Case Cost, 1-Norm
and -Norm Case
In this chapter we show how to nd a state feedback solution to CROC problems
when a worst case performance index is used. The following results will be used in
the next sections.
Lemma 16.1 Let f : R
s
R
n
R
nw
R and g : R
s
R
n
R
nw
R
ng
be functions
of (z, x, w) convex in w for each (z, x). Assume that the variable w belongs to the
polyhedron J with vertices w
i
NW
i=1
. Then the min-max multiparametric problem
J
(x) = min
z
max
wW
f(z, x, w)
subj. to g(z, x, w) 0 w J
(16.43)
is equivalent to the multiparametric optimization problem
J
(x) = min
,z

subj. to f(z, x, w
i
), i = 1, . . . , N
W
g(z, x, w
i
) 0, i = 1, . . . , N
W
.
(16.44)
Proof: Easily follows from the fact that the maximum of a convex function
over a convex set is attained at an extreme point of the set, cf. also [257].
Corollary 16.1 If f is convex and piecewise ane in (z, x), i.e. f(z, x, w) =
max
i=1,...,n
f
L
i
(w)z + H
i
(w)x + K
i
(w) and g is linear in (z, x) for all w J,
g(z, x, w) = L
g
(w)z + H
g
(w)x + K
g
(w) (with L
g
(), H
g
(), K
g
(), L
i
(), H
i
(),
K
i
(), i = 1, . . . , n
f
, convex functions), then the min-max multiparametric prob-
lem (16.43) is equivalent to the mp-LP problem
J
(x) = min
,z

subj. to L
j
( w
i
)z +H
j
( w
i
)x +K
j
( w
i
), i = 1, . . . , N
W
, j = 1, . . . , n
f
L
g
( w
i
)z +H
g
( w
i
)x K
g
( w
i
), i = 1, . . . , N
W
(16.45)
Remark 16.5 As discussed in Lemma 11.2, in the case g(z, x, w) = g1(z, x) +g2(w),
the second constraint in (16.44) can be replaced by g1(z, x) g, where g =
_
g1, . . . , gng
_
is a vector whose i-th component is

gi = max
wW
g
i
2
(w), (16.46)
and g
i
2
(w) denotes the i-th component of g2(w). Similarly, if f(z, x, w) = f1(z, x) +
f2(w), the rst constraint in (16.44) can be replaced by f1(z, x) +

f, where
fi = max
wW
f
i
2
(w). (16.47)
Clearly, this has the advantage of reducing the number of constraints in the multi-
parametric program from NWng to ng for the second constraint in (16.44) and from
NWn
f
to n
f
for the rst constraint in (16.44).
In the following subsections we show how to solve CROC problems in state
feedback form by using multiparametric linear programming.
16.4.1 Batch Approach: Open-Loop Predictions
Theorem 16.4 Consider the CROC-OL (16.8), (16.9), (16.11). Assume that the
parametric uncertainties are in the B matrix only (A(w
p
) = A). Then, there exists
a solution u
(0) = f
OL
0
(x(0)), f
OL
0
: R
n
R
m
, which is continuous and PPWA
f
OL
0
(x) = F
i
0
x +g
i
0
if x CR
i
0
, i = 1, . . . , N
r
0
(16.48)
i
0
= H
i
0
x k
i
0
, i = 1, . . . , N
r
0
feasible set A
OL
0
. Moreover f
0
can be found by solving an mp-LP. The same result
holds for u
(k) = f
OL
k
(x(0)), k = 1, . . . , N 1.
Proof: Since x
k
= A
k
x
0
+
k1
k=0
A
i
[B(w
p
k1i
)u
k1i
+ Ew
a
k1i
] is a linear
function of the disturbances w
a
= w
a
0
, . . . , w
a
N1
, and w
p
= w
p
0
, . . . , w
p
N1
for a
xed input sequence and x
0
, the cost function in the maximization problem (16.8) is
convex and piecewise ane with respect to the optimization vectors w
a
and w
p
and
the parameters U
0
, x
0
. The constraints in (16.11) are linear in U
0
and x
0
, for any
w
a
and w
p
. Therefore, by Corollary 16.1, problem (16.8)(16.11) can be solved by
specifying the constraints (16.11) only at the vertices of the set J
a
J
a
. . .J
a
and J
p
J
p
. . . J
p
and solving an mp-LP. The theorem follows from the
mp-LP properties described in Theorem 7.5.
Remark 16.6 Note that Theorem 16.4 does not hold when parametric uncertainties
are present also in the A matrix. In this case the predicted state x
k
is a nonlinear
function of the vector w
p
.
Remark 16.7 In case of CROC-OL with additive disturbances only (w
p
(t) = 0) the
number of constraints in (16.11) can be reduced as explained in Remark 16.5.
16.4.2 Recursive Approach: Closed-Loop Predictions
Theorem 16.5 There exists a state feedback control law u
(k) = f
k
(x(k)), f
k
:
A
k
R
n
| R
m
, solution of the CROC-CL (16.14)(16.17) with cost (16.20), (16.21)
and k = 0, . . . , N 1 which is time-varying, continuous and piecewise ane on
polyhedra
f
k
(x) = F
i
k
x +g
i
k
if x CR
i
k
, i = 1, . . . , N
r
k
(16.49)
i
k
= x R
n
: H
i
k
x K
i
k
, i = 1, . . . , N
r
k
are a
k
. Moreover f
i
, i = 0, . . . , N 1 can be found
by solving N mp-LPs.
Proof: Consider the rst step j = N 1 of dynamic programming applied to
the CROC-CL problem (16.14)(16.20) with cost (16.9)
J
N1
(x
N1
) = min
uN1
J
N1
(x
N1
, u
N1
) (16.50)
subj. to
_
_
_
x
N1
A, u
N1
|
A(w
p
N1
)x
N1
+B(w
p
N1
)u
N1
+Ew
a
N1
A
f
w
a
N1
J
a
, w
p
N1
J
p
(16.51)
J
N1
(x
N1
, u
N1
) = max
w
a
N1
W
a
, w
p
N1
W
p
_
_
_
|Qx
N1
|
p
+|Ru
N1
|
p
+
+|P(A(w
p
N1
)x
N1
+
+B(w
p
N1
)u
N1
+Ew
a
N1
)|
p
_
_
_
.
(16.52)
The cost function in the maximization problem (16.52) is piecewise ane and
convex with respect to the optimization vector w
a
N1
, w
p
N1
and the parameters
u
N1
, x
N1
. Moreover, the constraints in the minimization problem (16.51) are
linear in (u
N1
, x
N1
) for all vectors w
a
N1
, w
p
N1
. Therefore, by Corollary 16.1,
J
N1
(x
N1
), u
N1
(x
N1
) and A
N1
are computable via the mp-LP:
J
N1
(x
N1
) = min
,uN1
subj. to |Qx
N1
|
p
+|Ru
N1
|
p
+
+|P(A( w
p
h
)x
N1
+B( w
p
h
)u
N1
+E w
a
i
)|
p
(16.53a)
x
N1
A, u
N1
| (16.53b)
A( w
p
h
)x
N1
+B( w
p
h
)u
N1
+E w
a
i
A
N
(16.53c)
i = 1, . . . , n
W
a, h = 1, . . . , n
W
p.
where w
a
i
n
W
a
i=1
and w
p
h
n
W
p
h=1
are the vertices of the disturbance sets J
a
and
J
p
, respectively. By Theorem 7.5, J
N1
is a convex and piecewise ane function
of x
N1
, the corresponding optimizer u
N1
is piecewise ane and continuous,
and the feasible set A
N1
is a convex polyhedron. Therefore, the convexity and
linearity arguments still hold for j = N2, . . . , 0 and the procedure can be iterated
backwards in time j, proving the theorem. The theorem follows from the mp-LP
properties described in Theorem 7.5.
Remark 16.8 Let na and n
b
be the number of inequalities in (16.53a) and (16.53c),
respectively, for any i and h. In case of additive disturbances only (nW
p = 0) the
total number of constraints in (16.53a) and (16.53c) for all i and h can be reduced
from (na +n
b
)nW
anW
p to na +n
b
as shown in Remark 16.5.
Remark 16.9 The closed-loop solution u
(k) = f
k
(x(k)) can be also obtained by using
the modied batch approach with closed-loop prediction as discussed in Section 16.2.
The idea there is to augment the number of free inputs by allowing one sequence
u
i
0
, . . . u
i
N1
for each vertex i of the disturbance set V
a
V
a
. . . V
a
. .
N1
. The large
number of extreme points of such a set and the consequent large number of inputs
and constraints make this approach computationally hard.
16.4.3 Solution to CROC-CL and CROC-OL via mp-MILP*
Theorems 16.2-16.5 propose dierent ways of nding the PPWA solution to con-
strained robust optimal control by using multiparametric programs. The solution
approach presented in this section is more general than the one of Theorems 16.2-
16.5 as it does not exploit convexity, so that it may also be used in other contexts,
for instance for CROC-CL of hybrid systems, or for CROC of uncertain systems
of the type x(t + 1) = Ax(t) + Bu(t) + Ew
a
(t) where A and B belong to a nite
discrete set of known matrices.
Consider the multiparametric mixed-integer linear program (mp-MILP)
J
(x) = min
z
J(z, x) = c
z
subj. to Gz w +Sx.
(16.54)
where z = [z
c
, z
d
], z
c
R
nc
, z
d
0, 1
n
d
, s = n
c
+n
d
, z R
s
is the optimization
vector and x R
n
is the vector of parameters.
For a given polyhedral set / R
n
of parameters, solving (16.54) amounts to
determining the set /
/ of parameters for which (16.54) is feasible, the value

function J : /
R, and the optimizer function

1
z
: /
R
s
.
The properties of J
() and z
() have been analyzed in Section 7.4.1 and sum-

marized in Theorem 7.10. Below we state some properties based on these theorems.
Lemma 16.2 Let J : R
s
R
n
R be a continuous piecewise ane (possibly
nonconvex) function of (z, x),
J(z, x) = L
i
z +H
i
x +K
i
for [
z
x
] 1
i
, (16.55)
where 1
i
nJ
i=1
are polyhedral sets with disjoint interiors, 1 =
nJ
i=1
1
i
is a (possibly
non-convex) polyhedral set and L
i
, H
i
and K
i
are matrices of suitable dimensions.
1
In case of multiple solutions, we dene z
(x) as one of the optimizers.

Then the multiparametric optimization problem
J
(x) = min
z
J(z, x)
subj. to Cz c +Sx.
(16.56)
is an mp-MILP.
Proof: By following the approach of [39] to transform piecewise ane func-
tions into a set of mixed-integer linear inequalities, introduce the auxiliary binary
optimization variables
i
0, 1, dened as
[
i
= 1]
_
[
z
x
] 1
i
, (16.57)
where
i
, i = 1, . . . , n
J
, satisfy the exclusive-or condition
nJ
i=1
i
= 1, and set
J(z, x) =
nJ
i=1
q
i
(16.58)
q
i
= [L
i
z +H
i
x +K
i
]
i
(16.59)
where q
i
are auxiliary continuous optimization vectors. By transforming (16.57)
(16.59) into mixed-integer linear inequalities [39], it is easy to rewrite (16.56) as a
multiparametric MILP.
Next we present two theorems which describe how to use mp-MILP to solve
CROC-OL and CROC-CL problems.
Theorem 16.6 By solving two mp-MILPs, the solution U
0
(x
0
) to the CROC-
OL (16.8), (16.9), (16.11) with additive disturbances only (n
p
= 0) can be computed
in explicit piecewise ane form (16.48).
Proof: The objective function in the maximization problem (16.8) is convex
and piecewise ane with respect to the optimization vector w
a
= w
a
0
, . . . , w
a
N1
and the parameters U = u

0
, . . . , u
N1
, x
0
. By Lemma 16.2, it can be solved via
mp-MILP. By Theorem 7.10, the resulting value function J is a piecewise ane
function of U and x
0
and the constraints in (16.11) are a linear function of the
disturbance w
a
for any given U and x
0
. Then, by Lemma 16.2 and Corollary 16.1
the minimization problem is again solvable via mp-MILP, and the optimizer U
=
u
0
,. . . ,u
N1
is a piecewise ane function of x
0
.
Theorem 16.7 By solving 2N mp-MILPs, the solution of the CROC-CL (16.14)
(16.17) problem with cost (16.20), (16.21) and additive disturbances only (n
p
= 0)
can be obtained in state feedback piecewise ane form (16.49)
Proof: Consider the rst step j = N 1 of the dynamic programming solution
to CROC-CL. The cost-to-go at step N 1 is p(x
N
) from (16.16). Therefore, the
cost function in the maximization problem is piecewise ane with respect to both
the optimization vector w
a
N1
and the parameters u
N1
, x
N1
. By Lemma 16.2,
the worst case cost function (16.20) can be computed via mp-MILP, and, from
Theorem 7.10 we know that J
N1
(u
N1
, x
N1
) is a piecewise ane function. Then,
since constraints (16.51) are linear with respect to w
a
N1
for each u
N1
, x
N1
, we
can apply Corollary 16.1 and Remark 16.5 by solving LPs of the form (11.52).
Then, by Lemma 16.2, the optimal cost J
N1
(x
N1
) of the minimization problem
(16.14) is again computable via mp-MILP, and by Theorem 7.10 it is a piecewise
ane function of x
N1
. By virtue of Theorem 7.10, A
N1
is a (possible non-convex)
polyhedral set and therefore the above maximization and minimization procedures
can be iterated to compute the solution (16.49) to the CROC-CL problem.
16.5 Parametrizations of the Control Policies
From the previous sections it is clear that CROC-OL (16.8)(16.11) is conservative
since we are optimizing an open-loop control sequence that has to cope with all
possible future disturbance realizations, without taking future measurements into
account. The CROC-CL (16.14)(16.16) formulation overcomes this issue but it
can quickly lead to a problem so large that it becomes intractable.
This section presents an alternative approach which introduces feedback in the
system and, in some cases, can be more ecient than CROC-CL. The idea is
to parameterize the control sequence in the state vector and optimize over these
parameters. The approach is described next for systems with additive uncertainties.
Consider the worst case cost function as
J
0
(x(0), U
0
) = max
w
a
0
,...,w
a
N1
p(x
N
) +
N1
k=0
q(x
k
, u
k
)
subj. to
_
_
_
x
k+1
= Ax
k
+Bu
k
+Ew
a
k
w
a
k
J
a
,
k = 0, . . . , N 1
(16.60)
where N is the time horizon and U
0
= [u
0
, . . . , u
N1
]
R
s
, s = mN the vector of
the input sequence. Consider the robust optimal control problem
J
0
(x
0
) =min
U0
J
0
(x
0
, U
0
) (16.61)
subj. to
_
_
x
k
A, u
k
|
x
k+1
= Ax
k
+Bu
k
+Ew
a
k
x
N
A
f
k = 0, . . . , N 1
_
_
w
a
k
J
a
k = 0, . . . , N 1
(16.62)
Consider the parametrization of the control sequence
u
k
=
k
i=0
L
k,i
x
i
+g
i
, k 0, . . . , N 1 (16.63)
with the compact notation:
U
0
= Lx +g,
16.5 Parametrizations of the Control Policies 345
where x = [x
0
, x
1
, . . . , x
N
]
and
L =
_
_
L
0,0
0 0
.
.
.
.
.
.
.
.
.
.
.
.
L
N1,0
L
N1,N1
0
_
_, g =
_
_
g
0
.
.
.
g
N1
_
_ (16.64)
where L R
mNnN
and g R
mN
are unknown feedback control gain and oset,
respectively. With the parametrization (16.63) the robust control problem (16.60-
(16.62) becomes
J
Lg
0
(x(0), L, g) = max
w
a
0
,...,w
a
N1
p(x
N
) +
N1
k=0
q(x
k
, u
k
)
subj. to
_
_
x
k+1
= Ax
k
+Bu
k
+Ew
a
k
w
a
k
J
a
,
u
k
=
k
i=0
L
k,i
x
i
+g
i
k = 0, . . . , N 1
(16.65)
J
Lg
0
(x
0
) =min
L,g
J
Lg
0
(x
0
, L, g) (16.66)
subj. to
_
_
x
k
A, u
k
|
x
k+1
= Ax
k
+Bu
k
+Ew
a
k
u
k
=
k
i=0
L
k,i
x
i
+g
i
x
N
A
f
k = 0, . . . , N 1
_
_
.
w
a
k
J
a
k = 0, . . . , N 1
(16.67)
We denote with A
Lg
0
0
for which the robust optimal control
problem (16.66)-(16.67) is feasible, i.e.,
A
Lg
0
=
_
x
0
R
n
: T
Lg
0
(x
0
) ,=
_
T
Lg
0
(x
0
) = L, g : x
k
A, u
k
|, k = 0, . . . , N 1, x
N
A
f
w
a
k
J
a
k = 0, . . . , N 1,
where x
k+1
= Ax
k
+Bu
k
+Ew
a
k
, u
k
=
k
i=0
L
k,i
x
i
+g
i
(16.68)
Problem (16.65) looks for the worst value of the performance index J
Lg
0
(x
0
, L, g)
and the corresponding worst sequences w
a
as a function of x
0
and the controller
gain L and oset g.
Problem (16.66)(16.67) minimizes (over L and g) the worst performance sub-
ject to the constraint that the input sequence U
0
= Lx + g must be feasible for
all possible disturbance realizations. Notice that formulation (16.65)(16.67) is
based on a closed-loop prediction. Unfortunately the set T
Lg
0
(x
0
) is nonconvex,
in general [191]. Therefore, nding L and g for a given x
0
may be dicult.
Consider now the parametrization of the control sequence in past disturbances
u
k
=
k1
i=0
M
k,i
w
a
i
+v
i
, k 0, . . . , N 1, (16.69)
which can be compactly written as:
U
0
= Mw
a
+v
where
M =
_
_
0 0
M
1,0
0 0
.
.
.
.
.
.
.
.
.
.
.
.
M
N1,0
M
N1,N2
0
_
_
, v =
_
_
v
0
.
.
.
.
.
.
v
N1
_
_
. (16.70)
Notice that since
Ew
a
k
= x
k+1
Ax
k
Bu
k
, k 0, . . . , N 1.
the parametrization (16.69) can also be interpreted as a parametrization in the
states. The advantages of using (16.69) are explained next.
With the parametrization (16.69) the robust control problem (16.60-(16.62)
becomes
J
Mv
0
(x
0
, M, v) = max
w
a
0
,...,w
a
N1
p(x
N
) +
N1
k=0
q(x
k
, u
k
)
subj. to
_
_
x
k+1
= Ax
k
+Bu
k
+Ew
a
k
w
a
k
J
a
,
u
k
=
k1
i=0
M
k,i
w
a
i
+v
i
k = 0, . . . , N 1
(16.71)
J
Mv
0
(x
0
) =min
M,v
J
Mv
0
(x
0
, M, v) (16.72)
subj. to
_
_
x
k
A, u
k
|
x
k+1
= Ax
k
+Bu
k
+Ew
a
k
u
k
=
k1
i=0
M
k,i
w
a
i
+v
i
x
N
A
f
k = 0, . . . , N 1
_
_
w
a
k
J
a
k = 0, . . . , N 1
(16.73)
The problem (16.71)-(16.73) is now convex in the controller parameters M and v.
As solution we obtain u
(0) = f
0
(x(0)) = v
0
(x(0)).
We denote with A
Mv
0
0
for which the robust optimal
control problem (16.72)-(16.73) is feasible, i.e.,
A
Mv
0
=
_
x
0
R
n
: T
Mv
0
(x
0
) ,=
_
T
Mv
0
(x
0
) = M, v : x
k
A, u
k
|, k = 0, . . . , N 1, x
N
A
f
w
a
k
J
a
k = 0, . . . , N 1, where x
k+1
= Ax
k
+Bu
k
+Ew
a
k
,
u
k
=
k1
i=0
M
k,i
w
a
i
+v
i
(16.74)
The following result has been proven in [128], the convexity property has also
appeared in [191][Section 7.4].
Theorem 16.8 Consider the control parameterizations (16.63), (16.69) and the
corresponding feasible sets A
Lg
0
in (16.68) and A
Mv
0
in (16.74). Then,
A
Lg
0
= A
Mv
0
and T
Mv
0
(x
0
) is convex in M and v.
Note that in general A
Mv
0
and J
Mv
0
(x
0
) are dierent from the corresponding
CROC-CL solutions A
0
and J
0
(x
0
). In particular A
Mv
0
A
0
and J
Mv
0
(x
0
)
J
0
(x
0
).
Next we solve the problem in example 16.2 by using the approach discussed in
this section. The example will shed some light on how to compute the disturbance
feedback gains by means of convex optimization.
x
k+1
= x
k
+u
k
+w
k
(16.75)
k
[1, 1] and
w
k
[1, 1] be the feasible input and disturbance. The objective for player U is to
play two moves in order to keep the state x3 in the set .
f
= [1, 1].
Rewrite the terminal constraint as
x3 = x0 +u0 +u1 +u2 +w0 +w1 +w2 [1, 1]
w0 [1, 1], w1 [1, 1], w2 [1, 1].
(16.76)
The control inputs u are parameterized in past disturbances as in equation (16.69)
u0 = v0
u1 = v1 +M1,0w0
u2 = v2 +M2,0w0 +M2,1w1
(16.77)
Input constraints and terminal constraint are rewritten as
x0 +v0 +v1 +v2 + (1 +M1,0 +M2,0)w0 + (1 +M2,1)w1 +w2 [1, 1]
u0 = v0 [1, 1]
u1 = v1 +M1,0w0 [1, 1]
u2 = v2 +M2,0w0 +M2,1w1 [1, 1]
w0 [1, 1], w1 [1, 1], w2 [1, 1].
(16.78)
There are two approaches to convert the innite number of constraints (16.78) into
a set of nite number of constraint. The rst approach has been already presented
in this chapter and resorts to disturbance vertex enumeration. The second approach
uses duality. We will present both approaches next.
Enumeration method
Constraint (16.78) are linear in w0, w1 and w2 for xed v0, v1, v2, M1,0, M2,0,
M2,1. Therefore the worst case will occur at the extremes of the admissible dis-
turbance realizations, i.e., |(1, 1, 1), (1, 1, 1), (1, 1, 1), (1, 1, 1), (1, 1, 1),
(1, 1, 1), (1, 1, 1), (1, 1, 1). Therefore the constraints (16.78) can
be rewritten as:
x0 +v0 +v1 +v2 + (1 +M1,0 +M2,0) + (1 +M2,1) + 1 [1, 1]
x0 +v0 +v1 +v2 + (1 +M1,0 +M2,0) + (1 +M2,1) 1 [1, 1]
x0 +v0 +v1 +v2 (1 +M1,0 +M2,0) + (1 +M2,1) + 1 [1, 1]
x0 +v0 +v1 +v2 (1 +M1,0 +M2,0) + (1 +M2,1) 1 [1, 1]
x0 +v0 +v1 +v2 + (1 +M1,0 +M2,0) (1 +M2,1) + 1 [1, 1]
x0 +v0 +v1 +v2 + (1 +M1,0 +M2,0) (1 +M2,1) 1 [1, 1]
x0 +v0 +v1 +v2 (1 +M1,0 +M2,0) (1 +M2,1) + 1 [1, 1]
x0 +v0 +v1 +v2 (1 +M1,0 +M2,0) (1 +M2,1) 1 [1, 1]
u0 = v0 [1, 1]
u1 = v1 +M1,0 [1, 1]
u1 = v1 M1,0 [1, 1]
u2 = v2 +M2,0 +M2,1 [1, 1]
u2 = v2 +M2,0 M2,1 [1, 1]
u2 = v2 M2,0 +M2,1 [1, 1]
u2 = v2 M2,0 M2,1 [1, 1]
(16.79)
A feasible solution to (16.79) can be obtained by solving a linear optimization
problem for a xed initial condition x0. If we set x0 = 0.8, a feasible solution
to (16.79) is
v0 = 0.8, v1 = 0, v2 = 0
M1,0 = 1, M2,0 = 0, M2,1 = 1
(16.80)
which provides the controller
u0 = 0.8
u1 = w0
u2 = w1
(16.81)
which corresponds to
u0 = 0.8
u1 = x1
u2 = x2
(16.82)
Duality method
Duality can be used to avoid vertex enumeration. Consider the rst constraint
in (16.78):
x0 +v0 +v1 +v2 + (1 +M1,0 +M2,0)w0 + (1 +M2,1)w1 +w2 1,
w0 [1, 1], w1 [1, 1], w2 [1, 1].
(16.83)
We want to replace it with the most stringent constraint, i.e.
x0 +v0 +v1 +v2+J
(M1,0, M2,0, M2,1) 1 (16.84)

where
J
(M1,0, M2,0, M2,1)=maxw

0
,w
1
,w
2
(1 +M1,0 +M2,0)w0 + (1 +M2,1)w1 +w2
subj. to w0 [1, 1], w1 [1, 1], w2 [1, 1]
(16.85)
For xed M1,0, M2,0, M2,1 the optimization problem in (16.85) is a linear pro-
gram and can be replaced by its dual
x0 +v0 +v1 +v2 +d
(M1,0, M2,0, M2,1) 1 (16.86)

where
d
(M1,0, M2,0, M2,1) = min
u
i
,
l
i
u
0
+
u
1
+
u
2
+
l
0
+
l
1
+
l
2
subject to 1 +M1,0 +M2,0 +
l
0
u
0
= 0
1 +M2,1 +
l
1
u
1
= 0
1 +
l
2
u
2
= 0
u
i
0,
l
i
0, i = 0, 1, 2
(16.87)
where
u
i
and
l
i
are the dual variables corresponding to the the upper- and
lower-bounds of wi, respectively. From strong duality we know that for a given
v0, v1, v2, M1,0, M2,0, M2,1 problem (16.84)-(16.85) is feasible if and only if
there exists a set of
u
i
and
l
i
which together with v0, v1, v2, M1,0, M2,0, M2,1
solves the system of equalities and inequalities
x0 +v0 +v1 +v2+
u
0
+
u
1
+
u
2
+
l
0
+
l
1
+
l
2
1
1 +M1,0 +M2,0 +
l
0
u
0
= 0
1 +M2,1 +
l
1
u
1
= 0
1 +
l
2
u
2
= 0
u
i
0,
l
i
0, i = 0, 1, 2
(16.88)
We have transformed an innite dimensional set of constraints in (16.83) into
a nite dimensional set of linear constraints (16.88). By repeating the same
procedure for every constraint in (16.78) we obtain a set of linear constraints.
In particular,
x0 +v0 +v1 +v2 + (1 +M1,0 +M2,0)w0 + (1 +M2,1)w1 +w2 [1, 1]
w0 [1, 1], w1 [1, 1], w2 [1, 1]
is transformed into
x0 +v0 +v1 +v2 +
u
0
+
u
1
+
u
2
+
l
0
+
l
1
+
l
2
1
x0 v0 v1 v2 +
u
0
+
u
1
+
u
2
+
l
0
+
l
1
+
l
2
1
1 +M1,0 +M2,0 +
l
0
u
0
= 0
1 +M2,1 +
l
1
u
1
= 0
1 +
l
2
u
2
= 0
1 M1,0 M2,0 +
l
0
u
0
= 0
1 M2,1 +
l
1
u
1
= 0
1
l
2
u
2
= 0
u
i
,
l
i
,
u
i
,
l
i
0, i = 0, 1, 2,
(16.89)
and
u1 = v1 +M1,0w0 [1, 1]
w0 [1, 1], w1 [1, 1], w2 [1, 1]
is transformed into
v1 +
u
0
+
l
0
1
v1 +
u
0
+
l
0
1
M1,0 +
l
1
u
1
= 0
M1,0 +
l
1
u
1
= 0
u
0
,
l
0
,
u
0
,
l
0
0,
(16.90)
and
u2 = v2 +M2,0w0 +M2,1w1 [1, 1]
w0 [1, 1], w1 [1, 1], w2 [1, 1]
is transformed into
v2 +
u
0
+
u
1
+
l
0
+
l
1
1
v2 +
u
0
+
u
1
+
l
0
+
l
1
1
M2,0 +
l
0
u
0
= 0
M2,1 +
l
1
u
1
= 0
M2,0 +
l
0
u
0
= 0
M2,1 +
l
1
u
1
= 0
u
i
,
l
i
,
u
i
,
l
i
0, i = 0, 1
(16.91)
and the constraint
u0 = v0 [1, 1] (16.92)
remains unchanged. The solution to the set of linear of equalities and inequali-
ties (16.89)-(16.92) for a given x0 provides the solution to our problem.
sysStruct.A= [1 0.1; -0.2 1];
16.6 Example
Example 16.4 Consider the problem of robustly regulating to the system
x(t + 1) =
_
1 0.1
0.2 1
_
x(t) +
_
0.5
1
_
u(t) +w
a
(t)
| = |u R : 3 u 1
. = |x R
2
: 10 x 10, k = 0, . . . , 3.
The two-dimensional disturbance w
a
is restricted to the set V
a
= |v : |w
a
| 2.
We use the cost function
|PxN|
+
N1
k=0
(|Qx
k
|
+[Ru
k
[)
with N = 3, P = Q = [
1 1
0 1
], R = 1.8 and we set .
f
= ..
We compute three robust controllers CROC-OL, CROC-CL and CROC with distur-
bance feedback parametrization.
CROC-OL. The min-max state feedback control law u
(k) = f
OL
k
(x(0)), k = 1, . . . , N
1 is obtained by solving the CROC-OL (16.8)(16.11). The polyhedral partition cor-
responding to u
(0) consists of 64 regions and it is depicted in Figure 16.2(a).

CROC-CL. The min-max state feedback control law u
(k) = f
CL
k
(x(k)), k = 1, . . . , N
1 obtained by solving problem (16.14)(16.20) using the approach of Theorem 16.5.
The resulting polyhedral partition for k = 0 consists of 14 regions and is depicted in
Figure 16.2(b).
16.7 Robust Receding Horizon Control 351
CROC-Parameterized. The min-max state feedback control law u
(k) = f
par
k
(x(0)), k =
1, . . . , N1 obtained by solving problem (16.71)-(16.73) using the approach described
in Example 16.3. The resulting polyhedral partition for k = 0 consists of 136 re-
gions and is depicted in Figure 16.3(a) Figure 16.3(a) shows the set of feasible initial
states .
OL
0
, .0 and .
Mv
0
corresponding to the controllers CROC-OL, CROC-CL and
CROC-Parameterized, respectively.
2
x
1
x
2
(a) Solution to the CROC-OL
problem. Polyhedral partition of
the state-space. Download code.
x
1
x
2
(b) Solution to the CROC-CL
problem. Polyhedral partition of
the state-space.
Figure 16.2 Example (16.4). Polyhedral partition of the state-space corre-
sponding to the explicit solution of CROC-OL and CROC-CL at time t = 0.
16.7 Robust Receding Horizon Control
A robust receding horizon controller for system (16.1)-(16.4) which enforces the
constraints (16.2) at each time t in spite of additive and parametric uncertainties
can be obtained by setting
u(t) = f
0
(x(t)), (16.93)
where f
0
(x(0)) : R
n
R
m
is the solution to the CROC-OL or CROC-CL problems
discussed in the previous sections.
If f
0
is computed by solving CROC-CL (16.14)(16.16) (CROC-OL (16.8)
(16.11)), then the RHC law (16.93) is called a robust receding horizon controller
with closed-loop (open-loop) predictions. The closed-loop system obtained by con-
trolling (16.1)-(16.4) with the RHC (16.93) is
x(k +1) = A(w
p
)x(k) +B(w
p
)f
0
(x(k)) +Ew
a
= f
cl
(x(k), w
p
, w
a
), k 0 (16.94)
If the CROC falls in one of the classes presented in Section 16.1.1 then from the
theorems presented in this chapter we can immediately conclude that the robust
2
x
1
x
2
(a) Solution to the CROC-
Parametrized problem. Poly-
hedral partition of the state-
space. Download code.
x
1
x
2
A
0
= A
Mv
0
A
OL
0
(b) Sets .
OL
0
(dotted line), .0
(continuous line) and .
Mv
0
(con-
tinuous line). Download code.
Figure 16.3 Example (16.4). (Left) Polyhedral partition of the state-space
corresponding to the explicit solution of CROC-Parametrized at time t = 0.
(Right) Sets .
OL
0
, .0 and .
Mv
0
corresponding to the controllers CROC-OL,
CROC-CL and CROC-Parameterized, respectively.
RHC law (16.93) is piecewise ane and thus its on-line computation consists of a
function evaluation.
Convergence and persistent feasibility of the robust receding horizon controller
are not guaranteed as discussed in Chapter 13 for nominal receding horizon con-
trollers. In the robust RHC case it is desirable to obtain robust convergence to a
set O A
f
(rather than convergence to an equilibrium point) for all A
0
. In other
words, the goal is to design a RHC control law which drives any feasible state in
A
0
into the set O for all admissible disturbances and keeps the states inside the set
O for all future time and for all admissible disturbances. Clearly this is possible
only if O is a robust control invariant set for system (16.1)-(16.4).
We dene a distance of a point x R
n
from a nonempty set } R
n
as:
d(x, }) = inf
yY
d(x, y) (16.95)
The following theorem presents sucient conditions for convergence and per-
sistent feasibility in the robust case. It can be proven by using the arguments
of Theorem 13.2. Slightly modied versions can be found with proof and further
discussion in [245][p. 213-217, p. 618] and in [23][Section 7.6].
Theorem 16.9 Consider system (16.1)-(16.4), the closed-loop RHC law (16.93)
where f
0
(x) is obtained by solving the CROC-CL (16.14)(16.16) with cost (16.20), (16.21)
for x(0) = x and the closed-loop system (16.94). Assume that
(A0) There exist constants c
1
, c
2
, c
3
, c
4
> 0 such that
c
1
d(x, O) p(x) c
2
d(x, O) x A
0
(16.96)
16.7 Robust Receding Horizon Control 353
c
3
d(x, O) q(x, u) c
4
d(x, O) (x, u) A
0
| (16.97)
(A1) The sets A, A
f
, |, J
a
, J
p
are compact.
(A2) A
f
and O are robust control invariants, O A
f
A.
(A3) J
p
(x) 0 x A
f
where
J
p
(x) = min
uU
max
w
a
,w
p p(x
+
) p(x) +q(x, u)
subj. to
_
w
a
J
a
, w
p
J
p
x
+
= A(w
p
)x +B(w
p
)u +Ew
a
(16.98)
Then, for all x A
0
, lim
k
d(x(k), O) = 0.
Compare Theorem 16.9 with the nominal MPC stability results in Theorem 13.2.
Assumption (A0) characterizes stage cost q(x, u) and terminal cost p(x) in
both the nominal and the robust case. In the robust case q and p need to be
zero in O, positive and nitely determined outside O and continuous on the
border of O. Assumption (A0) of Theorem 16.9 guarantees these properties.
Note that this cannot be obtained, for instance, with the cost (16.22) in (16.8).
Assumption (A1) characterizes the initial sets in both the nominal and the
robust case. Notice that the compactness of J
a
and J
p
is required to have
a nite cost in the inner maximization problem. In unconstrained min-max
problems often the stage cost is augmented with the term
2
|w| to have a
well dened maximization problem [245][Sec. 3.3.3].
Assumption (A2) characterizes the terminal set in both the nominal and the
robust case. Recall that, as discussed earlier, the set O takes the role of the
origin for the nominal case. The set O must also be a robust invariant.
Assumption (A3) characterizes the terminal cost in both the nominal and the
robust case. The minimization problem in the nominal cases is replaced by
the min-max problem in the robust case. Robust control invariance of the
set A
f
in Assumption (A2) of Theorem 16.9 implies that problem (16.98) in
Assumption (A3) is always feasible.
Remark 16.10 Robust control invariance of the set O in Assumption (A2) of Theo-
rem 16.9 is also implicitly guaranteed by Assumption (A3) as shown next.
Because of Assumption (A0), Assumption (A3) in O becomes
min
u
max
w
a
,w
p
(p(A(w
p
)x +B(w
p
)u +Ew
a
) 0 x O.
From Assumption (A4), this can be veried only if it is equal to zero, i.e. if there
exists a u | such that A(w
p
)x + B(w
p
)u + Ew
a
O for all w
a
V
a
w
p
V
p
(since O is the only place where p(x) can be zero). This implies the robust control
invariance of O.
An extensive treatment of robust invariant sets can be found in [56, 57, 51, 58].
The proof to Theorem 11.2 can be fond in [170, 97] For the derivation of the algo-
rithms 11.4, 11.5 for computing robust invariant sets (and their nite termination)
see [11, 49, 170, 121, 164].
Min-max robust constrained optimal control was originally proposed by Witsen-
hausen [287]. In the context of robust MPC, the problem was tackled by Campo
and Morari [77], and further developed in [6] for SISO FIR plants. Kothare et
al. [177] optimize robust performance for polytopic/multi-model and linear frac-
tional uncertainty, Scokaert and Mayne [257] for additive disturbances, and Lee and
Yu [186] for linear time-varying and time-invariant state-space models depending
on a vector of parameters , where is either an ellipsoid or a polyhe-
dron. Other suboptimal CROC-Cl strategies have been proposed in [177, 28, 178].
For stability and feasibility of the robust RHC (16.1), (16.93) we refer the reader
to [40, 194, 202, 23].
The idea of the parametrization (16.69) appears in the work of Gartska & Wets
in 1974 in the context of stochastic optimization [115]. Recently, it re-appeared in
robust optimization work by Guslitzer and Ben-Tal [135, 45], and in the context
of robust MPC in the work of van Hessem & Bosgra, L ofberg and Goulart &
Kerrigan [279, 191, 129].
Part V
Constrained Optimal Control of
Hybrid Systems
Chapter 17
Models of Hybrid Systems
Hybrid systems describe the dynamical interaction between continuous and discrete
signals in one common framework (see Figure 17.1). In this chapter we focus our
attention on mathematical models of hybrid systems that are particularly suitable
for solving nite time constrained optimal control problems.
17.1 Models of Hybrid Systems
The mathematical model of a dynamical system is traditionally expressed through
dierential or dierence equations, typically derived from physical laws govern-
ing the dynamics of the system under consideration. Consequently, most of the
control theory and tools address models describing the evolution of real-valued sig-
nals according to smooth linear or nonlinear state transition functions, typically
dierential or dierence equations. In many applications, however, the system
to be controlled also contains discrete-valued signals satisfying Boolean relations,
if-then-else conditions, on/o conditions, etc., that also involve the real-valued sig-
nals. An example would be an on/o alarm signal triggered by an analog variable
passing over a given threshold. Hybrid systems describe in a common framework
the dynamics of real-valued variables, the dynamics of discrete variables, and their
interaction.
In this chapter we will focus on discrete-time hybrid systems, that we will
call discrete hybrid automata (DHA), whose continuous dynamics is described by
linear dierence equations and whose discrete dynamics is described by nite state
machines, both synchronized by the same clock [275]. A particular case of DHA
is the popular class of piecewise ane (PWA) systems [265]. Essentially, PWA
systems are switched ane systems whose mode depends on the current location
of the state vector, as depicted in Figure 17.2. PWA and DHA systems can be
translated into a form, denoted as mixed logical dynamical (MLD) form, that is
more suitable for solving optimization problems. In particular, complex nite time
hybrid dynamical optimization problems can be recast into mixed-integer linear or
quadratic programs as will be shown in Chapter 18.
358 17 Models of Hybrid Systems
discrete
dynamics
and logic
binary
outputs
events
mode
switches
binary
inputs
real-valued
outputs
real-valued
inputs
continuous
dynamics
Figure 17.1 Hybrid systems. Logic-based discrete dynamics and continuous
dynamics interact through events and mode switches
In Section 17.7 we will introduce the tool HYSDEL (HYbrid Systems DEscrip-
tion Language), a high level language for modeling and simulating DHA. Therefore,
DHA will represent for us the starting point for modeling hybrid systems. We will
show that DHA, PWA, and MLD systems are equivalent model classes, and in par-
ticular that DHA systems can be converted to an equivalent PWA or MLD form
for solving optimal control problems.
After introducing PWA systems, we will go through the steps needed for model-
ing a system as a DHA. We will rst detail the process of translating propositional
logic involving Boolean variables and linear threshold events over continuous vari-
ables into mixed-integer linear inequalities, generalizing several results available in
the literature, in order to get an equivalent MLD form of a DHA system. Finally, we
will briey present the tool HYSDEL, that allows describing the DHA in a textual
form and obtain equivalent MLD and PWA representations in MATLAB
TM
.
17.2 Piecewise Ane Systems
Piecewise Ane (PWA) systems [265, 144] are dened by partitioning the space
of states and inputs into polyhedral regions (cf. Figure 17.2) and associating with
each region dierent ane state-update and output equations:
x(t + 1) = A
i(t)
x(t) +B
i(t)
u(t) +f
i(t)
(17.1a)
y(t) = C
i(t)
x(t) +D
i(t)
u(t) +g
i(t)
(17.1b)
H
i(t)
x(t) +J
i(t)
u(t) K
i(t)
(17.1c)
17.2 Piecewise Ane Systems 359
C
7
C
4
C
2
C
1
C
3
C
6
C
5
x-space
Figure 17.2 Piecewise ane (PWA) systems. Mode switches are triggered
by linear threshold events.
where x(t) R
n
is the state vector at time t T and T = 0, 1, . . . is the set of
nonnegative integers, u(t) R
m
is the input vector, y(t) R
p
is the output vector,
i(t) 1 = 1, . . . , s is the current mode of the system, the matrices A
i(t)
, B
i(t)
,
f
i(t)
, C
i(t)
, D
i(t)
, g
i(t)
, H
i(t)
, J
i(t)
, K
i(t)
are constant and have suitable dimensions,
and the inequalities in (17.1c) should be interpreted component-wise. Each linear
inequality in (17.1c) denes a half-space in R
n
and a corresponding hyperplane,
that will be also referred to as guardline. Each vector inequality (17.1c) denes a
polyhedron c
i
= [
x
u
] R
n+m
: H
i
x +J
i
u K
i
in the state+input space R
n+m
that will be referred to as cell, and the union of such polyhedral cells as partition.
We assume that c
i
are full-dimensional sets of R
n+m
, for all i = 1, . . . , s.
A PWA system is called well-posed if it satises the following property [37]:
Denition 17.1 Let P be a PWA system of the form (17.1) and let c =
s
i=1
c
i
R
n+m
be the polyhedral partition associated with it. System P is called well-posed
if for all pairs (x(t), u(t)) c there exists only one index i(t) satisfying (17.1).
Denition 17.1 implies that x(t+1), y(t) are single-valued functions of x(t) and u(t),
and therefore that state and output trajectories are uniquely determined by the
initial state and input trajectory. A relaxation of denition 17.1 is to let polyhedral
cells c
i
share one or more hyperplanes. In this case the index i(t) is not uniquely
dened, and therefore the PWA system is not well-posed. However, if the mappings
(x(t), u(t)) x(t +1) and (x(t), u(t)) y(t) are continuous across the guardlines
that are facets of two or more cells (and, therefore, they are continuous on their
domain of denition), such mappings are still single valued.
17.2.1 Modeling Discontinuities
Discontinuous dynamical behaviors can be modeled by disconnecting the domain.
For instance, the state-update equation
x(t + 1) =
_
1
2
x(t) + 1 if x(t) 0
0 if x(t) > 0
(17.2a)
is discontinuous across x = 0. It can be modeled as
x(t + 1) =
_
1
2
x(t) + 1 if x(t) 0
0 if x(t)
(17.2b)
where > 0 is an arbitrarily small number, for instance the machine precision.
Clearly, system (17.2) is not dened for 0 < x(t) < , i.e., for the values of the state
that cannot be represented in the machine. However, the trajectories produced
by (17.2a) and (17.2b) are identical as long as x(t) > or x(t) 0, t N.
As remarked above, multiple denitions of the state-update and output func-
tions over common boundaries of sets c
i
is a technical issue that arises only when
the PWA mapping is discontinuous. Rather than disconnecting the domain, an-
other way of dealing with discontinuous PWA mappings is to allow strict inequal-
ities in the denition of the polyhedral cells in (17.1), or by dealing with open
polyhedra and boundaries separately as in [265]. We prefer to assume that in the
denition of the PWA dynamics (17.1) the polyhedral cells c
i(t)
are closed sets:
As will be clear in the next chapter, the closed-polyhedra description is mainly
motivated by the fact that numerical solvers cannot handle open sets.
Example 17.1 The following PWA system
1
_
_
x(t + 1) = 0.8
_
cos (t) sin (t)
sin (t) cos (t)
_
x(t) +
_
0
1
_
u(t)
y(t) =
_
0 1
_
x(t)
(t) =
_

3
if [ 1 0 ] x(t) 0
3
if [ 1 0 ] x(t) < 0
(17.3)
is discontinuous at x =
_
0
x
2
_
, x2 ,= 0. It can be described in form (17.1) as
_
_
x(t + 1) =
_
_
0.4
_
1
3 1
_
x(t) +
_
0
1
_
u(t) if
_
1 0
_
x(t) 0
0.4
_
1
3 1
_
x(t) +
_
0
1
_
u(t) if
_
1 0
_
x(t)
y(t) =
_
0 1
_
x(t)
(17.4)
for all x1 (, ] [0, +), x2 R, u R, and > 0.
Figure 17.3 shows the free response of the systems (open-loop simulation of the system
for a constant input u = 0) starting from the initial condition x(0) = [1 0] with
sampling time equal to 0.5s and = 10
6
.
1
time
x
1
time
m
o
d
e
Figure 17.3 Example 17.1: Free response of state x1 for x(0) = [1 0]. Down-
load code.
In case the partition c does not cover the whole space R
n+m
, well-posedness
does not imply that trajectories are persistent, i.e., that for all t N a successor
state x(t + 1) and an output y(t) are dened. A typical case of c ,= R
n+m
is
when we are dealing with bounded inputs and bounded states u
min
u(t) u
max
,
x
min
x(t) x
max
. By embedding such ranges in the inequalities (17.1c), the
system becomes undened outside the bounds, as no index i exists that satises
any of the set of inequalities (17.1c).
As will be clearer in the next chapter, when model (17.1) is used in an opti-
mal control formulation, any input sequence and initial state that are feasible for
the related optimization problem automatically dene unique trajectories over the
whole optimal control horizon.
PWA systems can model a large number of physical processes, as they can
model static nonlinearities through a piecewise ane approximation, or approxi-
mate nonlinear dynamics via multiple linearizations at dierent operating points.
Moreover, tools exist for obtaining piecewise ane approximations automatically
(see Section 17.8).
When the mode i(t) is an exogenous variable, condition (17.1c) disappears and
we refer to (17.1) as a switched ane system (SAS), see Section 17.3.1.
17.2.2 Binary States, Inputs, and Outputs
When dealing with hybrid systems, quite often one encounters some signals that
can only assume a binary value, namely either 0 or 1. In the most general form, let
us assume that the state vector x = [
xc
x
] where x
c
R
nc
are the continuous states,
x
R
n
are the binary states, and set n = n

c
+n
. Similarly, let y R
pc
0, 1
p
,
p = p
c
+p
, u R
mc
0, 1
m
, m = m
c
+m
. By dening a polyhedral partition

c
i
s1
i=0
of the sets of state and input space R
n+m
, for any x
0, 1 and u

0, 1 a sucient condition for the PWA system to be well posed is that the rows
and columns of matrices A
i
, B
i
, C
i
, D
i
corresponding to binary states and binary
outputs are zero and that the corresponding rows of matrices f
i
, g
i
are either 0
or 1, i.e., that the binary state update and output equations are binary piecewise
constant functions.
In the following sections we will treat 0-1 binary variables both as numbers
(over which arithmetic operations are dened) and as Boolean variables (over which
Boolean functions are dened, see Section 17.3.3). The variable type will be clear
from the context.
As an example, it is easy to verify that the hybrid dynamical system
x
c
(t + 1) = 2x
c
(t) +u
c
(t) 3u
(t) (17.5a)
x
(t + 1) = x
(t) u
(t) (17.5b)
where represents the logic operator and, can be represented in the PWA
form
_
x
c
x
_
(t + 1) =
_
_
_
2x
c
(t) +u
c
(t)
0
_
if x

1
2
, u

1
2
_
2x
c
(t) +u
c
(t) 3
0
_
if x

1
2
, u

1
2
+
_
2x
c
(t) +u
c
(t)
0
_
if x

1
2
+, u

1
2
_
2x
c
(t) +u
c
(t) 3
1
_
if x

1
2
+, u

1
2
+.
(17.5c)
by associating x
= 0 with x

1
2
and x
= 1 with x

1
2
+ for any 0 <
1
2
.
Note that, by assuming x
(0) 0, 1 and u
(t) 0, 1 for all t T, x
(t) will be
in 0, 1 for all t T.
Example 17.2 Consider the spring-mass-damper system
2
M x2 = u1 k(x1) b(u2)x2
where x1 and x2 = x1 denote the position and the velocity of the mass, respectively,
and u1 a continuous force input. The binary input u2 switches the friction coecient
b(u2) =
_
b1 if u2 = 1
b2 if u2 = 0.
The spring coecient switches to a dierent value at position xm
k(x1) =
_
k1x1 +d1 if x1 xm
k2x1 +d2 if x1 > xm.
Assume the system description is valid for 5 x1, x2 5, and 10 u1 10.
The system has four modes, depending on the binary input u2 and the position x1.
Assuming that the system parameters are M = 1, b1 = 1, b2 = 50, k1 = 1, k2 = 3,
d1 = 1, d2 = 7.5, xm = 1, after discretizing the dynamics in each mode with a
sampling time of 0.5 time units we obtain the following discrete-time PWA system
2
time
p
o
s
i
t
i
o
n
x
1
time
m
o
d
e
(a) Large damping (u2 = 0).
p
o
s
i
t
i
o
n
x
1
time
m
o
d
e
(b) Small damping (u2 = 1).
Figure 17.4 Example 17.2: Open-loop simulation of system (17.6) for u1 = 3
and zero initial conditions. Download code.
x(t+1) =
_
_
Mode 1
_
0.8956 0.0198
0.0198 0.0004
_
x(t) + [
0.1044
0.0198
] u1(t) +
_
0.0096
0.0198
_
if x1(t) 1, u2(t) 0.5
Mode 2
_
0.8956 0.0195
0.0584 0.0012
_
x(t) + [
0.1044
0.0195
] u1(t) +
_
0.0711
0.1459
_
if x1(t) 1 +, u2(t) 0.5
Mode 3
_
0.8956 0.3773
0.3773 0.5182
_
x(t) + [
0.1044
0.3773
] u1(t) +
_
0.1044
0.3773
_
if x1(t) 1, u2(t) 0.5
Mode 4
_
0.8956 0.3463
1.0389 0.3529
_
x(t) + [
0.1044
0.3463
] u1(t) +
_
0.7519
2.5972
_
if x(t) 1 +, u2(t) 0.5
(17.6)
for x1(t) [5, 1] [1 +, 5], x2(t) [5, 5], u1(t) [10, 10], and for any arbitrary
small > 0.
Figure 17.4 shows the open-loop simulation of the system for a constant continuous
input u1 = 3, starting from zero initial conditions and for = 10
6
.
Example 17.3 Consider the following SISO system
3
:
x1(t + 1) = ax1(t) +bu(t). (17.7)
A logic state x2 |0, 1 stores the information whether the state of system (17.7)
has ever gone below a certain lower bound x
lb
or not:
x2(t + 1) = x2(t)
[x1(t) x
lb
], (17.8)
3
x
1
x
2
time steps
u
-10
10
0
1
-10
10
(a) Scenario 1.
x
1
x
2
time steps
u
-10
10
0
1
-10
10
(b) Scenario 2.
Figure 17.5 Example 17.3: Open-loop simulation of system (17.10) for dif-
ferent sequences of the input u. Download code.
Assume that the input coecient is a function of the logic state:
b =
_
b1 if x2 = 0
b2 if x2 = 1.
(17.9)
The system can be described by the PWA model:
x(t+1) =
_
_
_
a 0
0 0
_
x(t) +
_
b2
0
_
u(t) +
_
0
1
_
if
_
1 0
_
x(t) x
lb
_
a 0
0 1
_
x(t) +
_
b1
0
_
u(t) if
_
1 0
0 1
_
x(t)
_
x
lb
+
0.5
_
_
a 0
0 1
_
x(t) +
_
b2
0
_
u(t) if x(t)
_
x
lb
+
0.5
_
(17.10)
for u(t) R, x1(t) (, x
lb
] [x
lb
+, +), x2 |0, 1, and for any > 0.
Figure 17.5 shows two open-loop simulations of the system, for a = 0.5, b1 = 0.1,
b2 = 0.3, x
lb
= 1, = 10
6
. The initial conditions is x(0) = [1, 0]. Note that when
the continuous state x1(t) goes below x
lb
= 1 at time t, then x
(t + 1) switches to
1 and the input has a stronger eect on the states from time t + 2 on. Indeed, the
steady state of x1 is a function of the logic state x2.
17.3 Discrete Hybrid Automata
As shown in Fig. 17.6, a discrete hybrid automaton (DHA) is formed by generating
the mode i(t) of a switched ane system through a mode selector function that
depends on (i) the discrete state of a nite state machine, (ii) discrete events
generated by the continuous variables of the switched ane system exceeding given
17.3 Discrete Hybrid Automata 365
E
1
s
y
v
d
F
d
u
v
S
M
x
u
d
i
.
.
.
!e(k)
!e(k)
!e(k)
u!(k)
u!(k)
x!(k)
i(k)
xc(k)
uc(k)
uc(k)
Mode
Selector
(MS)
Finite State
Machine
(FSM)
Event
Generator
(EG)
(SAS)
Switched
Affine
System
yc(k)
y!(k)
Figure 17.6 A discrete hybrid automaton (DHA) is the connection of a nite
state machine (FSM) and a switched ane system (SAS), through a mode
selector (MS) and an event generator (EG). The output signals are omitted
for clarity
linear thresholds (the guardlines), (iii) exogenous discrete inputs [275]. We will
detail each of the four blocks in the next sections.
17.3.1 Switched Ane System (SAS)
A switched ane system is a collection of ane systems:
x
c
(t + 1) = A
i(t)
x
c
(t) +B
i(t)
u
c
(t) +f
i(t)
, (17.11a)
y
c
(t) = C
i(t)
x
c
(t) +D
i(t)
u
c
(t) +g
i(t)
, (17.11b)
where t T is the time indicator, x
c
R
nc
is the continuous state vector, u
c
R
mc
is the exogenous continuous input vector, y
c
R
pc
is the continuous output vector,
A
i
, B
i
, f
i
, C
i
, D
i
, g
i
iI
is a collection of matrices of suitable dimensions, and the
mode i(t) 1 = 1, . . . , s is an input signal that determines the ane state update
dynamics at time t. An SAS of the form (17.11) preserves the value of the state
when a mode switch occurs, but it is possible to implement reset maps on an SAS
as shown in [275].
17.3.2 Event Generator (EG)
An event generator is an object that generates a binary vector
e
(t) 0, 1
ne
of event conditions according to the satisfaction of a linear (or ane) threshold
condition. Let h : R
nc
R
nc
0, 1
ne
be a vector function dened as
h
i
(x
c
, u
c
) =
_
1 if H
i
x
c
+J
i
u
c
+K
i
0
0 if H
i
x
c
+J
i
u
c
+K
i
> 0
where
i
denotes the i-th component of a vector or the i-th row of a matrix, and H,
J, K are constant matrices of suitable dimensions. Then, events are dened as
e
(t) = h(x
c
(t), u
c
(t)) (17.12)
In particular, state events are modeled as [
e
(t) = 1] [a
x
c
(t) b]. Note that
time events can be modeled as in (17.12) by adding the continuous time as an
additional continuous and autonomous state variable, (t + 1) = (t) + T
s
, where
T
s
is the sampling time, and by letting [
e
(t) = 1] [tT
s

0
], where
0
is a given
time. By doing so, the hybrid model can be written as a time-invariant one. Clearly
the same approach can be used for time-varying events
e
(t) = h(x
c
(t), u
c
(t), t), by
using time-varying event conditions h : R
nc
R
nc
T 0, 1
ne
.
17.3.3 Boolean Algebra
Before dealing in detail with the other blocks constituting the DHA and introduce
further notation, we recall here some basic denitions of Boolean algebra. A more
comprehensive treatment of Boolean calculus can be found in digital circuit design
texts, e.g. [85, 141]. For a rigorous exposition see e.g. [206].
A variable is a Boolean variable if 0, 1, where = 0 means something
is false, = 1 that it is true. A Boolean expression is obtained by combining
Boolean variables through the logic operators (not), (or), (and), (implied
by), (implies), and (i). A Boolean function f : 0, 1
n1
0, 1 is used
to dene a Boolean variable
n
as a logic function of other variables
1
, . . . ,
n1
:
n
= f(
1
,
2
, . . . ,
n1
) (17.13)
Given n Boolean variables
1
, . . . ,
n
, a Boolean formula F denes a relation
F(
1
, . . . ,
n
) (17.14)
that must hold true. Every Boolean formula F(
1
,
2
, . . . ,
n
) can be rewritten in
the conjunctive normal form (CNF)
(CNF)
m
j=1
_
_
_
_
iPj
i
_
_
_
_

iNj

i
_
_
_
_
(17.15)
N
j
, P
j
1, . . . , n, j = 1, . . . , m.
As mentioned in Section 17.2.2, often we will use the term binary variable
and Boolean variable without distinction. An improper arithmetic operation over
Boolean variables should be understood as an arithmetic operation over corre-
sponding binary variables and, vice versa, an improper Boolean function of binary
variables should be interpreted as the same function over the corresponding Boolean
variables. Therefore, from now on we will call binary variables both 0-1 variables
and Boolean variables.
17.3 Discrete Hybrid Automata 367

3
1
u
2
1
u
2
3
u
1

1
u
1

3

2
Red
Green Blue
Figure 17.7 Example of nite state machine
17.3.4 Finite State Machine (FSM)
A nite state machine (or automaton) is a discrete dynamic process that evolves
according to a Boolean state update function:
x
(t + 1) = f
(x
(t), u
(t),
e
(t)), (17.16a)
where x
0, 1
n
is the binary state, u
0, 1
m
is the exogenous binary input,
e
(t) is the endogenous binary input coming from the EG, and f
: 0, 1
n
0, 1
m
0, 1
ne
0, 1
n
is a deterministic Boolean function. In this text

we will only refer to synchronous nite state machines, where the transitions may
happen only at sampling times. The adjective synchronous will be omitted for
brevity.
An FSM can be conveniently represented using an oriented graph. An FSM
may also have an associated binary output
y
(t) = g
(x
(t), u
(t),
e
(t)), (17.16b)
where y
0, 1
p
and g
: 0, 1
n
0, 1
m
0, 1
ne
0, 1
p
.
Example 17.4 Figure 17.7 shows a nite state machine where u
= [u
1
u
2
]
is
the input vector, and = [1 . . . 3]
is a vector of signals coming from the event

generator. The Boolean state update function (also called state transition function)
is:
x
(t + 1) =
_
_
Red if ((x
(t) = Green) 3)
((x
(t) = Red) 1),

Green if ((x
(t) = Red) 1 u
2
)
((x
(t) = Blue) 2)
((x
(t) = Green) u
1
3),
Blue if ((x
(t) = Red) 1 u
2
)
((x
(t) = Green) (3 u
1
))
((x
(t) = Blue) 2)).

(17.17)
By associating a binary vector x
= [
x
1
x
2
] to each state (Red = [
0
0
], Green = [
0
1
], and
Blue = [
1
0
]), one can rewrite (17.17) as:
x
1
(t + 1) = ( x
1
x
2
1 u
2
)
(x
1
2) (x
2
3 u
1
),
x
2
(t + 1) = ( x
1
x
2
1 u
2
)
(x
1
2) (x
2
3 u
1
),
where the time index (t) has been omitted for brevity.
Since the Boolean state update function is deterministic, for each state the
conditions associated with all the outgoing arcs are mutually exclusive.
17.3.5 Mode Selector
In a DHA, the dynamic mode i(t) 1 = 1, . . . , s of the SAS is a function of the
binary state x
(t), the binary input u
(t), and the events

e
(t). With a slight abuse
of notation, let us indicate the mode i(t) through its binary encoding, i(t) 0, 1
ns
where n
s
= log
2
s, so that i(t) can be treated as a vector of Boolean variables.
Any discrete variable
1
, . . . ,
j
admits a Boolean encoding a 0, 1
d(j)
,
where d(j) is the number of bits used to represent
1
, . . . ,
j
. For example,
0, 1, 2 may be encoded as a 0, 1
2
by associating [
0
0
] 0, [
0
1
] 1,
[
1
0
] 2.
Then, we dene the mode selector by the Boolean function f
M
: 0, 1
n
0, 1
m
0, 1
ne
0, 1
ns
. The output of this function
i(t) = (x
(t), u
(t),
e
(t)) (17.18)
is called the active mode of the DHA at time t. We say that a mode switch occurs
at step t if i(t) ,= i(t 1). Note that, in contrast to continuous-time hybrid models
where switches can occur at any time, in our discrete-time setting, as mentioned
earlier, a mode switch can only occur at sampling instants.
17.3.6 DHA Trajectories
For a given initial condition
_
xc(0)
x
(0)
_
R
nc
0, 1
n
, and inputs
_
uc(t)
u
(t)
_
R
mc
0, 1
m
, t T, the state x(t) of the system is computed for all t T by recursively

iterating the set of equations:
e
(t) = h(x
c
(t), u
c
(t), t) (17.19a)
i(t) = (x
(t), u
(t),
e
(t)) (17.19b)
y
c
(t) = C
i(t)
x
c
(t) +D
i(t)
u
c
(t) +g
i(t)
(17.19c)
y
(t) = g
(x
(t), u
(t),
e
(t)) (17.19d)
x
c
(t + 1) = A
i(t)
x
c
(t) +B
i(t)
u
c
(t) +f
i(t)
(17.19e)
x
(t + 1) = f
(x
(t), u
(t),
e
(t)) (17.19f)
17.4 Logic and Mixed-Integer Inequalities 369
A denition of well-posedness of DHA can be given similarly to Denition 17.1
by requiring that the successor states x
c
(t + 1), x
(t + 1) and the outputs y

c
(t),
y
(t) are uniquely dened functions of x

c
(t), x
(t), u
c
(t), u
(t) dened by the DHA

equations (17.19).
DHA can be considered as a subclass of hybrid automata (HA) [8]. The main
dierence is in the time model: DHA are based on discrete time, HA on continuous
time. Moreover DHA models do not allow instantaneous transitions, and are de-
terministic, contrary to HA where any enabled transition may occur in zero time.
This has two consequences (i) DHA do not admit live-locks (innite switches in
zero time), (ii) DHA do not admit Zeno behaviors (innite switches in nite time).
Finally in DHA models, guards, reset maps and continuous dynamics are limited
to linear (or ane) functions.
17.4 Logic and Mixed-Integer Inequalities
Despite the fact that DHA are rich in their expressiveness and are therefore quite
suitable for modeling and simulating a wide class of hybrid dynamical systems,
they are not directly suitable for solving optimal control problems, because of their
heterogeneous discrete and continuous nature. In this section we want to describe
how DHA can be translated into dierent hybrid models that are more suitable
for optimization. We highlight the main techniques of the translation process, by
generalizing several results that appeared in the literature [123, 242, 285, 211, 82,
39, 153, 80, 276, 284, 217].
17.4.1 Transformation of Boolean Relations
Boolean formulas can be equivalently represented as integer linear inequalities.
For instance,
1

2
= 1 is equivalent to
1
+
2
1 [285]. Some equivalences are
reported in Table 17.1. The results of the table can be generalized as follows.
Lemma 17.1 For every Boolean formula F(
1
,
2
, . . . ,
n
) there exists a polyhedral
set P such that a set of binary values
1
,
2
, . . . ,
n
satises the Boolean formula
F if and only if = [
1

2
. . .
n
]
P.
Proof: Given a formula F, one way of constructing a polyhedron P is to rewrite
F in the conjunctive normal form (17.15), and then simply dene P as
P =
_
_
0, 1
n
:
1
iP1
i
+
iN1
(1
i
)
.
.
.
1
iPm
i
+
iNm
(1
i
)
_
_
(17.20)
The smallest polyhedron P associated with formula F has the following ge-
ometric interpretation: Assume to list all the 0-1 combinations of
i
s satisfying
relation Boolean linear constraints
AND
1

2

1
= 1,
2
= 1 or
1
+
2
2
OR
1
X
2

1
+
2
1
NOT
1

1
= 0
XOR
1
2

1
+
2
= 1
IMPLY
1

2

1
2
0
IFF
1

2

1
2
= 0
ASSIGNMENT
1
+ (1
3
) 1
3
=
1

2

3

1

2

2
+ (1
3
) 1
(1
1
) + (1
2
) +
3
1
Table 17.1 Basic conversion of Boolean relations into mixed-integer inequal-
ities. Relations involving the inverted literals can be obtained by sub-
stituting (1 ) for in the corresponding inequalities. More conversions
are reported in [208], or can be derived by (17.15)(17.20)
F (namely, to generate the truth table of F), and think of each combination as
an n-dimensional binary vector in R
n
, then P is the convex hull of such vec-
tors [82, 152, 209]. For methods to compute convex hulls, we refer the reader
to [107].
17.4.2 Translating DHA Components into Linear Mixed-Integer Relations
Events of the form (17.12) can be expressed equivalently as
h
i
(x
c
(t), u
c
(t), t) M
i
(1
i
e
), (17.21a)
h
i
(x
c
(t), u
c
(t), t) > m
i
i
e
, i = 1, . . . , n
e
, (17.21b)
where M
i
, m
i
are upper and lower bounds, respectively, on h
i
(x
c
(t), u
c
(t), t). As
we have pointed out in Section 17.2.1, from a computational viewpoint it is con-
venient to avoid strict inequalities. As suggested in [285], we modify the strict
inequality (17.21b) into
h
i
(x
c
(t), u
c
(t), t) + (m
i
)
i
e
(17.21c)
where is a small positive scalar (e.g., the machine precision). Clearly, as for the
case of discontinuous PWA discussed in Section 17.2.1, Equations (17.21) or (17.12)
are equivalent to (17.21c) only for h
i
(x
c
(t), u
c
(t), t) 0 and h
i
(x
c
(t), u
c
(t), t) .
Regarding switched ane dynamics, we rst rewrite the state-update equa-
tion (17.11a) as the following combination of ane terms and if-then-else condi-
17.5 Mixed Logical Dynamical Systems 371
tions:
z
1
(t) =
_
A
1
x
c
(t) +B
1
u
c
(t) +f
1
, if (i(t) = 1),
0, otherwise,
(17.22a)
.
.
.
z
s
(t) =
_
A
s
x
c
(t) +B
s
u
c
(t) +f
s
, if (i(t) = s),
0, otherwise,
(17.22b)
x
c
(t + 1) =
s
i=1
z
i
(t), (17.22c)
where z
i
(t) R
nc
, i = 1, . . . , s. The output equation (17.11b) admits a similar
transformation.
A generic if-then-else construct of the form
IF THEN z = a
1
x +b
1
u + f
1
ELSE z = a
2
x +b
2
u +f
2
, (17.23)
where 0, 1, z R, x R
n
, u R
m
, and a
1
, b
1
, f
1
, a
2
, b
2
, f
2
are constants of
suitable dimensions, can be translated into [43]
(m
2
M
1
) +z a
2
x +b
2
u +f
2
, (17.24a)
(m
1
M
2
) z a
2
x b
2
u f
2
, (17.24b)
(m
1
M
2
)(1 ) +z a
1
x +b
1
u +f
1
, (17.24c)
(m
2
M
1
)(1 ) z a
1
x b
1
u f
1
, (17.24d)
where M
i
, m
i
are upper and lower bounds on a
i
x +b
i
u +f
i
, i = 1, 2.
Finally, the mode selector function and binary state-update function of the au-
tomaton are Boolean functions that can be translated into integer linear inequalities
as described in Section 17.4.1. The idea of transforming a well-posed FSM into a
set of Boolean equalities was also presented in [227] where the authors performed
model checking using (mixed) integer optimization on an equivalent set of integer
inequalities.
17.5 Mixed Logical Dynamical Systems
Given a DHA representation of a hybrid process, by following the techniques de-
scribed in the previous section for converting logical relations into inequalities we
obtain an equivalent representation of the DHA as a mixed logical dynamical (MLD)
system [39] described by the following relations:
x(t + 1) = Ax(t) +B
1
u(t) +B
2
(t) +B
3
z(t) +B
5
, (17.25a)
y(t) = Cx(t) +D
1
u(t) +D
2
(t) +D
3
z(t) +D
5
, (17.25b)
E
2
(t) +E
3
z(t) E
1
u(t) +E
4
x(t) +E
5
, (17.25c)
where x R
nc
0, 1
n
is a vector of continuous and binary states, u R

mc
0, 1
m
are the inputs, y R

pc
0, 1
p
the outputs, 0, 1
r
are auxiliary
binary variables, z R
rc
are continuous auxiliary variables which arise in the
transformation (see Example 17.5), and A, B
1
, B
2
, B
3
, C, D
1
, D
2
, D
3
, E
1
,. . . ,E
5
are matrices of suitable dimensions. Given the current state x(t) and input u(t),
the time-evolution of (17.25) is determined by nding a feasible value for (t) and
z(t) satisfying (17.25c), and then by computing x(t + 1) and y(t) from (17.25a)
(17.25b).
As MLD models consist of a collection of linear dierence equations involving
both real and binary variables and a set of linear inequality constraints, they are
model representations of hybrid systems that can be easily used in optimization
algorithms, as will be described in Chapter 18.
A denition of well-posedness of the MLD system (17.25) can be given similarly
to Denition 17.1 by requiring that for all x(t) and u(t) within a given bounded
set the pair of variables (t), z(t) satisfying (17.25c) is unique, so that the succes-
sor state x(t + 1) and output y(t) are also uniquely dened functions of x(t), u(t)
through (17.25a)(17.25b). Such a well-posedness assumption is usually guaranteed
by the procedure described in Section 17.3.3 used to generate the linear inequali-
ties (17.25c). A numerical test for well-posedness is reported in [39, Appendix 1].
For a more general denition of well-posedness of MLD systems see [39].
Note that the constraints (17.25c) allow one to specify additional linear con-
straints on continuous variables (e.g., constraints over physical variables of the
system), and logical constraints over Boolean variables. The ability to include
constraints, constraint prioritization, and heuristics adds to the expressiveness and
generality of the MLD framework. Note also that despite the fact that the de-
scription (17.25) seems to be linear, clearly the nonlinearity is concentrated in the
integrality constraints over binary variables.
Example 17.5 Consider the following simple switched linear system [39]
x(t + 1) =
_
0.8x(t) +u(t) if x(t) 0
0.8x(t) +u(t) if x(t) < 0
(17.26)
where x(t) [10, 10], and u(t) [1, 1]. The condition x(t) 0 can be associated
to an event variable (t) |0, 1 dened as
[(t) = 1] [x(t) 0] (17.27)
By using the transformations (17.21a)(17.21c), equation (17.27) can be expressed
by the inequalities
m(t) x(t) m (17.28a)
(M +)(t) x(t) (17.28b)
where M = m = 10, and is an arbitrarily small positive scalar. Then (17.26) can
be rewritten as
x(t + 1) = 1.6(t)x(t) 0.8x(t) +u(t) (17.29)
17.6 Model Equivalence 373
By dening a new variable z(t) = (t)x(t) which, by (17.23)(17.24) can be expressed
as
z(t) M(t) (17.30a)
z(t) m(t) (17.30b)
z(t) x(t) m(1 (t)) (17.30c)
z(t) x(t) M(1 (t)) (17.30d)
the evolution of system (17.26) is ruled by the linear equation
x(t + 1) = 1.6z(t) 0.8x(t) +u(t) (17.31)
subject to the linear constraints (17.28) and (17.30). Therefore, the MLD equivalent
representation of (17.26) for x [10, ][0, 10] and u [1, 1] is given by collecting
Equations (17.31), (17.28) and (17.30).
17.6 Model Equivalence
In the previous chapters we have presented three dierent classes of discrete-time
hybrid models: PWA systems, DHA, and MLD systems. For what we described
in Section 17.3.3, under the assumption that the set of valid states and inputs
is bounded, DHA systems can always be equivalently described as MLD systems.
Also, a PWA system is a special case of a DHA whose threshold events and mode
selector function are dened by the PWA partition (17.1c). Therefore, a PWA
system with bounded partition c can always be described as an MLD system (an
ecient way of modeling PWA systems in MLD form is reported in [39]). The
converse result, namely that MLD systems (and therefore DHA) can be represented
as PWA systems, is less obvious. For any choice of , model (17.25) represented
an ane system dened over a polyhedral domain. Under the assumption of well
posedness these domains do not overlap. This result was proved formally in [34,
29, 118].
Such equivalence results are of interest because DHA are most suitable in the
modeling phase, but MLD systems are most suitable for solving open-loop nite
time optimal control problems, and PWA systems are most suitable for solving
nite time optimal control problems in state feedback form, as will be described in
Chapter 18.
17.7 The HYSDEL Modeling Language
A modeling language was proposed in [275] to describe DHA models, called HYbrid
System DEscription Language (HYSDEL). The HYSDEL description of a DHA is
an abstract modeling step. The associated HYSDEL compiler then translates the
description into several computational models, in particular into an MLD using
SYSTEM sample {
INTERFACE {
STATE {
REAL xr [-10, 10]; }
INPUT {
REAL ur [-2, 2]; }
}
IMPLEMENTATION {
AUX {
REAL z1, z2, z3;
BOOL de, df, d1, d2, d3; }
AD {
de = xr >= 0;
df = xr + ur - 1 >= 0; }
LOGIC {
d1 = ~de & ~df;
d2 = de;
d3 = ~de & df; }
DA {
z1 = {IF d1 THEN xr + ur - 1 };
z2 = {IF d2 THEN 2 * xr };
z3 = {IF (~de & df) THEN 2 }; }
CONTINUOUS {
xr = z1 + z2 + z3; }
}}
Table 17.2 Sample HYSDEL list of system (17.32)
the technique presented in Section 17.4, and PWA form using either the approach
of [118] or the approach of [29]. HYSDEL can generate also a simulator that runs as
a function in MATLAB
TM
. Both the HYSDEL compiler and the Hybrid Toolbox
can import and convert HYSDEL models.
In this section we illustrate the functionality of HYSDEL through a set of
examples. For more examples and the detailed syntax we refer the interested
reader to [148].
Example 17.6 Consider the DHA system:
SAS: x
c
(t) =
_
_
_
xc(t) +uc(t) 1, if i(t) = 1,
2xc(t), if i(t) = 2,
2, if i(t) = 3,
(17.32a)
EG:
_
e(t) = [xc(t) 0],
f
(t) = [xc(t) +uc(t) 1 0],
(17.32b)
MS: i(t) =
_
_
1, if
_
e(t)
f
(t)
_
= [
0
0
] ,
2, if e(t) = 1,
3, if
_
e(t)
f
(t)
_
= [
0
1
] .
(17.32c)
The corresponding HYSDEL list is reported in Table 17.2.
The HYSDEL list is composed of two parts. The rst one, called INTERFACE,
contains the declaration of all variables and parameters, so that it is possible to
make the proper type checks. The second part, IMPLEMENTATION, is composed
of specialized sections where the relations among the variables are described. These
sections are described next.
17.7 The HYSDEL Modeling Language 375
/* 2x2 PWA system */
SYSTEM pwa {
INTERFACE {
STATE { REAL x1 [-5,5];
REAL x2 [-5,5];
}
INPUT { REAL u [-1,1];
}
OUTPUT{ REAL y;
}
PARAMETER {
REAL alpha = 60*pi/180;
REAL C = cos(alpha);
REAL S = sin(alpha);
REAL MLD_epsilon = 1e-6; }
}
IMPLEMENTATION {
AUX { REAL z1,z2;
BOOL sign; }
AD { sign = x1<=0; }
DA { z1 = {IF sign THEN 0.8*(C*x1+S*x2)
ELSE 0.8*(C*x1-S*x2) };
z2 = {IF sign THEN 0.8*(-S*x1+C*x2)
ELSE 0.8*(S*x1+C*x2) }; }
CONTINUOUS { x1 = z1;
x2 = z2+u; }
OUTPUT { y = x2; }
}
}
Table 17.3 HYSDEL model of the PWA system described in Example 17.1
The HYSDEL section AUX contains the declaration of the auxiliary variables used
in the model. The HYSDEL section AD allows one to dene Boolean variables from
continuous ones, and is based exactly on the semantics of the event generator (EG)
described earlier. The HYSDEL section DA denes continuous variables according
to if-then-else conditions. This section models part of the switched ane system
(SAS), namely the variables zi dened in (17.22a)(17.22b). The CONTINUOUS
section describes the linear dynamics, expressed as dierence equations. This section
models (17.22c). The section LOGIC allows one to specify arbitrary functions of
Boolean variables.
Example 17.7 Consider again the PWA system described in Example 17.1. Assume
that [5, 5] [5, 5] is the set of states x(t) of interest and u(t) [1, 1]. By using
HYSDEL the PWA system (17.3) is described as in Table 17.3 and the equivalent
MLD form is obtained
x(t + 1) = [
1 0
0 1
] z(t) + [
0
1
] u(t)
y(t) = [ 0 1 ] x(t)
_
_
_
_
_
_
_
5
5
c
1
c
1
c
1
c
1
c
1
c
1
c
1
c
1
_
_
(t) +
_
_
_
_
_
_
0 0
0 0
1 0
1 0
1 0
1 0
0 1
0 1
0 1
0 1
_
_
z(t)
_
_
_
_
_
_
_
1 0
1 0
0.4 c
2
0.4 c
2
0.4 c
2
0.4 c
2
c
2
0.4
c
2
0.4
c
2
0.4
c
2
0.4
_
_
x(t) +
_
_
_
_
_
_
5
c
1
c
1
0
0
c
1
c
1
0
0
_
_
.
where c1 = 4(
3 + 1), c2 = 0.4
3, = 10
6
. Note that in Table 17.3 the OUTPUT
section allows to specify a linear map for the output vector y.
Example 17.8 Consider again the hybrid spring-mass-damper system described in
Example 17.2. Assume that [5, 5] [5, 5] is the set of states x and [10, 10] the set
of continuous inputs u1 of interest. By using HYSDEL, system (17.6) is described as
in Table 17.4 and an equivalent MLD model with 2 continuous states, 1 continuous
input, 1 binary input, 9 auxiliary binary variables, 8 continuous variables, and 58
mixed-integer inequalities is obtained.
Example 17.9 Consider again the system with a logic state described in Example 17.3.
The MLD model obtained by compiling the HYSDEL list of Table 17.5 is
x(t + 1) =
_
0 0
0 1
_
(t) +
_
1
0
_
z(t)
_
_
_
_
_
_
_
_
_
_
_
_
_
_
9 0
11 0
0 0
0 0
0 0
0 0
0 1
1 1
1 1
_
_
(t) +
_
_
_
_
_
_
_
_
_
_
_
_
_
_
0
0
1
1
1
1
0
0
0
_
_
z(t)
_
_
_
_
_
_
_
_
_
_
_
_
_
_
0
0
0.3
0.3
0.1
0.1
0
0
0
_
_
u(t) +
_
_
_
_
_
_
_
_
_
_
_
_
_
_
1 0
1 0
0.5 14
0.5 14
0.5 14
0.5 14
0 1
0 0
0 1
_
_
x(t) +
_
_
_
_
_
_
_
_
_
_
_
_
_
_
1
10
14
14
0
0
0
0
0
_
_
where we have assumed that [10, 10] is the set of states x1 and [10, 10] the set of
continuous inputs u of interest. In Table 17.5 the AUTOMATA section species the
state transition equations of the nite state machine (FSM) as a collection of Boolean
functions.
The lack of a general theory and of systematic design tools for systems having
such a heterogeneous dynamical discrete and continuous nature led to a consid-
erable interest in the study of hybrid systems. After the seminal work published
/* Spring-Mass-Damper System
*/
SYSTEM springmass {
INTERFACE { /* Description of variables and constants */
STATE {
REAL x1 [-5,5];
REAL x2 [-5,5];
}
INPUT { REAL u1 [-10,10];
BOOL u2;
}
PARAMETER {
/* Spring breakpoint */
REAL xm;
/* Dynamic coefficients */
REAL A111,A112,A121,A122,A211,A212,A221,A222;
REAL A311,A312,A321,A322,A411,A412,A421,A422;
REAL B111,B112,B121,B122,B211,B212,B221,B222;
REAL B311,B312,B321,B322,B411,B412,B421,B422;
}
}
IMPLEMENTATION {
AUX {
REAL zx11,zx12,zx21,zx22,zx31,zx32,zx41,zx42;
BOOL region;
}
AD { /* spring region */
region = x1-xm <= 0;
}
DA {
zx11 = { IF region & u2 THEN A111*x1+A112*x2+B111*u1+B112};
zx12 = { IF region & u2 THEN A121*x1+A122*x2+B121*u1+B122};
zx21 = { IF region & ~u2 THEN A211*x1+A212*x2+B211*u1+B212};
zx22 = { IF region & ~u2 THEN A221*x1+A222*x2+B221*u1+B222};
zx31 = { IF ~region & u2 THEN A311*x1+A312*x2+B311*u1+B312};
zx32 = { IF ~region & u2 THEN A321*x1+A322*x2+B321*u1+B322};
zx41 = { IF ~region & ~u2 THEN A411*x1+A412*x2+B411*u1+B412};
zx42 = { IF ~region & ~u2 THEN A421*x1+A422*x2+B421*u1+B422};
}
CONTINUOUS { x1=zx11+zx21+zx31+zx41;
x2=zx12+zx22+zx32+zx42;
}
}
}
Table 17.4 HYSDEL model of the spring-mass-damper system described in
Example 17.2. The A and B values are set according to equation (17.6).
/* System with logic state */
SYSTEM SLS {
INTERFACE { /* Description of variables and constants */
STATE {
REAL x1 [-10,10];
BOOL x2;
}
INPUT { REAL u [-10,10];
}
PARAMETER {
/* Lower Bound Point */
REAL xlb = -1;
/* Dynamic coefficients */
REAL a = .5;
REAL b1 =.1;
REAL b2 =.3;
}
}
IMPLEMENTATION {
AUX {BOOL region;
REAL zx1;
}
AD { /* PWA Region */
region = x1-xlb <= 0;
}
DA { zx1={IF x2 THEN a*x1+b2*u ELSE a*x1+b1*u};
}
CONTINUOUS { x1=zx1;
}
AUTOMATA { x2= x2 | region;
}
}
}
Table 17.5 HYSDEL model of the system with logic state described in Ex-
ample 17.3
in 1966 by Witsenhausen [286], who examined an optimal control problem for a
class of hybrid-state continuous-time dynamical systems, there has been a renewed
interest in the study of hybrid systems. The main reason for such an interest is
probably the recent advent of technological innovations, in particular in the domain
of embedded systems, where a logical/discrete decision device is embedded in a
physical dynamical environment to change the behavior of the environment itself.
Another reason is the availability of several software packages for simulation and
numerical/symbolic computation that support the theoretical developments.
Several modelling frameworks for hybrid systems have appeared in the litera-
ture. We refer the interested reader to [9, 124] and the references therein. Each
class is usually tailored to solve a particular problem, and many of them look largely
dissimilar, at least at rst sight. Two main categories of hybrid systems were suc-
cessfully adopted for analysis and synthesis [65]: hybrid control systems [192, 193],
where continuous dynamical systems and discrete/logic automata interact (see
Fig. 17.1), and switched systems [265, 66, 155, 290, 259], where the state-space
is partitioned into regions, each one being associated with a dierent continuous
dynamics (see Fig. 17.2).
Today, there is a widespread agreement in dening hybrid systems as dynamical
systems that switch among many operating modes, where each mode is governed
by its own characteristic dynamical laws, and mode transitions are triggered by
variables crossing specic thresholds (state events), by the lapse of certain time
periods (time events), or by external inputs (input events) [9]. In the literature,
systems whose mode only depends on external inputs are usually called switched
systems, the others switching systems.
Complex systems organized in a hierarchial manner, where, for instance, dis-
crete planning algorithms at the higher level interact with continuous control algo-
rithms and processes at the lower level, are another example of hybrid systems. In
these systems, the hierarchical organization helps to manage the complexity of the
system, as higher levels in the hierarchy require less detailed models (also called
abstractions) of the lower levels functions.
Hybrid systems arise in a large number of application areas and are attracting
increasing attention in both academic theory-oriented circles as well as in industry,
for instance in the automotive industry [16, 157, 61, 122, 74, 212]. Moreover,
many physical phenomena admit a natural hybrid description, like circuits involving
relays or diodes [27], biomolecular networks [7], and TCP/IP networks in [149].
In this book we have worked exclusively with dynamical systems formulated
in discrete time. This chapter therefore focused on hybrid models formulated in
discrete time. Though the eects of sampling can be neglected in most applications,
some interesting mathematical phenomena occurring in hybrid systems, such as
Zeno behaviors [158] do not exist in discrete time, as switches can only occur
at sampling instants. On the other hand, most of these phenomena are usually a
consequence of the continuous-time switching model, rather than the real behavior.
Our main motivation for concentrating on discrete-time models is that optimal
control problems are easier to formulate and to solve numerically than continuous-
time formulations.
In the theory of hybrid systems, several problems were investigated in the last
few years. Besides the issues of existence and computation of trajectories described
in Section 17.1, several other issues were considered. These include: equivalence of
hybrid models, stability and passivity analysis, reachability analysis and verica-
tion of safety properties, controller synthesis, observability analysis, state estima-
tion and fault detection schemes, system identication, stochastic and event-driven
dynamics. We will briey review some of these results in the next paragraphs and
provide pointers to some relevant literature references.
Equivalence of Linear Hybrid Systems
Under the condition that the MLD system is well-posed, the result showing that
an MLD systems admits an equivalent PWA representation was proved in [34]. A
slightly dierent and more general proof is reported in [29], where the author also
provides ecient MLD to PWA translation algorithms. A dierent algorithm for
obtaining a PWA representation of a DHA is reported in [118].
The fact that PWA systems are equivalent to interconnections of linear systems
and nite automata was pointed out by Sontag [266]. In [147, 37] the authors proved
the equivalence of discrete-time PWA/MLD systems with other classes of discrete-
time hybrid systems (under some assumptions) such as linear complementarity
(LC) systems [145, 278, 146], extended linear complementarity (ELC) systems [91],
and max-min-plus-scaling (MMPS) systems [92].
Stability Analysis
Piecewise quadratic Lyapunov stability is becoming a standard in the stability
analysis of hybrid systems [155, 93, 99, 231, 232]. It is a deductive way to prove
the stability of an equilibrium point of a subclass of hybrid systems (piecewise
ane systems). The computational burden is usually low, at the price of a con-
vex relaxation of the problem, that leads to possibly conservative results. Such
conservativeness can be reduced by constructing piecewise polynomial Lyapunov
functions via semidenite programming by means of the sum of squares (SOS) de-
composition of multivariate polynomials [228]. SOS methods for analyzing stability
of continuous-time hybrid and switched systems are described in [236]. For the gen-
eral class of switched systems of the form x = f
i
(x), i = 1, . . . , s, an extension of the
Lyapunov criterion based on multiple Lyapunov functions was introduced in [66].
The reader is also referred to the book of Liberzon [188].
The research on stability criteria for PWA systems has been motivated by the
fact that the stability of each component subsystem is not sucient to guarantee
stability of a PWA system (and vice versa). Branicky [66], gives an example where
stable subsystems are suitably combined to generate an unstable PWA system.
Stable systems constructed from unstable ones have been reported in [277]. These
examples point out that restrictions on the switching have to be imposed in order
to guarantee that a PWA composition of stable components remains stable.
Passivity analysis of hybrid models has received very little attention, except
for the contributions of [76, 196, 293] and [234], in which notions of passivity for
continuous-time hybrid systems are formulated, and of [30], where passivity and
synthesis of passifying controllers for discrete-time PWA systems are investigated.
Reachability Analysis and Verication of Safety Properties
Although simulation allows to probe a model for a certain initial condition and
input excitation, any analysis based on simulation is likely to miss the subtle phe-
nomena that a model may generate, especially in the case of hybrid models. Reach-
ability analysis (also referred to as safety analysis or formal verication), aims
at detecting if a hybrid model will eventually reach unsafe state congurations
or satisfy a temporal logic formula [8] for all possible initial conditions and in-
put excitations within a prescribed set. Reachability analysis relies on a reach set
computation algorithm, which is strongly related to the mathematical model of
the system. In the case of MLD systems, for example, the reachability analysis
problem over a nite time horizon (also referred to as bounded model checking) can
be cast as a mixed-integer feasibility problem. Reachability analysis was also in-
vestigated via bisimulation ideas, namely by analyzing the properties of a simpler,
more abstract system instead of those of the original hybrid dynamics [179].
Timed automata and hybrid automata have proved to be a successful modeling
framework for formal verication and have been widely used in the literature. The
starting point for both models is a nite state machine equipped with continuous
dynamics. In the theory of timed automata, the dynamic part is the continuous-time
ow x = 1. Ecient computational tools complete the theory of timed automata
and allow one to perform verication and scheduling of such models. Timed au-
tomata were extended to linear hybrid automata [8], where the dynamics is modeled
by the dierential inclusion a x b. Specic tools allow one to verify such mod-
els against safety and liveness requirements. Linear hybrid automata were further
extended to hybrid automata where the continuous dynamics is governed by dier-
ential equations. Tools exist to model and analyze those systems, either directly or
by approximating the model with timed automata or linear hybrid automata (see
e.g. the survey paper [264]).
Control
The majority of the control approaches for hybrid systems is based on optimal
control ideas (see the survey [289]). For continuous-time hybrid systems, most
authors either studied necessary conditions for a trajectory to be optimal [233,
270], or focused on the computation of optimal/suboptimal solutions by means
of dynamic programming or the maximum principle [125, 243, 142, 143, 79, 290,
189, 260, 263, 265, 272, 69, 140, 193, 125, 68, 250, 70] The hybrid optimal control
problem becomes less complex when the dynamics is expressed in discrete-time,
as the main source of complexity becomes the combinatorial (yet nite) number
of possible switching sequences. As will be shown in Chapter 18, optimal control
problems can be solved for discrete-time hybrid systems using either the PWA
or the MLD models described in this chapter. The solution to optimal control
problems for discrete-time hybrid systems was rst outlined by Sontag in [265].
In his plenary presentation [199] at the 2001 European Control Conference Mayne
presented an intuitively appealing characterization of the state feedback solution to
optimal control problems for linear hybrid systems with performance criteria based
on quadratic and piecewise linear norms. The detailed exposition presented in the
rst part of Chapter 18 follows a similar line of argumentation and shows that the
state feedback solution to the nite time optimal control problem is a time-varying
piecewise ane feedback control law, possibly dened over non-convex regions.
Model Predictive Control for discrete-time PWA systems and their stability and
robustness properties have been studied in [67, 59, 31, 185, 182, 183]. Invariant
sets computation for PWA systems has also been studied in [184, 4].
Observability and State Estimation
Observability of hybrid systems is a fundamental concept for understanding if a
state observer can be designed for a hybrid system and how well it will perform.
In [34] the authors show that observability properties (as well as reachability prop-
erties) can be very complex and present a number of counterexamples that rule out
obvious conjectures about inheriting observability/controllability properties from
the composing linear subsystems. They also provide observability tests based on
linear and mixed-integer linear programming.
State estimation is the reconstruction of the value of unmeasurable state vari-
ables based on output measurements. While state estimation is primarily required
for output-feedback control, it is also important in problems of monitoring and
fault detection [38, 17]. Observability properties of hybrid systems were directly
exploited for designing convergent state estimation schemes for hybrid systems
in [100].
Identication
Identication techniques for piecewise ane systems were recently developed [101,
253, 163, 36, 165, 280, 225], that allow one to derive models (or parts of models)
from input/output data.
Extensions to Event-driven and Stochastic Dynamics
The discrete-time methodologies described in this chapter were employed in [73] to
tackle event-based continuous-time hybrid systems with integral continuous dynam-
ics, called integral continuous-time hybrid automata (icHA). The hybrid dynamics
is translated into an equivalent MLD form, where continuous-time is an additional
state variable and the index t counts events rather than time steps. Extensions
of DHA to discrete-time stochastic hybrid dynamics was proposed in [32], where
discrete-state transitions depend on both deterministic and stochastic events.
Chapter 18
Optimal Control of Hybrid
Systems
In this chapter we study the nite time, innite time and receding horizon optimal
control problem for the class of hybrid systems presented in the previous chapter.
We establish the structure of the optimal control law and derive algorithms for its
computation. For nite time problems with linear and quadratic objective functions
we show that the time varying feedback law is piecewise ane. We present several
approaches for the computation of the optimal control law.
Consider the PWA system (17.1) subject to hard input and state constraints
Ex(t) +Lu(t) M (18.1)
for t 0, and rewrite its restriction over the set of states and inputs dened
by (18.1) as
x(t + 1) = A
i
x(t) +B
i
u(t) +f
i
if
_
x(t)
u(t)
_

c
i
(18.2)
where
c
i
s1
i=0
is the new polyhedral partition of the sets of state+input space
R
n+m
obtained by intersecting the sets c
i
in (17.1) with the polyhedron described
by (18.1). In this chapter we will assume that the sets c
i
are polytopes.
Dene the cost function
J
0
(x(0), U
0
) = p(x
N
) +
N1
k=0
q(x
k
, u
k
) (18.3)
where x
k
x
0
x
k+1
= A
i
x
k
+B
i
u
k
+f
i
if [
x
k
u
k
]

c
i
(18.4)
384 18 Optimal Control of Hybrid Systems
0
= [u
0
, . . . , u
N1
]
.
If the 1-norm or -norm is used in the cost function (18.3), then we set p(x
N
) =
|Px
N
|
p
and q(x
k
, u
k
) = |Qx
k
|
p
+|Ru
k
|
p
with p = 1 or p = and P, Q, R full
column rank matrices. Cost (18.3) is rewritten as
J
0
(x(0), U
0
) = |Px
N
|
p
+
N1
k=0
|Qx
k
|
p
+|Ru
k
|
p
(18.5)
If the squared Euclidian norm is used in the cost function (18.3), then we set
p(x
N
) = x
N
Px
N
and q(x
k
, u
k
) = x
k
Qx
k
+ u
k
Ru
k
with P _ 0, Q _ 0 and R 0.
Cost (18.3) is rewritten as
J
0
(x(0), U
0
) = x
N
Px
N
+
N1
k=0
x
k
Qx
k
+u
k
Ru
k
(18.6)
Consider the constrained nite time optimal control problem (CFTOC)
J
0
(x(0)) = min
U0
J
0
(x(0), U
0
) (18.7a)
subj. to
_
_
_
x
k+1
= A
i
x
k
+B
i
u
k
+f
i
if [
x
k
u
k
]

c
i
x
N
A
f
x
0
= x(0)
(18.7b)
where the column vector U
0
= [u
0
, . . . , u
N1
]
R
mcN
0, 1
m
N
, is the opti-
mization vector, N is the time horizon and A
f
is the terminal region.
In general, the optimal control problem (18.7) may not have a minimizer for
some feasible x(0). This is caused by discontinuity of the PWA system in the input
space. We will assume, however, that a minimizer U
0
(x(0)) exists for all feasible

x(0). Also the optimizer function U
0
may not be uniquely dened if the optimal
set of problem (18.7) is not a singleton for some x(0). In this case U
0
denotes one
of the optimal solutions.
We will denote by A
k
R
nc
0, 1
n
the set of states x

k
that are feasible
for (18.7):
A
k
=
_
_
x R
nc
0, 1
n
u R
mc
0, 1
m
,
i 1, . . . , s
[
x
u
]

c
i
and
A
i
x +B
i
u +f
i
A
k+1
_
_
,
k = 0, . . . , N 1,
A
N
= A
f
.
(18.8)
The denition of A
i
requires that for any initial state x
i
A
i
there exists a feasible
sequence of inputs U
i
= [u
i
, . . . , u
N1
] which keeps the state evolution in the
feasible set A at future time instants k = i + 1, . . . , N 1 and forces x
N
into
A
f
at time N. The sets A
k
for i = 0, . . . , N play an important role in the the
solution of the optimal control problem. They are independent of the cost function
and of the algorithm used to compute the solution to problem (18.7). As in the
case of linear systems (see Chapter 12.2) there are two ways to rigourously dene
18.2 Properties of the State Feedback Solution, 2-Norm Case 385
and compute the sets A
i
: the batch approach and the recursive approach. In this
chapter we will not discuss the details on the computation of A
i
. Also, we will not
discuss invariant and reachable sets for hybrid systems. While the basic concepts
are identical to the one presented in Section 11.1 for linear systems, the discussion
of ecient algorithms requires a careful treatment which goes beyond the scope of
this book. The interested reader is referred to the work in [130, 117] for a detailed
discussion on reachable and invariant sets for hybrid systems.
In the following we need to distinguish between optimal control based on the
squared 2-norm and optimal control based on the 1-norm or -norm. Note that
the results of this chapter also hold when the number of switches is weighted in the
cost function (18.3), if this is meaningful in a particular situation.
In this chapter we will make use of the following denition. Consider sys-
tem (18.2) and recall that, in general, x = [
xc
x
] where x
c
R
nc
are the continuous
states and x
R
n
are the binary states and u R

mc
0, 1
m
where u
c
R
mc
are the continuous inputs and u
R
m
are the binary inputs (Section 17.2.2). We

will make the following assumption.
Assumption 18.1 For the discrete-time PWA system (18.2) the mapping (x
c
(t), u
c
(t))
x
c
(t + 1) is continuous.
Assumption 18.1 requires that the PWA function that denes the update of the
continuous states is continuous on the boundaries of contiguous polyhedral cells,
and therefore allows one to work with the closure of the sets

c
i
without the need
of introducing multi-valued state update equations. With abuse of notation in the
next sections

c
i
will always denote the closure of

c
i
. Discontinuous PWA systems
will be discussed in Section 18.7.
18.2 Properties of the State Feedback Solution, 2-Norm
Case
Theorem 18.1 Consider the optimal control problem (18.7) with cost (18.6) and
let Assumption 18.1 hold. Then, there exists a solution in the form of a PWA state
feedback control law
u
k
(x(k)) = F
i
k
x(k) +g
i
k
if x(k) 1
i
k
, (18.9)
where 1
i
k
, i = 1, . . . , N
k
is a partition of the set A
k
of feasible states x(k), and the
closure

1
i
k
of the sets 1
i
k
has the following form:
1
i
k
=
_
x : x(k)
L
i
k
(j)x(k) +M
i
k
(j)
x(k) N
i
k
(j) ,
j = 1, . . . , n
i
k
_
, k = 0, . . . , N 1,
(18.10)
and
x(k + 1) = A
i
x(k) +B
i
u
k
(x(k)) +f
i
if
_
x(k)
u
k
(x(k))
_

c
i
, i = 1, . . . , s.
(18.11)
Proof: The piecewise linearity of the solution was rst mentioned by Sontag
in [265]. In [199] Mayne sketched a proof. In the following we will give the proof
for u
0
(x(0)), the same arguments can be repeated for u
1
(x(1)), . . . , u
N1
(x(N1)).
Case 1: (m
l
= n
l
= 0) no binary inputs and states
Depending on the initial state x(0) and on the input sequence U = [u
0
,. . .,
u
k
], the state x
k
is either infeasible or it belongs to a certain polyhedron
c
i
, k = 0, . . . , N 1. The number of all possible locations of the state
sequence x
0
, . . . , x
N1
is equal to s
N
. Denote by v
i
s
N
i=1
the set of all possible
switching sequences over the horizon N , and by v
k
i
the k-th element of the
sequence v
i
, i.e., v
k
i
= j if x
k

c
j
.
Fix a certain v
i
and constrain the state to switch according to the sequence
v
i
. Problem (18.3)-(18.7) becomes
J
vi
(x(0)) = min
{U0}
J
0
(U
0
, x(0)) (18.12a)
subj. to
_
_
x
k+1
= A
v
k
i
x
k
+B
v
k
i
u
k
+f
v
k
i
[
x
k
u
k
]

c
v
k
i
k = 0, . . . , N 1
x
N
A
f
x
0
= x(0)
(18.12b)
Problem (18.12) is equivalent to a nite time optimal control problem for a
linear time-varying system with time-varying constraints and can be solved
by using the approach described in Chapter 12. The rst move u
0
of its
solution is the PPWA feedback control law
u
i
0
(x(0)) =

F
i,j
x(0) + g
i,j
, x(0) T
i,j
, j = 1, . . . , N
ri
(18.13)
where T
i
=
N
ri
j=1
T
i,j
is a polyhedral partition of the convex set T
i
of feasible
states x(0) for problem (18.12). N
ri
is the number of regions of the polyhedral
partition of the solution and it is a function of the number of constraints in
problem (18.12). The upper index i in (18.13) denotes that the input u
i
0
(x(0))
is optimal when the switching sequence v
i
is xed.
The set A
0
of all feasible states at time 0 is A
0
=
s
N
i=1
T
i
and in general
it is not convex. Indeed, as some initial states can be feasible for dierent
switching sequences, the sets T
i
, i = 1, . . . , s
N
, in general, can overlap. The
solution u
0
(x(0)) to the original problem (18.3)-(18.7) can be computed in
the following way. For every polyhedron T
i,j
in (18.13),
1. If T
i,j
T
l,m
= for all l ,= i, l = 1, . . . , s
N
, and for all m ,= j,
m = 1, . . . , N
rl
, then the switching sequence v
i
is the only feasible one
for all the states belonging to T
i,j
and therefore the optimal solution is
given by (18.13), i.e.
u
0
(x(0)) =

F
i,j
x(0) + g
i,j
, x(0) T
i,j
. (18.14)
2. If T
i,j
intersects one or more polyhedra T
l1,m1
,T
l2,m2
, . . ., the states
belonging to the intersection are feasible for more than one switching
sequence v
i
, v
l1
, v
l2
, . . . and therefore the corresponding value functions
J
vi
, J
v
l
1
,J
v
l
2
, . . . in (18.12a) have to be compared in order to compute
the optimal control law.
Consider the simple case when only two polyhedra overlap, i.e. T
i,j
T
l,m
= T
(i,j),(l,m)
,= . We will refer to T
(i,j),(l,m)
as a double feasibility
polyhedron. For all states belonging to T
(i,j),(l,m)
the optimal solution
is:
u
0
(x(0)) =
_
F
i,j
x(0) + g
i,j
, x(0) T
(i,j),(l,m)
:
J
v
i
(x(0)) < J
v
l
(x(0))
F
l,m
x(0) + g
l,m
, x(0) T
(i,j),(l,m)
:
J
v
i
(x(0)) > J
v
l
(x(0))
_

F
i,j
x(0) + g
i,j
or
F
l,m
x(0) + g
l,m
x(0) T
(i,j),(l,m)
:
J
v
i
(x(0)) = J
v
l
(x(0))
(18.15)
Because J
vi
and J
v
l
are quadratic functions of x(0) on T
i,j
and T
l,m
re-
spectively, we nd the expression (18.10) of the control law domain. The
sets T
i,j
T
l,m
and T
l,m
T
i,j
are two single feasibility non-Euclidean
polyhedra which can be partitioned into a set of single feasibility polyhe-
dra, and thus be described through (18.10) with L
i
k
= 0.
In order to conclude the proof, the general case of n intersecting polyhedra
has to be discussed. We follow three main steps. Step 1: generate one poly-
hedron of n
th
-ple feasibility and 2
n
2 polyhedra, generally non-Euclidean
and possibly empty and disconnected, of single, double, . . ., (n 1)
th
-ple
feasibility. Step 2: the i
th
-ple feasibility non-Euclidean polyhedron is parti-
tioned into several i
th
-ple feasibility polyhedra. Step 3: any i
th
-ple feasibility
polyhedron with i > 1 is further partitioned into at most i subsets (18.10)
where in each one of them a certain feasible value function is greater than all
the others. The procedure is depicted in Figure 18.1 when n = 3.
Case 2: binary inputs, m
,= 0
The proof can be repeated in the presence of binary inputs, m
,= 0. In this
case the switching sequences v
i
are given by all combinations of region indices
and binary inputs, i.e., i = 1, . . . , (s m
)
N
. For each sequence v
i
there is an
associated optimal continuous component of the input as calculated in Case
1.
Case 3: binary states, n
l
,= 0
The proof can be repeated in the presence of binary states by a simple enu-
meration of all the possible n
N
discrete state evolutions.

1
2
3
1,3
1,2
1,2,3
(a) Step 1.
1
2
3
(b) Step 2 and Step 3.
Figure 18.1 Graphical illustration of the main steps for the proof of Theo-
rem 18.1 when 3 polyhedra intersect. Step 1: the three intersecting poly-
hedra are partitioned into: one polyhedron of triple feasibility (1,2,3), 2
polyhedra of double feasibility (1, 2) and (1, 3), 3 polyhedra of single fea-
sibility (1),(2),(3). The sets (1), (2) and (1,2) are neither open nor closed
polyhedra. Step 2: the single feasibility sets of (1), (2) and (3) are parti-
tioned into six polyhedra. Step 3: value functions are compared inside the
polyhedra of multiple feasibility.
From the result of the theorem above one immediately concludes that the value
function J
0
is piecewise quadratic:
J
0
(x(0)) = x(0)
H
i
1
x(0) +H
i
2
x(0) +H
i
3
if x(0) 1
i
0
, (18.16)
The proof of Theorem 18.1 gives useful insights into the properties of the sets
1
i
k
in (18.10). We will summarize them next.
Each set 1
i
k
has an associated multiplicity j which means that j switching
sequences are feasible for problem (18.3)-(18.7) starting from a state x(k) 1
i
k
.
If j = 1, then 1
i
k
is a polyhedron. In general, if j > 1 the boundaries of 1
i
k
can be described either by an ane function or by a quadratic function. In the
sequel boundaries which are described by quadratic functions but degenerate to
hyperplanes or sets of hyperplanes will be considered ane boundaries.
Quadratic boundaries arise from the comparison of value functions associated
with feasible switching sequences, thus a maximum of j 1 quadratic boundaries
can be present in a j-ple feasible set. The ane boundaries can be of three types.
Type a: they are inherited from the original j-ple feasible non-Euclidean
polyhedron. In this case across such boundaries the multiplicity of the feasi-
bility changes.
Type b: they are articial cuts needed to describe the original j-ple feasible
non-Euclidean polyhedron as a set of j-ple feasible polyhedra. Across type b
boundaries the multiplicity of the feasibility does not change.
Type c: they arise from the comparison of quadratic value functions which
degenerate in an ane boundary.
In conclusion, we can state the following lemma
Lemma 18.1 The value function J
k
1. is a quadratic function of the states inside each 1
i
k
2. is continuous on quadratic and ane boundaries of type b and c
3. may be discontinuous on ane boundaries of type a,
and the optimizer u
k
1. is an ane function of the states inside each 1
i
k
2. is continuous across and unique on ane boundaries of type b
3. is non-unique on quadratic boundaries, except possibly at isolated points.
4. may be non-unique on ane boundaries of type c,
5. may be discontinuous across ane boundaries of type a.
Based on Lemma 18.1 above one can highlight the only source of discontinuity
of the value function: ane boundaries of type a. The following corollary gives a
useful insight on the class of possible value functions.
Corollary 18.1 J
0
is a lower-semicontinuous PWQ function on A
0
.
Proof: The proof follows from the result on the minimization of lower-semicontinuous
point-to-set maps in [46]. Below we give a simple proof without introducing the
notion of point-to-set maps.
Only points where a discontinuity occurs are relevant for the proof, i.e., states
belonging to boundaries of type a. From Assumption 18.1 it follows that the feasible
switching sequences for a given state x(0) are all the feasible switching sequences
associated with any set 1
j
0
whose closure

1
j
0
contains x(0). Consider a state x(0)
belonging to boundaries of type a and the proof of Theorem 18.1. The only case of
discontinuity can occur when (i) a j-ple feasible set T
1
intersects an i-ple feasible
set T
2
with i < j, (ii) there exists a point x(0) T
1
, T
2
and a neighborhood
A(x(0)) with x, y A(x(0)), x T
1
, x / T
2
and y T
2
, y / T
1
. The proof
follows from the previous statements and the fact that J
0
(x(0)) is the minimum of
all J
vi
(x(0)) for all feasible switching sequences v
i
.
The result of Corollary 18.1 will be used extensively in the next sections. Even
if value function and optimizer are discontinuous, one can work with the closure
1
j
k
of the original sets 1
j
k
without explicitly considering their boundaries. In fact,
if a given state x(0) belongs to several regions

1
1
0
,. . . ,

1
p
0
, then the minimum value
among the optimal values (18.16) associated with each region

1
1
0
, . . . ,

1
p
0
allows us
to identify the region of the set 1
1
0
, . . . , 1
p
0
containing x(0).
Next we show some interesting properties of the optimal control law when we
restrict our attention to smaller classes of PWA systems.
Corollary 18.2 Assume that the PWA system (18.2) is continuous, and that E =
0 in (18.1) and A
f
= R
n
in (18.7) (which means that there are no state constraints,
i.e.,

P is unbounded in the x-space). Then, the value function J
0
in (18.7) is
continuous.
Proof: Problem (18.7) becomes a multi-parametric program with only input
constraints when the state at time k is expressed as a function of the state at time 0
and the input sequence u
0
, . . . , u
k1
, i.e., x
k
= f
PWA
(( (f
PWA
(x
0
, u
0
), u
1
), . . . , u
k2
), u
k1
).
J
0
in (18.3) will be a continuous function of x
0
and u
0
, . . . , u
N1
since it is the
composition of continuous functions. By assumptions the input constraints on
u
0
, . . . , u
N1
are convex and the resulting set is compact. The proof follows from
the continuity of J and Theorem 6.2.
Note that E = 0 is a sucient condition for ensuring that constraints (18.1)

are convex in the optimization variables u
0
, . . . , u
n
. In general, even for continuous
PWA systems with state constraints it is dicult to nd weak assumptions ensuring
the continuity of the value function J
0
. Ensuring the continuity of the optimal
control law u(k) = u
k
(x(k)) is even more dicult. A list of sucient conditions for
U
0
to be continuous can be found in [103]. In general, they require the convexity (or
a relaxed form of it) of the cost J
0
(U
0
, x(0)) in U
0
for each x(0) and the convexity
of the constraints in (18.7) in U
0
for each x(0). Such conditions are clearly very
restrictive since the cost and the constraints in problem (18.7) are a composition
of quadratic and linear functions, respectively, with the piecewise ane dynamics
of the system.
The next theorem provides a condition under which the solution u
k
(x(k)) of
the optimal control problem (18.3)-(18.7) is a PPWA state feedback control law.
Theorem 18.2 Assume that the optimizer U
0
(x(0)) of (18.3)-(18.7) is unique

for all x(0). Then the solution to the optimal control problem (18.3)-(18.7) is a
PPWA state feedback control law of the form
u
k
(x(k)) = F
i
k
x(k) +g
i
k
if x(k) T
i
k
k = 0, . . . , N 1, (18.17)
where T
i
k
, i = 1, . . . , N
r
k
, is a polyhedral partition of the set A
k
of feasible states
x(k).
Proof: In Lemma 18.1 we concluded that the value function J
0
(x(0)) is con-
tinuous on quadratic type boundaries. By hypothesis, the optimizer u
0
(x(0)) is
unique. Theorem 18.1 implies that

F
i,j
x(0) + g
i,j
=

F
l,m
x(0) + g
l,m
, x(0) be-
longing to the quadratic boundary. This can occur only if the quadratic boundary
degenerates to a single feasible point or to ane boundaries. The same arguments
can be repeated for u
k
(x(k)), k = 1, . . . , N 1.
Remark 18.1 Theorem 18.2 relies on a rather strong uniqueness assumption. Some-
times, problem (18.3)-(18.7) can be modied in order to obtain uniqueness of the
solution and use the result of Theorem 18.2 which excludes the existence of non-
convex ellipsoidal sets. It is reasonable to believe that there are other conditions
under which the state feedback solution is PPWA without claiming uniqueness.
18.3 Properties of the State Feedback Solution, 1, -Norm Case 391
Example 18.1 Consider the following simple system
1
_
_
x(t + 1) =
_
_
_
0 1
1 0
_
x(t) +
_
0.2
0
_
u(t) +
_
0.2
0.2
_
if x(t) c
1
= |x : [0 1]x 0
_
0 1
1 0
_
x(t) +
_
0.2
0
_
u(t)
_
0.2
0.2
_
if x(t) c
2
= |x : [0 1]x < 0
x(t) [0.5, 0.5] [0.5, 0.5]
u(t) [1000, 1000]
(18.18)
and the optimal control problem (18.7) with cost (18.6), N = 2, Q =
_
1 0.1
0.1 1
_
,
R = 10, P = 10Q, .
f
= ..
The possible switching sequences are v1 = |1, 1, v2 = |1, 2, v3 = |2, 1, v4 = |2, 2.
The solution to problem (18.12) is depicted in Figure 18.2. In Figure 18.3(a) the
four solutions are intersected. All regions are polyhedra of multiple feasibility. In
Figure 18.3(b) we plot with dierent colors the regions of the state-space partition
where the switching sequences v1 = |1, 1, v2 = |1, 2, v3 = |2, 1, v4 = |2, 2 are
optimal.
18.3 Properties of the State Feedback Solution, 1, -Norm
Case
The results of the previous section can be extended to piecewise linear cost func-
tions, i.e., cost functions based on the 1-norm or the -norm.
Theorem 18.3 Consider the optimal control problem (18.7) with cost (18.5), p =
1, and let Assumption 18.1 hold. Then there exists a solution in the form of a
PPWA state feedback control law
u
k
(x(k)) = F
i
k
x(k) +g
i
k
if x(k) T
i
k
, (18.19)
where T
i
k
, i = 1, . . . , N
r
k
is a polyhedral partition of the set A
k
of feasible states
x(k).
Proof: The proof is similar to the proof of Theorem 18.1. Fix a certain switch-
ing sequence v
i
, consider the problem (18.3)-(18.7) and constrain the state to switch
according to the sequence v
i
to obtain problem (18.12). Problem (18.12) can be
viewed as a nite time optimal control problem with a performance index based on
1-norm or -norm for a linear time varying system with time varying constraints
and can be solved by using the multi-parametric linear program as described in
Chapter 12.4. Its solution is a PPWA feedback control law
u
i
0
(x(0)) =

F
i,j
x(0) + g
i,j
, x T
i,j
, j = 1, . . . , N
ri
, (18.20)
1
x
1
x
2
(a) v1 = |1, 1
x
1
x
2
(b) v2 = |1, 2
x
1
x
2
(c) v2 = |2, 1
x
1
x
2
(d) v4 = |2, 2
Figure 18.2 Example 18.1: Problem (18.12) with cost (18.6) is solved for
dierent vi, i = 1, . . . , 4. Control law u
0
(x0) is PPWA. State space partition
is shown.
and the value function J
vi
is piecewise ane on polyhedra and convex. The rest
of the proof follows the proof of Theorem 18.1. Note that in this case the value
functions to be compared are piecewise ane and not piecewise quadratic.
18.4 Computation of the Optimal Control Input via Mixed
Integer Programming
In the previous section the properties enjoyed by the solution to hybrid optimal
control problems were investigated. Despite the fact that the proofs are construc-
tive (as shown in the gures), they are based on the enumeration of all the possible
switching sequences of the hybrid system, the number of which grows exponen-
tially with the time horizon. Although the computation is performed o-line (the
on-line complexity is the one associated with the evaluation of the PWA control
18.4 Computation of the Optimal Control Input via Mixed Integer Programming 393
x
1
x
2
(a) Feasibility domain obtained as
the union of the domains shown
in Figure 18.2. All regions are
polyhedra of multiple feasibility.
v
1
v
2
v
4
v
3
x
1
x
2
(b) Regions of the state-space
partition where the switching se-
quences v1 = |1, 1, v2 = |1, 2,
v3 = |2, 1, v4 = |2, 2 are opti-
mal.
Figure 18.3 Example 18.1: State-space partition corresponding to the opti-
mal control law of
law (18.17)), more ecient methods than enumeration are desirable. Here we show
that the MLD framework can be used to avoid the complete enumeration. In
fact, when the model of the system is an MLD model and the performance index is
quadratic, the optimization problem can be cast as a Mixed-Integer Quadratic Pro-
gram (MIQP). Similarly, 1-norm and -norm performance indices lead to Mixed-
Integer Linear Programs (MILPs) problems. In the following we detail the trans-
lation of problem (18.7) with cost (18.5) or (18.6) into a mixed integer linear or
quadratic program, respectively, for which ecient branch and bound algorithms
exist.
Consider the equivalent MLD representation (17.25) of the PWA system (18.2).
Problem (18.7) is rewritten as:
J
0
(x(0)) = min
U0
J
0
(x(0), U
0
) (18.21a)
subj. to
_
_
x
k+1
= Ax
k
+B
1
u
k
+B
2
k
+B
3
z
k
y(t) = Cx(t) +D
1
u(t) +D
2
(t) +D
3
z(t) +D
5
E
2
k
+E
3
z
k
E
1
u
k
+E
4
x
k
+E
5
x
N
A
f
x
0
= x(0)
(18.21b)
Note that the cost function (18.21a) is of the form
J
0
(x(0), U
0
) = |Px
N
|
p
+
N1
k=0
|Qx
k
|
p
+|Ru
k
|
p
+|Q
k
|
p
+|Q
z
z
k
|
p
(18.22)
when p = 1 or p = or
J
0
(x(0), U
0
) = x
N
Px
N
+
N1
k=0
x
k
Qx
k
+u
k
Ru
k
+
k
Q
k
+z
k
Q
z
z
k
(18.23)
when p=2.
The optimal control problem (18.21) with the cost (18.22) can be formulated as
a Mixed Integer Linear Program (MILP). The optimal control problem (18.21) with
the cost (18.23) can be formulated as a Mixed Integer Quadratic Program (MIQP).
The compact form for both cases is
min
H
1
+
H
2
x(0) +x(0)
H
3
x(0) +c
1
+c
2
x(0) +c
subj. to G w +Sx(0)
(18.24)
where H
1
, H
2
, H
3
, c
1
, c
2
, G, w, S are matrices of suitable dimensions, = [
c
,
d
]
where
c
,
d
represent continuous and discrete variables, respectively and H
1
, H
2
,
H
3
, are null matrices if problem (18.24) is an MILP.
The translation of (18.21) with cost (18.23) into (18.24) is simply obtained by
substituting the state update equation
x
k
= A
k
x
0
+
k1
j=0
A
j
(B
1
u
k1j
+B
2
k1j
+B
3
z
k1j
) (18.25)
The optimization vector in (18.24) is = u
0
, . . . , u
N1
,
0
, . . . ,
N1
, z
0
, . . . , z
N1
.
The translation of (18.21) with cost (18.22) into (18.24) requires the introduc-
tions of slack variables as shown in Section 12.4 for the case of linear systems. In
particular, for p = , Q
z
= 0 and Q
= 0, the sum of the components of any

vector
u
0
, . . . ,
u
N1
,
x
0
, . . . ,
x
N
that satises
1
m
u
k
Ru
k
, k = 0, 1, . . . , N 1
1
m
u
k
Ru
k
, k = 0, 1, . . . , N 1
1
n
x
k
Qx
k
, k = 0, 1, . . . , N 1
1
n
x
k
Qx
k
, k = 0, 1, . . . , N 1
1
n
x
N
Px
N
,
1
n
x
N
Px
N
,
(18.26)
represents an upper bound on J
0
(x(0)), where 1
k
is a column vector of ones of
length k, and where x(k) is expressed as in (18.25). Similarly to what was shown
in [77], it is easy to prove that the vector =
u
0
, . . . ,
u
N1
,
x
0
, . . . ,
x
N
, u(0), . . . , u(N
1) that satises equations (18.26) and simultaneously minimizes
J() =
u
0
+. . . +
u
N1
+
x
0
+. . . +
x
N
(18.27)
also solves the original problem, i.e., the same optimum J
0
(x(0)) is achieved.
Therefore, problem (18.21) with cost (18.22) can be reformulated as the follow-
ing MILP problem
min
J()
subj. to 1
m
u
k
Ru
k
, k = 0, 1, . . . , N 1
1
n
x
k
Q(A
k
x
0
+
k1
j=0
A
j
(B
1
u
k1j
+
B
2
k1j
+B
3
z
k1j
)) k = 0, . . . , N 1
1
n
x
N
P(A
N
x
0
+
N1
j=0
A
j
(B
1
u
k1j
+
B
2
k1j
+B
3
z
k1j
))
x
k+1
= Ax
k
+B
1
u
k
+B
2
k
+B
3
z
k
, k 0
E
2
k
+ E
3
z
k
E
1
u
k
+E
4
x
k
+E
5
, k 0
x
N
A
f
x
0
= x(0)
(18.28)
where the variable x(0) in (18.28) appears only in the constraints as a parameter
vector.
Given a value of the initial state x(0), the MIQP (18.24) or the MILP (18.28)
can be solved to get the optimizer
(x(0)) and therefore the optimal input U
0
(0).
In the next Section 18.5 we will show how multiparametric programming can be
also used to eciently compute the piecewise ane state feedback optimal control
law (18.9) or (18.19).
Example 18.2 Consider the problem of steering in three steps the simple piecewise
ane system presented in Example 17.1 to a small region around the origin
2
. The
system state update equations are
_
_
x(t + 1) =
_
_
0.4
_
1
3 1
_
x(t) +
_
0
1
_
u(t) if
_
1 0
_
x(t) 0
0.4
_
1
3 1
_
x(t) +
_
0
1
_
u(t) if
_
1 0
_
x(t)
y(t) =
_
0 1
_
x(t)
(18.29)
x(t) [5, 5] [5, 5]
u(t) [1, 1]
(18.30)
The MLD representation of system (18.29)-(18.30) was reported in Example 17.7.
The nite time constrained optimal control problem (18.7) with cost (18.5) with
p = , N = 3, P = Q = [
1 0
0 1
], R = 1, and .
f
= [0.01, 0.01] [0.01, 0.01], can be
solved by considering the optimal control problem (18.21) with cost (18.22) for the
equivalent MLD representation and solving the associated MILP problem (18.24).
The resulting MILP has 27 variables ([[ = 27) which are x R
2
, |0, 1, z R
2
,
u R, y R over 3 steps plus the real-valued slack variables
u
0
,
u
1
,
u
2
and
x
1
,
x
2
,
x
3
(note that
x
0
is not needed). The number of mixed-integer equality constraints
is 9 resulting from the 3 equality constraints of the MLD systems over 3 steps. The
number of mixed-integer inequality constraints is 110.
2
The nite time constrained optimal control problem (18.7) with cost (18.6), N = 3,
P = Q = [
1 0
0 1
], R = 1, and .
f
= [0.01 0.01] [0.01 0.01], can be solved by
considering the optimal control problem (18.21) with cost (18.23) for the equivalent
MLD representation and solving the associated MIQP problem (18.24).
The resulting MIQP has 21 variables ([[ = 21) which are x R
2
, |0, 1, z R
2
,
u R, y R over 3 steps. The number of mixed-integer equality constraints is
9 resulting from the 3 equality constraints of the MLD systems over 3 steps. The
number of mixed-integer inequality constraints is 74.
18.4.1 Mixed-Integer Optimization Methods
With the exception of particular structures, mixed-integer programming problems
involving 0-1 variables are classied as AP-complete, which means that in the
worst case, the solution time grows exponentially with the problem size.
Despite the combinatorial nature of these problems, excellent solvers for MILPs
and MIQPs have become available over the last 10-15 years. As discussed in Chap-
ter 3.4, the most obvious way to solve an MIQP (MILP) is to enumerate all the
integer values and solve the corresponding QPs (LPs). By comparing the QPs
(LPs) optimal costs one can derive the optimizer and the optimal cost of the MIQP
(MILP). This is far from being ecient. The book [106] is a good reference for a
long list of algorithmic approaches to eciently solve mixed integer programming
problems.
One simple idea called Branch and Bound has become the basis of many
ecient solvers. We will explain the basic ideas behind Branch and Bound solvers
with the help of a simple example.
Example 18.3 Illustrative Example for Branch and Bound Procedure We
refer to Figure 18.4. Assume that we are solving an MILP with three 0-1 integer
variables x
d,1
, x
d,2
, x
d,3
.
1. In a rst step we relax the integer constraints, i.e., we allow the variables
x
d,1
, x
d,2
, x
d,3
to vary continuously between 0 and 1. We nd the minimum
of the Relaxed LP to be 10, which must be a lower bound on the solution.
2. We x one of the integer variables x
d,1
at 0 and 1, respectively, and leave the
others relaxed. We solve the two resulting LPs and obtain 13 and 12, which are
new improved lower bounds on the solutions along the two branches.
3. Heuristically we decide to continue on the branch with x
d,1
= 1, because the
lower bound is lower than with x
d,1
= 0. We now x one of the remaining
integer variables x
d,2
at 0 and 1, respectively, and leave x
d,3
relaxed. We solve
the two resulting LPs and obtain 16 and 14, which are new improved lower
bounds on the solution along the two branches.
4. Heuristically we decide again to continue on the branch with x
d,2
= 1, because
the lower bound is lower than with x
d,2
= 0. We now x the remaining integer
variable x
d,3
at 0 and 1, respectively. We solve the two resulting LPs and obtain
15 and 17. We can conclude now that 15 is an upper bound on the solution and
the integer choices leading to 17 are non-optimal. We can also stop the further
10
13 12
19 16 14
15 17
relaxed LP
x
d,1
= 1 x
d,1
= 0
x
d,2
= 1 x
d,2
= 0 x
d,2
= 1 x
d,2
= 0
x
d,3
= 1 x
d,3
= 0
LB=10
infeasible 19 > UB 16 > UB
UB=15 15 < UB = 17
optimal
solution
Figure 18.4 Illustration of Branch and Bound procedure.
exploration of node 16 because this lower bound is higher than the established
upper bound of 15.
5. We explore node 13 and nd the integer choices either to be infeasible or to lead
to a lower bound of 19 which is again higher than the established upper bound
of 15. Thus no further explorations are necessary and 15 is conrmed to be the
optimal solution.
More generally, the Branch and Bound algorithm for MILPs involves solving
and generating a set of LP subproblems in accordance with a tree search, where
the nodes of the tree correspond to LP subproblems. Branching generates child-
nodes from parent-nodes according to branching rules, which can be based, for
instance, on a-priori specied priorities on integer variables, or on the amount by
which the integer constraints are violated. Nodes are labeled either as pending, if
the corresponding LP problem has not yet been solved, or fathomed, if the node
has already been fully explored. The algorithm stops when all nodes have been
fathomed.
The success of the branch and bound algorithm relies on the fact that whole
subtrees can be excluded from further exploration by fathoming the corresponding
root nodes. This happens if the corresponding LP subproblem is either infeasible
or an integer solution is obtained. In the second case, the corresponding value of
the cost function serves as an upper bound on the optimal solution of the MILP
problem, and is used to further fathom any other nodes having either a greater
optimal value or lower bound.
18.5 State Feedback Solution via Batch Approach
Multiparametric programming [111, 98, 41, 62] has been used to compute the PWA
form of the optimal state feedback control law u
(x(k)) for linear system. By gener-

alizing the results of the previous chapters to hybrid systems, the state vector x(0),
which appears in the objective function and in the linear part of the right-hand side
of the constraints (18.24), can be considered as a vector of parameters. Then, for
performance indices based on the -norm or 1-norm, the optimization problem can
be treated as a multiparametric MILP (mp-MILP), while for performance indices
based on the 2-norm, the optimization problem can be treated as a multiparametric
MIQP (mp-MIQP). Solving an mp-MILP (mp-MIQP) amounts to expressing the
solution of the MILP (MIQP) (18.24) as a function of the parameters x(0) .
In Section 7.4.1 we have presented an algorithm for solving mp-MILP problems,
while, to the authors knowledge, there does not exist an ecient method for solving
general mp-MIQPs. In Section 18.6 we will present an algorithm that eciently
solves the specic mp-MIQPs derived from optimal control problems for discrete-
time hybrid systems.
18.6 State Feedback Solution via Recursive Approach
In this section we propose an ecient algorithm for computing the solution to
the nite time optimal control problem for discrete-time linear hybrid systems.
It is based on a dynamic programming recursion and a multiparametric linear
or quadratic programming solver. The approach represents an alternative to the
mixed-integer parametric approach presented in the previous section.
The PWA solution (18.9) will be computed proceeding backwards in time using
two tools: a linear or quadratic multi-parametric programming solver (depending
on the cost function used) and a special technique to store the solution which will be
illustrated in the next sections. The algorithm will be presented for optimal control
based on a quadratic performance criterion. Its extension to optimal control based
on linear performance criteria is straightforward.
18.6.1 Preliminaries and Basic Steps
Consider the PWA map dened as
: x 1
i
F
i
x +g
i
for i = 1, . . . , N
R
, (18.31)
where 1
i
, i = 1, . . . , N
R
are subsets of the xspace. Note that if there exist
l, m 1, . . . , N
R
such that for x 1
l
1
m
, F
l
x + g
l
,= F
m
x + g
m
the map
(18.31) is not single valued.
Denition 18.1 Given a PWA map (18.31) we dene f
PWA
(x) =
o
(x) as the
18.6 State Feedback Solution via Recursive Approach 399
ordered region single-valued function associated with (18.31) when
o
(x) = F
j
x +g
j
[ x 1
j
and i < j : x / 1
i
,
j 1, . . . , N
R
,
and write it in the following form
o
(x) =
_
F
1
x +g
1
if x T
1
.
.
.
F
NR
x +g
NR
if x T
NR
.
Note that given a PWA map (18.31) the corresponding ordered region single-valued
function
o
changes if the order used to store the regions 1
i
and the corresponding
ane gains change. For illustration purposes consider the example depicted in
Figure 18.5, where x R, N
R
= 2, F
1
= 0, g
1
= 0, 1
1
= [2, 1], F
2
= 1, g
2
= 0,
1
2
= [0, 2].
In the following, we assume that the sets 1
k
i
in the optimal solution (18.9) can
overlap. When we refer to the PWA function u
k
(x(k)) in (18.9) we will implicitly
mean the ordered region single-valued function associated with the map (18.9).
Example 18.4 Let J
1
: T1 R and J
2
: T2 R be two quadratic functions,
J
1
(x) = x
L1x + M
1
x + N1 and J
2
(x) = x
L2x + M
2
x + N2, where T1 and T2
are convex polyhedra and J
i
(x) = + if x / Ti, i |1, 2. Let u
1
: T1 R
m
,
u
2
: T2 R
m
be vector functions. Let T1 T2 = T3 ,= and dene
J
(x) = min|J
1
(x), J
2
(x) (18.32)
u
(x) =
_
u
1
(x) if J
1
(x) J
2
(x)
u
2
(x) if J
1
(x) J
2
(x)
(18.33)
where u
(x) can be a set valued function. Let L3 = L2 L1, M3 = M2 M1,

N3 = N2 N1. Then, corresponding to the three following cases
(i) J
1
(x) J
2
(x) x T3
(ii) J
1
(x) J
2
(x) x T3
(iii) x1, x2 T3[J
1
(x1) < J
2
(x1) and J
1
(x2) > J
2
(x2)
the expressions (18.32) and a real-valued function that can be extracted from (18.33)
can be written equivalently as:
(i)
J
(x) =
_
J
1
(x) if x T1
J
2
(x) if x T2
(18.34)
u
(x) =
_
u
1
(x) if x T1
u
2
(x) if x T2
(18.35)
(ii) as in (18.34) and (18.35) by switching the indices 1 and 2
x
(
x
)
F
2
x +g
2
F
1
x +g
1
(a) Multi valued PWA map .
x
12
(x) =
_
F
1
x +g
1
if x 1
1
F
2
x +g
2
if x 1
2
(b) Ordered region single valued
function 12.
x
21
(x) =
_
F
2
x + g
2
if x 1
2
F
1
x + g
1
if x 1
1
(c) Ordered region single valued
function 21.
Figure 18.5 Illustration of the ordered region single valued function.
(iii)
J
(x) =
_
min|J
1
(x), J
2
(x) if x T3
J
1
(x) if x T1
J
2
(x) if x T2
(18.36)
u
(x) =
_
u
1
(x) if x T3
|x [ x
L3x +M
3
x +N3 0
u
2
(x) if x T3
|x [ x
L3x +M
3
x +N3 0
u
1
(x) if x T1
u
2
(x) if x T2
(18.37)
where (18.34), (18.35), (18.36), and (18.37) have to be considered as PWA and
PPWQ functions in the ordered region sense.
Example 18.4 shows how to
avoid the storage of the intersections of two polyhedra in case (i) and (ii)
avoid the storage of possibly non convex regions T
1
T
3
and T
2
T
3
work with multiple quadratic functions instead of quadratic functions dened
over non-convex and non-polyhedral regions.
The three points listed above will be the three basic ingredients for storing and
simplifying the optimal control law (18.9). Next we will show how to compute it.
Remark 18.2 In Example 18.4 the description (18.36)-(18.37) of Case (iii) can always
be used but the on-line evaluation of the control u
(x) is rather involved requiring

a series of set membership and comparison evaluations. To assess if the simpler
description of u
(x) of Case (i) or Case (ii) could be used instead, one needs to solve
indenite quadratic programs of the form
minx x
L3x +M
3
x +N3
subj. to x T3
(18.38)
which are nontrivial, in general.
18.6.2 Multiparametric Programming with Multiple Quadratic Functions
Consider the multi-parametric program
J
(x) = min
u
l(x, u) +q(f(x, u))
s.t. f(x, u) T,
(18.39)
where T R
n
is a compact set, f : R
n
R
m
R
n
, q : T R, and l : R
n
R
m
R
is a convex quadratic function of x and u. We aim at determining the region A
of variables x such that the program (18.39) is feasible and the optimum J
(x) is
nite, and at nding the expression u
(x) of (one of) the optimizer(s). We point

out that the constraint f(x, u) T implies a constraint on u as a function of x
since u can assume only values where f(x, u) is dened.
Next we show how to solve several forms of problem (18.39).
Lemma 18.2 (one to one problem) Problem (18.39) where f is linear, q is
quadratic and strictly convex, and T is a polyhedron can be solved by one mp-QP.
Proof: See Chapter 7.3.1.
Lemma 18.3 (one to one problem of multiplicity d) Problem (18.39) where
f is linear, q is a multiple quadratic function of multiplicity d and T is a polyhedron
can be solved by d mp-QPs.
Proof: The multi-parametric program to be solved is
J
(x) = min
u
l(x, u) + minq
1
(f(x, u)), . . . , q
d
(f(x, u))
subj. to f(x, u) T
(18.40)
and it is equivalent to
J
(x) = min
_
_
min
u
l(x, u) +q
1
(f(x, u)),
subj. to f(x, u) T,
.
.
.,
min
u
l(x, u) +q
d
(f(x, u))
subj. to f(x, u) T
_
_
. (18.41)
The i
th
sub-problems in (18.41)
J
i
(x) = min
u
l(x, u) +q
i
(f(x, u)) (18.42)
subj. to f(x, u) T (18.43)
is a one to one problem and therefore it is solvable by an mp-QP. Let the solution
of the i-th mp-QPs be
u
i
(x) =

F
i,j
x + g
i,j
, x T
i,j
, j = 1, . . . , N
ri
, (18.44)
where T
i
=
N
ri
j=1
T
i,j
is a polyhedral partition of the convex set T
i
of feasible x for
the ith sub-problem and N
ri
is the corresponding number of polyhedral regions.
The feasible set A satises A = T
1
= . . . = T
d
since the constraints of the d
sub-problems are identical.
The solution u
(x) to the original problem (18.40) is obtained by comparing

and storing the solution of d mp-QP subproblems (18.42)-(18.43) as explained
in Example 18.4. Consider the case d = 2, and consider the intersection of the
polyhedra T
1,i
and T
2,l
for i = 1, . . . , N
r1
, l = 1, . . . , N
r2
. For all T
1,i
T
2,l
=
T
(1,i),(2,l)
,= the optimal solution is stored in an ordered way as described in
Example 18.4, while paying attention to the fact that a region could be already
stored. Moreover, when storing a new polyhedron with the corresponding value
function and optimizer, the relative order of the regions already stored must not
be changed. The result of this Intersect and Compare procedure is
u
(x) = F
i
x +g
i
if x 1
i
,
1
i
= x : x
L
i
(j)x +M
i
(j)
x N
i
(j), j = 1, . . . , n
i
,
(18.45)
where 1 =
NR
j=1
1
j
is a polyhedron and the value function
J
(x) =

J
j
(x) if x T
j
, j = 1, . . . , N
D
, (18.46)
where

J
j
(x) are multiple quadratic functions dened over the convex polyhedra
T
j
. The polyhedron T
j
can contain several regions 1
i
or can coincide with one of
them. Note that (18.45) and (18.46) have to be considered as PWA and PPWQ
functions in the ordered region sense.
If d > 2 then the value function in (18.46) is intersected with the solution
of the third mp-QP sub-problem and the procedure is iterated by making sure
not to change the relative order of the polyhedra and corresponding gain of the
solution constructed in the previous steps. The solution will still have the same
form (18.45) (18.46).
Lemma 18.4 (one to r problem) Problem (18.39) where f is linear, q is a
lower-semicontinuous PPWQ function dened over r polyhedral regions and strictly
convex on each polyhedron, and T is a polyhedron, can be solved by r mp-QPs.
Proof: Let q(x) = q
i
if x T
i
the PWQ function where the closures

T
i
of
T
i
are polyhedra and q
i
strictly convex quadratic functions. The multi-parametric
program to solve is
J
(x) = min
_
_
min
u
l(x, u) +q
1
(f(x, u)),
subj. to f(x, u)

T
1
f(x, u) T
.
.
.,
min
u
l(x, u) +q
r
(f(x, u))
subj. to f(x, u)

T
r
f(x, u) T
_
_
. (18.47)
The proof follows the lines to the proof of the previous theorem with the excep-
tion that the constraints of the i-th mp-QP subproblem dier from the j-th mp-QP
subproblem, i ,= j.
The lower-semicontinuity assumption on q(x) allows one to use the closure of the
sets T
i
in (18.47).The cost function in problem (18.39) is lower-semicontinuous since
it is a composition of a lower-semicontinuous function and a continuous function.
Then, since the domain is compact, problem (18.47) admits a minimum. Therefore
for a given x, there exists one mp-QP in problem (18.47) which yields the optimal
solution. The procedure based on solving mp-QPs and storing the results as in
Example 18.4 will be the same as in Lemma 18.3 but the domain 1 =
N
R
j=1
1
j
of
the solution can be a non-Euclidean polyhedron.
If f is PPWA dened over s regions then we have a s to X problem where X
can belong to any of the problems listed above. In particular, we have an s to
r problem of multiplicity d if f is PPWA and dened over s regions and q is a
multiple PPWQ function of multiplicity d, dened over r polyhedral regions. The
following lemma can be proven along the lines of the proofs given before.
Lemma 18.5 Problem (18.39) where f is linear and q is a lower-semicontinuous
PPWQ function of multiplicity d, dened over r polyhedral regions and strictly
convex on each polyhedron, is a one to r problem of multiplicity d and can be solved
by r d mp-QPs.
An s to r problem of multiplicity d can be decomposed into s one to r problems of
multiplicity d. An s to one problem can be decomposed into s one to one problems.
18.6.3 Algorithmic Solution of the Bellman Equations
In the following we will substitute the PWA system equations (18.2) with the
shorter form
x(k + 1) =

f
PWA
(x(k), u(k)) (18.48)
where

f
PWA
:

c R
n
and

f
PWA
(x, u) = A
i
x +B
i
u +f
i
if [
x
u
]

c
i
, i = 1, . . . , s,
and
_
c
i
_
is a polyhedral partition of

c.
Consider the dynamic programming formulation of the CFTOC problem (18.6)-
(18.7),
J
j
(x(j)) = min
uj
x
j
Qx
j
+u
j
Ru
j
+J
j+1
(

f
PWA
(x(j), u
j
))
(18.49)
subj. to

f
PWA
(x(j), u
j
) A
j+1
(18.50)
for j = N 1, . . . , 0, with terminal conditions
A
N
= A
f
(18.51)
J
N
(x) = x
Px, (18.52)
where A
j
is the set of all states x(j) for which problem (18.49)(18.50) is feasible:
A
j
= x R
n
[ u,

f
PWA
(x, u) A
j+1
. (18.53)
Equations (18.49)-(18.53) are the discrete-time version of the well known Hamilton-
Jacobi-Bellman equations for continuous-time optimal control problems.
Assume for the moment that there are no binary inputs and binary states,
m
= n
= 0. The Bellman equations (18.49)(18.52) can be solved backwards in

time by using a multi-parametric quadratic programming solver and the results of
the previous section.
Consider the rst step of the dynamic program (18.49)(18.52)
J
N1
(x
N1
) = min
{uN1}
x
N1
Qx
N1
+u
N1
Ru
N1
+J
N
(

f
PWA
(x
N1
, u
N1
))
(18.54)
subj. to

f
PWA
(x
N1
, u
N1
) A
f
. (18.55)
The cost to go function J
N
(x) in (18.54) is quadratic, the terminal region A
f
is
a polyhedron and the constraints are piecewise ane. Problem (18.54)(18.55) is
an s to one problem that can be solved by solving s mp-QPs From the second
step j = N 2 to the last one j = 0 the cost to go function J
j+1
(x) is a lower-
semicontinuous PPWQ with a certain multiplicity d
j+1
, the terminal region A
j+1
is a polyhedron (in general non-Euclidean) and the constraints are piecewise ane.
Therefore, problem (18.49)(18.52) is an s to N
r
j+1
problem with multiplicity d
j+1
(where N
r
j+1
is the number of polyhedra of the cost to go function J
j+1
), that
can be solved by solving sN
r
j+1
d
j+1
mp-QPs (Lemma 18.5). The resulting optimal
solution will have the form (18.9) considered in the ordered region sense.
In the presence of binary inputs the procedure can be repeated, with the dif-
ference that all the possible combinations of binary inputs must be enumerated.
Therefore a one to one problem becomes a 2
m
to one problem and so on. In the

presence of binary states the procedure can be repeated either by enumerating
them all or by solving a dynamic programming algorithm at time step k from a
relaxed state space to the set of binary states feasible at time k + 1.
Next we summarize the main steps of the dynamic programming algorithm
discussed in this section. We use boldface characters to denote sets of polyhedra,
i.e., R = 1
i
i=1,...,|R|
, where 1
i
is a polyhedron and [R[ is the cardinality of
the set R. Furthermore, when we say SOLVE an mp-QP we mean to compute and
store the triplet S
k,i,j
of expressions for the value function, the optimizer, and the
polyhedral partition of the feasible space.
Algorithm 18.1
input: CFTOC problem (18.3)(18.7)
output: Solution (18.9) in the ordered region sense
let RN = |.
f
let J
N,1
(x) = x
Px
for k = N 1, . . . , 1,
for i = 1, . . . , [R
k+1
[,
for j = 1, . . . , s,
let S
k,i,j
= |
solve the mp-QP
S
k,i,j
minu
k
x
k
Qx
k
+u
k
Ru
k
+J
k+1,i
(Ajx
k
+Bju
k
+fj)
subj. to
_
Ajx
k
+Bju
k
+fj 1
k+1,i
[
x
k
u
k
]

c
j
.
end
end
let R
k
= |1
k,i,j,l
i,j,l
. Denote by 1
k,h
its elements, and by J
k,h
and u
k,h
(x) the
associated costs and optimizers, with h |1, . . . , [R
k
[
keep only triplets (J
k,h
(x), u
k,h
(x), 1
k,h
) for which x 1
k,h
: x / 1
k,d
, d ,= h
OR x 1
k,h
: J
k,h
(x) < J
k,d
(x), d ,= h
create multiplicity information and additional regions for an ordered region solution
as explained in Example 18.4
end
In Algorithm 18.1, the structure S
k,i,j
stores the matrices dening quadratic
function J
k,i,j,l
(), ane function u
k,i,j,l
(), and polyhedra 1
k,i,j,l
, for all l:
S
k,i,j
=
_
l
__
J
k,i,j,l
(x), u
k,i,j,l
(x), 1
k,i,j,l
__
, (18.56)
where the indices in (18.56) have the following meaning: k is the time step, i
indexes the piece of the cost-to-go function that the DP algorithm is considering,
j indexes the piece of the PWA dynamics the DP algorithm is considering, and l
indexes the polyhedron in the mp-QP solution of the (k, i, j)th mp-QP problem.
The KEEP only triplets Step of Algorithm 18.1 aims at discarding regions
1
k,h
that are completely covered by some other regions that have lower cost. Obvi-
ously, if there are some parts of the region 1
k,h
that are not covered at all by other
regions (rst condition) we need to keep them. Note that comparing the cost func-
tions is, in general, non-convex optimization problem. One might consider solving
the problem exactly, but since algorithm works even if some removable regions are
kept, we usually formulate LMI relaxation of the problem at hand.
The output of Algorithm 18.1 is the state feedback control law (18.9) considered
in the ordered region sense. The online implementation of the control law requires
simply the evaluation of the PWA controller (18.9) in the ordered region sense.
18.6.4 Examples
Example 18.5 Consider the control problem of steering the piecewise ane sys-
tem (18.29) to a small region around the origin
3
. The constraints and cost function
of Example (18.2) are used. The state feedback solution u
0
(x(0)) was determined
by using the approach presented in the previous section. When the innity norm
is used in the cost function, the feasible state-space .0 at time 0 is divided into 84
polyhedral regions and it is depicted in Fig. 18.6(a). When the squared Euclidean
norm is used in the cost function, the feasible state-space .0 at time 0 is divided into
92 polyhedral regions and is depicted in Fig. 18.6(b).
x
1
x
2
(a) Partition of the feasible state-
space .0 when the innity norm
is used in the cost function
(N
r
3 = 84)
x
1
x
2
(b) Partition of the feasible state-
space .0 when the squared Eu-
clidian norm is used in the cost
function (N
r
3 = 92)
Figure 18.6 Example 18.5: state space control partition for u
0
(x(0))
Note that as explained in Section 18.6.2 for the case of the squared Euclidian norm,
the optimal control law is stored in a special data structure where:
1. The ordering of the regions is important.
2. The polyhedra can overlap.
3. The polyhedra can have an associated value function of multiplicity d > 1.
Thus, d quadratic functions have to be compared on-line in order to compute
the optimal control action.
Example 18.6 Consider the hybrid spring-mass-damper system described in Exam-
3
ple 17.2
4
x(t+1) =
_
_
_
0.90 0.02
0.02 0.00
_
x(t) + [
0.10
0.02
] u1(t) +
_
0.01
0.02
_
if x1(t) 1, u2(t) 0.5
_
0.90 0.02
0.06 0.00
_
x(t) + [
0.10
0.02
] u1(t) +
_
0.07
0.15
_
if x1(t) 1 +, u2(t) 0.5
_
0.90 0.38
0.38 0.52
_
x(t) + [
0.10
0.38
] u1(t) +
_
0.10
0.38
_
if x1(t) 1, u2(t) 0.5
_
0.90 0.35
1.04 0.35
_
x(t) + [
0.10
0.35
] u1(t) +
_
0.75
2.60
_
if x(t) 1 +, u2(t) 0.5
(18.57)
x(t) [5, 5] [5, 5]
u(t) [1, 1].
(18.58)
We solve the nite time constrained optimal control problem (18.7) with cost (18.6)
with N = 3, P = Q = [
1 0
0 1
], R = [
0.2 0
0 1
]. The state feedback solution was de-
termined by using the approach presented in the previous section in two cases:
without terminal constraint (Figure 18.7(a)) and with terminal constraint .
f
=
[0.01 0.01] [0.01 0.01] (Figure 18.7(b)).
x
1
x
2
(a) Partition with no terminal
constraint (N
r
3 = 183).
x
1
x
2
(b) Partition with terminal con-
straint (N
r
3 = 181).
Figure 18.7 Example 18.6: state space optimal control partition
18.7 Discontinuous PWA systems
Without Assumption 18.1 the optimal control problem (18.3)-(18.7) may be feasible
but may not admit an optimizer for some x(0) (the problem in this case would be
to nd an inmum rather than the minimum).
4
18.8 Receding Horizon Control 409
Under the assumption that the optimizer exists for all states x(k), the ap-
proach explained in the previous sections can be applied to discontinuous systems
by considering three elements. First, the PWA system (18.2) has to be dened on
each polyhedron of its domain and all its lower dimensional facets. Secondly, dy-
namic programming has to be performed from and to any lower dimensional
facet of each polyhedron of the PWA domain. Finally, value functions are not
lower-semicontinuous, which implies that Lemma 18.4 cannot by used. Therefore,
when considering the closure of polyhedral domains in multi-parametric program-
ming (18.47), a post-processing is necessary in order to remove multi-parametric
optimal solutions which do not belong to the original set but only to its closure.
The tedious details of the dynamic programming algorithm for discontinuous PWA
systems are not included here but can be immediately extracted from the results
of the previous sections.
In practice, the approach just described for discontinuous PWA systems can
easily be numerically prohibitive. The simplest approach from a practical point of
view is to introduce gaps between the boundaries of any two polyhedra belonging
to the PWA domain (or, equivalently, to shrink by a quantity the size of every
polyhedron of the original PWA system). This way, one deals with PWA systems
dened over a disconnected union of closed polyhedra. By doing so, one can use
the approach discussed in this chapter for continuous PWA systems. However, the
optimal controller will not be dened at the points in the gaps. Also, the computed
solution might be arbitrarily dierent from the original solution to problem (18.3)-
(18.7) at any feasible point x. Despite this, if the dimension of the gaps is
close to the machine precision and comparable to sensor/estimation errors, such an
approach can be very appealing in practice. To the best of our knowledge in some
cases this approach is the only one that is computationally tractable for computing
controllers for discontinuous hybrid systems fullling state and input constraints
that are implementable in real-time.
Without Assumption 18.1, problem (18.3)-(18.7) is well dened only if an op-
timizer exists for all x(0). In general, this is not easy to check. The dynamic
programming algorithm described here could be used for such a test but the de-
tails are not included in this book.
18.8 Receding Horizon Control
Consider the problem of regulating to the origin the PWA system (18.2). Receding
Horizon Control (RHC) can be used to solve such a constrained regulation problem.
The control algorithm is identical to the one outlined in Chapter 13 for linear
systems. Assume that a full measurement of the state x(t) is available at the
current time t. Then, the nite time optimal control problem
J
t
(x(t)) = min
U
tt+N|t
J
t
(x(t), U
tt+N|t
) (18.59a)
subj. to
_
_
_
x
t+k+1|t
= A
i
x
t+k|t
+B
i
u
t+k|t
+f
i
if
_
x
t+k|t
u
t+k|t

c
i
x
t+N|t
A
f
x
t|t
= x(t)
(18.59b)
is solved at each time t, where U
tt+N|t
= u
t|t
, . . . , u
t+N1|t
.
Let U
t
= u
t|t
, . . . , u
t+N1|t
be the optimal solution of (18.59) at time t. Then,
the rst sample of U
t
is applied to system (18.2):
u(t) = u
t|t
. (18.60)
The optimization (18.59) is repeated at time t+1, based on the new state x
t+1|t+1
=
x(t + 1), yielding a moving or receding horizon control strategy.
Based on the results of previous sections the state feedback receding horizon
controller (18.59)(18.60) can be immediately obtained in two ways: (i) solve the
MIQP/MILP (18.24) for x
t|t
= x(t) or (ii) by setting
u(t) = f
0
(x(t)), (18.61)
where f
0
: R
n
R
nu
is the piecewise ane solution to the CFTOC (18.59)
computed as explained in Section 18.6. Clearly, the explicit form (18.61) has the
advantage of being easier to implement, and provides insight on the type of action
of the controller in dierent regions of the state space.
18.8.1 Stability and Feasibility Issues
As discussed in Chapter 13 the feasibility and stability of the receding horizon
controller (18.59)(18.60) is, in general, not guaranteed.
Theorem 13.2 in Chapter 13 can be immediately modied for the hybrid case:
persistent feasibility and Lyapunov stability are guaranteed if the terminal con-
straint A
f
is a control invariant set (assumption (A2) in Theorem 13.2) and the
terminal cost p(x
N
) is a control Lyapunov function (assumption (A3) in Theo-
rem 13.2). In the hybrid case the computation of control invariant sets and control
Lyapunov functions is computationally more involved. As in the linear case, A
f
= 0
satises the aforementioned properties and it thus represents a very simple, yet re-
strictive, choice for guaranteeing persistent feasibility and Lyapunov stability.
18.8.2 Examples
Example 18.7 Consider the problem of regulating the piecewise ane system (18.29)
to the origin
5
. The nite time constrained optimal control (18.7) is solved with
5
N = 3, P = Q = [
1 0
0 1
], R = 1, and .
f
= [0.01 0.01] [0.01 0.01]. Its state
feedback solution (18.9) u
(x(0)) = f
0
(x(0)) at time 0 is implemented in a receding
horizon fashion, i.e. u(x(k)) = f
0
(x(k)).
The state feedback control law with cost (18.5) with p = has been computed in
Example 18.5 and it consists of 84 polyhedral regions. None of them has multiplicity
higher than 1. Figure 18.8 shows the corresponding closed-loop trajectories starting
from the initial state x(0) = [2 2]
.
The state feedback control law with cost (18.6) has been computed in Example 18.5
and it consists of 92 polyhedral regions, some of them have multiplicity higher than 1.
Figure 18.9 shows the corresponding closed-loop trajectories starting from the initial
state x(0) = [2 2]
.
time
x
1
a
n
d
x
2
(a) State trajectories (x1 dashed
line and x2 solid line).
time
u
(b) Optimal input.
Figure 18.8 Example 18.7: MPC control of system (18.29) when the innity
norm is used in the objective.
time
x
1
a
n
d
x
2
time
u
(b) Optimal input.
Figure 18.9 Example 18.7: MPC control of system (18.29) when the squared
Euclidian norm is used in the objective.
Example 18.8 Consider the problem of regulating the hybrid spring-mass-damper
system (18.57) described in Examples 17.2 and 18.6 to the origin
6
. The nite time
constrained optimal control problem (18.7) with cost (18.6) is solved with N = 3,
P = Q = [
1 0
0 1
], R = [
0.2 0
0 1
]. Its state feedback solution (18.9) u
(x(0)) = f
0
(x(0)) at
time 0 is implemented in a receding horizon fashion, i.e. u(x(k)) = f
0
(x(k)).
The state feedback solution was determined in Example 18.6 for the case of no termi-
nal constraint (Figure 18.7(a)). Figure 18.10 depicts the corresponding closed-loop
trajectories starting from the initial state x(0) = [3 4]
.
The state feedback solution was determined in Example 18.6 for terminal constraint
.
f
= [0.01, 0.01] [0.01, 0.01] (Figure 18.7(b)). Figure 18.11 depicts the corre-
sponding closed-loop trajectories starting from the initial state x(0) = [3 4]
.
Comparing the two controllers, we nd that the terminal constraint leads to a more
aggressive behaviour. In addition, in the case with terminal constraint we observe the
coordination of the binary and continuous inputs. The small damping coecient is
switched on to increase the eect of the continuous input. The damping is increased
again once the steady-state value is approached.
time
x
1
a
n
d
x
2
time
u
1
a
n
d
u
2
(b) Optimal inputs (u1 solid line
and the binary input u2 with a
dashed line)
Figure 18.10 Example 18.8: MPC control of system (18.57) without terminal
constraint
6
time
x
1
a
n
d
x
2
time
u
1
a
n
d
u
2
(b) Optimal inputs (u1 solid line
and the binary input u2 with a
dashed line)
Figure 18.11 Example 18.8: MPC control of system (18.57) with terminal
constraint
List of Figures
2.1 Constraints, feasible set and gradients of Example 2.1. . . . . . . . . . . 18
2.2 Graphical interpretation of KKT conditions [26]. . . . . . . . . . . . . . 19
3.1 Polytope. The planes (here lines) dening the halfspaces are a
i
x bi = 0. 22
3.2 Linear Program Graphical Interpretation, ki < ki1. . . . . . . . . . . . 23
3.3 Primal Degeneracy in a Linear Program. . . . . . . . . . . . . . . . . . 26
3.4 Example 3.1 LP with Primal and Dual Degeneracy. The vectors max and
min point in the direction for which the objective improves. The feasible
regions are shaded. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
3.5 Convex PWA Function Described as the Max of Ane Functions. . . . . 29
3.6 Graphical Interpretation of the Quadratic Program Solution . . . . . . . 30
4.1 Upper and lower bounds on functions: The quadratic upper bound stems
from L-smoothness of f (cf. Denition 4.2), while the quadratic lower
bound is obtained from strong convexity of f (cf. Theorem 4.1). In this
example, f(z) = z
2
+
1
2
z + 1 and we have chosen L = 3 and = 1 for
illustration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
4.2 First 10 iterations of the gradient method with constant step size 1/L
using the tight, a slightly underestimated and a strongly underestimated
Lipschitz constant L for Example 4.1. Also shown are the iterates of the
fast gradient method using the tight Lipschitz constant. The contour lines
indicate the function value of each iterate. Note that the gradients are
orthogonal to the contour lines. . . . . . . . . . . . . . . . . . . . . . . 42
4.3 Classic gradient method (left) and fast gradient method (right) for
the constrained case. After computing the standard update rule
from the unconstrained case, a projection onto the feasible set is
applied. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
4.4 Illustration of interior point method for example problem (4.34) . . . 57
4.5 Ance-scaling (pure Newton) and centering search directions of classic
primal-dual methods. The Newton direction makes only progress towards
the solution when it is computed from a point close to the central path,
hence centering must be added to the pure Newton step to ensure that
the iterates stay suciently close to the central path. . . . . . . . . . . . 61
416 LIST OF FIGURES
4.6 Search direction generation in predictor-corrector methods. The ane
search direction can be used to correct for linearization errors, which re-
sults in better performance in practice due to closer tracking of the central
path. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
5.1 Illustration of the convex hull of three points x
1
= [1, 1]
, x
1
= [1, 0]
,
x
3
= [0, 1]
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
5.2 Illustration of a cone spanned by the three points x
1
= [1, 1, 1]
, x
2
=
[1, 2, 1]
, x
3
= [1, 1, 2]
. . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
5.3 -polyhedron. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
5.4 -polytope. The planes (here lines) dening the halfspaces are a
i
xbi =
0. The arrows represent the direction of ai into the valid halfspace. . . . 72
5.5 Illustration of a polytope in 1- and 1- representation. . . . . . . . . 74
5.6 Illustration of the convex hull operation. Download code. . . . . . . 79
5.7 Illustration of the envelope operation. Download code. . . . . . . . . 79
5.8 Illustration of the Chebychev ball contained in a polytope T. Download
code. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
5.9 Illustration of the projection of a 3-dimensional polytope T onto the plane
x3 = 0. Download code. . . . . . . . . . . . . . . . . . . . . . . . . . . 82
5.10 Two dimensional example: partition of the rest of the space }1
0
. . 83
5.11 Illustration of the Pontryagin dierence and Minkowski sum opera-
tions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
5.12 Illustration that (T Q) Q T. Download code. . . . . . . . . . 86
5.13 Graphical illustration of Algorithm 5.3. Download code. . . . . . . . 90
6.1 Example 6.1: Optimizer z
(x) as a function of
the parameter x. Download code. . . . . . . . . . . . . . . . . . . . . . 97
6.2 Example 6.3: Solution. . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
6.3 Example 6.4: problem (6.10). (a)-(c) Projections of the point-to-set map
R(x) for three values of the parameter x: x1 < x2 < x3; (d) Value function
J
(x). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
6.4 Example 6.5: Open and not closed point-to-set map R(x). . . . . . . . . 102
6.5 Example 6.6: Closed and not open point-to-set map R(x). . . . . . . . . 102
6.6 Example 6.8: Solution. . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
the parameter x. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
7.2 Polyhedral partition of the parameter space corresponding to the solution
of Example 7.3. Download code. . . . . . . . . . . . . . . . . . . . . . . 117
7.3 Example 7.4: Polyhedral partition of the parameter space corresponding
to the solution. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
7.4 Example 7.4: A possible sub-partitioning of the degenerate region CR2
where the regions

CR
{2,5}
(Fig. 7.4(a)),

CR
{2,4}
(Fig. 7.4(b)) and
CR
{2,1}
(Fig. 7.5(a)) are overlapping. Note that below each picture
the feasible set and the level set of the value function in the z-space is
depicted for a particular choice of the parameter x indicated by a point
marked with . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
LIST OF FIGURES 417
7.5 Example 7.4: A possible sub-partitioning of the degenerate region CR2
where the regions

CR
{2,5}
(Fig. 7.4(a)),

CR
{2,4}
(Fig. 7.4(b)) and
CR
{2,1}
(Fig. 7.5(a)) are overlapping. Note that below each picture
the feasible set and the level set of the value function in the z-space is
depicted for a particular choice of the parameter x indicated by a point
marked with . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
7.6 A possible solution to Example 7.4 where the regions

CR
{2,5}
and

CR
{2,3}
are non-overlapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
7.7 A possible solution for Example 7.4 where

CR
{2,4}
is obtained by inter-
secting

CR
{2,4}

CR
{2,5}
, and

CR
{2,1}
by inter-
secting

CR
{2,1}

CR
{2,5}
and

CR
{2,4}
. . . . . . 123
7.8 Example of a critical region explored twice . . . . . . . . . . . . . . . . 124
the parameter x. Download code. . . . . . . . . . . . . . . . . . . . . . 128
7.10 Polyhedral partition of the parameter space corresponding to the solution
of Example 7.6. Download code. . . . . . . . . . . . . . . . . . . . . . . 132
7.11 Solution to the mp-MIQP (7.82). . . . . . . . . . . . . . . . . . . . . . 144
8.1 Lyapunov function (8.41). . . . . . . . . . . . . . . . . . . . . . . . . . 161
10.1 Example 10.1, -norm objective function, -horizon controller. Piece-
wise linear optimal cost (value function) and corresponding polyhedral
partition. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181
11.1 Example 11.1: one-step controllable set Pre(.) . for system (11.8)
under constraints (11.9). Download code. . . . . . . . . . . . . . . . 188
11.2 Example 11.1: one-step reachable set for system (11.8). Download code. . 189
11.3 Example 11.2: one-step controllable set Pre(.) . for system (11.15)
under constraints (11.16). Download code. . . . . . . . . . . . . . . . . . 190
11.4 Example 11.2: one-step reachable sets for system (11.15) under constraints
(11.16). Download code. . . . . . . . . . . . . . . . . . . . . . . . . . . 191
11.5 Example 11.3: one-step controllable sets /j(S) for system (11.15) under
constraints (11.16) for N=1,2,3,4. Note that the sets are shifted along the
x-axis for a clearer visualization. Download code. . . . . . . . . . . . . . 192
11.6 Example 11.3: N-step reachable sets for system (11.15) under constraints
(11.16). Download code. . . . . . . . . . . . . . . . . . . . . . . . . . . 192
11.7 Maximal Positive Invariant Set of system (11.8) under constraints (11.9). Down-
load code. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195
11.8 Maximal Control Invariant Set of system (11.15) subject to constraints (11.16). Down-
load code. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196
11.9 Example 11.8: one-step robust controllable set Pre(., V) . for system
(11.32) under constraints (11.33)-(11.34). Download code. . . . . . . . . . 202
11.10Example 11.8: one-step robust reachable set for system (11.32) under
constraints (11.34). Download code. . . . . . . . . . . . . . . . . . . . . 203
11.11Example 11.9: one-step robust controllable set Pre(., V) . for system
(11.42) under constraints (11.43)-(11.44). Download code. . . . . . . . . . 204
11.12Example 11.9: one-step robust reachable set for system (11.42) under
constraints (11.43)-(11.44). Download code. . . . . . . . . . . . . . . . . 205
418 LIST OF FIGURES
11.13Example 11.10: one-step robust controllable set Pre(., V). for system
(11.53) under constraints (11.54). Download code. . . . . . . . . . . . . . 207
11.14Example 11.10: one-step robust reachable set Pre(., V) . for system
(11.60) under constraints (11.61). Download code. . . . . . . . . . . . . . 208
11.15Maximal Robust Positive Invariant Set of system (11.65) subject to con-
straints (11.66). Download code. . . . . . . . . . . . . . . . . . . . . . . 210
11.16Maximal Robust Control Invariant Set of system (11.42) subject to con-
straints (11.43). Download code. . . . . . . . . . . . . . . . . . . . . . . 212
12.1 Example 12.1: Propagation of the feasible sets .i to arrive at c (shaded). Down-
load code. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221
12.2 CLQR Algorithm: Region Exploration. . . . . . . . . . . . . . . . . . . 229
12.3 Example 12.2: Double Integrator, 2-norm objective function, horizon N =
6. Partition of the state space for the time-varying optimal control law.
Polyhedra with the same control law were merged. Download code. . . . . 244
12.4 Example 12.3: Double Integrator, 2-norm objective function, horizon N =
. Partition of the state space for the time invariant optimal control
law. Download code. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245
12.5 Example 12.4: Double Integrator, -norm objective function, horizon
N = 6. Partition of the state space for the time-varying optimal control
law. Polyhedra with the same control law were merged. Download code. . 246
12.6 Example 12.5: Double Integrator, -norm objective function, horizon
N = . Partition of the state space for the time invariant optimal control
law. Download code. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247
12.7 Example 12.6: Double Integrator, minimum-time objective function. Par-
tition of the state space and control law. Maximal LQR invariant set O
LQR
is the central shaded region in (a). Download code. . . . . . . . . . . . . 247

13.1 Receding Horizon Idea. . . . . . . . . . . . . . . . . . . . . . . . . . . . 249
13.2 Example 13.1. Closed-loop trajectories for the initial state x(0)=[-4.5,2]
(boxes) and x(0)=[-4.5,3] (circles). Download code. . . . . . . . . . . . . 254
13.3 Double integrator Example (13.1). Download code. . . . . . . . . . . . . 255
13.4 Example 13.2. Closed-loop trajectories for dierent settings of horizon N
and weight R. Boxes (Circles) are initial points leading (not leading) to
feasible closed-loop trajectories. Download code. . . . . . . . . . . . . . 256
13.5 Example 13.2. Maximal positive invariant sets O for dierent parameter
settings: Setting 1 (origin), Setting 2 (dark-gray) and Setting 3 (gray and
dark-gray). Also depicted is the maximal control invariant set c (white
and gray and dark-gray). Download code. . . . . . . . . . . . . . . . . . 257
13.6 Example (13.3): 2-norm objective function, receding horizon controller.
Horizon N, terminal region .
f
. Download code. . . . . . . . . . . . . . . 265
13.7 Double integrator Example (13.4). Download code. . . . . . . . . . . . . 267
14.1 Examples of interpolation. . . . . . . . . . . . . . . . . . . . . . . . . . 288
14.2 Illustration of the Convex-Hull Property of Barycentric Interpolation. The
rst image shows a number of samples of a function, the second the convex
hull of these sample points and the third, a barycentric interpolation that
lies within this convex hull. . . . . . . . . . . . . . . . . . . . . . . . . 289
LIST OF FIGURES 419
14.3 Illustration of the relationship between the optimal value function J
(x),
the value function evaluated for the sub-optimal function z(x) and the
convex hull of the optimal value at the sample points. . . . . . . . . . . . 291
14.4 Polyhedron T constructed to be an outer approximation of epi J
,
and
an inner approximation of epi J
. . . . . . . . . . . . . . . . . . . . . . 294
14.5 Example of one iteration of the implicit double-description algorithm.
Square: Point vmax at which the worst-case error occurs. Circle: Point
z
(vmax); Projection of vmax onto the set. Red line: Separating hyper-
plane. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 295
14.6 Several steps of the Double Description algorithm with the termination
criterion extreme(P) epi J
(x) satised in the last step. . . . . . . . . . 300

14.7 Plot of the second order interpolant in 2D centered at zero, with a width
of one. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 301
14.8 Example 14.1: Optimal cost J
(x) (light gray), and stability function

J
(x) +(x) (gray). . . . . . . . . . . . . . . . . . . . . . . . . . . . . 301

14.9 Example 14.1: Error function V (top). Points marked are the vertices
with worst-case t. Partition (triangulation) of the sampled points (mid-
dle). Upper-bound on approximate value function (convex hull of sample
points) (bottom). Download code. . . . . . . . . . . . . . . . . . . . 302
14.10Example 14.1: Approximate control law using triangulation. . . . . . . . 305
14.11Example 14.1: Error function V (top). Points marked are the vertices
with worst-case t. Partition (triangulation) of the sampled points (mid-
dle). Upper-bound on approximate value function (convex hull of sample
points) (bottom). Download code. . . . . . . . . . . . . . . . . . . . 307
14.12Example 14.1: Partition generated by the second-order interpolant method.
Download code. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 308
14.13Example 14.1: Approximate control law using second-order interpolants. . 308
14.14Example 14.1: Approximate value function

J(x) using second-order in-
terpolants. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 308
15.1 Example for Algorithm 15.3 in one dimension: For a given point x T3
(x = 5) we have J
(x) = max(T
1
x +V
1
, . . . , T
4
x +V
4
). . . . . . . . . . 312
15.2 Example for Algorithm 15.4 in one dimension: For a given point x T2
(x = 4) we have O2(x) = [1 1]
= S2, while O1(x) = 1 ,= S1 = 1,

O3(x) = [1 1]
,= S3 = [1 1]
, O4(x) = 1 ,= S4 = 1. . . . . . . . . 314
15.3 Illustration for Algorithm 15.5 in two dimensions. . . . . . . . . . . . . . 317
16.1 Batch Approach with Closed-Loop Prediction. State propagation and
associated input variables. The disturbance set has two vertices w
1
and
w
2
(nW
a = 2) and the horizon is N = 4. The continuous line represents
the occurrence of w
1
, while the dashed line represents the occurrence of
w
2
. Over the horizon there are eight possible scenarios associated to eight
extreme disturbance realizations. The state x
j
3
denotes the predicted state
at time 3 associated to the j-th disturbance scenario. . . . . . . . . . . . 336
16.2 Example (16.4). Polyhedral partition of the state-space corresponding to
the explicit solution of CROC-OL and CROC-CL at time t = 0. . . . . . 351
420 LIST OF FIGURES
16.3 Example (16.4). (Left) Polyhedral partition of the state-space correspond-
ing to the explicit solution of CROC-Parametrized at time t = 0. (Right)
Sets A
OL
0
, A
0
and A
Mv
0
corresponding to the controllers CROC-OL,
CROC-CL and CROC-Parameterized, respectively. . . . . . . . . . . 352
17.1 Hybrid systems. Logic-based discrete dynamics and continuous dynamics
interact through events and mode switches . . . . . . . . . . . . . . . . 358
17.2 Piecewise ane (PWA) systems. Mode switches are triggered by linear
threshold events. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 359
17.3 Example 17.1: Free response of state x1 for x(0) = [1 0]. Download code. . 361
17.4 Example 17.2: Open-loop simulation of system (17.6) for u1 = 3 and zero
initial conditions. Download code. . . . . . . . . . . . . . . . . . . . . . 363
17.5 Example 17.3: Open-loop simulation of system (17.10) for dierent se-
quences of the input u. Download code. . . . . . . . . . . . . . . . . . . 364
17.6 A discrete hybrid automaton (DHA) is the connection of a nite
state machine (FSM) and a switched ane system (SAS), through a
mode selector (MS) and an event generator (EG). The output signals
are omitted for clarity . . . . . . . . . . . . . . . . . . . . . . . . . . 365
17.7 Example of nite state machine . . . . . . . . . . . . . . . . . . . . . 367
18.1 Graphical illustration of the main steps for the proof of Theorem 18.1
when 3 polyhedra intersect. Step 1: the three intersecting polyhedra are
partitioned into: one polyhedron of triple feasibility (1,2,3), 2 polyhe-
dra of double feasibility (1, 2) and (1, 3), 3 polyhedra of single feasibility
(1),(2),(3). The sets (1), (2) and (1,2) are neither open nor closed poly-
hedra. Step 2: the single feasibility sets of (1), (2) and (3) are partitioned
into six polyhedra. Step 3: value functions are compared inside the poly-
hedra of multiple feasibility. . . . . . . . . . . . . . . . . . . . . . . . . 388
18.2 Example 18.1: Problem (18.12) with cost (18.6) is solved for dierent vi,
i = 1, . . . , 4. Control law u
0
(x0) is PPWA. State space partition is shown. 392
18.3 Example 18.1: State-space partition corresponding to the optimal control
law of . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 393
18.4 Illustration of Branch and Bound procedure. . . . . . . . . . . . . . . . 397
18.5 Illustration of the ordered region single valued function. . . . . . . . . . 400
18.6 Example 18.5: state space control partition for u
0
(x(0)) . . . . . . . . . 407
18.7 Example 18.6: state space optimal control partition . . . . . . . . . . . . 408
18.8 Example 18.7: MPC control of system (18.29) when the innity norm
is used in the objective. . . . . . . . . . . . . . . . . . . . . . . . . . 411
18.9 Example 18.7: MPC control of system (18.29) when the squared Eu-
clidian norm is used in the objective. . . . . . . . . . . . . . . . . . . 411
18.10Example 18.8: MPC control of system (18.57) without terminal constraint412
18.11Example 18.8: MPC control of system (18.57) with terminal constraint 413
References
[1] L. Chisci A. Bemporad and E. Mosca. On the stabilizing property of
SIORHC. Automatica, 30(12):20132015, 1994.
[2] J. Acevedo and E.N. Pistikopoulos. A multiparametric programming ap-
proach for linear process engineering problems under uncertainty. Ind. Eng.
Chem. Res., 36:717728, 1997.
[3] I. Adler and R.D.C. Monteiro. A geometric view of parametric linear pro-
gramming. Algorithmica, 8(2):161176, 1992.
[4] A. Alessio, M. Lazar, A. Bemporad, and W.P.M.H. Heemels. Squaring the
circle: An algorithm for obtaining polyhedral invariant sets from ellipsoidal
ones. Automatica, 43(12):20962103.
[5] Alessandro Alessio and Alberto Bemporad. A survey on explicit model predic-
tive control. In Lalo Magni, DavideMartino Raimondo, and Frank Allg
Awer,
editors, Nonlinear Model Predictive Control, volume 384 of Lecture Notes in
Control and Information Sciences, pages 345369. Springer Berlin Heidel-
berg, 2009.
[6] J.C. Allwright and G.C. Papavasiliou. On linear programming and robust
model-predictive control using impulse-responses. Systems & Control Letters,
18:159164, 1992.
[7] R. Alur, C. Belta, F. Ivancic, V. Kumar, M. Mintz, G.J. Pappas, H. Rubin,
and J. Schug. Hybrid modeling and simulation of biomolecular networks.
In M.D. Di Benedetto and A. Sangiovanni Vincentelli, editors, Hybrid Sys-
tems: Computation and Control, volume 2034 of Lecture Notes in Computer
Science, pages 1933. Springer-Verlag, 2001.
[8] R. Alur, C. Courcoubetis, T.A. Henzinger, and P.H. Ho. Hybrid automata:
An algorithmic approach to the specication and verication of hybrid sys-
tems. In A.P. Ravn R.L. Grossman, A. Nerode and H. Rischel, editors, Hybrid
Systems, volume 736 of Lecture Notes in Computer Science, pages 209229.
Springer Verlag, 1993.
[9] P.J. Antsaklis. A brief introduction to the theory and applications of hy-
brid systems. Proc. IEEE, Special Issue on Hybrid Systems: Theory and
Applications, 88(7):879886, July 2000.
422 References
[10] Larry Armijo. Minimization of functions having lipschitz continuous rst
partial derivatives. Pacic Journal of mathematics, 16(1):13, 1966.
[11] J.P. Aubin. Viability theory. Systems & Control: Foundations & Applica-
tions. Birkhauser, 1991.
[12] D. Avis. lrs: A revised implementation of the reverse search vertex enumera-
tion algorithm. Polytopes, Combinatorics and Computation, pages 177198,
2000.
[13] T.A. Badgwell and K.R. Muske. Disturbance model design for linear model
predictive control. In Proceedings of the American Control Conference, vol-
ume 2, pages 16211626, 2002.
[14] V.L. Bageshwar and F. Borrelli. On a property of a class of oset-free model
predictive controllers. Automatic Control, IEEE Transactions on, 54(3):663
669, march 2009.
[15] E. Balas. Projection with a minimum system of inequalities. Computational
Optimization and Applications, 10:189193, 1998.
[16] A. Balluchi, L. Benvenuti, M. Di Benedetto, C. Pinello, and A. Sangiovanni-
Vincentelli. Automotive engine control and hybrid systems: Challenges and
opportunities. Proc. IEEE, 88(7):888912, 2000.
[17] A. Balluchi, L. Benvenuti, M.D. Di Benedetto, and A. Sangiovanni-
Vincentelli. Design of observers for hybrid systems. In C.J. Tomlin and M.R.
Greenstreet, editors, Hybrid Systems: Computation and Control, volume
2289 of Lecture Notes in Computer Science, pages 7689. Springer-Verlag,
Berlin Heidelberg New York, 2002.
[18] B. Bank, J. Guddat, D. Klatte, B. Kummer, and K. Tammer. Non-Linear
Parametric Optimization. Akademie-Verlag, Berlin, 1982.
[19] M. Baotic. An ecient algorithm for multi-parametric quadratic program-
ming. Technical Report AUT02-04, Automatic Control Laboratory, ETH
Zurich, Switzerland, February 2002.
[20] M. Baotic. Gradient of the value function in parametric convex optimization
problems. Technical report, Faculty of Electrical Engineering and Computing.
University of Zagreb, Croatia, 2013. Technical Report.
[21] M. Baotic, F. Borrelli, A. Bemporad, and M. Morari. Ecient on-line com-
putation of constrained optimal control. SIAM Journal on Control and Op-
timization, 5:24702489, September 2008.
[22] M. Baotic and F.D. Torrisi. Polycover. Technical Report AUT03-11,
Automatic Control Laboratory, ETHZ, Switzerland, 2003. Available from
http://control.ee.ethz.ch/research/publications/publications.msql?
[23] M. Baric. Constrained Control - Computations, Performance and Robustness.
PhD thesis, ETH Zurich, Switzerland, October 2008.
References 423
[24] T. Basar and G.J. Olsder. Dynamic Noncooperative Game Theory, Second
Edition. Classics in Applied Mathematics. SIAM, Philadelphia, 1998.
[25] Farhad Bayat, Tor Arne Johansen, and Ali Akbar Jalali. Using hash tables to
manage the time-storage complexity in a point location problem: Application
to explicit model predictive control. Automatica, 47(3):571 577, 2011.
[26] M.S. Bazaraa, H.D. Sherali, and C.M. Shetty. Nonlinear Programming
Theory and Algorithms. John Wiley & Sons, Inc., New York, second edition,
1993.
[27] A.G. Beccuti, G. Papafotiou, R. Frasca, and M. Morari. Explicit hybrid
model predictive control of the DC-DC boost converter. In IEEE PESC,
Orlando, Florida, USA, June 2007.
[28] A. Bemporad. Reducing conservativeness in predictive control of constrained
systems with disturbances. In Proc. 37th IEEE Conf. on Decision and Con-
trol, pages 13841391, Tampa, Florida, USA, 1998.
[29] A. Bemporad. Ecient conversion of mixed logical dynamical systems into an
equivalent piecewise ane form. IEEE Transactions on Automatic Control,
49(5):832838, 2004.
[30] A. Bemporad, G. Bianchini, and F. Brogi. Passivity analysis and passication
of discrete-time hybrid systems. IEEE Transactions on Automatic Control,
54(4):10041009, 2008.
[31] A. Bemporad, F. Borrelli, and M. Morari. Piecewise linear robust model
predictive control. In Proc. European Control Conf., Porto, Portugal, October
2001.
[32] A. Bemporad and S. Di Cairano. Optimal control of discrete hybrid stochastic
automata. In M. Morari and L. Thiele, editors, Hybrid Systems: Computation
and Control, number 3414 in Lecture Notes in Computer Science, pages 151
167. Springer-Verlag, 2005.
[33] A. Bemporad, A. Casavola, and E. Mosca. Nonlinear control of constrained
linear systems via predictive reference management. IEEE Transactions on
Automatic Control, AC-42(3):340349, 1997.
[34] A. Bemporad, G. Ferrari-Trecate, and M. Morari. Observability and con-
trollability of piecewise ane and hybrid systems. IEEE Transactions on
Automatic Control, 45(10):18641876, 2000.
[35] A. Bemporad, K. Fukuda, and F.D. Torrisi. Convexity recognition of the
union of polyhedra. Computational Geometry, 18:141154, 2001.
[36] A. Bemporad, A. Garulli, S. Paoletti, and A. Vicino. A bounded-error ap-
proach to piecewise ane system identication. IEEE Transactions on Au-
tomatic Control, 50(10):15671580, October 2005.
424 References
[37] A. Bemporad, W.P.M.H. Heemels, and B. De Schutter. On hybrid systems
and closed-loop MPC systems. IEEE Transactions on Automatic Control,
47(5):863869, May 2002.
[38] A. Bemporad, D. Mignone, and M. Morari. Moving horizon estimation for
hybrid systems and fault detection. In Proc. American Contr. Conf., pages
24712475, Chicago, IL, June 1999.
[39] A. Bemporad and M. Morari. Control of systems integrating logic, dynamics,
and constraints. Automatica, 35(3):407427, March 1999.
[40] A. Bemporad and M. Morari. Robust model predictive control: A survey. In
A. Garulli, A. Tesi, and A. Vicino, editors, Robustness in Identication and
Control, number 245 in Lecture Notes in Control and Information Sciences,
pages 207226. Springer-Verlag, 1999.
[41] A. Bemporad, M. Morari, V. Dua, and E.N. Pistikopoulos. The explicit linear
quadratic regulator for constrained systems. Automatica, 38(1):320, 2002.
[42] A. Bemporad, M. Morari, V. Dua, and E.N. Pistikopoulos. The Explicit
Linear Quadratic Regulator for Constrained Systems. Automatica, 38(1):3
20, January 2002.
[43] A. Bemporad, F.D. Torrisi, and M. Morari. Discrete-time hybrid model-
ing and verication of the batch evaporator process benchmark. European
Journal of Control, 7(4):382399, 2001.
[44] Alberto Bemporad and Carlo Filippi. An algorithm for approximate multi-
parametric convex programming. Computational Optimization and Applica-
tions, V35(1):87108, 2006.
[45] A. Ben-Tal, A. Goryashko, E. Guslitzer, and A. Nemirovski. Adjustable
robust solutions of uncertain linear programs. Mathematical Programming,
99:351376, 2004.
[46] C. Berge. Topological Spaces. Dover Publications, inc., Mineola, New York,
1997.
[47] A.B. Berkelaar, K. Roos, and T. Terlaky. The optimal set and optimal
partition approach to linear and quadratic programming. In T. Gal and
H.J. Greenberg, editors, Advances in Sensitivity Analysis and Parametric
Programming, volume 6 of International Series in Operations Research &
Management Science, chapter 6. Kluwer Academic Publishers, 1997.
[48] Dimitri P Bertsekas. Nonlinear Programming. Athena Scientic, Belmont,
Massachusetts, 2
nd
edition, 1999.
[49] D.P. Bertsekas. Control of Uncertain Systems with a setmembership de-
scription of the uncertainty. PhD thesis, MIT, 1971.
[50] D.P. Bertsekas. Innite-time reachability of state-space regions by using feed-
back control. IEEE Transactions on Automatic Control, 17:604613, October
1972.
References 425
[51] D.P. Bertsekas. Dynamic Programming and Optimal Control. Athena Scien-
tic, Belmont, Massachusetts, 1995.
[52] D.P. Bertsekas. Dynamic Programming and Optimal Control, volume II.
Athena Scientic, Belmont, Massachusetts, 2nd edition, 2001.
[53] D.P. Bertsekas and I.B. Rhodes. On the minimax reachability of target sets
and target tubes. Automatica, 7:233247, 1971.
[54] L.T. Biegler and V.M. Zavala. Large-scale nonlinear programming using
ipopt: An integrating framework for enterprise-wide dynamic optimization.
Computers & Chemical Engineering, 33(3):575 582, 2009. Selected Papers
from the 17th European Symposium on Computer Aided Process Engineering
held in Bucharest, Romania, May 2007.
[55] R.R. Bitmead, M. Gevers, and V. Wertz. Adaptive Optimal Control: The
Thinking Mans GPC. International Series in Systems and Control Engineer-
ing. Prentice Hall, 1990.
[56] F. Blanchini. Ultimate boundedness control for uncertain discrete-time sys-
tems via set-induced Lyapunov functions. IEEE Transactions on Automatic
Control, 39(2):428433, February 1994.
[57] F. Blanchini. Set invariance in control a survey. Automatica, 35(11):1747
1768, November 1999.
[58] F. Blanchini and S. Miano. Set-Theoretic Methods in Control. Birkhauser,
2009.
[59] F. Borrelli, M. Baotic, A. Bemporad, and M. Morari. Dynamic program-
ming for constrained optimal control of discrete-time linear hybrid systems.
Automatica, 41:17091721, October 2005.
[60] F. Borrelli, M. Baotic, J. Pekar, and G. Stewart. On the computation of
linear model predictive control laws. Automatica, 46(6):1035 1041, 2010.
[61] F. Borrelli, A. Bemporad, M. Fodor, and D. Hrovat. An MPC/hybrid sys-
tem approach to traction control. IEEE Trans. Contr. Systems Technology,
14(3):541552, May 2006.
[62] F. Borrelli, A. Bemporad, and M. Morari. A geometric algorithm for multi-
parametric linear programming. J. Opt. Theory and Applications, 118(3),
September 2003.
[63] Stephen Boyd, Neal Parikh, Eric Chu, Borja Peleato, and Jonathan Eckstein.
Distributed optimization and statistical learning via the alternating direction
method of multipliers. Foundations and Trends in Machine Learning, 3(1):1
122, 2011.
[64] Stephen Boyd and Lieven Vandenberghe. Convex Optimization. Cambridge
University Press, March 2004.
426 References
[65] M.S. Branicky. Studies in hybrid systems: modeling, analysis, and control.
PhD thesis, LIDS-TH 2304, Massachusetts Institute of Technology, Cam-
bridge, MA, 1995.
[66] M.S. Branicky. Multiple Lyapunov functions and other analysis tools for
switched and hybrid systems. IEEE Transactions on Automatic Control,
43(4):475482, April 1998.
[67] M.S. Branicky, V.S. Borkar, and S.K. Mitter. A unied framework for hybrid
control: model and optimal control theory. IEEE Transactions on Automatic
Control, 43(1):3145, 1998.
[68] M.S. Branicky and S.K. Mitter. Algorithms for optimal hybrid control. In
Proc. 34th IEEE Conf. on Decision and Control, New Orleans, USA, Decem-
ber 1995.
[69] M.S. Branicky and G. Zhang. Solving hybrid control problems: Level sets and
behavioral programming. In Proc. American Contr. Conf., Chicago, Illinois
USA, June 2000.
[70] M. Buss, O. von Stryk, R. Bulirsch, and G. Schmidt. Towards hybrid optimal
control. AT - Automatisierungstechnik, 48:448459, 2000.
[71] Kerrigan E. C. and Maciejowski J. M. Soft constraints and exact penalty
functions in model predictive control. In Proc. UKACC International Con-
ference (Control 2000), Cambridge, UK, September 2000.
[72] R. Cagienard, P. Grieder, E.C. Kerrigan, and M. Morari. Move blocking
strategies in receding horizon control. Journal of Process Control, 17(6):563
570, 2007.
[73] S. Di Cairano, A. Bemporad, and J. J ulvez. Event-driven optimization-based
control of hybrid systems with integral continuous-time dynamics. Automat-
ica, 45:12431251, 2009.
[74] S. Di Cairano, A. Bemporad, I. Kolmanovsky, and D. Hrovat. Model pre-
dictive control of magnetically actuated mass spring dampers for automotive
applications. Int. J. Control, 80(11):17011716, 2007.
[75] E.F. Camacho and C. Bordon. Model Predictive Control. Advanced Text-
books in Control and Signal Processing. Springer, London, 1999.
[76] M.K. Camlibel, W.P.M.H. Heemels, and J.M. Schumacher. On linear passive
complementarity systems. European Journal of Control, 8(3), 2002.
[77] P.J. Campo and M. Morari. Robust model predictive control. In Proc.
American Contr. Conf., volume 2, pages 10211026, 1987.
[78] P.J. Campo and M. Morari. Model predictive optimal averaging level control.
AIChE Journal, 35(4):579591, 1989.
References 427
[79] C.G. Cassandras, D.L. Pepyne, and Y.Wardi. Optimal control of a class of
hybrid systems. IEEE Transactions on Automatic Control, 46(3):3981415,
2001.
[80] T.M. Cavalier, P.M. Pardalos, and A.L. Soyster. Modeling and integer pro-
gramming techniques applied to propositional calculus. Computers & Oper-
ations Research, 17(6):561570, 1990.
[81] S.N. Cernikov. Contraction of nite systems of linear inequalities (in Rus-
sian). Doklady Akademiia Nauk SSSR, 152(5):10751078, 1963. (English
translation in Soviet Mathematics - Doklady, Vol. 4, No. 5 (1963), pp.1520
1524).
[82] V. Chandru and J.N. Hooker. Optimization methods for logical inference.
Wiley-Interscience, 1999.
[83] H. Chen and F. Allgower. A quasi-innite horizon nonlinear model predictive
control scheme with guaranteed stability. Automatica, 304(10):12051218,
1998.
[84] D. Chmielewski and V. Manousiouthakis. On constrained innite-time lin-
ear quadratic optimal control. Systems & Control Letters, 29(3):121130,
November 1996.
[85] D. Christiansen. Electronics Engineers Handbook, 4th edition. IEEE Press/
McGraw Hill, Inc., 1997.
[86] F.J. Christophersen. Optimal Control and Analysis for Constrained Piecewise
Ane Systems. Dr. sc. ETH Zurich thesis, ETH Zurich, Zurich, Switzerland,
August 2006. Available from http://control.ee.ethz.ch.
[87] F.J. Christophersen and M. Morari. Further Results on Innity Norms as
Lyapunov Functions for Linear Systems. IEEE Trans. on Automatic Control,
52(3):547553, March 2007.
[88] D.W. Clarke, C. Mohtadi, and P.S. Tus. Generalized Predictive Control
part i. the basic algorithm. Automatica, 23(2):137148, March 1987.
[89] C.R. Cuttler and B.C. Ramaker. Dynamic matrix controla computer con-
trol algorithm. In Proc. American Contr. Conf., volume WP5-B, San Fran-
cisco, USA, 1980.
[90] G.B. Dantzig, J. Folkman, and N. Shapiro. On the continuity of the min-
imum set of a continuous function. Journal of Mathematical Analysis and
Applications, 17:519548, 1967.
[91] B. De Schutter and B. De Moor. The extended linear complementarity
problem and the modeling and analysis of hybrid systems. In P. Antsaklis,
W. Kohn, M. Lemmon, A. Nerode, and S. Sastry, editors, Hybrid Systems V,
volume 1567 of Lecture Notes in Computer Science, pages 7085. Springer,
1999.
428 References
[92] B. De Schutter and T. van den Boom. Model predictive control for max-plus-
linear discrete event systems. Automatica, 37(7):10491056, July 2001.
[93] R. DeCarlo, M. Branicky, S. Pettersson, and B. Lennartson. Perspectives
and results on the stability and stabilizability of hybrid systems. Proceedings
of the IEEE, 88(7):10691082, 2000.
[94] Olivier Devolder, Fran cois Glineur, and Yurii Nesterov. First-order methods
of smooth convex optimization with inexact oracle. Mathematical Program-
ming, pages 139, 2013.
[95] A. Domahidi, A. Zgraggen, M.N. Zeilinger, M. Morari, and C.N. Jones. E-
cient interior point methods for multistage problems arising in receding hori-
zon control. In Conference on Decision and Control (CDC), pages 668674,
Maui, HI, USA, December 2012.
[96] J.A. De Don a. Input Constrained Linear Control. PhD thesis, Control Group,
Department of Engineering, University of Cambridge, Cambridge, 2000.
[97] C.E.T. Dorea and J.C. Hennet. (a,b)-invariant polyhedral sets of linear
discrete-time systems. J. Optim. Theory Appl., 103(3):521542, 1999.
[98] V. Dua and E.N. Pistikopoulos. An algorithm for the solution of multipara-
metric mixed integer linear programming problems. Annals of Operations
Research, 99:123139, 2000. 10.1023/A:1019241000636.
[99] G. Ferrari-Trecate, F.A. Cuzzola, D. Mignone, and M. Morari. Analysis of
discrete-time piecewise ane and hybrid systems. Automatica, 38:21392146,
2002.
[100] G. Ferrari-Trecate, D. Mignone, and M. Morari. Moving horizon estimation
for hybrid systems. IEEE Transactions on Automatic Control, 47(10):1663
1676, 2002.
[101] G. Ferrari-Trecate, M. Muselli, D. Liberati, and M. Morari. A clustering
technique for the identication of piecewise ane systems. Automatica,
39(2):205217, February 2003.
[102] H.J. Ferreau and M. Diehl H.G. Bock. An online active set strategy to
overcome the limitations of explicit mpc. International Journal of Robust
and Nonlinear Control, 18:816830, 2008.
[103] A.V. Fiacco. Sensitivity analysis for nonlinear programming using penalty
methods. Mathematical Programming, 10(3):287311, 1976.
[104] A.V. Fiacco. Introduction to sensitivity and stability analysis in nonlinear
programming. Academic Press, London, U.K., 1983.
[105] R. Fletcher. Practical Methods of Optimization. Wiley-Interscience Publica-
tion, 1987. Second Edition.
[106] C.A. Floudas. Nonlinear and Mixed-Integer Optimization. Oxford University
Press, 1995.
References 429
[107] K. Fukuda. cdd/cdd+ Reference Manual. Institute for operations Research
ETH-Zentrum, ETH-Zentrum, CH-8092 Zurich, Switzerland, 0.61 (cdd) 0.75
(cdd+) edition, December 1997.
[108] K. Fukuda. Polyhedral computation FAQ, 2000. On
line document. Both html and ps versions available from
http://www.ifor.math.ethz.ch/staff/fukuda.
[109] K. Fukuda, T.M. Liebling, and C. L utolf. Extended convex hull. In Proceed-
ings of the 12th Canadian Conference on Computational Geometry. Freder-
icton, New Brunswick, Canada, page 57
1
2
64, July 2000.
[110] K. Fukuda and A. Prodon. Double description method revisited. Combina-
torics and Computer Science, 1120:91
1
2
111, 1996.
[111] T. Gal. Postoptimal Analyses, Parametric Programming, and Related Topics.
de Gruyter, Berlin, 2nd edition, 1995.
[112] T. Gal and H.J. Greenberg (Eds.). Advances in Sensitivity Analysis and
Parametric Programming, volume 6 of International Series in Operations
Research & Management Science. Kluwer Academic Publishers, 1997.
[113] T. Gal and J. Nedoma. Multiparametric linear programming. Management
Science, 18:406442, 1972.
[114] C.E. Garca and A.M. Morshedi. Quadratic programming solution of dynamic
matrix control (QDMC). Chem. Eng. Communications, 46:7387, 1986.
[115] S.J. Gartska and R.J.B. Wets. On decision rules in stochastic programming.
Mathematical Programming, 7:117143, 1974.
[116] S.I. Gass and T.L. Saaty. The computational algorithm for the parametric
objective function. Naval Research Logistics Quarterly, 2:3945, 1955.
[117] T. Geyer. Low Complexity Model Predictive Control in Power Electronics and
Power Systems. PhD thesis, Swiss Federal Institute of Technology (ETH),
ETH-Zentrum, CH-8092 Zurich, Switzerland, March 2005.
[118] T. Geyer, F.D. Torrisi, and M. Morari. Ecient Mode Enumeration of Com-
positional Hybrid Models. In A. Pnueli and O. Maler, editors, Hybrid Sys-
tems: Computation and Control, volume 2623 of Lecture Notes in Computer
Science, pages 216232. Springer-Verlag, 2003.
[119] T. Geyer, F.D. Torrisi, and M. Morari. Optimal complexity reduction of
polyhedral piecewise ane systems. Automatica, 44:17281740, July 2008.
[120] E.G. Gilbert and K.T. Tan. Linear systems with state and control constraints:
The theory and application of maximal output admissible sets. IEEE Trans-
actions on Automatic Control, 36(9):10081020, September 1991.
[121] E.G. Gilbert and K.T. Tan. Linear systems with state and control constraints:
the theory and applications of maximal output admissible sets. IEEE Trans-
actions on Automatic Control, 36(9):10081020, 1991.
430 References
[122] N. Giorgetti, A. Bemporad, H.E. Tseng, and D. Hrovat. Hybrid model pre-
dictive control application towards optimal semi-active suspension. Int. J.
Control, 79(5):521533, 2006.
[123] F. Glover. Improved linear integer programming formulations of nonlinear
integer problems. Management Science, 22(4):455460, 1975.
[124] R. Goebel, R.G. Sanfelice, and A.R. Teel. Hybrid dynamical systems. IEEE
Control Systems Magazine, 29(2):2893, 2009.
[125] K. Gokbayrak and C.G. Cassandras. A hierarchical decomposition method
for optimal control of hybrid systems. In Proc. 38th IEEE Conf. on Decision
and Control, pages 18161821, Phoenix, AZ, December 1999.
[126] Jacob E. Goodman and Joseph ORourke, editors. Handbook of Discrete and
Computational Geometry, chapter Voronoi Diagrams and Delaunay Trian-
gulations, Steven Fortune, pages 513528. Chapman & Hall, 2
nd
edition,
2004.
[127] G.C. Goodwin and K. S. Sin. Adaptive Filtering, Prediction and Control.
Prentice-Hall, Englewood Clis, NJ, 1984.
[128] Paul J. Goulart, Eric C. Kerrigan, and Jan M. Maciejowski. Optimization
over state feedback policies for robust control with constraints. Automatica,
42(4):523 533, 2006.
[129] P.J. Goulart, E.C. Kerrigan, and J.M. Maciejowski. Optimization over state
feedback policies for robust control with constraints. Automatica, 42(4):523
533, 2006.
[130] P. Grieder. Ecient Computation of Feedback Controllers for Constrained
Systems. Dr. Sc. Techn. thesis, Swiss Federal Institute of Technology (ETH),
ETH-Zentrum, CH-8092 Zurich, Switzerland, 2004.
[131] P. Grieder, F. Borrelli, F.D. Torrisi, and M. Morari. Constrained in-
nite horizon linear quadratic regulator. Technical Report AUT02-66,
Automatic Control Laboratory, ETH Zurich, Switzerland, July 2002.
http://control.ee.ethz.ch.
[132] P. Grieder, F. Borrelli, F.D. Torrisi, and M. Morari. Computation of the
constrained innite time linear quadratic regulator. Automatica, 40:701708,
April 2004.
[133] P. Grieder, M. Kvasnica, M. Baotic, and M. Morari. Low complexity control
of piecewise ane systems with stability guarantee. In Proc. of the American
Control Conference, pages 11961201, Boston, USA, June 2004.
[134] B. Gr unbaum. Convex Polytopes. Springer-Verlag, 2nd edition, 2003.
[135] E. Guslitzer. Uncertainty-immunized solutions in linear programming. Mas-
ters thesis, Technion (Israel Institute of Technology), Haifa, Israel, 2002.
References 431
[136] P.O. Gutman. Online use of a linear programming controller. In G. Ferrate
and E.A. Puente, editors, Software for Computer Control 1982. Proceedings of
the Third IFAC/IFIP Symposium, pages 313318. Pergamon, Oxford, 1983.
[137] P.O. Gutman. A linear programming regulator applied to hydroelectric reser-
voir level control. Automatica, 22(5):533541, 1986.
[138] P.O. Gutman and M. Cwikel. Admissible sets and feedback control for
discrete-time linear dynamical systems with bounded control and states.
IEEE Transactions on Automatic Control, AC-31(4):373376, 1986.
[139] P.O. Gutman and M. Cwikel. An algorithm to nd maximal state constraint
for discrete-time linear dynamical systems with bounded control and states.
IEEE Transactions on Automatic Control, AC-32(3):251254, 1987.
[140] A. Hassibi and S. Boyd. Quadratic stabilization and control of piecewise-
linear systems. In Proc. American Contr. Conf., Philadelphia, Pennsylvania
USA, June 1998.
[141] J.P. Hayes. Introduction to Digital Logic Design. Addison-Wesley Publishing
Company, Inc., 1993.
[142] S. Hedlund and A. Rantzer. Optimal control of hybrid systems. In Proc.
38th IEEE Conf. on Decision and Control, pages 39723976, Phoenix, AZ,
December 1999.
[143] S. Hedlund and A. Rantzer. Convex dynamic programming for hybrid sys-
tems. IEEE Transactions on Automatic Control, 47(9):15361540, September
2002.
[144] W.P.H.M Heemels, B. de Schutter, and A. Bemporad. On the equivalence of
classes of hybrid dynamical models. In Proc. 40th IEEE Conf. on Decision
and Control, pages 364369, Orlando, Florida, 2001.
[145] W.P.M.H. Heemels. Linear complementarity systems: a study in hybrid dy-
namics. PhD thesis, Dept. of Electrical Engineering, Eindhoven University
of Technology, The Netherlands, 1999.
[146] W.P.M.H. Heemels, J.M. Schumacher, and S. Weiland. Linear complemen-
tarity systems. SIAM Journal on Applied Mathematics, 60(4):12341269,
2000.
[147] W.P.M.H. Heemels, B. De Schutter, and A. Bemporad. Equivalence of hybrid
dynamical models. Automatica, 37(7):10851091, July 2001.
[148] M. Herceg, M. Kvasnica, C.N. Jones, and M. Morari. Multi-Parametric Tool-
box 3.0. In Proc. of the European Control Conference, pages 502510, Z urich,
Switzerland, July 1719 2013. http://control.ee.ethz.ch/
~
mpt.
[149] J.P. Hespanha, S. Bohacek, K. Obraczka, and J. Lee. Hybrid modeling of
TCP congestion control. In M.D. Di Benedetto and A. Sangiovanni Vin-
centelli, editors, Hybrid Systems: Computation and Control, volume 2034 of
Lecture Notes in Computer Science, pages 291304. Springer-Verlag, 2001.
432 References
[150] Jean-Baptiste Hiriart-Urruty and Claude Lemarechal. Fundamentals of Con-
vex Analysis. Springer, 2001.
[151] W.M. Hogan. Point-to-set maps in mathematical programming. SIAM Re-
view, 15(3):591603, July 1973.
[152] J.N. Hooker. Logic-Based Methods for Optimization: Combining Optimiza-
tion and Constraint Satisfaction. Wiley, New York, 2000.
[153] T. Huerlimann. Reference Manual for the LPL Modeling Language, Ver-
sion 4.42. Departement for Informatics, Universite de Fribourg, Switzerland,
http://www2-iiuf.unifr.ch/tcs/lpl/TonyHome.htm, 2001.
[154] A. Jadbabaie, Y. Jie, and J. Hauser. Stabilizing Receding Horizon Control of
nonlinear systems: a control Lyapunov function approach. In Proc. American
Contr. Conf., June 1999.
[155] M. Johannson and A. Rantzer. Computation of piece-wise quadratic Lya-
punov functions for hybrid systems. IEEE Transactions on Automatic Con-
trol, 43(4):555559, 1998.
[156] T.A. Johansen and A. Grancharova. Approximate explicit constrained linear
model predictive control via orthogonal search tree. Automatic Control, IEEE
Transactions on, 48(5):810815, May 2003.
[157] T.A. Johansen, J. Kalkkuhl, J. L udemann, and I. Petersen. Hybrid control
strategies in ABS. In Proc. American Contr. Conf., Arlington, Virginia, June
2001.
[158] K.H. Johansson, M. Egerstedt, J. Lygeros, and S. Sastry. On the regulariza-
tion of Zeno hybrid automata. System & Control Letters, 38:141150, 1999.
[159] C. Jones, P. Grieder, and S. Rakovic. A Logarithmic-Time Solution to
the Point Location Problem for Closed-Form Linear MPC. In IFAC World
Congress, Prague, Czech Republic, July 2005.
[160] C.N. Jones, M. Baric, and M. Morari. Multiparametric Linear Programming
with Applications to Control. European Journal of Control, 13(2-3):152170,
March 2007.
[161] C.N. Jones, E.C. Kerrigan, and J.M. Maciejowski. Equality set pro-
jection: A new algorithm for the projection of polytopes in halfspace
representation. Technical Report CUED Technical Report CUED/F-
INFENG/TR.463, Department of Engineering, Cambridge University, UK,
2004. http://www-control.eng.cam.ac.uk/ cnj22/.
[162] C.N. Jones and M. Morari. Polytopic approximation of explicit model predic-
tive controllers. IEEE Transactions on Automatic Control, 55(11):25422553,
November 2010.
[163] P. Julian, M. Jordan, and A. Desages. Canonical piecewise-linear approxi-
mation of smooth functions. IEEE Trans. Circuits and Systems I: Fun-
damental Theory and Applications, 45(5):567571, May 1998.
References 433
[164] A.A. Julius and A.J. van der Schaft. The maximal controlled invariant set of
switched linear systems. In Proc. of the 41st IEEE Conference on Decision
and Control, Las Vegas, Nevada, USA, December 2002.
[165] A. Juloski, S. Weiland, and M.Heemels. A bayesian approach to identication
of hybrid systems. Proc. 43th IEEE Conf. on Decision and Control, 2004.
[166] Narendra Krishna Karmarkar. A new polynomial-time algorithm for linear
programming. Combinatorica, 4:373395, 1984.
[167] S. Keerthi and E. Gilbert. Computation of minimum-time feedback con-
trol laws for discrete-time systems with state-control constraints. Automatic
Control, IEEE Transactions on, 32(5):432435, 1987.
[168] S.S. Keerthi and E.G. Gilbert. Optimal innite-horizon feedback control laws
for a general class of constrained discrete-time systems: stability and moving-
horizon approximations. J. Opt. Theory and Applications, 57:265293, 1988.
[169] S.S. Keerthi and K. Sridharan. Solution of parametrized linear inequalities
by Fourier elimination and its applications. J. Opt. Theory and Applications,
65(1):161169, 1990.
[170] E.C. Kerrigan. Robust Constraints Satisfaction: Invariant Sets and Predic-
tive Control. PhD thesis, Department of Engineering, University of Cam-
bridge, Cambridge, England, 2000.
[171] E.C. Kerrigan and J.M. Maciejowski. Designing model predictive controllers
with prioritised constraints and objectives. In Computer Aided Control Sys-
tem Design, 2002. Proceedings. 2002 IEEE International Symposium on,
pages 33 38, 2002.
[172] E.C. Kerrigan and J.M. Maciejowski. On robust optimization and the optimal
control of constrained linear systems with bounded state disturbances. In In
Proc. 2003 European Control Conference, Cambridge, UK, September 2003.
[173] H.K. Khalil. Nonlinear Systems. Prentice Hall, 2nd edition, 1996.
[174] H. Kiendl, J. Adamy, and P. Stelzner. Vector norms as Lyapunov functions
for linear systems. IEEE Transactions on Automatic Control, 37(6):839842,
June 1992.
[175] D. Klatte and G. Thiere. Error bounds for solutions of linear equations and
inequalities. ZOR - Mathematical Methods of Operations Research, 41:191
214, 1995.
[176] I. Kolmanovsky and E.G. Gilbert. Theory and computation of disturbance
invariant sets for discrete-time linear systems. Mathematical Problems in
Egineering, 4:317367, 1998.
[177] M.V. Kothare, V. Balakrishnan, and M. Morari. Robust constrained model
predictive control using linear matrix inequalities. Automatica, 32(10):1361
1379, 1996.
434 References
[178] B. Kouvaritakis, J.A. Rossiter, and J. Schuurmans. Ecient robust predictive
control. IEEE Transactions on Automatic Control, 45(8):15451549, 2000.
[179] G. Laerriere, G.J. Pappas, and S. Sastry. Reachability analysis of hybrid
systems using bisimulations. In Proc. 37th IEEE Conf. on Decision and
Control, pages 16231628, Tampa, Florida, USA, 1998.
[180] J.P. LaSalle. Stability theory for dierence equations. In Jack Hale, editor,
Studies in Ordinary Dierential Equations, volume 14 of MAA Studies in
Mathematics, pages 131. 1977.
[181] J.P. LaSalle. The Stability and Control of Discrete Processes, volume 62 of
Applied Mathematical Sciences. Springer-Verlag, New York, 1986.
[182] M. Lazar and W.P.M.H. Heemels. Predictive control of hybrid sys-
tems: Input-to-state stability results for sub-optimal solutions. Automatica,
45(1):180185, 2009.
[183] M. Lazar, W.P.M.H. Heemels, and A.R. Teel. Lyapunov functions, stability
and input-to-state stability subtleties for discrete-time discontinuous systems.
IEEE Transactions on Automatic Control, 54(10):24212425, 2009.
[184] M. Lazar, W.P.M.H. Heemels, S. Weiland, and A. Bemporad. Stabilizing
model predictive control of hybrid systems. IEEE Transactions on Automatic
Control, 51(11):18131818, 2006.
[185] M. Lazar, D. M. de la Pe na, W.P.M.H. Heemels, and T. Alamo. On input-
to-state stability of min-max nonlinear model predictive control. Systems &
Control Letters, 57:3948, 2008.
[186] J. H. Lee and Z. Yu. Worst-case formulations of model predictive control for
systems with bounded parameters. Automatica, 33(5):763781, 1997.
[187] F.L. Lewis and V.L. Syrmos. Optimal Control. John Wiley & Sons, Inc.,
New York, 1995.
[188] D. Liberzon. Switching in Systems and Control. Systems and Control: Foun-
dations and Application. Birkhauser, Boston, MA, June 2003.
[189] B. Lincoln and A. Rantzer. Optimizing linear system switching. In Proc.
40th IEEE Conf. on Decision and Control, pages 20632068, Orlando, FL,
USA, 2001.
[190] Y.-C. Liu and C. B. Brosilow. Simulation of large scale dynamic systems
I. modular integration methods. Computers & Chemical Engineering,
11(3):241253, 1987.
[191] J. L ofberg. Minimax approaches to robust model predictive control. PhD
thesis, Linkoping University, Sweden, April 2003.
References 435
[192] J. Lygeros, D.N. Godbole, and S. Sastry. A game theoretic approach to
hybrid system design. In R. Alur and T. Henzinger, editors, Hybrid Systems
III, volume 1066 of Lecture Notes in Computer Science, pages 112. Springer
Verlag, 1996.
[193] J. Lygeros, C. Tomlin, and S. Sastry. Controllers for reachability specica-
tions for hybrid systems. Automatica, 35(3):349370, 1999.
[194] J.M. Maciejowski. Predictive Control with Constraints. Prentice Hall, 2002.
[195] U. Maeder, F. Borrelli, and M. Morari. Linear oset-free Model Predictive
Control. Automatica, 45(10):2214 2222, 2009.
[196] S. Mahapatra. Stability of Hybrid Haptic Systems. PhD thesis, University of
Illinois, Chicago, Illinois, 2003.
[197] O.L. Mangasarian and J.B. Rosen. Inequalities for stochastic nonlinear pro-
gramming problems. Operations Research, 12(1):143154, 1964.
[198] Jacob Mattingley and Stephen Boyd. CVXGEN: A code generator for em-
bedded convex optimization. Optimization and Engineering, 13:127, 2012.
[199] D.Q. Mayne. Constrained Optimal Control. European Control Conference,
Seminario de Vilar, Porto, Portugal, Plenary Lecture, September 2001.
[200] D.Q. Mayne. Control of constrained dynamic systems. European Jornal of
Control, 7:8799, 2001.
[201] D.Q. Mayne, J.B. Rawlings, C.V. Rao, and P.O.M. Scokaert. Constrained
model predictive control: Stability and optimality. Automatica, 36:789814,
2000.
[202] D.Q. Mayne, J.B. Rawlings, C.V. Rao, and P.O.M. Scokaert. Constrained
model predictive control: Stability and optimality. Automatica, 36(6):789
814, June 2000.
[203] T.A. Meadowcroft, G. Stephanopoulos, and C. Brosilow. The Modular Multi-
variable Controller: 1: Steady-state properties. AIChE Journal, 38(8):1254
1278, 1992.
[204] S. Mehrotra and R.D.C. Monteiro. Parametric and range analysis for interior
point methods. Technical report, Dept. of Systems and Industrial Engineer-
ing, University of Arizona, Tucson, USA, 1992.
[205] Sanjay Mehrotra. On the implementation of a primal-dual interior point
method. SIAM Journal on Optimization, 2(4):575601, November 1992.
[206] E. Mendelson. Introduction to mathematical logic. Van Nostrand, 1964.
[207] J.A. Mendez, B. Kouvaritakis, and J.A. Rossiter. State space approach to
interpolation in MPC. Int. J. Robust Nonlinear Control, 10(1):2738, January
2000.
436 References
[208] D. Mignone. Control and Estimation of Hybrid Systems with Mathematical
Optimization. PhD thesis, Automatic Control Labotatory - ETH, Zurich,
2002.
[209] D. Mignone, A. Bemporad, and M. Morari. A framework for control, fault
detection, state estimation and verication of hybrid systems. Proc. American
Contr. Conf., pages 134138, June 1999.
[210] R. Milman and E.J. Davison. A fast mpc algorithm using nonfeasible active
set methods. Journal of Optimization Theory and Applications, 139(3):591
616, 2008.
[211] G. Mitra, C. Lucas, and S. Moody. Tool for reformulating logical forms into
zero-one mixed integer programs. European Journal of Operational Research,
71:262276, 1994.
[212] R. M obus, M. Baotic, and M. Morari. Multi-objective adaptive cruise con-
trol. In O. Maler and A. Pnueli, editors, Hybrid Systems: Computation and
Control, number 2623 in Lecture Notes in Computer Science, pages 359374.
Springer-Verlag, 2003.
[213] M. Morari and G. Stephanopoulos. Minimizing unobservability in inferential
control schemes. Int. J. Control, 31:367377, 1980.
[214] M. Morari and G. Stephanopoulos. Studies in the synthesis of control struc-
tures for chemical processes; Part III: Optimal selection of secondary mea-
surements within the framework of state estimation in the presence of per-
sistent unknown disturbances. AIChE J., 26:247260, 1980.
[215] Katta G. Murty. Linear Programming. Wiley-Interscience Publication, 1983.
First Edition.
[216] K.R. Muske and T.A. Badgwell. Disturbance modeling for oset-free linear
model predictive control. Journal of Process Control, 12:617632, 2002.
[217] G.L. Nemhauser and L.A. Wolsey. Integer and Combinatorial Optimization.
Wiley, 1988.
[218] Yurii Nesterov. Introductory Lectures on Convex Optimization. A Basic
Course. Springer, 2004.
[219] Yurii Nesterov. Smooth minimization of non-smooth functions. Mathematical
Programming, 103(1):127152, 2005.
[220] Yurii Nesterov and Arkadi Nemirovski. Interior-Point Polynomial Algorithms
in Convex Programming. Society for Industrial Mathematics, 1994.
[221] Jorge Nocedal and Stephen J. Wright. Numerical Optimization. Springer,
New York, 2
nd
edition, 2006.
[222] James M Ortega and Werner C Rheinboldt. Iterative solution of nonlinear
equations in several variables, volume 30. Society for Industrial and Applied
Mathematics, 1987.
References 437
[223] G. Pannocchia. Robust disturbance modeling for model predictive control
with application to multivariable ill-conditioned processes. J. Process Con-
trol, 13(8):693701, 2003.
[224] G. Pannocchia and J.B. Rawlings. Disturbance models for oset-free model
predictive control. AIChE Journal, 49(2):426437, 2003.
[225] S. Paoletti. Identication of Piecewise Ane Models. PhD thesis, Dept.
Information Engineering, University of Siena, Italy, 2004.
[226] Neal Parikh and Stephen Boyd. Proximal algorithms. Foundations and
Trends in optimization, 1(3):123231, 2013.
[227] T. Park and P.I. Barton. Implicit model checking of logic-based control
systems. AIChE Journal, 43(9):22462260, 1997.
[228] P.A. Parrilo. Structured semidenite programs and semialgebraic geometry
methods in robustness and optimization. Ph.D. thesis, California Institute of
Technology, Pasadena, CA, USA, 2000.
[229] Panagiotis Patrinos and Haralambos Sarimveis. Convex parametric piecewise
quadratic optimization: Theory and Algorithms. Automatica, 47(8):1770
1777, 2011.
[230] D. M. De La Pena, A. Bemporad, and C. Filippi. Robust explicit MPC based
on approximate multiparametric convex programming. Automatic Control,
IEEE Transactions on, 51(8):13991403, 2006.
[231] S. Pettersson and B. Lennartson. Stability and robustness for hybrid systems.
In Proc. 35th IEEE Conf. on Decision and Control, pages 12021207, Kobe,
Japan, 1996.
[232] S. Pettersson and B. Lennartson. Exponential stability of hybrid systems
using piecewise quadratic lyapunov functions resulting in LMIs. In Proc.
IFAC World Congress, pages 103108, Beijing, China, July 1999.
[233] B. Piccoli. Necessary conditions for hybrid optimization. In Proc. 38th IEEE
Conf. on Decision and Control, Phoenix, Arizona USA, December 1999.
[234] A. Pogromski, M. Jirstrand, and P. Spangeus. On stability and passivity of a
class of hybrid systems. In Proc. 37th IEEE Conf. on Decision and Control,
pages 37053710, Tampa, Florida, USA, 1998.
[235] A. Polanski. On innity norms as Lyapunov functions for linear systems.
IEEE Transactions on Automatic Control, 40(7):12701274, July 1995.
[236] S. Prajna and A. Papachristodoulou. Analysis of switched and hybrid sys-
tems beyond piecewise quadratic methods. In Proceedings of the American
Control Conference, volume 4, pages 27792784, 2003.
[237] A.I. Propoi. Use of linear programming methods for synthesizing sampled
1
2
-
data automatic systems. Automation and Remote Control, 24(7):837844,
1963.
438 References
[238] S.J. Qin and T.A. Badgwell. An overview of industrial model predictive
control technology. In Chemical Process Control - V, volume 93, no. 316,
pages 232256. AIChE Symposium Series - American Institute of Chemical
Engineers, 1997.
[239] S.J. Qin and T.A. Badgwell. A survey of industrial model predictive control
technology. Control Engineering Practice, 11:733764, 2003.
[240] S.V. Rakovic, P. Grieder, M. Kvasnica, D.Q. Mayne, and M. Morari. Com-
putation of invariant sets for piecewise ane discrete time systems subject to
bounded disturbances. In IEEE Conference on Decision and Control, pages
14181423, December 2004.
[241] S.V. Rakovic, E.C. Kerrigan, and D.Q. Mayne. Reachability computations
for constrained discrete-time systems with state- and input-dependent dis-
turbances. In Proc. of the Conf. on Decision and Control, Maui, Hawaii,
USA, pages 39053910, December 2003.
[242] R. Raman and I.E. Grossmann. Relation between MILP modeling and logical
inference for chemical process synthesis. Computers Chem. Engng., 15(2):73
84, 1991.
[243] A. Rantzer and M. Johansson. Piecewise linear quadratic optimal control.
IEEE Transactions on Automatic Control, 45(4):629637, April 2000.
[244] Christopher V. Rao, Stephen J. Wright, and James B. Rawlings. Application
of Interior-Point methods to model predictive control. Journal of Optimiza-
tion Theory and Applications, 99(3):723757, December 1998.
[245] James B. Rawlings and David Q. Mayne. Model Predictive Control: Theory
and Design. Nob Hill Publishing, 2009.
[246] J. Richalet, A. Rault, J.L. Testud, and J. Papon. Algorithmic control of
industrial processes. In Fourth IFAC symposium on identixcation and system
parameter estimation, volume WP5-B, pages 11191167, 1976.
[247] J. Richalet, A. Rault, J.L. Testud, and J. Papon. Model predictive heuristic
control-application to industrial processes. Automatica, 14:413428, 1978.
[248] S. Richter. Computational Complexity Certication of Gradient Methods
for Real-Time Model Predictive Control. PhD thesis, ETH Zurich, Zurich,
Switzerland, November 2012.
[249] S. Richter, C. Jones, and M. Morari. Certication Aspects of the Fast Gra-
dient Method for Solving the Dual of Parametric Convex Programs. Mathe-
matical Methods of Operations Research, 77(3):305321, January 2013.
[250] P. Riedinger, F.Kratz, C. Iung, and C. Zanne. Linear quadratic optimization
for hybrid systems. In Proc. 38th IEEE Conf. on Decision and Control,
Phoenix, Arizona USA, December 1999.
References 439
[251] S.M. Robinson. Some continuity properties of polyhedral multifunctions.
Mathematical Proggraming Study, 14:206214, 1981.
[252] S.M. Robinson and R.H. Day. A sucient condition for continuity of optimal
sets in mathematical programming. Journal of Mathematical Analysis and
Applications, 45:506511, 1974.
[253] J. Roll, A. Bemporad, and L. Ljung. Identication of piecewise ane systems
via mixed-integer programming. Automatica, 40(1):3750, 2004.
[254] L.O. Santos, P.A.F.N.A. Afonso, J.A.A.M. Castro, N.M.C. Oliveira, and L.T.
Biegler. On-line implementation of nonlinear mpc: an experimental case
study. Control Engineering Practice, 9(8):847 857, 2001.
[255] M. Schechter. Polyhedral functions and multiparametric linear programming.
J. Opt. Theory and Applications, 53(2):269280, May 1987.
[256] C. Scherer and S. Weiland. Linear Matrix Inequalities in Control. Tech-
nical report, Delft Center for Systems and Control, Delft University
of Technology, Delft, The Netherlands, January 2005. Available from
http://www.dcsc.tudelft.nl/
~
cscherer/2416/lmi05.pdf.
[257] P.O.M. Scokaert and D.Q. Mayne. Min-max feedback model predictive con-
trol for constrained linear systems. IEEE Transactions on Automatic Control,
43(8):11361142, 1998.
[258] P.O.M. Scokaert and J.B. Rawlings. Constrained linear quadratic regulation.
[259] C. Seatzu, D. Corona, A. Giua, and A. Bemporad. Optimal control of
continuous-time switched ane systems. IEEE Transactions on Automatic
Control, 51(5):726741, 2006.
[260] C. Seatzu, D. Corona, A. Giua, and A. Bemporad. Optimal control of
continuous-time switched ane systems. Automatic Control, IEEE Transac-
tions on, 51(5):726 741, may 2006.
[261] M.M. Seron, J.A. DeDon a, and G.C. Goodwin. Global analytical model pre-
dictive control with input constraints. In Proc. 39th IEEE Conf. on Decision
and Control, pages 154159, 2000.
[262] J. Serra. Image Analysis and Mathematical Morphology, Vol II: Theoretical
advances. Academic Press, 1988.
[263] M.S. Shaikh and P.E. Caines. On the optimal control of hybrid systems:
optimization of trajectories, switching times and location schedules. In 6th
Int. Workshop on Hybrid Systems: Computation and Control, Prague, The
Czech Republic, 2003.
[264] B.I. Silva, O. Stursberg, B.H. Krogh, and S. Engell. An assessment of the
current status of algorithmic approaches to the verication of hybrid sys-
tems. In Proc. 40th IEEE Conf. on Decision and Control, pages 28672874,
Orlando, Florida, December 2001.
440 References
[265] E.D. Sontag. Nonlinear regulation: The piecewise linear approach. IEEE
Transactions on Automatic Control, 26(2):346358, April 1981.
[266] E.D. Sontag. Interconnected automata and linear systems: A theoretical
framework in discrete-time. In R. Alur, T.A. Henzinger, and E.D. Sontag,
editors, Hybrid Systems III - Verication and Control, number 1066 in Lec-
ture Notes in Computer Science, pages 436448. Springer-Verlag, 1996.
[267] J. Spjotvold, E.C. Kerrigan, C.N. Jones, P. Tondel, and T.A Johansen. On
the facet-to-facet property of solutions to convex parametric quadratic pro-
grams. Automatica, 42(12):22092214, December 2006.
[268] R. Suard, J. L ofberg, P. Grieder, M. Kvasnica, and M. Morari. Ecient
computation of controller partitions in multi-parametric programming. In
Proc. 43th IEEE Conf. on Decision and Control, pages 36433648, Bahamas,
December 2004.
[269] S. Summers, C.N. Jones, J. Lygeros, and M. Morari. A multiresolution ap-
proximation method for fast explicit model predictive control. Automatic
Control, IEEE Transactions on, 56(11):25302541, Nov 2011.
[270] H.J. Sussmann. A maximum principle for hybrid optimal control problems.
In Proc. 38th IEEE Conf. on Decision and Control, Phoenix, Arizona USA,
December 1999.
[271] M. Sznaier and M.J. Damborg. Suboptimal control of linear systems with
state and control inequality constraints. In Proc. 26th IEEE Conf. on Deci-
sion and Control, volume 1, pages 761762, 1987.
[272] C.J. Tomlin, J. Lygeros, and S.S. Sastry. A game theoretic approach to
controller design for hybrid systems. Proceeding of IEEE, 88, July 2000.
[273] P. Tndel, T.A. Johansen, and A. Bemporad. An algorithm for multi-
parametric quadratic programming and explicit MPC solutions. In Proc.
40th IEEE Conf. on Decision and Control, December 2001.
[274] P. Tndel, T.A. Johansen, and A. Bemporad. Evaluation of piecewise ane
control via binary search tree. Automatica, 39(5):945950, 2003.
[275] F.D. Torrisi and A. Bemporad. HYSDEL A tool for generating compu-
tational hybrid models. IEEE Trans. Contr. Systems Technology, 12(2):235
249, March 2004.
[276] M.L. Tyler and M. Morari. Propositional logic in control and monitoring
problems. Automatica, 35(4):565582, 1999.
[277] V.I. Utkin. Variable structure systems with sliding modes. IEEE Transac-
tions on Automatic Control, 22(2):212222, April 1977.
[278] A.J. van der Schaft and J.M. Schumacher. Complementarity modelling of
hybrid systems. IEEE Transactions on Automatic Control, 43:483490, 1998.
References 441
[279] D.H. vanHessem and O.H. Bosgra. A conic reformulation of model predictive
control including bounded and stochastic disturbances under state and input
constraint. In Proc. 41th IEEE Conf. on Decision and Control, pages 4643
4648, Las Vegas, Nevada, USA, 2002.
[280] R. Vidal, S. Soatto, Y. Ma, and S. Sastry. An algebraic geometric approach
to the identication of a class of linear hybrid systems. In Proc. 42th IEEE
Conf. on Decision and Control, pages 167172, Maui, Hawaii, 2003.
[281] D.W. Walkup and R.J.-B. Wets. A lipschitzian characterizations of convex
polyhedra. Proceeding of the American Mathematical Society, 20:167173,
1969.
[282] Y. Wang and S. Boyd. Fast model predictive control using online optimiza-
tion. IEEE Transactions on Control Systems Technology, 18(2):267 278,
March 2010.
[283] J. Warren, S. Schaefer, A. Hirani, and M. Desbrun. Barycentric coordinates
for convex sets. Advances in Computational Mathematics, 27(3):319338,
2007.
[284] H.P. Williams. Logical problems and integer programming. Bulletin of the
Institute of Mathematics and Its Applications, 13:1820, 1977.
[285] H.P. Williams. Model Building in Mathematical Programming. John Wiley
& Sons, Third Edition, 1993.
[286] H. Witsenhausen. A class of hybrid-state continuous-time dynamic systems.
[287] H.S. Witsenhausen. A min-max control problem for sampled linear systems.
[288] Stephen J. Wright. Primal-Dual Interior Point Methods. SIAM, 1997.
[289] X. Xu and P.J. Antsaklis. Results and perspectives on computational meth-
ods for optimal control of switched systems. In O. Maler and A. Pnueli,
editors, Hybrid Systems: Computation and Control, number 2623 in Lecture
Notes in Computer Science, pages 540555. Springer-Verlag, 2003.
[290] X. Xu and P.J. Antsaklis. Optimal control of switched systems based on
parameterization of the switching instants. IEEE Transactions on Automatic
Control, 49(1):216, 2004.
[291] L.A. Zadeh and L.H. Whalen. On optimal control and linear programming.
IRE Trans. Automatic Control, 7:4546, 1962.
[292] E. Zariou and M. Morari. A general controller synthesis methodology based
on the IMC structure and the H
2
-, H
- and -optimal control theories.

Computers & Chemical Engineering, 12(7):757765, 1988.
442 References
[293] M. Zefran, F. Bullo, and M. Stein. A notion of passivity for hybrid systems.
In Proc. 40th IEEE Conf. on Decision and Control, pages 768773, Orlando,
FL, 2001.
[294] G.M. Ziegler. Lectures on Polytopes. Graduate Texts in Mathematics.
Springer-Verlag, 1994.

Model Predictive Control

Загружено:

Сведения о документе

Оригинальное название

Авторское право

Доступные форматы

Поделиться этим документом

Поделиться или встроить документ

Параметры публикации

Этот документ был вам полезен?

Это неприемлемый материал?

Авторское право:

Доступные форматы

Model Predictive Control

Загружено:

Авторское право:

Доступные форматы

Predictive Control

for linear and hybrid systems

such that the objective function is optimized over some

det(A) Determinant of matrix A

Largest absolute value of the vector x R

Minimal robust positively invariant set

Maximal robust positively invariant set

Maximal robust control invariant set

Maximal robust stabilizable set

Maximal positively invariant set O

for LTI system under LQR feedback

z S and ( > 0 z S : f(z) f

= we say that the problem is unbounded below. If the set S is empty

exists, then we rewrite problem (1.2) as

is called an optimizer or global optimizer or optimal solution. Minimizer or

z +r is both convex and concave.

z +r is convex if and only if H _ 0.

z +r is strictly convex if and only if

as k increases. These algorithms iteratively use and/or solve condi-

d < 0, then there exists a > 0 such that f( z+d) <

d < 0. Theorem 2.1

be the optimal value. Denote by Z

. Next we will show how to generate a lower

). The dual cost d(u, v) is the opti-

is called the optimal duality gap. The weak duality

are innite. For example, if the primal

= +, i.e., the primal problem is

we can give a bound on how suboptimal a given

f(z). Therefore z is -suboptimal, with equal to the primal-dual

[d(u, v), f(z)] and d

[d(u, v), f(z)].

) are primal and dual feasible

) < 0), then the

) be primal and dual optimal points, respec-

) which satises condi-

) by using KKT conditions. They mainly dier on the type of

. If problem (2.1) satises Slaters condition

is optimal if and only if there are (u

) that, together with z

be a feasible solution and A = i : g

. Suppose cost and constraints f, g

for all i and that h

) for j = 1, . . . , p, are linearly

) are primal and dual optimal points, then they satisfy

) that, together with z

, satisfy the KKT

. The optimization problem does not satisfy Slaters conditions

) (which represents the direction of steepest

the optimal value and by Z

the set of optimizers of

> and the optimizer is unique.

z are represented by the parallel lines.

then there exists z

such that [A(z

on an optimal facet. Lemma 3.1 states the simple fact that if

on the same facet where

such that the LICQ does not hold at z

(a) Case 1, optimizer in interior of T.

(b) Case 2, optimizer at the boundary of T.

z+r are represented by the ellipsoids. All the points z belonging

(Gz w). (3.28)

(Gz w) is convex. Therefore

u) and, substituting this