Selected Topics in Signals, Systems and Control

Vol. 12, September 2001

Design strategies for iterative learning control based on
optimal control

Rob Tousain, Eduard van der Meche and Okko Bosgra
Mechanical Engineering Systems and Control Group
Delft University of Technology, Mekelweg 2, 2628 CD, Delft, The Netherlands
E-mail: r.l.tousain@wbmt.tudelft.nl
E-mail: e.g.vandermeche@wbmt.tudelft.nl
Abstract. This paper deals with the analysis and synthesis of Iterative Learning Con-
trol (ILC) systems using a lifted representation of the plant. In this lifted representation
the system dynamics are described by a static map whereas the learning dynamics are
described by a dierence equation. The properties of the lifted system and in particular
the role of non minimum phase zeros and system delays are investigated. Based on the
internal model principle a general, integrating update law is suggested. Two methods
are proposed for the design of the learning gain, based on optimal control theory. In the
latter, a multi-objective design, the convergence speed is optimized subject to a bound
on the closed loop variance due to stochastic initial conditions, process disturbances and
measurement noise. Ecient tailor-made solutions to the design problems are presented,
making optimal use of the specic and nice structure of the lifted system ILC represen-
tation. The potential of the design methods is demonstrated on a realistic example.
Keywords. Iterative Learning Control, optimal control, variance control, multi-objective
control, Algebraic Riccati equation.
1 Introduction
Many control systems perform the same task over
and over again. The use of feedforward control for
this kind of problems is common practice and will
generally yield much improved performance com-
pared to pure feedback control. One way to achieve
high-quality feedforward controls is through the ap-
plication of Iterative Learning Control (ILC), an it-
erative update scheme which improves the quality
of the feedforward signal from trial to trial. Itera-
tive Learning Control can be particularly eective in
case of imperfect knowledge on the plant. Literature
on ILC is already quite extensive. The work on ILC
was pioneered by Arimoto et al. (1984). Conver-
gence properties for linear and nonlinear ILC have
been studied and many design strategies for ILC

This paper will be presented at the 40th IEEE Confer-

ence on Decision and Control, December 4-7, 2001, Orlando,
Florida. Copyright of this paper remains with IEEE.
schemes were proposed. For a detailed and recent
review we refer to (Bien, 1998).
The paper builds on the lifted system representa-
tion which was, to our knowledge, rst introduced
in relation to Iterative Learning Control by (Phan,
1988). In this representation of the learning con-
trol system, the system dynamics are described by
a static map whereas the dynamics of the learn-
ing mechanism are described by a dierence equa-
tion in the repetition domain. Several authors used
the same system representation, mainly for analy-
sis of the learning convergence, see e.g. (Moore,
1998; Amann et al., 1996). However, a systematic
model-based synthesis of learning controllers on the
basis of the lifted system representation seems to
be lacking. This paper provides two main contribu-
tions. First, the properties of the lifted system rep-
resentation will be investigated in relation to ILC
design. The role of system delays and (non min-
imum phase) zeros, which were not investigated in
1
e.g. (Phan, 1988; Moore, 1998; Amann et al., 1996),
shall be discussed. Secondly, this paper aims to
show that the analysis and design of learning up-
date schemes can be done using standard classical
and modern (optimal) control methods. This holds
even in case of non repeating disturbances and mea-
surement noise, the eect of which can be accounted
for in the ILC design. Two design methods shall
be presented. The optimal control design problems
they entail are of very large dimension in general.
Ecient methods for solving them will be presented.
Only linear SISO discrete time systems are handled.
Extensions to linear multi-variable systems are often
obvious.
The paper is organized as follows. First, the lifted
system representation and the corresponding ILC
design problem will be explained in Section 2. Then,
in Section 3 we will, using the internal model prin-
ciple, investigate the steady state convergence prop-
erties of the ILC and propose a general update law
which will yield asymptotic convergence if the rep-
etition dynamics are stabilized. The stabilization is
done by means of static output feedback, the design
of which is the main subject of Sections 4 and 5.
Section 4 presents a minimum variance approach to
ILC design based on the well-known Kalman lter;
this approach enables to trade o convergence speed
against the minimization of the eect of stochas-
tic disturbances. A single tuning factor determines
how both objectives are compromised, however, the
trade-o between convergence speed and variance
minimization is still done in an intuitive fashion.
Therefore, in Section 5 we propose a multi-objective
formulation of the ILC design problem where the
convergence rate is optimized subject to a prespeci-
ed bound on the closed loop variance. The design
methods described in the paper are demonstrated on
a representative example of a fourth order mechan-
ical system in Section 6. Finally, Section 7 presents
some conclusions.
2 Lifted system representation
A typical feedback control system is given in Fig-
ure 1. P is the plant, K a feedback controller, y
the measured output, u
K
the controller output, w
disturbances, v measurement noise and r the refer-
ence signal. Finally, u is a feedforward control. The
SISO LTI plant P is given by the following state-
space equation (without loss of generality no direct
feed through is assumed)
P :
_
x
p
k+1
= Ax
p
k
+B(u
K,k
+u
k
) +B
d
w
k
y
k
= Cx
p
k
+v
k
(1)
where w and v are assumed Gaussian white noise
with covariance matrices respectively W and V .
K P

e
u
K
u
v
w
y
r
Fig. 1: Feedback control system with additional
feedforward
The controller K is given as follows
K :
_
x
c
k+1
= A
K
x
c
k
+B
K
(r
k
y
k
)
u
K,k
= C
K
x
c
k
+D
K
(r
k
y
k
)
(2)
With K and P as dened above, the closed loop
system is given as follows
T :
_
x
k+1
= Ax
k
+B v
k
e
k
= Cx
k
+D v
k
(3)
with
_
A B
C D
_
=
_
A B
1
B
2
B
3
B
4
C D
1
D
2
D
3
D
4
_
=
_
_
ABD
K
C BC
K
B B
d
BD
K
BD
K
B
K
C A
K
0 0 B
K
B
K
C 0 0 0 1 1
_
_
(4)
and where v
k
= [u
k
, w
k
, v
k
, r]
T
are the external sig-
nals. We now consider the behavior of this system
for a specic task, which is to follow a desired ref-
erence signal r of nite length N, starting from an
initial condition x
0
. We assume that this same task
is repeated over a sequence of trials, the aim of It-
erative Learning Control being to compute a feed-
forward signal u in such a way as to converge to a
suciently small error e.
The technique of lifting refers to a construction
whereby one lifts a continuous-time signal to a
discrete-time one, see (Bamhieh et al., 1991). The
idea of signal and system lifting is at the basis of
sampled data system analysis and control and it is
extremely well applicable to the analysis and syn-
thesis of ILC schemes. For discrete time systems
lifting boils down to a very straightforward con-
struction, as will be shown next. We introduce the
lifted input u
l
to represent the input to the system
in the l
th
trial: u
l
R
N
:= [u
l
0
, u
l
1
, . . . , u
l
N1
]
T
. The
lifted representation of the error in trial l is given by
e
l
R
N
:= [e
l
0
, e
l
1
, . . . , e
l
N1
]
T
. Based on the lifted
representation of all the signals the lifted system is
given by
P : e
l
= P
1
u
l
+P
2
w
l
+P
3
v
l
+P
4
r +P
x
x
l
0
(5)
2
where P
1
: R
N
R
N
, P
2
: R
Nnw
R
N
, P
3
:
R
N
R
N
, and P
4
: R
nx
R
N
are Toeplitz matri-
ces given by
P
i
:
_
_
_
_
_
G
i
0
0 . . . 0
G
i
1
G
i
0
. . . 0
.
.
.
.
.
.
.
.
.
.
.
.
G
i
N1
G
i
N2
. . . G
i
0
_
_
_
_
_
(6)
and P
x
is given by
P
x
:
_
C
T
(CA)
T
. . . (CA
N1
)
T
_
T
(7)
G
i
0
= D
i
, G
i
k
= CA
k1
B
i
, k = 1, 2, . . ., are the
Markov parameters of system (3) Note that the
lifted representation of the plant is a static, multi-
variable system with N outputs.
2.1 Properties of the lifted system
The lifted system representation given by (5) was
used earlier by e.g. (Phan, 1988; Moore, 1998).
However, so far not much attention has been paid
to the properties of such systems in relation to ILC.
From traditional ILC design it is known that special
attention needs to be paid to the role of system de-
lays and non-minimum phase zeros. Their eect on
the characteristics of the lifted system will be stud-
ied next. We will consider the map P
1
only, since its
role is essential in the design of learning controllers.
Obviously, the matrix P
1
will loose rank in case one
or more of the leading Markov parameters are zero.
Suppose G
1
0
, . . . , G
1
d1
= 0, G
1
d
= 0. Then, the rank
of P
1
will be N d. Further, if the system (3) has
zeros outside the unit disk, then the gap between
the largest and the smallest non-zero singular value
of P
1
will generally be large. This can be seen as
follows. Let the non-minimum phase zeros of (3) be
given by a
1
, a
2
, . . . , a
nz
. Then, by the denition of
zeros, for each zero a
i
there exists an initial condi-
tion x
i
and an input u
i
= [a
0
i
, a
1
i
, . . . , a
N1
i
]
T
such
that
y = P
1
u
i
+P
x
x
i
= 0 (8)
Using this expression, an upper bound for the small-
est non-zero singular value of P
1
is given by
(P
1
) <
P
x
x
i

u
i

(9)
Since the norm of u
i
increases fast for increasing N,
the upper bound on (P
1
) will tend to zero for large
N. The rank deciency of P
1
and the large gap be-
tween the smallest and the largest non-zero singular
values implies that there exists a subspace of input
signals that is not, respectively poorly observable in
the measured output. This is undesirable for techni-
cal reasons which will become clear in the remainder
of this paper. One way to improve the condition of
P
1
is by removing one or more columns, correspond-
ing to a shortening of the length of the input vector.
Another way will be presented next.
Reduced system representation Let P
1

R
NN
have rank N d. Also, let its singular value
decomposition be given by P
1
= UV
T
, with =
diag(
1
,
2
, . . . ,
N
). According to Mirsky (1960)
the best rank r approximation of P
1
(r N d)
in any unitarily invariant matrix norm is given by
P
r
1
= Udiag(
1
, . . . ,
r
, 0, . . . , 0)V
T
. The null space
of P
r
1
is spanned by the last N r columns of
V . This suggests a parametrization of the inputs
u = U
p
u where U
p
contains the rst r columns of
V and u R
r
is the new input. The generalized
reduced representation of the (feedforward part of
the) lifted system is then given by y =

P
1
u where

P
1
= P
r
1
U
p
. Using this type of model reduction, we
can remove those input directions which do not con-
tribute to the system output signicantly. The re-
duction proposed here is done on the basis of system
knowledge only, a further reduction can be achieved
if knowledge on the reference signal is used as well,
see e.g. (Hamamoto and Sugie, 1999). In the re-
mainder of the paper we will stick to the notation
y = P
1
u, with P
1
of dimensions N times N
u
, for
the feedforward part of the lifted system, however
all results apply to any rank r order approximation
of P
1
.
2.2 Learning control design problem
In literature on ILC the initial condition x
0
is of-
ten assumed zero, or at least constant for each trial.
This assumption is not very realistic and not nec-
essary in our approach. Instead, we will assume x
0
to be a random, zero mean variable with covariance
X, where X is the stationary closed loop variance
of closed loop system (3):
X = AXA
T
+
_
B
d
WB
T
d
0
0 B
K
V B
T
K
_
(10)
The random variables w
l
, v
l
and x
l
0
are uncorre-
lated, hence an alternative, compact formulation of
the lifted system (5) is given by
P : e
l
= P
4
r +P
1
u
l
+G v (11)
where v R
N
is a random variable with covariance
matrix I and G is dened by the Choleski factor
of the composite covariance matrix, i.e. GG
T
=
P
2
WP
T
2
+P
3
V P
T
3
+P
x
XP
T
x
. This compact, lifted
representation of the plant is depicted in Figure 2.
The ILC design problem now can be dened as to
nd a strict causal feedback control law u = L(e)
3
L
P

u
v
r
e
Fig. 2: The lifted plant; dashed line: interconnec-
tion with learning controller.
such that the feedback interconnection of L and P
is stable and such that e converges to a suciently
small value. We emphasize: learning control is feed-
back control!
3 The internal model principle -
asymptotic convergence
We will now discuss the structure of L. Typical
update laws in ILC are of the form:
L : u
l+1
= u
l
+Ke
l
(12)
To explain why this is a proper structure we use the
internal model principle. In absence of noise, we
want our closed loop system to track the xed
1
ref-
erence input r asymptotically. The internal model
principle states that we can achieve this if we in-
clude a model of the reference dynamics, for a con-
stant reference this is a bank of integrators, in the
controller. This is exactly what happens if we use an
update law like (12). The internal model principle
assumes that the number of inputs will be at least
equal to the number of outputs. Recall that P
1
will
generally not be square, so that the internal model
principle does not apply. We will next investigate
the steady state error properties of the non-square
ILC scheme.
We rst combine the integrating dynamics of the
control law with the expression of the lifted plant to
derive a new extended plant as follows:

P :
_
u
l+1
= u
l
+ u,
e
l
= P
4
r +P
1
u
l
+G v
(13)

P describes the dynamics of the learning control

scheme in the repetition domain. For any stabiliz-
ing control u
l
= Ke
l
and in the absence of noise,
the learning iteration will converge to a xed point
u

which is given by
KP
4
r +KP
1
u

= 0 (14)
1
The repeating reference signal constitutes a vector of con-
stant reference inputs for the lifted system.
Since P
1
has full column rank, we can choose K
such that KP
1
is non-singular. Then, the steady
state tracking error is given by
e

= P
4
r P
1
(KP
1
)
1
KP
4
r (15)
which is not equal to zero in general. Zero error
tracking can be achieved i r is chosen such that
P
4
r = P
1
z, for any z R
N
u
. One way to achieve
this is by minimizing in least square sense the dif-
ference between r and an ideally desired reference
r

, e.g.
min
r,z
(r r

)
T
Q

(r r

)
s.t. P
4
r +P
1
z = 0
(16)
Remark 1 In the previous, we considered feedback
laws of the form u
l+1
= u
l
+ Ke
l
. In literature,
e.g. (de Roover, 1997), often a so-called robustness
lter Q is included, yielding u
l+1
= Qu
l
+ Ke
l
. It
can easily be shown that this learning scheme will
converge to zero steady state o-set only if Q = I.
For this reason, we decided to omit Q. It is our
proposition that zero steady state o-set needs not
be be sacriced for the sake of robustness and noise
attenuation; in this paper, we will show how we deal
with noise through a proper design of K.
4 Minimum variance learning
A straightforward design strategy for the learning
gain is by static decoupling and pole placement, see
e.g. (Amann et al., 1996). However, such a design
strategy does not take the eect of the stochastic
initial condition, process disturbances and measure-
ment noise into account explicitly. In this section,
we propose a design method which enables the user
to trade-o convergence speed against the amplica-
tion of noise and disturbances. The proposed min-
imum variance static output feedback design prob-
lem is given as follows
min
K
J = lim
M
1
M
E
_

M
l=1
u
l
T
u
l
_
s.t. u
l+1
= u
l
+ u
l
+Fw
l
e
l
= P
1
u
l
+G v
l
u
l
= Ke
l
(17)
where the variable w
l
with covariance I is added for
regularization of the problem. Note that omitting
w
l
would yield a singular optimal control problem.
If F is chosen equal to I with a scalar , then the
choice of determines the trade-o between the am-
plication of noise and the convergence speed. The
gain that solves (17) can be obtained by solving the
corresponding estimation Riccati equation. Using
the interpretation of K as the Kalman gain, we can
develop some intuition for the choice of . Choosing
4
close to zero will yield a low-gain K, hence will
result in slow convergence and a low variance of the
controls and the error. If we choose 0 then the
solution will tend to the dead beat control solution
discussed in e.g (Phan, 1988). Hence, we can use
to tune the performance of the learning controller.
The solution of the estimation Riccati equation
for problem (17) can be computed very eciently
via the transformation to a purely diagonal design
problem. The transformation proceeds as follows.
Let the generalized singular value decomposition of
(P
1
F)
T
and G
T
be given as follows
(P
1
F)
T
= U
P
CX
T
, G
T
= U
G
SX
T
(18)
where U
P
and U
G
are unitary matrices, X is a
square matrix of dimension N, S is a diagonal
matrix of dimension N, and C = [

C 0] where

C R
N
u
N
u
is a diagonal matrix. We introduce the
state transformation u = T
1
u, and the output trans-
formation e = T
2
e, with transformation matrices
given by T
1
= U
T
P
F
1
, and T
2
= X
1
respectively.
Then, the problem of nding the Kalman gain K
for the system in problem (17) is equivalent to the
problem of nding a Kalman gain

K = T
1
KT
2
for
the system u
l+1
= u
l
+ w
l
, e
l
= C
T
u
l
+ S v
l
, where
the covariances of w and v are I. It can be easily
shown that because the last N N
u
rows of C
T
are zero and S is diagonal the last N N
u
rows
of the Kalman gain for the transformed system will
be zero as well. Hence, the Kalman gain is given
by

K = [

K 0], where

K is the Kalman gain for
the square and diagonal system which is obtained
by truncating the last N N
u
rows of the output.
For this diagonal problem, the Riccati solution and
the corresponding Kalman gain can be obtained by
simple element-wise computations.
5 Multi-objective design: conver-
gence speed vs. noise attenuation
Ultimately, we would like to synthesize a learning
gain in such a fashion that we can directly control
the trade-o between noise attenuation and conver-
gence speed. In this section such a design strategy
is proposed. If r is chosen such that P
4
r = P
1
z,
z R
N
u
then the lifted system representation can
be transformed into
u
l+1
= u
l
+ u
l
e
l
= P
1
u
l
+Gv
l
(19)
where u = u+z and with a non-zero initial condition
u
0
= z. We seek a static output control u
l
=
Ke
l
such that e converges to a small value according
to some control objectives. Let us rst investigate
the objectives we would like to achieve.
Variance reduction One of the goals in the
multi-objective design is to limit the steady state
variance of the feedforward signal and of the error.
Traditional H
2
control minimizes the trace of the co-
variance. However, because of the interpretation of
u as the lifted input or, in case an input parametriza-
tion is used (see Section 2.1), as the coecients in
the input parametrization, it is in our case more ap-
propriate to consider the variance of each state as
an individual objective. This amounts to dening
the following objectives:
J

i
= lim
M
1
M
E
_

M
l=1
u
l2
i
_
, i = 1, . . . , N
u
(20)
Convergence speed Next to limiting the eect
of noise on the feedforward and error signal we re-
quire a certain speed of convergence of the learning
controller. One way of doing this is by constrain-
ing the closed loop eigenvalues to lie in a prescribed
region within the unit circle. However, a fast conver-
gence of the learning error in for example a 2-norm
sense, does not necessarily mean that all eigenvalues
need to lie within a prescribed region. Also, the en-
forcement of all closed loop eigenvalues may lead to
a high gain feedback which can result in poor vari-
ance results. Therefore, we will consider a dierent
objective here, namely a deterministic LQR objec-
tive. In the presence of noise, this objective can be
dened as follows
J

l=1
E( u
l
)
T
QE( u
l
)+E( u
l
)
T
RE( u
l
) (21)
where E( u
l
) represents the deterministic component
in u
l
due to the eect of the non-zero initial condi-
tion u
0
= z.
Multi-objective design The proposed multi-
objective design problem consists of nding a static
output feedback K such that the LQR objective is
minimized while satisfying a prespecied bound on
the closed loop variance. Mathematically:
min
K
J

s.t. J

i
, i = 1, . . . , N
u
(22)
This type of multi-objective problem has no
straightforward solution in general. However, if we
choose Q = I and R = I, then the optimal solu-
tion can be found easily. To demonstrate this, we
diagonalize the system by applying the input/state
transformation u = T
1
u, u = T
1
u and the out-
put transformation e = T
2
e. The state/input trans-
formation is given by T
1
= U
T
P
and the output trans-
formation is given by T
2
= [

C
1
O]X
1
, where U
P
,
X,

C and O are given by the generalized singular
value decomposition of P
T
1
and G
T
:
P
T
1
= U
P
[

C O]X
T
, G
T
= U
G
SX
T
(23)
5
This renders the following transformed system
u
l+1
= u
l
+ u
l
e
l
= u
l
+

C
1
S v
l
(24)
The objective functions, expressed in the trans-
formed variables become
J

i
= lim
M
1
M
E
_

M
l=1
u
l2
i
_
, i = 1, . . . , N
u
(25)
J

l=1
E( u
l
)
T
E( u
l
) +E( u
l
)
T
E( u
l
) (26)
The control problem dened by system (24) and ob-
jectives (25,26) is diagonal, hence its solution will be
a diagonal feedback. Let the i-th diagonal element
of

C, S and

K be denoted respectively c
i
, s
i
and k
i
.
Then, J

and J

can be expressed in terms of the so-

lution of the scalar, closed loop Lyapunov functions
of the individual loops,
J

i
= p

i
=
s
2
i
k
2
i
c
2
i
(k
2
i
+ 2k
i
)
(27)
J

=
N
u

i=1
p

i
=
N
u

i=1
_
(k
2
i
+ 1)
k
2
i
+ 2k
i
_
(28)
for 2 < k
i
< 0. The minimization of J

boils
down to the minimization of the individual contri-
butions p

i
, hence the optimal feedback gain is found
by solving a series of N
u
convex (!) scalar, optimiza-
tion problems:
min
k
i
(k
2
i
+ 1)
k
2
i
+ 2k
i
s.t.
s
2
i
k
2
i
c
2
i
(k
2
i
+ 2k
i
)
(29)
2 < k
i
< 0
By inspection, the solution is given by
k
i
= max(k

i
, k

i
) (30)
where k

i
is obtained by solving the variance con-
straint p

i
to equality, and k

i
minimizes p

i
:
k

i
=
2c
2
i
c
2
i
+s
2
i
(31)
k

i
=
1

1 + 4
2
(32)
The feedback gain for the original problem is found
by back transformation: K = T
1

KT
2
.
6 Example
As an example, we consider the servo control prob-
lem for a typical fourth order mechanical system.
1 10 100
10
10
10
8
10
6
10
4
10
2
10
0
svdnumber []
s
v
d

v
a
l
u
e

[

]
Fig. 3: Singular values of P
1
.
The plant and the feedback controller are respec-
tively given by
_
A B
C D
_
=
_
_
_
_
3.9050 1.4510 0.9732 0.4970 0.0019
4.000 0 0 0 0
0 1.000 0 0 0
0 0 0 0.5000 0
10
3
(0.4780 1.3065 1.3049 0.2381) 0
_
_
_
_
_
A
K
B
K
C
K
D
K
_
=
_
0.9048 1
0.8564 10
_
(33)
with a sampling time of 0.5 ms. A servo task with
a duration of 200 samples is considered. Measure-
ment noise with a covariance of 10
4
is assumed.
The singular values of P
1
are plotted in Figure 3.
The last 4 singular values of P
1
are computed to
be [5.79 10
9
, 2.26 10
9
, 5.23 10
24
, 0]. In ac-
cordance with the analysis presented in Section 2.1
this suggest that the leading Markov parameter of
the process sensitivity system is zero, which is in-
deed the case. Also, the very small non-zero singular
value rightly indicates the presence of a non mini-
mum phase zero. Further, from Figure 3 it is obvi-
ous that there exists a large input subspace which
contributes negligibly to the output. This subspace
corresponds to the high-frequency input signals: in-
deed the process sensitivity of the closed loop system
has a high-frequency roll-o. We will base our ILC
design on a 40
th
order norm-optimal approximation
of P
1
as in Section 2.1. The servo task is dened as
to move with constant velocity from position 0 to
position 1 in 25 ms, starting at time 25 ms. Based
on these operating specications we compute a refer-
ence signal via a least squares optimization problem
like (16) where we add a set of inequality constraints
to account for input saturation (at -4 respectively
+4). The resulting reference trajectory and feed-
forward signal are displayed in Figure 4. We now
consider the design and implementation of a min-
imum variance learning controller as described in
Section 4. In Figure 5 the convergence of the out-
put and the feedforward are displayed for the case
6
0 20 40 60 80 100
0.5
0
0.5
1
1.5
p
o
s
i
t
i
o
n
0 20 40 60 80 100
5
0
5
time [ms]
f
e
e
d
f
o
r
w
a
r
d
Fig. 4: Reference trajectory and nominal feedfor-
ward.

2
(u)
2
(e)
0.03 5.76 2.08e-2
0.1 19.23 2.11e-2
0.3 57.56 2.17e-3
Table 1: Steady state input and error variance for
the three Kalman designs.
of measurement noise and non-zero initial conditions
for a choice of F = I, with equal to 0.1. The
decay of the summed square error for this design is
plotted in Figure 6, together with the decay for two
other choices of , 0.03 and 0.3. The steady state
input and error variances for the three designs are
given in Table 1. Note that reducing yields a sig-
nicantly decreased eect of the measurement noise
on the input variance, however only at the expense
of a signicant reduction in convergence speed.
Next, a multi-objective design is done. In Figure 7
the convergence of the output and the feedforward
are displayed for = 0.1 and = 1. Note that,
despite the tight variance constraint, a reasonably
quick convergence is established. The decay of the
summed square error for designs with respectively
= 0.01, 0.1 and 1 is plotted in Figure 6. Note
that the initial decay is fast for all three designs.
Compare this to Figure 6 where convergence speed
is largely sacriced for the sake of a limited closed
loop covariance. This clearly indicates the benets
of the multi-objective design approach. Finally, the
closed loop input and error variance for the three
designs is given in Table 2.
0 20 40 60 80 100
0.5
0
0.5
1
1.5
p
o
s
i
t
i
o
n
0 20 40 60 80 100
5
0
5
time [ms]
f
e
e
d
f
o
r
w
a
r
d
Fig. 5: Output and feedforward signal for Kalman
ILC design with = 0.1, solid: trial 1,
dashed: trial 2, dotted: trial 10.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
10
1
10
0
10
1
iteration number
s
u
m
m
e
d

s
q
u
a
r
e
d

e
r
r
o
r
s
Fig. 6: Decay of summed square error for the rst
20 iterations using Kalman ILC design,
solid: = 0.03, dashed: = 0.1, dotted:
= 0.3.
7 Conclusions
In this paper we presented a systematic approach
to the analysis and synthesis of Iterative Learning
Controllers based on a so-called lifted system rep-
resentation. In this representation the dynamics
of the learning mechanism are described by a set
of dierence equations. The actual plant dynamics
transform to a static map in the lifted domain. It is
shown that system delays and non-minimum phase
zeros lead to respectively singularity and a bad con-
dition of this static map. This can be resolved using
a reduced order representation of the lifted system.
By applying the internal model principle we showed
that the integrating learning update law is a good
structural choice since it provides zero error steady
state tracking in the absence of noise for any stabi-
lizing gain. A Kalman lter based design method for
this learning gain was presented next, which takes
the presence of stochastic initial conditions, process
noise and measurement noise into account explic-
7
0 20 40 60 80 100
0.5
0
0.5
1
1.5
p
o
s
i
t
i
o
n
0 20 40 60 80 100
5
0
5
time [ms]
f
e
e
d
f
o
r
w
a
r
d
Fig. 7: Output and feedforward signal for the multi
objective ILC design with = 0.1, = 1.
Solid: trial 1, dashed: trial 2, dotted: trial
10.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
10
1
10
0
10
1
iteration number
s
u
m
m
e
d

s
q
u
a
r
e
d

e
r
r
o
r
s
Fig. 8: Decay of summed square error for the rst
20 iterations using multi-objective ILC de-
sign with = 1. Solid: = 0.01, dashed:
= 0.1, dotted: = 1.
itly. Finally, we presented a multi-objective design
approach in which a deterministic quadratic crite-
rion is minimized, penalizing the error as well as
the change in the feedforward signals, subject to a
bound on the steady state covariance of every sin-
gle input. Ecient solutions, based on diagonalizing
state and output transformations were proposed for
the two optimal control methods presented. The de-
sign strategies are tested on a realistic example, a
fourth order mechanical system. The results clearly
demonstrate the potential of the multi-objective de-
sign.
References
Amann, N., Owens, D.H. and E. Rogers (1996). Iter-
ative learning control using optimal feedback and
feedforwad actions, Int. Journal of Control, 65,
277-293.
Arimoto, S. Kawamura, S. and F. Miyazaki (1984).

2
(u)
2
(e)
0.01 0.37 2.09e-2
0.1 3.29 2.14e-2
1 22.48 2.18e-2
Table 2: Steady state input and error variance for
the three multi-objective designs with =
1.
Bettering operation of robots by learning, Journal
of Robotic Systems, 1, 123-140.
Bamieh, B., Boyd Pearson, J., Francis, B.A. and A.
Tannenbaum (1991). A lifting technique for linear
periodic systems with applications to sampled-
data control, Systems & Control Letters, 17, 79-
88.
Bien, Z. and J. Xu (1998). Iterative learning con-
trol: analysis, design, integration and applica-
tions. Kluwer Academic Publishers, Boston.
Roover, D. de (1997). Motion control of a wafer
stage; a design approach for speeding up IC pro-
duction. Ph.D. Thesis, Delft Univ. Techn., The
Netherlands.
Golub, G.H. and C.F. van Loan (1983). Matrix com-
putations. The John Hopkins University Press,
Baltimore.
Hamamoto, K. and T. Sugie (1999). An itera-
tive learning control algorithm within prescribed
input-output subspace, Proc. of the 14th World
Congress of IFAC, 501-506.
Mirsky, L. (1960). Symmetric gauge functions and
unitarily invariant norms, Q. Journal of Mathe-
matics, (2), 11, 50-59.
Moore, K.L. (1998). Multi-loop control approach to
designing iteratively learning controllers, Proc. of
the 37
th
IEEE CDC, 666-671.
Phan, M.A. and R.W. Longman (1988). A math-
ematical theory of learning control for lin-
ear discrete multivariable systems, Proc. of the
AIAA/AAS Astrodyna-mics Specialist Confer-
ence, Minneapolis, MN, 740-746.
8