(George A. Rovithakis Manolis A. Christodoulou) Adaptive Control With Recurrent High Order Neural Networks

Advances in Industrial Control
Springer
London
Berlin
Heidelberg
New York
Barcelona
Hong Kong
Milan
Paris
Santa Clara
Singapore
Tokyo
Other titles published in this Series:
Control ofModern Integrated Power Systems
E. Mariani and S.S. Murthy
Advanced Load Dispatch for Power Systems: Principles, Practices and Economies
E. Mariani and S.S. Murthy
Supervision and Control for Industrial Processes
Bjorn Sohlberg
Modelling and Simulation ofHuman Behaviour in System Control
Pietro Carlo Cacciabue
Modelling and Identification in Robotics
Krzysztof Kozlowski
Spacecraft Navigation and Guidance
Maxwell Noton
Robust Estimation and Failure Detection
Rami Mangoubi
Adaptive Internal Model Control
Aniruddha Datta
Price-Based Commitment Decisions in the Electricity Market
Eric Allen and Marija Hie
Compressor Surge and Rotating Stall
Jan Tommy Gravdahl and Olav Egeland
Radiotherapy Treatment Planning
Oliver Haas
Feedback Control Theory For Dynamic Traffic Assignment
Pushkin Kachroo and Kaan 6zbay
Control Instrumentation for Wastewater Treatment Plants
Reza Katebi, Michael A. Johnson and Jacqueline Wilkie
Autotuning ofPID Controllers
Cheng-Ching Yu
Robust Aeroservoelastic Stability Analysis
Rick Lind & Marty Brenner
Performance Assessment of Control Loops:Theory and Applications
Biao Huang & Sirish L. Shah
Data Mining and Knowledge Discovery for Process Monitoring and Control
XueZ. Wang
Advances in PID Control
Tan Kok Kiong, Wang Quing-Guo & Hang Chang Chieh with Tore J. Hagglund
George A. Rovithakis
and Manolis A. Christodoulou
Adaptive Control
with Recurrent High-order
Neural Networks
Theory and Industrial Applications
With 30 Figures
Springer
George A. Rovithakis, PhD
Manolis A. Christodoulou, PhD
Department of Electronic and Computer Engineering,
Technical University of Crete,
GR-73100 Chania,
Crete,
Greece.
British Library Cataloguing in Publication Data

Rovithakis, George A.
Adaptive control with recurrent high-order neural networks
: theory and industrial applications. - (Advances in
industrial control)
l.Adaptive control systems 2. Neural networks (Computer
science)
I. Title II. Christodoulou, Manolis A.
629.8'36
ISBN-13: 978-1-4471-1201-3 e-ISBN-13: 978-1-4471-0785-9
DOl: 10.1007/978-1-4471-0785-9
Library of Congress Cataloging-in-Publication Data
A catalog record for this book is available from the Library of Congress
Apart from any fair dealing for the purposes of research or private study, or criticism or review, as
permitted under the Copyright, Designs and Patents Act 1988, this publication may only be reproduced,
stored or transmitted, in any form or by any means, with the prior permission in writing of the
publishers, or in the case of repro graphic reproduction in accordance with the terms of licences issued
by the Copyright Licensing Agency. Enquiries concerning reproduction outside those terms should be
sent to the publishers.
Springer-Verlag London Limited 2000

Softcover reprint of the hardcover I st edition 2000
The use of registered names, trademarks, etc. in this publication does not imply, even in the absence of a
specific statement, that such names are exempt from the relevant laws and regulations and therefore
free for general use.
The publisher makes no representation, express or implied, with regard to the accuracy of the
information contained in this book and cannot accept any legal responsibility or liability for any errors
or omissions that may be made.
Typesetting: Camera ready by authors
69/3830-543210 Printed on acid-free paper SPIN 10728731

Advances in Industrial Control
Series Editors
Professor Michael J. Grimble, Professor ofIndustrial Systems and Director

Professor Michael A. Johnson, Professor of Control Systems and Deputy Director
Industrial Control Centre
Department of Electronic and Electrical Engineering
University of Strathdyde
Graham Hills Building
50 George Street
GlasgowG11QE
United Kingdom
Series Advisory Board
Professor Dr-Ing J. Ackermann

DLR Institut fur Robotik und Systemdynamik
Postfach 1116
D82230 WeBling
Germany
Professor I.D. Landau

Laboratoire d'Automatique de Grenoble
ENSIEG, BP 46
38402 Saint Martin d'Heres
France
Dr D.C. McFarlane
Department of Engineering
University of Cambridge
Cambridge CB2 1QJ
United Kingdom
Professor B. Wittenmark
Department of Automatic Control
Lund Institute of Technology
PO Box 118
S-221 00 Lund
Sweden
Professor D.W. Clarke

Department of Engineering Science
University of Oxford
Parks Road
Oxford OX1 3PJ
United Kingdom
Professor Dr -Ing M. Thoma
Institut fiir Regelungstechnik
Universitiit Hannover
Appelstr. 11
30167 Hannover
Germany
Professor H. Kimura
Department of Mathematical Engineering and Information Physics
Faculty of Engineering
The University of Tokyo
7-3-1 Hongo
Bunkyo Ku
Tokyo 113
Japan
Professor A.J. Laub

College of Engineering - Dean's Office
University of California
One Shields Avenue
Davis
California 95616-5294
United States of America
Professor J.B. Moore

Department of Systems Engineering
The Australian National University
Research School of Physical Sciences
GPO Box4
Canberra
ACT 2601
Australia
Dr M.K. Masten
Texas Instruments
2309 Northcrest
Plano
TX 75075
United States of America
Professor Ton Backx

AspenTech Europe B.V.
DeWaal32
NL-5684 PH Best
The Netherlands
SERIES EDITORS' FOREWORD
The series Advances in Industrial Control aims to report and encourage

technology transfer in control engineering. The rapid development of control
technology has an impact on all areas of the control discipline. New theory, new
controllers, actuators, sensors, new industrial processes, computer methods,
new applications, new philosophies ... , new challenges. Much of this
development work resides in industrial reports, feasibility study papers and the
reports of advanced collaborative projects. The series offers an opportunity for
researchers to present an extended exposition of such new work in all aspects of
industrial control for wider and rapid dissemination.
Neural networks is one of those areas where an initial burst of
enthusiasm and optimism leads to an explosion of papers in the journals and
many presentations at conferences but it is only in the last decade that
significant theoretical work on stability, convergence and robustness for the use
of neural networks in control systems has been tackled. George Rovithakis and
Manolis Christodoulou have been interested in these theoretical problems and in
the practical aspects of neural network applications to industrial problems. This
very welcome addition to the Advances in Industrial Control series provides a
succinct report of their research.
The neural network model at the core of their work is the Recurrent
High Order Neural Network (RHONN) and a complete theoretical and
simulation development is presented. Different readers will find different
aspects of the development of interest. The last chapter of the monograph
discusses the problem of manufacturing or production process scheduling.
Based on the outcomes of a European Union ESPRIT funded project, a full
presentation of the application of the RHONN network model to the scheduling
problem is given. Ultimately, the cost implication of reduced inventory holdings
arising from the RHONN solution is discussed. Clearly, with such an excellent
mix of theoretical development and practical application, this monograph will
appeal to a wide range of researchers and readers from the control and
production domains.
M.J. Grimble and M.A. Johnson

Industrial Control Centre
Glasgow, Scotland, UK
PREFACE
Recent technological developments have forced control engineer~ to deal with

extremely complex systems that include uncertain, and possibly unknown,
nonlinearities, operating in highly uncertain environments. The above, to-
gether with continuously demanding performance requirements, place con-
trol engineering as one of the most challenging technological fields. In this
perspective, many "conventional" control schemes fail to provide solid de-
sign procedures, since they mainly require known mathematical models of
the system and/or make assumptions that are often violated in real world
applications. This is the reason why a lot of research activity has been con-
centrated on "intelligent" techniques recently.
One of the most significant tools that serve in this direction, is the so called
artificial neural networks (ANN). Inspired by biological neuronal systems,
ANNs have presented superb learning, adaptation, classification and function-
approximation properties, making their use in on line system identification
and closed-loop control promising.
Early enrolment of ANNs in control exhibit a vast number of papers
proposing different topologies and solving various application problems. Un-
fortunately, only computer simulations were provided at that time, indicating
good performance. Before hitting real-world applications, certain properties
like stability, convergence and robustness of the ANN-based control archi-
tectures, must be obtained although such theoretical investigations though
started to appear no earlier than 1992.
The primary purpose of this book is to present a set of techniques, which
would allow the design of
controllers able to guarantee stability, convergence and robustness for dy-
namical systems with unknown nonlinearities
real time schedulers for manufacturing systems.
To compensate for the significant amount of uncertainty in system struc-
ture, a recently developed neural network model, named Recurrent High Or-
der Neural Network (RHONN), is employed. This is the major novelty of this
book, when compared with others in the field. The relation between neural
and adaptive control is also clearly revealed.
It is assumed that the reader is familiar with a standard undergraduate
background in control theory, as well as with stability and robustness con-
X Preface
cepts. The book is the outcome of the recent research efforts of its authors.
Although it is intended to be a research monograph, the book is also useful
for an industrial audience, where the interest is mainly on implementation
rather than analyzing the stability and robustness of the control algorithms.
Tables are used to summarize the control schemes presented herein.
Organization of the book. The book is divided into six chapters. Chap-
ter 1 is used to introduce neural networks as a method for controlling un-
known nonlinear dynamical plants. A brief history is also provided. Chapter
2 presents a review of the recurrent high-order neural network model and an-
alyzes its approximation capabilities based on which all subsequent control
and scheduling algorithms are developed. An indirect adaptive control scheme
is proposed in Chapter 3. Its robustness owing to unmodeled dynamics is an-
alyzed using singular perturbation theory. Chapter 4 deals with the design
of direct adaptive controllers, whose robustness is analyzed for various cases
including unmodeled dynamics and additive and multiplicative external dis-
turbances. The problem of manufacturing systems scheduling is formulated
in Chapter 5. A real time scheduler is developed to guarantee the fulfillment
of production demand, avoiding the buffer overflow phenomenon. Finally, its
implementation on an existing manufacturing system and comparison with
various conventional scheduling policies is discussed in Chapter 6.
The book can be used in various ways. The reader who is interested in
studying RHONN's approximation properties and its usage in on-line system
identification, may read only Chapter 2. Those interested in neuroadaptive
control architectures should cover Chapters 2, 3 and 4, while for those wishing
to elaborate on industrial scheduling issues, Chapters 2, 5 and 6 are required.
A higher level course intended for graduate students that are interested in
a deeper understanding of the application of RHONNs in adaptive control
systems, could cover all chapters with emphasis on the design and stability
proofs. A course for an industrial audience, should cover all chapters with
emphasis on the RHONN based adaptive control algorithms, rather than
stability and robustness.
Chania, Crete, Greece George A. Rovithakis
August 1999 Manolis A. Christodoulou
CONTENTS
1. Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 General Overview ...................................... 1
1.2 Book Goals & Outline .................................. 7
1.3 Notation.............................................. 8
2. Identification of Dynamical Systems Using

Recurrent High-order Neural Networks. . . . . . . . . .. . . . . . . . . 9
2.1 The RHONN Model. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 10
2.1.1 Approximation Properties . . . . . . . . . . . . . . . . . . . . . . . .. 13
2.2 Learning Algorithms. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 15
2.2.1 Filtered Regressor RHONN . . . . . . . . . . . . . . . . . . . . . . .. 16
2.2.2 Filtered Error RHONN ........................... 19
2.3 Robust Learning Algorithms. . . .. . . .. . . . . . . . . . . . . . . .. . . .. 20
2.4 Simulation Results. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 25
Summary .................................................. 27
3. Indirect Adaptive Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 29

3.1 Identification .......................................... 29
3.1.1 Robustness of the RHONN Identifier Owing to Un-
modeled Dynamics. .. . . .. . . .. . . .. . . .. . . .. . . . . . . .. 31
3.2 Indirect Control. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 35
3.2.1 Parametric Uncertainty ........................... 36
3.2.2 Parametric plus Dynamic Uncertainties ............. 39
3.3 Test Case: Speed Control of DC Motors. . . . . . . . . . . . . . . . . .. 43
3.3.1 The Algorithm. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 44
3.3.2 Simulation Results ............................... 46
Summary .................................................. 48
4. Direct Adaptive Control. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 53

4.1 Adaptive Regulation - Complete Matching. . .. . . .. . . .. . . .. 53
4.2 Robustness Analysis. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 61
4.2.1 Modeling Error Effects . . . . . . . . . . . . . . . . . . . . . . . . . . .. 62
4.2.2 Model Order Problems. . . . . . . . . . . . . . . . . . . . . . . . . . .. 71
4.2.3 Simulations...................................... 80
XII Contents
4.3 Modeling Errors with Unknown Coefficients. . . . . . . . . . . . . . .. 83

4.3.1 Complete Model Matching at Ixl = 0.. .. . . .. . . .. . . .. 93
4.4 Tracking Problems. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 95
4.4.1 Complete Matching Case ............. , . . .. . . .. . . .. 97
4.4.2 Modeling Error Effects ............................ 102
4.5 Extension to General Affine Systems ...................... 108
4.5.1 Adaptive Regulation .............................. 110
4.5.2 Disturbance Effects ............................... 123
Summary .................................................. 134
5. Manufacturing Systems Scheduling ....................... 137

5.1 Problem Formulation ................................... 145
5.1.1 Continuous Control Input Definition ........... , .... 146
5.1.2 The Manufacturing Cell Dynamic Model ............ 147
5.2 Continuous-time Control Law ............................ 151
5.2.1 The Ideal Case ................................... 152
5.2.2 The Modeling Error Case .......................... 153
5.3 Real-time Scheduling ................................... 155
5.3.1 Determining the Actual Discrete Dispatching Decision 155
5.3.2 Discretization Effects ............................. 157
5.4 Simulation Results ...................................... 159
Summary .................................................. 163
6. Scheduling using RHONNs: A Test Case ................. 165

6.1 Test Case Description ................................... 166
6.1.1 General Description .............................. 166
6.1.2 Production Planning & Layout in SHW ............. 166
6.1.3 Problem Definition ............................... 168
6.1.4 Manufacturing Cell Topology ...................... 169
6.1.5 RHONN Model Derivation ........................ 171
6.1.6 Other Scheduling Policies .......................... 173
6.2 Results & Comparisons ................................. 174
Summary .................................................. 183
References .................................................... 184
Index ......................................................... 191

CHAPTER 1
INTRODUCTION
1.1 General Overview
Man has two principal objectives in the scientific study of his environment:
he wants to understand and to control. The two goals reinforce each other,
since deeper understanding permits firmer control, and, on the other hand,
systematic application of scientific theories inevitably generates new problems
which require further investigation, and so on.
It might be assumed that a fine-grained descriptive theory of terrestrial
phenomena would be required before an adequate theory of control could
be constructed. In actuality this is not the case, and indeed, circumstances
themselves force us into situations where we must exert regulatory and cor-
rective influences without complete knowledge of basic causes and effects. In
connection with the design of experiments, space travel, economics, and the
study of cancer, we encounter processes which are not fully understood. Yet
design and control decisions are required. It is easy to see that in the treat-
ment of complex processes, attempts at complete understanding at a basic
level may consume so much time and so large a quantity of resources as to
impede us in more immediate goals of control.
Artificial Neural Networks have been studied for many years with the hope
of achieving human-like performance in solving certain problems in speech
and image processing. There has been a recent resurgence in the field of neu-
ral networks owing to the introduction of new network topologies, training
algorithms and VLSI implementation techniques. The potential benefits of
neural networks such as parallel distributed processing, high computation
rates, fault tolerance and adaptive capability, have lured researchers from
other fields such as controls, robotics etc. to seek solutions to their compli-
cated problems.
Several types of neural networks appear to offer promise for use in con-
trol systems. These include the multilayer neural network trained with the
commonly attributed to Rumelhart et al., [97], the recurrent neural networks
such as the feedback network of Hopfield, [38], the cerebellar model articula-
tion controller (CMAC) model of Albus, [2], the content-addressable memory
ofKohonen, [55], and the Gaussian node network of Moody and Darken, [69].
The choice of which neural network to use and which training procedure to
G. A. Rovithakis et al., Adaptive Control with Recurrent High-order Neural Networks

2 1. Introduction
invoke is an important decision and varies depending on the intended appli-

cation.
The type of neural network most commonly used in control systems is
the feedforward multilayer neural network, where no information is fed back
during operation. There is, however, feedback information available during
training. Typically, supervised learning methods, where the neural network
is trained to learn input-output patterns presented to it, are used. Most of-
ten, versions of the backpropagation (BP) algorithm are used to adjust the
neural network weights during training. This is generally a slow and very
time consuming process, because the algorithm usually takes a long time
to converge. However, other optimization methods such as conjugate direc-
tions and quasi-Newton have also been implemented; see [36]. Most often,
the individual neuron-activation functions are sigmoidal, but also signum -or
radial-basis Gaussian functions are also used. Note that there are additional
systems and control results involving recurrent networks, as discussed later.
Theoretical studies by several research groups [16],[24], [35],[40], demon-
strated that multilayer neural networks with just one hidden layer can ap-
proximate any continuous function uniformly over a compact domain, by
simply adjusting the synaptic weights, such that a functional of the error
between the neural network output and the output of the unknown map, is
minimized.
The procedure of training a neural network to represent the forward dy-
namics of a plant is called forward modeling. The neural network model is
placed in parallel with the plant and the error between the plant and the net-
work outputs - the prediction error - is used as the network training signal.
At this point, we should mention that the plant can be single-input single-
output or multi-input multi-output, continuous or discrete, linear or nonlin-
ear. For the neural network training, discrete samples of the plant inputs
and outputs are often used. We assume that the plant is described by the
nonlinear difference equation:
yP(k + 1) = f(yP(k), '" yP(k - n + 1); u(k), '" u(k - m + 1)).
Thus, the system output yP at time k + 1 depends on the past n output
values and the past m values of the input u. An obvious approach for system
modeling is to choose the input-output structure of the neural network to be
the same as that of the system. Denoting the output of the network as ym,
we then have:
ym(k + 1) = fapr(yP(k), '" yP(k - n + 1); u(k), '" u(k - m + 1)).
Here, fapr represents the nonlinear input output map of the network, that
is, the approximation of f. We can readily see that the input to the network
includes the past values of the real system output, hence, the system has
no feedback. If we assume that after a certain training period the network
gives a good representation of the plant, that is ym ~ yP, then for subsequent
post-training purposes the network output together with its delay values can
1.1 General Overview 3
be fed back and used as part of the network input. In this way, the network
can be used independently of the plant. Such a network model is described
by
ym(k + 1) = fapr(ym(k), .. , ym(k - n + 1); u(k), .. , u(k - m + 1.
u
PLANT
Fig. 1.1. Plant identification with a multilayer neural network
Suppose now that the information we have about the plant is in the form
of an input-output table, which makes the problem of identification look like a
typical pattern recognition problem; then, for the training of the plant model
the current and previous inputs to the plant, as well as the previous outputs
of the plant should be used again. Other possibilities for the training include
the plant states and derivatives of the plant states.
For this reason, if a feedforward multilayer neural network is used and the
training is done with the BP algorithm, then we realize that since we need
discrete outputs of the plant model, a discrete or discretized continuous plant
has to be considered, as discussed before. This can be illustrated in Figure 1.1.
The arrow that passes through the neural model is indicative of the fact that
the output error is used to train the neural network. As mentioned before,
we see that the discrete inputs of the plant, as well as the discrete outputs of
the plant are used for the training. The number of delays of previous inputs
and outputs is unknown; since we have no information about the structure
of the plant this number has to be determined experimentally. As far as the
training signal is concerned, it has been suggested, [41],[74], that a random
signal uniformly distributed over certain ranges should be used.
Instead of training a neural network to identify the forward dynamics of
the plant, a neural network can be trained to identify the inverse dynamics of
the plant. The neural network's input is the plant's output, and the desired
neural network output is the plant's input. The error difference between the
actual input of the plant and the output of the neural network is to be
4 1. Introduction
minimized and can be used to train the neural network. The desired output
of the neural network is the current input to the plant. When modeling the
inverse dynamics of the plant with a neural network, the assumption is being
made, either implicitly or explicitly, that the neural network can approximate
the inverse of the plant well. This, of course, means that the inverse exists
and it is unique; if not unique then care should be taken with the ranges of
the inputs to the network. It also means that the inverse is stable.
u y
NEURAL
PLANT
CONTROLLER
Fig. 1.2. A neural network controller
REFERENCE 1 reference model output

ym
MODEL J
+
/
NEURAL e
~ PLANT \.
r reference input
u yI'
NEURAL
PLANT
DELAY CONTROLLER plant
output
~
r DELAY
L
f
l
DELAY }-
Fig. 1.3. An open loop control configuration
Once an identification neural model of the plant is available, this model

can be used for the design of the controller, as shown below. A neural net-
work can be used as a conventional controller in both open and closed loop
1.1 General Overview 5
configurations. The training of a neural network as an open loop controller is

shown in Figure 1.2. The error e = Y - Yd is used to train the neural network.
Since we do not have a desired output for the neural controller, the error
at the output of the plant is backpropagated through the plant to account
for this. The backpropagation of the error can be done by several methods
as stated in [36]. The most convenient way appears to be by using a neural
model of the plant. A neural network is first trained to provide a model of the
nonlinear plant in question as discussed before. This can be used in parallel
with the plant, with errors at the plant output backpropagated through its
neural model. The computed error in the input of the plant is the error at
the output of the controller. Finally, the BP algorithm is used on this error
to train the neural controller. As we can see in Figure 1.3, the inputs to the
neural controller include the current and previous reference inputs, previous
outputs of the neural controller, as well as previous outputs of the reference
model. In this figure, the existence of a reference model has been assumed, so
that the task of the controller is to force the plant to the output designated
by the reference model.
At this point, we should mention that for the construction of the neu-
ral model of the controller there exist further possibilities beside the mean
squared error between the output of the reference model and the output of
the actual plant. Other terms that can be included are the mean squared
error between the reference input and the real output, r - Yp, as well as the
input u to the plant. The inclusion of u in the cost function is desirable, in
order to preserve control energy. In the same way, the rate of u can also be
included, so that the transition from one extreme value for u to another can
be avoided. On the other hand, each one of the terms that participate in
the cost function can be assigned a weight, so that their contribution to the
minimizing function varies, depending on the specific application.
In order that a neural network architecture be able to approximate the
behavior of a dynamical system in some sense, it is clear that it should con-
tain some form of dynamics, or stated differently, feedback connections. In
the neural network literature, such networks are known as . They were orig-
inally designed for pattern recognition applications. A static neural network
can also be made a dynamic one, by simply connecting the past neural out-
puts as inputs to the neural network, thus making the neural network a very
complicated and highly nonlinear dynamical system. A more efficient way to
introduce dynamics with the aid of feedforward multilayer neural networks
was proposed in [74]. They connect stable linear dynamical systems with
static multilayer networks. The connections need not be only serial; parallel,
and feedback connections and combinations of the three types are also per-
mitted. Similar to the static multilayer networks, the synaptic weights are
adjusted according to a gradient descent rule.
The main problem with the dynamic neural networks that are based on
static multilayer networks is that the synaptic weights appear nonlinearly in
6 1. Introduction
the mathematical representation that governs their evolution. This leads to

a number of significant drawbacks. First, the learning laws that are used,
require a high amount of computational time. Second, since the synaptic
weights are adjusted to minimize a functional of the approximation error and
the weights appear nonlinearly, the functional possesses many local minima
so there is no way to ensure the convergence of the weights to the global min-
imum. Moreover, due to the highly nonlinear nature of the neural network
architecture, basic properties like stability, convergence and robustness, are
very difficult to verify. The fact that even for linear systems such adapta-
tion methods can lead to instability was also shown in [3],[50],[78]. On the
other hand, the recurrent networks possessing a linear-in-the-weights prop-
erty, make the issues of proving stability and convergence feasible and their
incorporation into a control loop promising.
The most significant problem in generalizing the application of neural
networks in control, is the fact that the very interesting simulation results that
are provided, lack theoretical verification. Crucial properties like stability,
convergence and robustness of the overall system must be developed and/or
verified. The main reason for the existence of the above mentioned problem,
is the mathematical difficulties associated with nonlinear systems controlled
by highly nonlinear neural network controllers. In view of the mathematical
difficulties encountered in the past in the adaptive control of linear systems,
(which remained as an active problem until the early 1980's [22],[68],[71],[30)),
it is hardly surprising that the analytical study of nonlinear adaptive control
using neural networks, is a difficult problem indeed, but progress has been
made in this area and certain important results have begun to emerge, aiming
to bridging the gap between theory and applications.
The problem of controlling an unknown nonlinear dynamical system, has
been attacked from various angles using both direct and indirect adaptive
control structures and employing different neural network models. A beautiful
survey of the above mentioned techniques, can be found in a paper by Hunt
et al. [42], in which links between the fields of control science and neural
networks were explored and key areas for future research were proposed, but
all works share the key idea, that is, since neural networks can approximate
static and dynamic, highly nonlinear systems arbitrarily well, the unknown
system is substituted by a neural network model, which is of known structure
but contains a number of unknown parameters (synaptic weights), plus a
modeling error term. The unknown parameters may appear both linearly or
nonlinearly with respect to the network nonlinearities, thus transforming the
original problem into a nonlinear robust adaptive control problem.
Recent advances in nonlinear control theory and, in particular, feedback
linearization techniques, [47],[76], created a new and challenging problem,
which came to be known as adaptive nonlinear control. It was formulated to
deal with the control of systems containing both unknown parameters and
known nonlinearities. Several answers to this problem have been proposed in
1.2 Book Goals & Outline 7
the literature with typical examples [70],[105),[102],[52], [53],(54),[6],[83], [64).

A common assumption made in the above works is that of linear parameter-
ization. Although sometimes it is quite realistic, it constraints considerably
the application field. An attempt to relax this assumption and provide global
adaptive output feedback control for a class of nonlinear systems, determined
by specific geometric conditions, is given by Marino and Tomei in their recent
paper [65).
The above discussion makes apparent that adaptive control research, thus
far, has been directed towards systems with special classes of parametric
uncertainties. The need to deal with increasingly complex systems, to ac-
complish increasingly demanding design requirements and the need to attain
these requirements with less precise advanced knowledge of the plant and its
environment, inspired much work that came mostly from the area of neural
networks but with obvious and strong relation to the adaptive control field
[9), [10], [81), [85)-[96], [98)-[100]' [61), [62).
1.2 Book Goals & Outline
As the first results in neural control started to appear, it became increasingly

clear that in order to achieve global admittance within the control systems
society and before thinking of real world applications, much more was needed
than merely presenting some simulation results.
The purpose of this book, is to present a rigorous mathematical framework
to analyze and design closed loop control systems based on neural networks
especially on those of the specific structure termeded recurrent high-order
neural nets (RHONNs). The proposed neurocontrol schemes will be applied
to nonlinear systems possessing highly uncertain and possibly unknown non-
linearities. Owing to the great amount of uncertainty allowed, the controller
should be able to handle various robustness issues like modeling errors, un-
modeled dynamics and external disturbances acting both additively and mul-
tiplicatively. Since the scope of the book series is strongly related to industrial
applications, the presented theory will be extended to cover issues of schedul-
ing manufacturing cells.
To accomplish the aforementioned goals, the presentation of this book
proceeds as follows:
Chapter 2 introduces the RHONN structure and analyze its approxima-

tion capabilities. It is seen that the proposed neural network scheme may
approximate general nonlinear systems, whose vector fields satisfy a local
Lipschitz condition arbitrarily well. We go beyond the existence theorem
and present stable learning algorithms for tuning the RHONN weights,
using Lyapunov theory. Simulations performed on a robotic manipulator
conclude this chapter.
8 1. Introduction
Chapter 3 deals with the problem of controlling affine in the control non-
linear dynamical systems, attacking it from an indirect adaptive control
point of view. Modified accordingly, the learning algorithms developed in
Chapter 2 are employed for on-line system identification. Subsequently, the
RHONN model acquired, is used for control. The scheme is tested for both
parametric and dynamic uncertainties, operated within a singular pertur-
bation theory. Simulations performed on a nonlinearly operated Dc motor,
highlight certain performance issues.
Chapter 4 introduces the problem of controlling nonlinear dynamical sys-
tems through direct adaptive control techniques. The algorithms developed
may handle various destabilizing mechanisms like modeling errors, external
disturbances and unmodeled dynamics, without the need of singular per-
turbation theory. Both regulation and tracking issues are examined. The
results are also extended to cover the case where the number of measured
states is different from the number of control inputs.
Chapter 5 discusses the issues of manufacturing systems modeling and
control, using recurrent high order neural networks. Appropriately de-
signed RHONN-based controllers are used to output the required schedule,
guaranteeing achievement of production demand, while keeping all system
buffers bounded.
Finally, Chapter 6, applies the theoretical framework developed in Chap-
ter 5 to solve a real test case. Calculation of various performance indices
indicates near optimal operation.
1.3 Notation
The following notations and definitions will extensively be used throughout
the book. I denotes the identity matrix. I . I denotes the usual Euclidean
norm of a vector. In cases where y is a scalar, I y I denotes its absolute value.
If A is a matrix, then IIAII denotes the Frobenius matrix norm [29], defined
as
IIAI12 = L laijl2 = tr{AT A},
ij
where tr{.} denotes the trace of a matrix. Now let d(t) be a vector function
of time. Then
IIdll2 ~ (1 00
Id(rWdr)1/2,
and
Ildlloo ~ sup Id(t)l.
t~O
We will say that d E L2 when IIdl1 2is finite. Similarly, we will say that d E Loo
when IIdlloo is finite.
CHAPTER 2
IDENTIFICATION OF DYNAMICAL SYSTEMS USING
RECURRENT HIGH-ORDER NEURAL NETWORKS
The use of multilayer neural networks for pattern recognition and for model-
ing of "static" systems is currently well-known (see, for example, [1]). Given
pairs of input-output data (which may be related by an unknown algebraic
relation, a so-called "static" function) the network is trained to learn the par-
ticular input-output map. Theoretical work by several researchers, including
Cybenko [16], and Funahashi [24], have proven that, even with one hidden
layer, neural networks can approximate any continuous function uniformly
over a compact domain, provided the network has a sufficient number of
units, or neurons. Recently, interest has been increasing towards the usage of
neural networks for modeling and identification of dynamical systems. These
networks, which naturally involve dynamic elements in the form of feedback
connections, are known as recurrent neural networks.
Several training methods for recurrent networks have been proposed in
the literature. Most of these methods rely on the gradient methodology and
involve the computation of partial derivatives, or sensitivity functions. In this
respect, they are extensions of the backpropagation algorithm for feedforward
neural networks [97]. Examples of such learning algorithms include the recur-
rent backpropagation [80], the backpropagation-through-time algorithms [106],
the real-time recurrent learning algorithm [107]' and the dynamic backprop-
agation [75] algorithms. The last approach is based on the computation of
sensitivity models for generalized neural networks. These generalized neural
networks, which were originally proposed in [74], combine feedforward neural
networks and dynamical components in the form of stable rational transfer
functions.
Although the training methods mentioned above have been used success-
fully in many empirical studies, they share some fundamental drawbacks. One
drawback is the fact that, in general, they rely on some type of approximation
for computing the partial derivative. Furthermore, these training methods re-
quire a great deal of computational time. A third disadvantage is the inability
to obtain analytical results concerning the convergence and stability of these
schemes.
Recently, there has been a concentrated effort towards the design and
analysis of learning algorithms that are based on the Lyapunov stability the-
ory [81],[100], [10], [9],[61], [98], [99], [85], [57] targeted at providing stability,

10 2. RHONNs for Identification of Dynamical Systems
convergence and robustness proofs, in this way, bridging the existed gap be-
tween theory and applications.
In this chapter we discuss the identification problem which consists of
choosing an appropriate identification model and adjusting its parameters
according to some adaptive law, such that the response of the model to an
input signal (or a class of input signals), approximates the response of the real
system to the same input. Since a mathematical characterization of a system
is often a prerequisite to analysis and controller design, system identifica-
tion is important not only for understanding and predicting the behavior of
the system, but also for obtaining an effective control law. For identification
models we use recurrent high-order neural networks. High-order networks are
expansions of the first-order Hopfield [39] and Cohen-Grossberg [12] models
that allow higher-order interactions between neurons. The superior storage
capacity of has been demonstrated in [77, 4], while the stability properties
of these models for fixed-weight values have been studied in [18, 51]. Fur-
thermore, several authors have demonstrated the feasibility of using these
architectures in applications such as grammatical inference [28] and target
detection [63].
The idea of recurrent neural networks with dynamical components dis-
tributed throughout the network in the form dynamical neurons and their
application for identification of dynamical systems was proposed in [57]. In
this chapter, we combine distributed recurrent networks with high-order con-
nections between neurons. In Section 1 we show that recurrent high-order
neural networks are capable of modeling a large class of dynamical systems.
In particular, it is shown that if enough higher-order connections are allowed
in the network then there exist weight values such that the input-output be-
havior of the RHONN model approximates that of an arbitrary dynamical
system whose state trajectory remains in a compact set. In Section 2, we
develop weight adjustment laws for system identification under the assump-
tion that the system to be identified can be modeled exactly by the RHONN
model. It is shown that these adjustment laws guarantee boundedness of all
the signals and weights and furthermore, the output error converges to zero.
In Section 3, this analysis is extended to the case where there is a nonzero
mismatch between the system and the RHONN model with optimal weight
values. In Section 4, we apply this methodology to the identification of a
simple robotic manipulator system and in Section 5 some final conclusions
are drawn.
2.1 The RHONN Model

Recurrent neural network (RNN) models are characterized by a two way con-
nectivity between units (i.e., neurons). This distinguishes them from feedfor-
ward neural networks, where the output of one unit is connected only to units
2.1 The RHONN Model 11
of the next layer. In the most simple case, the state history of each neuron is
governed by a differential equation of the form:
Xi = -aiXi + bi L WijYj , (2.1)

j
where Xi is the state of the i-th neuron, ai, bi are constants, Wij is the synaptic
weight connecting the j-th input to the i-th neuron and Yj is the j-th input
to the above neuron. Each Yj is either an external input or the state of a
neuron passed through a sigmoid function (i.e., Yj = s(Xj)), where s(.) is the
sigmoid nonlinearity.
The dynamic behavior and the stability properties of neural network mod-
els of the form (2.1) have been studied extensively hy various researchers
[39], [12], [51], [18]. These studies exhibited encouraging results in applica-
tion areas such as associative memories, but they also revealed the limitations
inherent in such a simple model.
In a recurrent second order neural network, the input to the neuron is
not only a linear combination of the components Yj, but also of their product
Yj Yk One can pursue this line further to include higher order interactions
represented by triplets Yj Yk YI, quadruplets, etc. forming the recurrent high
order neural networks (RHONNs).
Let us now consider a RHONN consisting of n neurons and m inputs. The
state of each neuron is governed by a differential equation of the form:

Xi = -aiXi + bi [ L
k=I
L Wik IT
jE1k
Yj 1
dj(k)
, (2.2)
where {II, h, ... , h} is a collection of L not-ordered subsets of {I, 2, ... , m+

n}, ai, bi are real coefficients, Wik are the (adjustable) synaptic weights of
the neural network and dj (k) are non-negative inegers. The state of the i-th
neuron is again represented by Xi and Y = [VI, Y2, ... ,Ym+n]T is the input
vector to each neuron defined by:
S(XI)
YI
S(X2)
Y2
s(x n )
y= Yn (2.3)
UI
Yn+I
U2
Yn+m
Um
where U = [UI, U2, ... , umJT is the external input vector to the network. The
function s(.) is monotone-increasing, differentiable and is usually represented
by sigmoids of the form:
a
s(X) = l+e- fJx -"I, (2.4)
where the parameters a, {3 represent the bound and slope of sigmoid's curva-
ture and "I is a bias constant. In the special case where a = {3 = 1, "I = 0, we
obtain the logistic function and by setting a = (3 = 2, "I = 1, we obtain the
I
hyperbolic tangent function; these are the sigmoid activation functions most
commonly used in neural network applications.
We now introduce the L-dimensional vector z, which is defined as
Zl]
z [ TIjElt Yjy1;(I)
d;(2)
z-
_
[ .. 2 _
-
jE/2
. . (2.5)
. ..
ZL TIjElL Yjdj(L)
Hence, the RHONN model (2.2) becomes
Xi = -aiXi + bi [t
k=l
WikZkj. (2.6)
Moreover, if we define the adjustable parameter vector as

Wi = bi[Wil Wi2 ... WiLf,
then (2.6) becomes

.
Xi = -aiXi + WiT Z. (2.7)
The vectors {Wi : i = 1,2, ... , n} represent the adjustable weights of the
network, while the coefficients {ai : i = 1,2, ... , n} are part of the underlying
network architecture and are fixed during training.
In order to guarantee that each neuron Xi is bounded-input bounded-
output (BIBO) stable, we shall assume that ai > 0, Vi = 1,2, ... , n. In the
special case of a continuous time Hopfield model [39], we have ai =

R;le;,
where R; > and Ci > are the resistance and capacitance connected at
the i-th node of the network respectively.
The dynamic behavior of the overall network is described by expressing
(2.7) in vector notation as:
(2.8)
where X = [Xl, X2, ... , Xn]T E ~n, W = [WI, W2, ... , WnY
E ~Lxn and A =
diag{ -aI, -a2, ... , -an} is a n x n diagonal matrix. Since ai > 0, Vi =
1,2, ... , n, A is a stability matrix. Although it is not written explicitly, the
vector Z is a function of both the neural network state X and the external
input u.
2.1 The RHONN Model 13
2.1.1 Approximation Properties
Consider now the problem of approximating a general nonlinear dynamical

system whose input-output behavior is given by
x = F(X, u), (2.9)
where X E ~n is the system state, u E ~m is the system input and F :
~n+m --+ ~n is a smooth vector field defined on a compact set Y c ~n+m.
The approximation problem consists of determining whether by allow-
ing enough higher-order connections, there exist weights W, such that the
RHONN model approximates the input-output behavior of an arbitrary dy-
namical system of the form (2.9).
In order to have a well-posed problem, we assume that F is continuous
and satisfies a local Lipschitz condition such that (2.9) has a unique solution
-in the sense of Caratheodory [34]- and (x(t), u(t)) E Y for all t in some
time interval JT = {t : 0 ~ t ~ T}. The interval JT represents the time
period over which the approximation is to be performed. Based on the above
assumptions we obtain the following result.
Theorem 2.1.1. Suppose that the system (2.9) and the model (2.8) are
initially at the same state x(O) = X(O); then for any c > 0 and any finite
T > 0, there exists an integer L and a matrix W* E ~Lxn such that the state
x(t) of the RHONN model (2.8) with L high-order connections and weight
values W = W* satisfies
sup Ix(t) - x(t)1 ~ C.
O~t~T
Proof: From (2.8), the dynamic behavior of the RHONN model is described
by
i: = Ax + WT z( x, u) . (2.10)
Adding and subtracting AX, (2.9) is rewritten as
X=AX+G(X,u), (2.11)
where G(X, u) = F(X, u) - AX. Since x(O) = X(O), thE; state error e = x - X
satisfies the differential equation
e=Ae+WTz(x,u)-G(x,u), e(O)=O. (2.12)
By assumption, (X(t), u(t)) E Y for all t E [0, T], where Y is a compact subset
of ~n+m. Let
Ye = {(X, u) E ~n+m : I(x, u) - (Xy, uy)1 ~ c, (Xy, uy) E y}. (2.13)
It can be seen readily that Ye is also a compact subset of ~n+m and Y c
Yeo In simple words Ye is c larger than y, where c is the required degree
of approximation. Since z is a continuous function, it satisfies a Lipschitz
condition in Ye, i.e., there exists a constant I such that for all (Xl, u), (X2' u) E
Ye
(2.14)
In what follows, we show that the function WT z( x, u) satisfies the condi-
tions of the Stone-Weierstrass Theorem and can approximate any continuous
function over a compact domain, therefore.
From (2.2), (2.3) it is clear that z(x, u) is in the standard polynomial
expansion with the exception that each component of the vector X is prepro-
cessed by a sigmoid function s(.). As shown in [14], preprocessing of input
via a continuous invertible function does not affect the ability of a network
to approximate continuous functions; therefore, it can be shown readily that
if L is sufficiently large, then there exist weight values W = W* such that
W*T z( x, u) can approximate G( x, u) to any degree of accuracy, for all (x, u)
in a compact domain. Hence, there exists W = W* such that
sup IW*T z(X, u) - G(X, u)1 :::; 8, (2.15)
(x,u)EY.
where 8 is a constant to be designed in the sequel.
The solution of (2.12) is
e(t) = lot eA(t-r) [W*T z(x(r), u(r)) - G(x(r), u(r))] dr,
= lot eA(t-r) [W*T z(x(r), u(r)) - W*T z(x(r), u(r))] dr
+ lot eA(t-r) [W*T z(X( r), u( r)) - G(X( r), u( r))] dr. (2.16)
Since A is a diagonal stability matrix, there exists a positive constant a
such that IleAtl1 :::; e- at for all t 2: o. Also, let L = IIlW*II. Based on the
aforementioned definitions of the constants a, L, c:, let 8 in (2.15) be chosen
as
c:a L
8=2e--;;->0. (2.17)
First consider the case where (x(t), u(t)) E Ye for all t E [0, T]. Starting from
(2.16), taking norms on both sides and using (2.14), (2.15) and (2.17), the
following inequalities hold for all t E [0, T]:
le(t)1 :::; lot lIeA(t-r)lIlw*T z(x(r), u(r)) - W*T z(x(r), u(r))1 dr
+ lot lIeA(t-r)lIlw*T z(x(r), u(r)) - I

G(x(r), u(r)) dr,
:::; lot e-a(t-r) Lle( r)ldr + lot 8e- a(t-r)dr ,
:::; ~e-~ + L lot e-a(t-r)le(r)ldr.

2.2 Learning Algorithms 15
Using the Bellman-Gronwall Lemma [34], we obtain
le(t)1 :::; ~e-~ + eL lot e-a(t-r)dT,

:::; ~. (2.18)
Now suppose (for the sake of contradiction), that (x, u) does not belong to
Ye for all t E [0, T]; then, by the continuity of x{t), there exist a T*, where
0< T* < T, such that (x{T*), u(T*)) E aYe where aYe denotes the boundary
of Yeo If we carry out the same analysis for t E [0, T*] we obtain that in this
intervallx(t) - x(t)1 :::; ~, which is clearly a contradiction. Hence, (2.18) holds
for all t E [0, T].
The aforementioned theorem proves that if sufficiently large number of
connections is allowed in the RHONN model then it is possible to approx-
imate any dynamical system to any degree of accuracy. This is strictly an
existence result; it does not provide any constructive method for obtaining
the optimal weights W*. In what follows, we consider the learning problem
of adjusting the weights adaptively, such that the RHONN model identifies
general dynamic systems.
2.2 Learning Algorithms
In this section we develop weight adjustment laws under the assumption that
the unknown system is modeled exactly by a RHONN architecture of the
form (2.8). This analysis is extended in the next section to cover the case
where there exists a nonzero mismatch between the system and the RHONN
model with optimal weight values. This mismatch is referred to as modeling
error.
Although the assumption of no modeling error is not very realistic, the
identification procedure of this section is useful for two reasons:
the analysis is more straightforward and thus easier to understand,
the techniques developed for the case of no modeling error are also very
important in the design of weight adaptive laws in the presence of modeling
errors.
Based on the assumption of no modeling error, there exist unknown weight
vectors wf, i = 1,2, ... , n, such that each state Xi of the unknown dynamic
system (2.9) satisfies:
Xi = -aiXi + wfz{x, u), Xi{O) = X? (2.19)
where X? is the initial i-th state of the system. In the following, unless there
is confusion, the arguments of the vector field z will be omitted.
As is standard in system identification procedures, we will assume that
the input u(t) and the state X(t) remain bounded for all t 2: O. Based on
the definition of z(X, u), as given by (2.5), this implies that z(x, u) is also
bounded. In the subsections that follow we present different approaches for
estimating the unknown parameters wf of the RHONN model.
2.2.1 Filtered Regressor RHONN
The following lemma is useful in the development of the adaptive identifica-

tion scheme presented in this subsection.
Lemma 2.2.1. The system described by
Xi = -aiXi + wtz(X, u), Xi(O) = X? (2.20)
can be expressed as
(i(O) = 0, (2.21 )
(2.22)
Proof: From (2.21) we have
(i = 1t e-ai(t-T)z(x(r),u(r))dr;
therefore,
wtT (i + e-aitx? = e-aitx? + 1t e-ai(t-T)wt T z(X( r), u( r))dr . (2.23)
Using (2.20), the right hand side of (2.23) is equal to Xi(t) and this concludes
the proof.
Using Lemma 2.2.1, the dynamical system described by (2.9) is rewritten
as
Xi=Wi.T(i+(j, i = 1,2, .. . ,n, (2.24)

where (i is a filtered version of the vector z (as described by (2.5)) and
(i := eait X? is an exponentially decaying term which appears if the system
is in a nonzero initial state. By replacing the unknown weight vector wi in
(2.24), by its estimate Wi and ignoring the exponentially decaying term (i,
we obtain the RHONN model
Xi = wT(j, i=1, 2, ... ,no (2.25)
The exponentially decaying term (i(t) can be omitted in (2.25) since, as we
shall see later, it does not affect the convergence properties of the scheme.
The state error ei = Xi - Xi between the system and the model satisfies
(2.26)
where <Pi = Wi - wi is the weight estimation error. The problem now is to
derive suitable adaptive laws for adjusting the weights Wi for i = 1, ... n.
This can be achieved by using well-known optimization techniques for mini-

mization of the quadratic cost functional
1 1
'2 L f;] .
n n 2
J(Wl, 00 .Wn ) = '2 ~::>; = [(Wi - wif (i - (2.27)
;=1 i=l
Depending on the optimization method that is employed, different weight

adjustment laws can be derived. Here we consider the gradient and the least-
squares methods [45]. The gradient method yields
i= 1,2,oo.,n, (2.28)
where r; is a positive definite matrix referred to as the adaptive gain or
learning rate. With the we obtain
{ i=1,2,oo.,n, (2.29)
where P(O) is a symmetric positive definite matrix. In the above formulation,

the least-squares algorithm can be thought of as a gradient algorithm with a
time-varying learning rate.
The stability and convergence properties of the weight adjustment laws
given by (2.28) and (2.29) are well-known in the adaptive control literature
(see, for example, [31, 73]). For tutorial purposes and for completeness we
present the stability proof for the gradient method here.
Theorem 2.2.1. Consider the RHONN model given by (2.25) whose pa-
rameters are adjusted according to (2.28). Then for i = 1,2, ... , n
(a) e;, rPi E Loo (ei and rP are uniformly bounded)
(b) limt--+oo ei(t) = 0
Proof: Consider the Lyapunov function candidate
(2.30)
Using (2.28) and (2.26), the time derivative of V in (2.30) is expressed as
~ ( -eirPiT (; - '2fi
V. = L..J 1 2) ,
.=1
(2.31)
Since V ::; 0, we obtain that rPi E Loo. Moreover, using (2.26) and the bound-
edness of (i, we have that ei is also bounded. To show that ei(t) converges to
zero, we first note that since V is a non-increasing function of time and also
bounded from below, the limt_oo V(t) = Voo exists; therefore, by integrating
both sides of (2.31) from t = 0 to 00, and taking bounds we obtain
[00 n
in I>?(T) dT :::; 2 (V(O) - Voo ) ,
o ;=1
so for i = 1, ... n, e;(t) is square integrable. Furthermore, using (2.26)

ei(t) = <Pi (i + <Pi (i - f; ,
. 'T T
= -ei(T riC; - ai<Pr (i + <pr Z - fi .

Since ei, (;, <Pi, z, fi are all bounded, fi E Loo. Hence, by applying Barbalat's
Lemma [73] we obtain that limt_oo ei(t) = O.
Remark 2.2.1. The stability prooffor the least-squares algorithm (2.29)

proceeds along the same lines as in the proof of Theorem 2.2.1 by considering
the Lyapunov function
V= ~
2
t
;=1
(<pr Pi- 1 <Pi + 1
t
00
f}(T) dT) .
A problem that may be encountered in the application of the least-squares

algorithm is that P may become arbitrarily small and thus slow down adap-
tation in some directions [45, 31]. This so-called problem can be prevented by
using one of various modifications which prevent P(t) from going to zero. One
such modification is the so-called, where if the smallest eigenvalue of P(t)
becomes smaller than PI then P(t) is reset to P(t) = pol, where Po ~ PI > 0
are some design constants.
Remark 2.2.2. The above theorem does not imply that the weight esti-
mation error <Pi = wi - wi converges to zero. In order to achieve convergence
of the weights to their correct value the additional assumption of persis-
tent excitation needs to be imposed on the regressor vector (i. In particular,
(;(t) E ~L is said to be persistently exciting if there exist positive scalars c,
d and T such that for all t ~ 0
cI:::; I t
t+T
(i(T)(T(T) dT :::; dI, (2.32)
where I is the L x L identity matrix.
Remark 2.2.3. The learning algorithms developed above can be extended

to the case where the underlying neuron structure is governed by the higher-
order Cohen-Grossberg model [12, 18]
Xi = -ai(xi) [bi(Xi) + t Wik II y1 (k)] ,

k=1 j Elk
j (2.33)
where ai(), bi () satisfy certain conditions required for the boundedness of

the state variables [18]. It can be seen readily that in (2.33) the differential
equation is still linear in the weights and hence a similar parameter estimation
procedure can be applied.
The filtered-regressor RHONN model considered in this subsection relies
on filtering the vector z, which is sometimes referred to as the regressor vector.
By using this filtering technique, it is possible to obtain a very simple alge-
braic expression for the error (as given by (2.26)), which allows the application
of well-known optimization procedures for designing and analyzing weight ad-
justment laws but there is an important drawback to this method, namely
the complex configuration and heavy computational demands required in the
filtering of the regressor. Generally, the dimension of the regressor will be
larger than the dimension of the system, i.e., L > n, it might be very ex-
pensive computationaly to employ so many filters. In the next subsection
we consider a simpler structure that requires only n filters and hence, fewer
computations.
2.2.2 Filtered Error RHONN
In developing this identification scheme we start again from the differential

equation that describes the unknown system, i.e.,
i=1,2, ... ,n. (2.34)
Based on (2.34), the identifier is now chosen as
i=1,2, ... ,n, (2.35)
where Wi is again the estimate of the unknown vector wi. In this case the
state error ei := Xi - Xi satisfies
.
ei
",T
= -aiei+'Pi z, i=1,2, ... ,n, (2.36)
where <Pi = Wi-wi. The weights Wi, for i = 1,2, ... , n, are adjusted according
to the learning laws
(2.37)
where the adaptive gain rj is a positive definite L x L matrix. In the special
case that ri = IiI, where /i > 0 is a scalar, then rj in (2.37) can be replaced
by Ii.
The next theorem shows that this identification scheme has similar con-
vergence properties as the filtered regressor RHONN model with the gradient
method for adjusting the weights.
Theorem 2.2.2. Consider the filtered error RHONN model given by (2.35)
whose weights are adjusted according to (2.37). Then Jor i = 1,2, ... , n
(a) ej, <Pi E .coo
(b) limt--+oo ei(t) = 0

Proof. Consider the Lyapunov function candidate
~ (e;2 + cP;T
V = 2"1 L..... ri-1 cPi ) (2.38)
i=1
Then, using (2.36), (2.37), and the fact that ~i = Wi, the time derivative of
V in (2.38) satisfies
(2.39)
Since V S 0, from (2.38) we obtain that ei, cPi E 00 for i = 1, .. . n. Using

this result in (2.36) we also have that ei E 00' Now, by employing the
same techniques as in the proof of Theorem 2.2.1 it can be shown readily
that ei E 2, i.e., e;(t) is square integrable; therefore, by applying Barbalat's
Lemma we obtain that limt--+oo e;(t) = O.
Table 2.1. Filtered-regressor RHONN identifier
System Model: x = F(X, u), X E !Rn , u E !Rm

Parametric Model: (i = -ai(i + Z, i = 1,2, ... , n
Xi = wtT(i +e-a;t x ?, i = 1,2, ... ,n
RHONN Identifier Model: Xi = WrCi, i = 1,2, ... ,n
Identification Error: ei = Xi - Xi, i = 1,2, ... ,n
Weight Estimation Error: 4>i = Wi - wt, i = 1,2, ... , n
Learning Laws:
a) Gradient Algorithm: Wi = -ri(iei, i = 1,2, ... , n
b) Least-Squares Algorithm: Wi = -Pi(iei, i = 1,2, ... , n
.
Pi = -Pi(i(iT Pi, i = 1,2, ... , n
Conditions: Xi(O) = X?, (i(O) = 0
Pi(O) a symmetric positive
definite matrix i = 1,2, ... ,n
2.3 Robust Learning Algorithms

The derivation of the learning algorithms developed in the previous section
made the crucial assumption of no modeling error. Equivalently, it was as-
sumed that there exist weight vectors wi, for i = 1, ... n, such that each state
of the unknown dynamical system (2.9) satisfies
2.3 Robust Learning Algorithms 21
Table 2.2. Filtered-error RHONN identifier
System Model: X = F(X,u), X E iR n , u E iR m

Parametric Model: Xi = -aiXi + wt T z, i = 1,2, ... ,n
RHONN Identifier Model: Xi = -aiXi + wT z, i = 1,2, ... , n
Identification Error: ei = Xi - Xi, i = 1,2, ... , n
Weight Estimation Error: rPi = Wi - wt, i = 1,2, ... ,n
Learning Law: Wi = -rizei i = 1,2, ... , n
Xi. = -aiXi +wi.T z ( X,U ) . (2.40)

In many cases this assumption will be violated. This is mainly due to
an insufficient number of higher-order terms in the RHONN model. In such
cases, if standard adaptive laws are used for updating the weights, then the
presence of the modeling error in problems related to learning in dynamic
environments, may cause the adjusted weight values (and, consequently, the
error ej = Xi - X;) to drift to infinity. Examples of such behavior, which
is usually referred to as , can be found in the adaptive control literature of
linear systems [73, 45].
In this section we shall modify the standard weight adjustment laws in
order to avoid the parameter drift phenomenon. These modified weight ad-
justment laws will be referred to as robust learning algorithms.
In formulating the problem it is noted that by adding and subtracting
aiXi + wiT z(x, u), the dynamic behavior of each state of the system (2.9)
can be expressed by a differential equation of the form
Xi = -aiXi + wiT z(X, u) + Vi(t) , (2.41)
where the modeling error viet) is given by
Viet) := Fi(X(t), u(t)) + aix(t) - wiT z(X(t), u(t)) . (2.42)
The function Fi(X, u) denotes the i-th component of the vector field F(X, u),
while the unknown optimal weight vector wi is defined as the value of the
weight vector Wi that minimizes the Loo-norm difference between Fi(X, u) +
ajX and wTz(X, u) for all (X, u) EYe ~n+m, subject to the constraint
that IWil ~ Mi, where Mi is a large design constant. The region Y denotes
the smallest compact subset of ~n+m that includes all the values that (X, u)
can take, i.e., (X(t), u(t)) E Y for all t ~ O. Since by assumption u(t) is
uniformly bounded and the dynamical system to be identified is BIBO stable,
the existence of such Y is assured. It is pointed out that in our analysis we
do not require knowledge of the region y, nor upper bounds for the modeling
error viet).
In summary, for i = 1,2, ... , n, the optimal weight vector wi is defined
as
w; := arg min. { sup IFi(X, u) + aiX - wi z(x, u)l} (2.43)

Iw,lSM, (x,u)EY
The reason for restricting wi to a ball of radius Mi is twofold: firstly, to

avoid any numerical problems that may arise owing to having weight values
that are too large, and secondly, to allow the use of the iT-modification [45],
which will be developed below to handle the parameter drift problem.
The formulation developed above follows the methodology of [81] closely.
Using this formulation, we now have a system of the form (2.41) instead of
(2.40). It is noted that since X(t) and u(t) are bounded, the modeling error
Vi(t) is also bounded, i.e., SUPt>O IVi(t)1 :S Vi for some finite constant Vi.
In what follows we develop robust learning algorithms based on the filtered
error RHONN identifier; however, the same underlying idea can be extended
readily to the filtered-regressor RHONN. Hence, the identifier is chosen as in
(2.35), i.e.,
i= 1,2, ... ,n (2.44)
where Wi is the estimate of the unknown optimal weight vector wi. Using
(2.41), (2.44), the state error ei = Xi - Xi satisfies
ej = -aiej + A,T
'f'i Z - Vi , (2.45)
where <Pi = wi - wi. Owing to the presence of the modeling error Vi, the
learning laws given by (2.37) are modified as follows:
if
if IWil > Mi
.
IW'I' -< M (2.46)
where iTj is a positive constant chosen by the designer. The above weight
adjustment law is the same as (2.37) if Wi belongs to a ball of radius Mi.
In the case that the weights leave this ball, the weight adjustment law is
modified by the addition of the leakage term iTjriWj, whose objective is to
prevent the weight values from drifting to infinity. This modification is known
as the [45].
In the following theorem we use the vector notation V := [Vi ... vnf and
e := [ei ... enf.
Theorem 2.3.1. Consider the filtered error RHONN model given by (2.44)
whose weights are adjusted according to (2.46). Then for i = 1, ... n
(aJ ej, <Pi E 00
(b J there exist constants A, J.l such that
it le(rW dr :S A + J.l it Iv(rW dr.
Proof: Consider the Lyapunov function candidate

2.3 Robust Learning Algorithms 23
V ~ (2
= -1 L...J T -1 )
ei + i r i i . (2.47)
2 i=1
Using (2.45) and (2.46) it can be shown that
(2.48)
where I~. is the indicator function defined as I~ i = 1 if 1Wi 1> Mi and I~ i = 0

if IWi 1 :::; Mi. Since i = Wi - wi, we have that
T 1 T 1( T T *)
i Wi = "2i i +"2 i i + 2i Wi ,
1 2 1 2 1 *2
= "21il + "2l wd - "2IWi 1 .
Since, by definition, Iwi 1 :::; Mi and IWil > Mi for I~i = 1, we have that
I~i ~i (lw;j2 _ Iwi 12) ;::: 0;
therefore, (2.48) becomes
n
V< L...J (-a.e~ - 1*
- '" 1 1 Wi
(Ti
2 1),.1
'1'1
2 - e.v.)
1 1 , (2.49)
i=1
(2.50)
where
a := min {ai, )..maX~~i-1) ; i = 1,2, ... , n} ,

and )..max (ri- 1 ) > 0 denotes the maximum eigenvalue of r i- 1 Since
if Iw1
1 _< MI
otherwise
we obtain that (1 - I~.) TI;j2 :::; (TiM? Hence (2.50) can be written in the
form
V:::; -aV +K,
where K := 2::7=1 ((TiM? + vl!2ai) and Vi is an upper bound for Vi; therefore,
for V ;::: Vo = Kia, we have V :::; 0, which implies that V E 'coo. Hence
ei, i E 'coo
To prove the second part, we note that by completing the square in (2.49)
we obtain

< ~n
V -L.J (-ae.
'I
2
- ev-)
I.
< ~n (
-L.J
ai 2
--e
2'
v
+-'
2.
2) (2.51 )
i=l i=l a,
Integrating both sides of (2.51) yields
V(t) - V(O) ~ t (-
.
ai
2
it e;( r) dr + ~
2a ,
it v;( r) dr) ,
it
0 0
1=1
~ _ amin !e(rW dr + _1_ t !v(rW dr,

2 0 2amin Jo
it it
where amin:= min{ai ; i = 1, .. . n}; therefore,
2
!e(rW dr ~ - . [V(O) - V(t)] + - 21 - !v(rW dr,
o a mm amin
1t
0
~ .x + J.L !v(r)!2 dr,
where .x := (2/amin) SUPt>o (V(O) - V(t)] and J.L := l/a~in' This proves part
(b) and concludes the proof of Theorem 2.3.1.
In simple words the above theorem states that the weight adaptive law
(2.46) guarantees that ei and 1>i remain bounded for all i = 1, ... n, and
furthermore, the "energy" of the state error e(t) is proportional to the "en-
ergy" of the modeling error v(t). In the special case that the modeling error
is square integrable, i.e., v E 2, then e(t) converges to zero asymptotically.
Remark 2.3.1. It is noted that the O'-modification causes the adaptive
law (2.46) to be discontinuous; therefore standard existence and uniqueness
results of solutions to differential equations are in general not applicable.
In order to overcome the problem of existence and uniqueness of solutions,
the trajectory behavior of Wi(t) can be made "smooth" on the discontinuity
hypersurface {Wi E WL !w;j = Md by modifying the adaptive law (2.46)
to
-rizei if {!Wi! < M;} or
{!w;j = Mi and wr rizei > O}
if {!w;j = M;} and (2.52)
Wi=
{-O'iWr riw ~ wr rizei ~ O}
-rizei - O'iriWi if {!Wi! > M;} or {!Wi! = M;}
and {wr rizei < -O'iWr riW}
As shown in [82], the adaptive law (2.52) retains all the properties of (2.46)
and, in addition, guarantees the existence of a unique solution, in the sense
of Caratheodory [34]. The issue of existence and uniqueness of solutions in
adaptive systems is treated in detail in [82].
2.4 Simulation Results 25
Table 2.3. Robust learning algorithms
System Model:
X = F(X, u),
Parametric Model:
Xi = -aiXi + wtT z + Vi(t),
RHONN Identifier Model:
Xi = -aiXi + wT z,
Identification Error:
ei = Xi - Xi,
Weight Estimation Error:
>i = Wi - wt,
Modeling Error:
Fi(X(t), u(t)) + aiXi(t) - wt T z(X(t), u(t))
Robust Learning Algorithms:
a) Switching u-modification:
tV _ {-rize i if !w;/ :S Mi
1- -rizei _ UiriWi if !w;/ >Mi
b) u-mod with existence
and uniqueness of solutions:
-rizei if {!Wi! < M;} or
{!w;/ = Mi and wT F;zei > o}
if {!Wi! = M;} and
tVi=
{ -uiwT riW :S wT rizei :S o}
if {!w;/ > M;} or {!Wi! = M;}
and {wT rizei < -uiwT F;w}
XE ~n, u E ~m, i = 1,2, ... ,n
2.4 Simulation Results
In this section we present simulation results of nonlinear system identification.

The efficiency of an identification procedure depends mainly on the following:
a) the error convergence and speed of convergence
b) stability in cases of abrupt input changes
c) performance of the identification model after the training stops
All three factors are checked during our simulations. We have used a recurrent
second-order neural network based on the filtered-error scheme described
by (2.36) and the weight adjustment laws given by (2.37). The particular
sigmoidal nonlinearity employed is the function (2.4) with a = 4, J3 = 0.1,

'Y = 2.
Now, consider an n-degree-of-freedom robotic manipulator which is de-
scribed by the following nonlinear vector differential equation
r(t) = M(w(t),p)w(t) + C(w(t), w(t),p)w(t) + G(w(t)), (2.53)
where
r(t) is an n x 1 vector of joint torques
w(t) is an n x 1 vector containing the joint variables
M( w(t), p) represents the contribution of the inertial forces to the dynam-
ical equation; hence the matrix M represents the inertia matrix of the
manipulator
C( w(t), w(t), p) represents the Coriolis forces
G( w( t)) represents the gravitational forces
p is a parameter vector whose elements are functions of the geometric and
inertial characteristics of the manipulator links and the payload, i.e., p
depends on the lengths and moments of inertia of each individual link and
the payload.
It is noted that the parameter vector p can be constant in time (for
example in the case of constant payload) or it can be varying as a function
of time, p = p(t), as in the case of changing payload. An introduction to the
derivation of the dynamical model of a robotic manipulator can be found in
[15].
For simplicity in our case we assume that the manipulator consists of
n = 2 degrees of freedom and more especially of two revolute joints whose
axes are parallel. In this case the parameter vector is chosen as
PI = It + h + lac + L~M2 + L~(Ma + M4 + Mp) + P2,
P2 = 1a + 14 + 1p + L~M4 + L~Mp ,
Pa = LIL4M4 + LIL2Mp ,
where the geometric and inertial parameter values are provided by the fol-
lowing table.
It =0.2675, rotor 1 inertia
12 =0.360, arm 1 inertia about c.g.
Ia=0.0077, motor 2 rotor inertia
Iac=0.040,motor 2 stator inertia
14=0.051, arm 2 inertia about c.g.
1p=0.046, payload inertia
M I =73.0, motor 1 mass
M2=10.6 , arm 1 mass
Ma=12.0, motor 2 mass
M 4=4.85, arm 2 mass
M p=6.81, payload mass
Summary 27
Ll =0.36, arm 1 length

L 2=0.24, arm 1 radius of gyration
L 3 =0.139, arm 2 radius of gyration
L 4 =0,099
The system matrices M and C can be written as:
M( (t) ) _ ((1,0, 2COSW2)p (0, 1, COSW2)P)
W ,p - (0,1, 2COSW2)p (0,0, O)p ,
C(w(t), w(t),p) = ((0, 0, -~2s~nw2)p (0,0, -(WI + w2)SinW2)p)
(0,0, -Wlsmw2)p (0,0, O)p
The above mathematical model and the particular numerical values of
the robot parameters have been taken from [8]. It is noted that in this robot
model there are no gravitational forces affecting the robot dynamics.
The RHONN identifier consists of four dynamic neurons, two for the an-
gular positions WI and W2 and two for the angular velocities WI and W2. The
objective here is to train the network so as to identify the robot model.
The training starts at time t = O. The learning rate is I = 0.05 for all
parameters and the sampling takes place every 0.001 seconds. The training
is as follows: for the first 2 seconds or 2000 steps the input torques for both
joints 1 and 2 are generated as random data in the range [-1,1]. For the next
two seconds or steps from 2001 to 4000 the input torques are sin(0.5t 3 ) for
joint 1 and cos(0.5t 3 ) for joint 2 and for the next two seconds or steps from
4001 to 6000 the input torques are sin(0.001t2) and cos(0.002t2) for joints
1 and 2 respectively. The above training input waveforms were repeatedly
applied every 6000 steps until the 96000 step or the 96th second. After this
instant the training ended and the same inputs as for the first 6000 steps
were applied to the neural network and the robot model. Figure 2.1 shows
WI and W2, the outputs of the first and second joint respectively. The solid
line corresponds to the robot model and the dashed line corresponds to the
RHONN model. It is seen that after the 96th second when the training stops
there is a small error; however the network outputs follow closely the outputs
of the robot model.
Summary
In this chapter we have studied the stability, convergence and approximation
properties of the recurrent high-order neural networks (RHONN) as models
of nonlinear dynamical systems. The overall structure of the RHONN con-
sists of dynamical elements distributed throughout the network in the form of
dynamical neurons, which are interconnected by high-order connections be-
tween them. We have shown that if a sufficiently large number of high-order
connections between neurons is allowed then the RHONN model is capable
of approximating the input-output behavior of general dynamical systems to
any degree of accuracy.
:l 4
~
0r-----______---------r
2
~71--~~~---793~--~~~--~9~S----%~---9~7----~98----~~~~~lOO
Tune ill secoDds
4r---~----~--~--~----~--~----~--~--~
.S
.!i, 2
) o
j. -2
:S
~~1----~~--~93-----~~---9~S----%~---9~7----~98----~~~--~lOO
Time ita secoads
Fig. 2.1. Identification performance of the first and second joint outputs, Wi and
W2 respectively. The solid line corresponds to the robot model and the dashed line
corresponds to the RHONN model. The training stops at the 96th second, after
which the adjustable weights are kept fixed
Based on the linear-in-the-weights property of the RHONN model, we

have developed identification schemes and derived weight-adaptive laws for
the adjustment of weights. The convergence and stability properties of these
weight-adaptIve laws have been analyzed. We showed that in the case of no
modeling error, the state error between the system and RHONN model con-
verges to zero asymptotically. In the case that modeling errors are present, we
proposed the O"-modification as a method of guaranteeing the stability of the
overall scheme. Using the O"-modification we showed that the state error and
the weight estimation error remain bounded and the residual state error is
proportional to the magnitude of the modeling error. The feasibility of apply-
ing these techniques has been demonstrated by considering the identification
of a simple rigid robotic system.
CHAPTER 3
INDIRECT ADAPTIVE CONTROL
This chapter is devoted to the development of indirect adaptive Gontrol tech-

niques (based on RHONNs), for controlling nonlinear dynamical systems,
with highly uncertain and possibly unknown nonlinearities.
The approach is comprised of an identification model, whose parameters
are updated on-line in such a way that the error between the actual system
output and the model output is approximately zero. The controller receives
information from the identifier and outputs the necessary signal, which forces
the plant to perform a prespecified task.
The learning laws developed in the previous chapter can also be used
herein in the building up of the identification part of the architecture. These
algorithms are enriched further to increase robustness however, especially in
the case of model order mismatch.
The contents of this chapter is based on [85]. Since the actual system
is assumed to be completely unknown, we propose a two phase algorithm.
In phase one, a RHONN is employed to perform "black box" identification
around a known operational point. Many cases that lead to modeling errors
(i.e., parametric, dynamic uncertainties), are taken into consideration. Stabil-
ity of the identification scheme plus convergence of the identification error to
within a small neighborhood of zero, is guaranteed with the aid of Lyapunov
and singular perturbations theories.
The successful completion of phase one, implies that a model of the origi-
nally unknown nonlinear dynamical system has been obtained. Thus, we are
free to proceed to the control phase of our algorithm, in which an appropriate
state feedback is constructed to achieve asymptotic regulation of the output,
while keeping bounded all signals in the closed loop. A block diagram of the
indirect adaptive control architecture is pictured in Figure 3.1. The algo-
rithm has also been applied successfully to control the speed of a DC motor,
operated in a nonlinear fashion providing an application insight in this way.
3.1 Identification
We consider affine in the control, nonlinear dynamical systems of the form
i: = f(x) + G(x)u, (3.1)
30 3. Indirect Adaptive Control
I 1
RHON
+
u e
CONTROLLER
-
UNK OWN
SYSTEM
Fig. 3.1. The two-stage control algorithm architecture
where the state x E ~n, is assumed to be completely measured, the control

u is in ~n , f is an unknown smooth vectorfield called the drift term and G is
a matrix with columns the unknown smooth controlled vectorfields gi , i =
1,2, ... , n G = [g1 g2 ... gn]. The above class of continuous-time nonlinear
systems are called affine, because in (3.1) the control input appears linear with
respect to G. The main reason for considering this class of nonlinear systems
rather than the general one treated in Chapter 2 is that most of the systems
encountered in engineering, are by nature or design, affine. Furthermore, we
note that non affine systems of the form
x = f(x, u),
can be converted into affine, by passing the input through integrators [76], a
procedure known as dynamic extension. The following mild assumptions are
also imposed on (3.1), to guarantee the existence and uniqueness of solution
for any finite initial condition and u E U.
Assumption 3.1.1. Given a class U of admissible inputs, then for any
u E U and any finite initial condition, the state trajectories are uniformly
bounded for any finite T> O. Hence, 1x(T) 1< 00.
Assumption 3.1.2. The vectorfields f, gi i = 1,2, ... , n are continuous
with respect to their arguments and satisfy a local Lipschitz condition so that
the solution x(t) of (3.1) is unique for any finite initial condition and u E U.
Following the discussion and analysis of Chapter 2, an affine RHONN
model of the form
i = Ax + BWS(x) + BW1 S'(x)u, (3.2)
can be used to describe (3.1). In (3.2) x E ~n, the inputs u E U C ~n, W
is a n x n matrix of synaptic weights, A is a n x n stable matrix which for
3 .1 Identification 31
simplicity can be taken to be diagonal, B is a n x n matrix with elements

the scalars bi for all i = 1,2, ... , n and WI is a n x n diagonal matrix of
synaptic weights of the form WI = diag[wllw2 1 .. . wn 1]. Finally, S(x) is
a n-dimensional vector and S' (x) is a n x n diagonal matrix, with elements
combinations of sigmoid functions. For more details concerning the RHONN
structure and its approximation capabilities, the reader is referred to Chapter
2.
In the case where only parametric uncertainties are present we can prove
using techniques analogous to the ones presented in Chapter 2 the theorem:
Theorem 3.1.1. Consider the identification scheme
e = Ae + BWS(x) + BW1 S'(x)u,

The learning law
Wij = -biPiS( x j )ei ,

Wil = -bis'(Xi)Piuiei ,
for all i, j = 1,2, ... , n guarantees the following properties
e, X, W, WI E L oo , e E L2 .
limt ..... oo e(t) = 0, limt ..... oo W(t) = 0, limt ..... oo W l(t) =0
The robust learning algorithms developed in Chapter 2 can also be used
herein to cover for the existence of modeling errors.
3.1.1 Robustness of the RHONN Identifier Owing to Unmodeled

Dynamics
In the previous section we assumed that there exist weight values W*, wt
such that a nonlinear dynamical system can be completely described by a
neural network of the form
x = Ax + BW*S(x) + BW{S'(x)u,
where all matrices are as defined previously. It is well known however, that the
model can be of lower order than the plant, owing to the presence of unmod-
eled dynamics. In the following, we extend our theory within the framework
of singular perturbations, to include the case where dynamic uncertainties
are present. For more details concerning singular perturbation theory, the
interested reader is referred to [56]. Now we can assume that the unknown
plant can be completely described by
x = Ax + BW*S(x) + BW{S'(x)u
+F(x, W, W 1 )Aol BoWou + F(x, W, W 1 )z,
J1.z = Aoz + BoWou, zE ~r (3.3)
where z is the state of the unmodeled dynamics and J1. > 0 a small singular
perturbation scalar. If we define the error between the identifier states and
the real system states as e = x- x then from (3.2) and (3.3) we obtain the
error equation
e = Ae + BWS(x) + BW1S'(x)u
-F(x, W, Wl)Ai) 1 BoWou - F(x, w, Wl)Z,
J-tz = Aoz + Bo Wou, z E ~r , (3.4)
where F(x, W, W l ), BoWou, BWS(x), BW1S'(x)u, are bounded and differen-
tiable with respect to their arguments for every W E Bw a ball in ~nxn,
Wl E BWI a ball in ~n and all x E Bx a ball in ~n. Further, we assume that
the unmodeled dynamics are asymptotically stable for all x E Bx. In other
words we assume that there exists a constant v > 0 such that
Re A{A o } ::; -v < o.
z
Note that is large since J-t is small and hence, the unmodeled dynamics are
fast. For a singular perturbation from J-t > 0 to J-t = 0 we obtain
z = -Ai)lBoWou.
Since the unmodeled dynamics are asymptotically stable the existence of Ai) 1
is assured. As it is well known from singular perturbation theory, we express
the state z as
z=h(x,TJ)+'f/, (3.5)
where h(x, TJ) is defined as the quasi-steady-state of z and 'f/ as its fast tran-
sient. In our case
h(x, 'f/) = -Ai) 1 BoWou
Substituting (3.5) into (3.4) we obtain the singularly perturbed model as
e = Ae + BWS(x) + BW1S'(x)u - F(x, W, Wl)'f/,
J-try= Ao'f/ - J-th(e, W, Wb 'f/, u), (3.6)
where we define
. - - 8h . oh':' 8h,:. 8h .
h(e, W, Wl,TJ,u) = J'}e
ve
+ --
oW
W + ---Wl + J'}u.
OWl vU
Notice, however, that in the control case, u is a function of e, W, Wl therefore
making h(e, W, Wb 'f/, u) to be equal to
. - - 8h 8h,:. oh':'
h(e, W, Wl,TJ,u) = J'}e + -- W + ---Wl
ve 8W OWl
Remark 3.1.1. F(x, W, Wl)Ai) 1 BoWou, F(x, w,
Wl)z in (3.3) can be
viewed as correction terms in the input vectorfields and in the drift term
of
x = Ax + BW*S(x) + BW[S'(x)u,
in the sense that the unknown system can now be described by a neural
network plus the correction terms.
3 .1 Identification 33
Before proceeding any further we need to prove the following lemma.

Lemma 3.1.1. It is true that h(e, W, vV1, TJ, u) is bounded by
Ilh(e, W, WI, TJ, u)11 :s; pdlell + p211TJII,
provided that the following inequalities hold
Ilhw WII :s; kollell ,

Ilhwl WIll :s; k111ell,
IlheBW1S'(x)ull :s; k211ell ,
IlheBWS(x)11 :s; k311ell,
IlheF(x, W, W1)11 :s; P2 ,
IlheAell :s; k411ell ,
Ilh"ull :s; k511ell ,
and
PI = ko + kl + k2 + k3 + k4 + k5 .
Proof" Differentiating h(e, W, WI, TJ, u) we obtain
. .
h(e, W, WI, TJ, u) = hee + hwW + hWl WI + h"u,
or
. .
-F(x, W, Wt)TJ + hw W + hWl WI + h"u;
therefore,
Ilh(e, W, WI, TJ, u)11 :s; IlheAel1 + IlheBWS(x)11 + IlheBW1S'(x)ull
+llheF(x, W, W1)TJII + IlhwWII
+llh w1 WIll + Ilh"ull ,
:s; k411ell + k311ell + k211ell + IlheF(x, W, W1)IIIITJII
+kollell + k11lell + k511 ell ,
:s; k411ell + k311ell + k211ell + P2111J11
+kollell + k11lell + k5 l1 ell
Hence,
which concludes the proof.

We are now able to prove the following theorem
Theorem 3.1.2. The equilibrium of the singularly perturbed model IS

asymptotically stable for all
1
jlE(O, ),
C1C2 + 2C3
and an estimate of its region of attraction is
S = {e, W, Wl,T}: Vee, W, Wl,T}):::; c},
where c is the largest constant such that the set {e, W, W l : V( e, W, W!, 0) :::;
c} is contained to Be X Bw X Bw, . Furthermore, the following properties are
guaranteed
e,x,T}, W, W l E Loo , e,T} E L2
limt--+oo e~t) = 0, limt--+oo T}(~) = 0
limt--+oo Wet) = 0, limt--+oo W let) = 0
Proof: Let us take the Lyapunov function candidate
- -
Vee, W, W l , T}) = 21 ClT IT
Pe + 2C2T} PoT}
1 -T- 1 -T-
+2cltr{W W} + 2cltr{Wl Wd, (3.7)
where P, Po > 0 are chosen to satisfy the Lyapunov equation

PA+ATp=-I,
PoAo + A6' Po = - I .
Observe that (3.7) is a weighted sum composed of a slow and a fast part.
Taking the time derivative of (3.7) and using the learning law
Wij = -biPis(xj )ei ,
Win+l = -bis'(xi)Piuiei ,
for all i = 1,2, ... , n, we obtain, as in a previous subsection that
V = - clllel12
2
- ~1IT}112
2jl
- cle T P F(:r, W, Wl)T}
T . --
-C2T} Poh(e, W, W l , T}, u),
:::; _ c; II el1 2_ ;~ 11T}11 2
+llcleT P F(x, W, W1T} + C2T}T Poh(e,
.
W, W l , T}, u)ll
--
Employing Lemma 3.1.1 we obtain

.
V :::; -
Cl
2"llell 2- 2jlllT}11
C2 2+ clile T P F(x, W, Wl)IIIIT}11
+C211T}Poll(Plllell + p211T}11),
which finally takes the form
3.2 Indirect Control 35
. CI 2 1 2
V:::; -Zllell - C2(2jj - C3)111711 + CIC21I ellll17ll, (3.8)
IIPF(x, W, Wdll:::; C2,
IIPollpl :::; CI ,
IIPollp2 :::; C3;
therefore,
.
V:::; - [lI ellll17111
[-T
T C2(JJl
-~
- C3)
] [Ilell]
111711 (3.9)
The 2 x 2 matrix in (3.9) is positive definite, when

1
jj< - - - -
CIC2 + 2C3
Then V is negative semidefinite. Since V :::; 0 we conclude that V E L oo , which
implies e, 17, W, WI E Loo. Furthermore, e, x = e+x , W = W + W* , WI =
WI + wt are also bounded. Since V is a non-increasing function of time and
bounded from below, the limt ..... oo V = Voo exists so by integrating V from 0
to 00 we have
CI roo IIel12dt + C2( 2.. _ C3) roo 1117112dt
2 Jo 2jj Jo
-cIc21 lI ellll17ll dt :::; [V(O) - Vool < 00,
which implies that e, 17 E L 2 .Furthermore
e = Ae + BWS(x) + BWIS'(x)u - F(x, W, W I )17 ,
jjiJ = Ao17 - jjh(e, W, WI, 17, u).
Since u, A o, h(e, W, WI, 17, u) are bounded, e E Loo and iJ E Loo. Since
e E L2nLoo,17 E L2nLoo, using Barbalat's Lemma we conclude that
= =
limt ..... oo e(t) 0, limt ..... oo 17(t) O. Now using the bo~nde~ness ofu, S(x), S'(x)
and the convergence of e(t) to zero, we have that W , WI also converges to
zero.
Remark 3.1.2. Again, we cannot conclude anything about the conver-
gence of the weights to their optimal values from the above analysis. Remark
2.2.2 provides some details concerning the problem.
3.2 Indirect Control

In this section we investigate the regulation problem. The unknown nonlinear
dynamical system which, by now, has already been identified around an op-
erational point, is regulated to zero adaptively. The adaptive control scheme
falls into the indirect category. The purpose of the identification stage is to
give the green light to the control stage, (if identification fails, control is not
performed). Furthermore, it provides adequate initial values to the control
stage, therefore, leading to a better transient response of the output.
3.2.1 Parametric Uncertainty
In this subsection we assume that the unknown system can be modeled ex-
actly by a dynamical neural network of the form
x = Ax + BW*S(x) + BWtS'(x)u (3.10)
where all matrices are as defined previously. Define the error between the
identifier states and the real system states as
e=x-x,
then from (3.2) and (3.10) we obtain the error equation
e = Ae + BWS(x) + BWIS'(x)u, (3.11)
where
W=W-W*,
WI = WI - wt
Our purpose is to find suitable control and learning laws to drive both e and
x to zero, while all other signals in the closed loop remain bounded. At each
time instant the actual system is modeled by a RHONN of the form
1: = Ax + BWS(x) + BWI 5'(x)u, (3.12)
where Wand WI are the synaptic weight estimates that are provided by the
RHONN identifier. Taking u to be equal to
u = -[WIS'(x)tIWS(x) , (3.13)
and substituting it into (3.2.3) we finally obtain
x = Ax. (3.14)
Again the Lyapunov synthesis method is used to derive stable adaptive laws
so if we take the Lyapunov function candidate,
- - 1 T 1 T 1 -T - 1 -T-
V(e, x, W, Wt) = Ze Pe + Zx Px + ztr{W W} + ztr{WI Wd,
where P > 0 is chosen to satisfy the Lyapunov equation
PA+ATp=-I,
we obtain, (following the same procedure as in Section 3.1.1), that the learn-
ing laws
Wij = -biPiS(Xj )ei ,

Win+l = -bis'(Xi)PiUiei ,
for all i, j = 1, 2, ... , n make
Furthermore, it is trivial to verify that the learning laws above can be written
in matrix form as
W= -EBP50 ,
WI = -BP5'U E,
where all matrices are defined as follows
P = diag[Pl,P2, ... ,Pn],
B = diag[b l , b2, ... , bn] ,
E = diag[el, e2, .. , en],
U = diag[ul, U2, .. . , un] ,
_[S(~t) ... s(~n)l
50 - : :'
S(Xl) ... s(xn)
To apply the control law (3.13), we have to assure the existence of (WI5'(x ))-1.
Since WI and 5' (x) are diagonal matrices and s' (x;) I 0, Vi = 1,2, ... , n
all we need to establish is Win+I(t) I 0 , Vi 2: 0 , Vi = 1,2, ... , n. Hence
WI (t) is confined, through the use of a projection algorithm [73], [32], [45]
to the set W = {WI: IIWIII :S w m } where Wm is a positive constant. Fur-
thermore, WI = WI - W{ and W{ contains the initial values of WI that
identification provides. In particular, the standftrd adaptive laws are modi-
fied to
W= -EBP50 ,
-BP5'UE if WI E W or {IIWIII = Wm
and tr{-BP5'UEWd:S O}
WI = -BP5'UE
+tr{BP5'U EWde+~:'11 )2WI if {IIWIII = Wm and
tr{-BP5'UEWd > O}
therefore, if the initial weights are chosen such that IIW(O)n+111 :S W m , then
we have that IIWIII :S Wm for all t 2: O. This can be readily established by
noting that whenever IIW(t)n+111 = Wm then
dIIW(t)n+111 2 < 0
(3.15)
dt -
which implies that the weights WI are directed towards the inside or the ball
{WI: IIW11I $ w m }. A proof of the (3.15), can be found in Chapter 4. Now
we can prove the following theorem.
Theorem 3.2.1. Consider the control scheme (3.11),(3.13)'(3.14). The
learning law
W= -EBPSo ,
-BPS'UE if WI E W or {IIW11I = Wm
and tr{-BPS'UEWd $ O}
-BPS'UE
+tr{BPS'UEWd(1+~W11I)2Wl if {IIWlll = Wm and
m tr{-BPS'UEWd>O}
guarantees the following properties
e,x, W, WI E Loo , e,x E L2
limt_oo e~t) = 0, liffit_oo x(~) = 0
limt-+oo W(t) = 0, limt_oo W l(t) = 0
Proof: With the adaptive laws mentioned above V becomes
V= _~lIeIl2 - ~llxll2 + Intr{tr{BPS'UEWt}( 1 +~~11l)2WrWd,
~ _~lIeIl2 _ ~lIx112 + Intr{BPS'UEWd( 1 +~~11l)2tr{WrWd,
$ _~lIeIl2 _ ~lIxll2 + Intr{BPS'U EWd( 1 ::m )2I1WlI12,
::; _~lIeIl2 - ~lIxll2 + Intr{BPS'U EWd(l + wm)2,
where In, is an indicato! function defined as In = 1 if the conditions UWllI =
Wm and tr{ -BPS'U EWd > 0 are satisfied. Now since tr{BPS'U EWd < 0
then Intr{BPS'U EWd x(l + wm )2 < o. Hence, V ::; 0 so the additional
terms introduced by the projection can only make Vmore negative. Since V is
negative semidefinite we have that V E L oo , which implies e, X, W, WI E Loo.
Since V is a non-increasing function of time and bounded from below, the
limt_oo V = Voo exists; therefore, by integrating V from 0 to 00 we have
-In(l + wm)21O tr{BPS'U EWddt
1 2(lI 1l + II xll
o
00 1
e 2 2 )dt ::; [V(O) - Voo ] < 00,
which implies that e,:i E L 2 By definition the sigmoid functions S(x), S'(x)
are bounded for all x and by assumption all inputs to the reference model
are also bounded. Hence, from (3.13) we have that u is bounded and from
n
(3.11),(3.14) e, i: E Loo. Since e,:i E L2 Loo and e, i: E L oo , using Barbalat's
= =
Lemma [84], we conclude that limt .... oo e(t) limt .... oo x(t) o. Now using the
boundedness of u, Sex), S'(x) and the convergence of e(t) to zero, we have
that W, WI also converge to zero.
Remark 3.2.1. The analysis above implies that the projection modifica-
tion guarantees bounded ness of the weights, without affecting the rest of the
stability properties established in the absence of projection.
3.2.2 Parametric plus Dynamic Uncertainties
In this subsection, we examine a more general case where parametric and

dynamic uncertainties are present. To analyse the problem, the complete sin-
gular perturbation model (3.6) is used so the control scheme is now described
by the following set of nonlinear differential equations
f- = Ae + BWS(x) + BWIS'(x)u - F(x, W, WI) 7] ,
i: = Ax,
J1:r, = Ao7]/-lh(e, W, WI, 7]),
-
U = -[WIS'(x)]-IWS(x). (3.16)
Before proceeding any further, we need to prove the following lemma.
Lemma 3.2.1. It is true that h(e, W, WI, 7], u) is bounded by
IIh(e, W, WI, 7], u)1I ::; pdlell + pzll7]lI,
IIhw WII
::; kollell ,
IIhwl WIll ::; kIliell,
IIheBWIS'(x)ull ::; kzllell,
IIheBWS(x)1I ::; k3l1ell,
IIheF(x, W, WI)II ::; P2,
IIheAell ::; k411ell ,
and
PI = ko + kl + k2 + k3 + k4 .
Proof: Differentiating h(e, W, WI, 7], u) we obtain
h(e, W, WI, 7]) = lief- + hwW + hWl WI,

or
. .
+h W- W + hW
- WI
i'
therefore,
IIk(e, W, WI. 17, u)1I ::; IIheAell + IIheBWS(:c)11 + IIheBW1S'(:c)ull

+lIheF(:c, W, Wd1711 + IIhwwlI + IIhwl w111,
::; k411ell + kallell + k211e1l + IIheF(:c, W, W1)111I1711
+kollell + k1l1ell,
::; k411ell + kallell + k211ell + P2111711 + kollell + k111 ell
Hence,
k(e, W, W1 , 17, u) ::; P111ell + P2111711,
which concludes the proof.
We are now able to prove the following theorem
Theorem 3.2.2. The control scheme (3.16), is asymptotically stable for
all
J1. E (0, J1.o),
where J1.o = H21'l1'!+1'a). Furthermore, the learning law
W = -EBPSo ,
-BPS'UE ifW1 E W or
IIW1 11 = Wm and
tr{-BPS'UEWd
::;0
-BPS'UE
+tr{BPS'UEWd(1+~:lll)2W1 ifllW111 = Wm and
tr{-BPS'UEWd
>0
guarantees the following properties:
e, X, 17, W, W1 E L cx )! e, X, 17 E L2
limt .... oo e~t) = 0, limt .... oo x(~) = 0, limt .... oo 17(t) = 0
limt .... oo W(t) = 0, limt .... oo Wl(t) = 0
Proof: Let's take the Lyapunov function candidate
-- 1 T 1 T 1 T
V(e, W, W1,17) = 2/1e Pe+ 2/1X PX+ 2/217 Po17 (3.17)
1 -T- 1 -T-
+2/1tr{W W} + 2/1tr{W1 Wd,
where P, Po > 0 are chosen to satisfy the Lyapunov equations
PA+ATp = -I,
PoAo + A5' Po = -1.
Observe that (3.17) is a weighted sum, composed of a slow and a fast part.
Taking the time derivative of (3.17) we obtain
or
v = ~l(-eTe+ST(x)WTBPe+uTS'(X)WIBpe
-r? FT(X, W, W1)Pe) + ~l ((ST(X)WT BPe)T
+(UT S'(x)W1BPef - (7]T FT(X, W, W1)Pef) - ~l XT X
12 1 T .T - - .T - - T
+-(--7] 7]-h (e,W,Wl,7])Po7]-(h (e,W,W1,7])Po7]) )
2 J.l
:.. T _ :.. T _
+/ltr{W W}+/2tr{Wl Wd
Now since sT(x)WT Bpe, uTS'(x)W1Bpe, 7]T FT(x, W, W1)Pe,
hT(e, W, WI, 7])Po7], are scalars, we have that
ST(x)WTBpe = (ST(x)WTBPe)T,
uT S'(x)W1Bpe = (u T S'(x)W1BPef ,
7]T FT(x, W, Wt)Pe = (7]T FT(x, W, W1)Pef,
- Wl,
hT (e, W, - 7])P TJ = (hT (e, W,
- Wl,
- 7])P 7]) T ;
o o
therefore, V becomes
V= ~l (-lIeJJ2 + 2ST(x)W BPe + 2uT S'(x)W1BPe

T T 12 1 --
-2TJ F (x, W, W1)Pe) + T( -;JJTJII 2 - 27]TPoh(e,

W, WI, TJ))
/1 :.. T _ :.. T -
-TJJxIl2 + Iltr{W W} + 12tr{Wl Wd. (3.18)
If we apply the learning laws we have obtained in the previous subsection
(3.18) becomes
V = - IlJJeJJ2 - IlJJxJJ2 - '2 11 7]JJ2 - IlTJT FT(x, W, W1)Pe

2 2 2J.l
T - -
-/2 h (e, W, Wi, TJ)P0 7]
+Intr{tr{BPS'U EWd( 1+ JJWlJJ )2W[Wd ,
Wm
~ - '; JJeJJ2 - ~lJJx1l2 - ;~ JJTJW + IlJJTJJJIIF(x, W, Wl)JJJJPJJJJeJJ

Wl, TJ)JJJJPoJJJJ7]1I
+/2I1h(e, W,
+Intr{BPS'U EWd( 1+ Wm )2JJWlW.
Wm
Employing Lemma 3.2.1 we obtain
v ~ - ~lllel12 - ~lllxl12 - ;~ 1177112 + 'Y1II77IIIIF(x, W, W1)IIIIPllliell

+/'2(pt!lell + p21177IDllPollll7711
+Intr{tr{BPS'U EWd(l + W m ?,
~ - ~111e1l2 _ ~111x112 - ;~ 1177112 + 2/'1I2I1ellll7711 + /'2/3117711 2
+Intr{BPS'U EWd(l + W m )2 ,
provided that
IIF(x, W, WdllllPll ~ /'2,
Plll p oll ~ /'1 ,
P211PoII ~ /'3 ,
which finally takes the form
V ~ - /'; lIel1 2- ~111x112 -/,2(2~ -/'3)II77W + 2/'1I2I1ellll7711

+Intr{BPS'U EWd(l + w m )2 ,
or
V ~ - [lI ellll77l1ll xlll [-! ~2(IZ2

/'2
D D
- /'3) ~1
1.!.
2
+Intr{BPS'U EWd(l + w m )2 . (3.19)

The 3 x 3 matrix in (3.19) is positive definite when
1 1
P < -( ).
- 2 2/'1/'2 + /'3
Then V is negative semidefinite. Since V ~ D we conclude that V E L oo ,
which implies that e, X, 77, W, WI E Loo. Furthermore, W = W + W* , WI =
WI + wt are also bounded. Since V is a non-increasing function of time and
bounded from below, the limt-+oo V = Voo exists; therefore, by integrating V
from D to 00 we have
/'1
2
roo IIell2dt + /'2 ( -.!..
Jo 2p
-/'3) roo 117711 2dt + /'21 roo IIxl1 2dt
Jo Jo
-2/'112 1 00
Ilellll77l1dt - In(l + w m )2 100
tr{BPS'U EWddt
~ [V(D) - Vool < 00 ,
which implies that e, x, 77 E L 2 Furthermore,
e = Ae + BWS(x) + BW1S'(x)u - F(x, W, Wd77,
x = Ax,
pi] = A o77- pk(e, W, WI, 77),
u = -[W1 S'(X)t 1WS(x).
3.3 Test Case: Speed Control of DC Motors 43
Since u, Ao, h(e, W, WI), S(x), S'(x) are bounded, e,~, iJ E Loo- Since
n
e, X, 'T] E L2 L oo , using Barbalat's Lemma we conclude that limt ..... oo e(t) =
0, limt ..... oo x(t) = 0, limt ..... oo 'T](t) = o. Now, using the bound.edne~s of u, S(x),
S'(x) and the convergence of e(t) to zero, we have that W , WI also con-
verges to zero.
Table 3.1. Indirect neural adaptive control scheme
Actual System: :i; = f(x) + G(x)u

Error System: e = Ae + BWS(x) + BW1S'(x)u - F(x, W, Wt)1),
x=Ax,
I'-i!
= A o1) -I'-h(e, W, W1 , 1)),
e=x-x
Control Law: u = -[W1S'(X)]-lWS(X)
Adaptive Laws: W= -EBPSo ,
-BPS'UE if W 1 E Wor
IIW11I = Wm and
tr{ -BPS'U EW1}
::;0
-BPS'UE
+tr{BPS'U EWde+~:tll )2Wl if IIWll1 = Wm and
tr{ -BPS'U EW1}
>0
Properties: e, X, 1), W, W1 E L>e
e, X, 1) E L2
limt_oo e(t) = O,limt_oo x(t) = 0, limt_oo 1)(t) = 0
. .
limt_oo W(t) = 0, limt_oo W1(t) = 0
Conditions: x,x,u E !R n , I'- E (0, ~ (2"Yl""Y~+"Y.))
3.3 Test Case: Speed Control of DC Motors
In this section we apply our theory to solve the problem of controlling the
speed of a DC motor. A separately excited DC motor, can be described by
the following set of normalized nonlinear differential equations [60], [58], [7],
[21].
dla
Tadt = -Ia - 4>Q + Va
dQ
Tmdt = 4>Ia - KoQ- mL (3.20)
d4>
Trdi = -If + Vf
4> = alf
1 + /3If
where the physical meaning of all variables and parameters are provided in
Table 3.2. The state equations for this system may be written in the familiar
form
x= f(x) + g(x)u,
where the states are chosen to be the armature current the angular speed
and the stator flux x = [la Q 4>Y. As control inputs we have the armature
and the field voltage, u = [Va Vf y. With this choice, we have
f(x) =
g(x) =
[t n
Traditionally, the angular velocity of a DC motor is controlled with changes
in its armature voltage, while keeping constant the field excitation. Thus, the
nonlinear model (3.20) is linearized and reduced to
dla
Tadt = -la - 4>Q + Va,
Tm dd~ = 4>Ia - KoQ - mL,
now with 4> a constant value parameter. This method of linearization and
reduction can not be used when one has to alter the field excitation to ful-
fil additional requirements imposed to the control system however, as for
example in the loss minimization problem [66].
The regulation problem of a DC motor is translated as follows: Find a
state feedback to force the angular velocity Q and the armature current la,
to go to zero, while the magnetic flux 4>, varies.
3.3.1 The Algorithm
To achieve such a goal, assuming that the dynamics (3.20) are unknown, we
first assume that the following RHONN model describes, within a degree of
accuracy, the unknown dynamics (3.20)
Table 3.2. Physical interpretation of all DC motor variables-parameters
Parameters Physical meaning

Variables
T.a -- h
Ro. electric time constant
T. - Jn n mechanic time constant
m - TLn
T - NJ~n field time constant
f - VJn
Ko = ~nn normalized viscous friction coefficient
Ln
mL a speed independent component of the applied load torque
Dn nominal no load speed
tPn nominal flux
Vn = ctPnDn nominal armature voltage
I an = l:.u..
Ra extrapolated stalled rotor current at nominal voltage
TLn = ctPnlan extrapolated stalled rotor torque
I fn = ~R nominal field current at nominal field voltage
J
c a constant
rotor resistance
rotor self-inductance
stator self-inductance
number of turns, stator winding
moment of inertia of the motor load system,
referred to the motor shaft
armature voltage
field voltage
armature current
field current
viscous friction coefficient
angular speed
magnetic flux
~l = -alXl + wtls(xt) + wtzs(xz) + wtn+ls'(Xt)Ul,

~z = -azxz + w;zs(xz) + W;n+1S'(XZ)Uz, (3.21 )
where Xl = fa and Xz = iP. Observe that the second differential equation
in (3.21) depends only on Xz and Uz. The above choice is motivated by the
fact that the actual differential equation that describes the evolution of the
magnetic flux iP with time, depends only on iP and Vj, as it can be seen
from (3.20). In this way we employ all a priori information, to develop a
model that matches the dynamic behavior of the unknown plant as closely
as possible. Estimates are provided by the learning laws developed in Section

3.1 because the parameters wtl' wt2, wtn+l' w~2 and W~n+l are unknown. The
role of identification is twofold. First, it gives us an insight about the system
and the green light to proceed to the control phase. If identification fails, the
system under consideration cannot be controlled by our algorithm. Second, it
provides the control phase with appropriate initial conditions, thus improving
the transient behavior of our system.
To regulate the motor speed to zero we apply the following static state
feedback to the control inputs of our system.
1
UI = -
Wln+lS
I() [WllS(Xt)
Xl
+ WI2 S(X2)] ,
1
U2 = - W2n+IS I( X2 )[WI2 S(X2)]. (3.22)
3.3.2 Simulation Results
We simulated a lKW DC motor with a normalized model as in (3.20) with

parameter values that can be seen in Table 3.3. Our two stage algorithm, was
Table 3.3. DC motor parameters
Parameter Value
1
Ta
148.88
1
Tm
42.91
~ 0.0129
Tm
Tj 31.88
mL 0.0
Q' 2.6
(3 1.6
applied. In the identification phase, we use the RHONN shown previously

with parameters
ai = 15,
bi = 65,
for all i= 1, 2, while the parameters of the sigmoid functions were
k = 1.2,
1= 1.0.
The inputs were chosen to be
0.4 -,-------------------------------------------,
0.2
'-
o
t: 0.0
(l)
-0.2
- 0 .4 -+-,..,-..."......,.,......,-,..,-..,-,-...,..,.--r-,....,........,..,....,..,r--r-...-r---r-r-r-.---I
0.0 1.0 2.0 3.0 4.0 5.0 6.0

time (in seconds)
Fig. 3.2. Evolution of el
Ul = Uz = 1 + 0.8sin(0.00lt) .
All initial values were set to zero, except that of the magnetic flux which
was taken to be equal to 0.98. Figures 3.2, 3.3, give the evolution of the errors
el and e2 respectively.
In the control phase, we applied the static-state feedback obtained in
the previous subsection, with the initial values that can be seen in Table
3.4. Figures 3.4, 3.5 give the evolution of the angular velocity and armature
current respectively. As can be seen, both [l and Ia converge to zero very
fast as is desired.
0.8
0.4
L
o
:: 0.0
(J)
-0.4
-0.8
0.0 1.0 2.0 3.0 4.0 5.0 6.0
time (in seconds)
Fig. 3.3. Evolution of e2
Summary
The purpose of this chapter is to use indirect adaptive control techniques to

unknown nonlinear dynamical systems regulation, using RHONNs. Since the
plant is unknown, a two-step algorithm was considered. In step one, a dynam-
ical neural network identifier is employed to perform "black box" identifica-
tion. Many cases that lead to modeling errors, (i.e., parametric and dynamic
uncertainties), are taken into consideration. Convergence ofthe identification
error and weights, (under a sufficiency of excitation condition), to zero and
to a constant value respectively, is proved. Since our main concern lies in
the control problem, only a rough estimation of the region where the weights
should belong is needed to proceed to the control phase of our algorithm,
in which a static-state feedback is developed, to achieve asymptotic regula-
Summary 49
0.32 . , . - - - - - - - - - - - - - - - - - - - ,
0.24
l:J
Q)
Q)
0..
if)
....
0.16
o
......
o
2
0 .08
0.0 0.3 0.6 0.9 1.2 1.5 1.8

ti m e (in seconds)
Fig. 3.4. Convergence of the angular velocity to zero from 0.3 initially
0.15 -r-- - - - -- - - - - - - - - - - - - - - ,
-
-
0.075 -
.....,
c
CIJ
~ 0.0-
u
CIJ
'-
:::l
~ -0.075 -
'-
<I:
-0.15 - V
- 0 .2 2 5 -l-T"""T"""T""""T""T1' 1'T"""T
I I 'I--r-r
I I 'I--r-r
11 '1-,---,-
1 ---,--.------.-1---,--.------.----,---1
0.0 0. 1 0.2 0.3 0.4 0 .5
time (in seconds)
Fig. 3.5. Convergence of the armature current to zero

Summary 51
Table 3.4. Initial values of all variables
Variable Initial Value
Wll (t) 0.037802

W12 (t) 0.236919
W13(t) -0.135972
W21 (t) 0.067767
W22(t) 0.281422
W23(t) -0.116260
.!t(t) 0.3
Ia(t) 0.3
<P( t) 0.98
tion. Again, convergence of the error to zero and boundedness of all signals
in the closed loop, are proved. Finally, only the states of the unknown plant
which are related to the reduced-order model, are assumed to be available
for measurement.
CHAPTER 4
DIRECT ADAPTIVE CONTROL
This chapter is devoted to the development of direct adaptive neurocon-

trollers for affine in the control nonlinear dynamical systems possessing un-
known nonlinearities. The recurrent high-order neural networks are used as
models of the unknown plant, practically transforming the original unknown
system into a RHONN model which is of known structure, but contains a
number of unknown constant-value parameters, known as synaptic weights.
When the RHONN model matches the unknown plant, we provide a com-
prehensive and rigorous analysis of the stability properties of the closed loop
system. Convergence of the state to zero plus boundedness of all other sig-
nals in the closed loop is guaranteed without the need of parameter (weight)
convergence, which is assured only if a sufficiency-of-excitation condition is
satisfied. Moreover, certain sources of instability mechanisms, namely model-
ing errors, uncertainty in model order and external disturbances acting both
additively and multiplicatively, are also considered. Modifications on the con-
trol and update laws are provided, to guarantee a certain robustness level.
The work presented in this chapter, (which is based on [86]-[92]), is written
in progressive complexity. The first four sections deal with the case where
the number of control inputs (m) equals the number of measurable states
(n) and hence, can also be viewed as extensions of the two phase indirect
adaptive control scheme presented in Chapter 3, since more general system
classes are now covered. The last section is devoted to general affine systems
with n i- m. Finally, simulation results are provided to highlight performance
issues.
4.1 Adaptive Regulation - Complete Matching
In this section we investigate the adaptive regulation problem when the mod-
eling error term is zero, or in other words, when we have complete model
matching. Under this assumption the unknown system can be written as
x = -Ax + W*S(x) + wts'(x)u, ( 4.1)
where x E ~n is the system state vector, u E ~n are the control inputs, W* is
a n x n matrix of synaptic weights, W[ is a n x n diagonal matrix of synaptic
weights of the form W[ = [wtl' wt2' ... , wtn] and A is a n x n matrix with
54 4. Direct Adaptive Control
positive eigenvalues which for simplicity can be taken as diagonal. Finally,

S ( x) is a n-dimensional vector and S' ( x) is a n x n diagonal matrix with
elements Si (x), sH x) respectively, both representing high-order connections
of sigmoid functions. For more details concerning the RHONN structure, the
reader is referred to Chapter 2.
Let us take a function h (x) from lRn to lR+ of class C 2 , whose deri vati ve
with respect to time is
ah T
h = ax [-Ax + W*S(x) + wts'(x)u].
The above equation, which is linear with respect to W* and wt, can be
written as:
ah T ah T ah T
h + ax Ax = ax W*S(x) + ax wts'(x)u. (4.2)
Define
t:;. ah T ah T , . ah T
II = ax WS(x) + ax WiS (x)u - h - ax Ax,
where Wand Wi are estimates of W* and wt respectively, obtained by
update laws which are to be designed in the sequel. This signal cannot be
measured since h is unknown. To circumvent this problem, we use the fol-
lowing filtered version of II
e+lI":e=lI,
ah T ah T ah T
= ax WS(x) + ax WiS'(x)u-h- ax Ax,
. ah T
= -h + ax [-Ax + WS(x) + WiS'(x)u] , (4.3)
with K, a strictly positive constant. To implement (4.3), we take

t:;.
e=(-h. (4.4)
Employing (4.4), (4.3) can be written as
ah T
(+ 11":( = II":h + ax [-Ax + WS(x) + WiS'(x)u] , (4.5)
with the state ( E lR. This method is referred to as error filtering. Further-
more, we choose h(x) to be
1 2
h(x) = 21xl .
Hence, (4.5) becomes
(+ 11":( = II":h - x T Ax + xTWS(x) + XTWiS'(x)u. (4.6)
To continue, consider the Lyapunov-like function
4.1 Adaptive Regulation - Complete Matching 55
12 1 -T- 1 -T-
="2e + "2 tr {W W} + "2tr{WI Wd, (4.7)
where
W=W-W*,
WI = W I - wt,
If we take the derivative of (4.7) with respect to time we obtain
'=e~+tr{WTW}+tr{W[Wd. (4.8)
Employing (4.3), (4.8) becomes
, = -",e + e [-h - x T Ax + xTWS(x) + XTWIS'(x)u]

+tr{WTW} + tr{W[Wd, (4.9)
which together with (4.2) gives
, = -",e + e[-xTW*S(x) - xTwts'(x)u + xTWS(x)]
+e [XTWIS'(x)u] + tr{WTW} + tr{W[Wd,
or equivalently
, = -",e+exTWS(x)+eXTWIS'(x)u+tr{WTW}+tr{W[Wd .(4.10)
Hence, if we choose
tr{WTW} = -exTWS(x), (4.11)
. T - T - ,
tr{WI Wd = -ex WIS (x)u, (4.12)
, becomes

= -",e 2
~ o. (4.13)
It can be easily verified that (4.11), (4.12) can be written in terms of elements,
W;j = -eX;Sj(x), (4.14)
Wi! = -ex;s:(x)u;, (4.15)
for all i, j = 1,2, ... , n, and in matrix form as
W = -exsT (x), (4.16)
WI = -ex'S'(x)U, (4.17)
where
x' = diag[xl. X2, ... , x n] ,
U = diag[ul. U2, .. , un].
Now we can prove the following lemma
Lemma 4.1.1. Consider the system

i: = -Ax + W*S(x) + W{S'(x)u,
( = -K( + Kh - xT Ax + xTWS(x) + XTWIS'(x)u,
~=(-h,
1 2
h = 2"lxl .
The update laws
W = _~xST(x),
WI = -~x'S'(x)U,
guarantee the following properties
.~, Ixl, w, WI,( E Loo
I~I E L2
limt-+oo ~(t) = 0, limt-+oo W(t) = 0, limt-+oo WI(t) = 0
provided that u E Loo.
Proof: From (4.13) we have that E Loo which implies e, W, WI E Loo. Since
u E Loo then x E L oo , hence h E Loo. Furthermore, ~ = ( - h, hence ( E Loo.
Since is a monotone decreasing function of time and bounded from below,
the limt-+oo (t) = 00 exists so by integrating from 0 to 00 we have
1 00 1
1~12dt = -[(0) - 00] < 00 ,
r
which implies that I~I E L 2 We also have that
~ = -K~ + xTWS(x) + XTWIS'(x)u.
Hence, eE Loo provided that u E Loo. Having in mind that ~ E L2 nLoo and
eE Loo , applying Barbalat's Lemma [84] we conclude that limt ..... oo~(t) = o.
Now, using the boundedness of u, S( x ), s' (x), x and the convergence of ~ (t)
to zero, we have that W, WI also converge to zero.
Remark 4.1.1. From Lemma 4.1.1 we cannot conclude anything about
the convergence of the weights Wand WI to their optimal values W* and
W{ respectively. In order to guarantee convergence, S( x), S' (x)u need to
satisfy a persistency-of-excitation condition. A signal z(t) E ~n is persistently
exciting in ~n ifthere exist positive constants fio, fil, T such that
fioI~ t I t+T
Z(T)zT(T)dT~fiII \ft~O,
however, such a condition cannot be verified a priori since S(x) and S'(x)u
are nonlinear functions of the state x.
To proceed further, we observe that h can be written as
h = x T [-Ax + WS(x) + WIS'(x)u] - xTWS(x) - XTWIS'(x)u.
Hence, if we choose the control input u to be

u = - [WIS'(X)r l WS(x), (4.18)
then it becomes
it = _x T Ax - xTWS(x) - XTWIS'(x)u. (4.19)
Moreover, (4.19) can be written
. c 2
h :::; - 21x I - ~ - r;;C (4.20)
where
c = 2n.Amin(A) ,
where .Amin(A) denotes the minimum eigenvalue of the A matrix. Observe
that (4.20) is equivalent to
it :::; -ch - ~ - r;;~ . (4.21 )
Furthermore,
h =(-C
hence, (4.21) becomes
( :::; -c( + c~ - r;;C
:::; -c(+(c+r;;)I~I, (4.22)
however, in order to apply the control law (4.18) we have to assure the
existence of [WIS'(X)r l . Since, WI and S'(x) are diagonal matrices and
sHx) > 0, Vi = 1,2, ... , n all we need to establish is Wil(i) :/; 0 Vi:::: 0, Vi =
1,2, ... ,n. Hence, Wil(i), i = 1,2, ... ,n are confined through the use of a
projection algorithm [73],[32],[45],[46] to the set W' = {Wil : 0 < E :::; Wil :::;
w m } where E, w m are appropriately chosen positive constants i . In particular,
the standard update law defined by (4.15) is modified to
-~XiS;(X)Ui if Wil E W' or wilsgn(wil) = E
and ~XiS;(x)uisgn(wil) :::; 0
o ifwilsgn(wil) = E and
~XiS;(x)uisgn(wil) > 0
(4.23)
-~XiS;(X)Ui if Wil E W' or wilsgn(wil) = wm
and ~XiS;(x)uisgn(wil) :::: 0
o if wilsgn(wil) = wm and
~XiS;(x)uisgn(wil) < 0
for all i,j = 1,2, ... , n, where the update law is written in terms of elements
for easier understanding. The following lemma presents the properties of the
projection algorithm in detail.
1 Observe that Wil can be also confined to be negative. However, the above choice
does not harm the generality.
Lemma 4.1. 2. The update law (4.15) with the projection modification
(4.23), can only make .c
more negative and in addition guarantee that
Wi1 E W' for all i = 1,2, ... , n, provided that Wi1(0) E W' and wf1 E W'.
Proof: In order to prove that the projection modification given by (4.23) can
only make i more negative, we observe that the update law (4.23) has the
same form as the one without the projection except for the additional term
eXisi(X)Ui if wi1 sgn (wf1) = C
R = 1 and eXisi(x)Uisgn(wf1)
or if wi1sgn(wf1) = w m
and eXisi(x)Uisgn(wf1)
>0
<0
o otherwise
in the expression for Wi1 for all i = 1,2, ... , n.
Thus, .c is augmented by the following quantity
n
Ra = 2.: eXis;(X)UiWil.
i=1
Furthermore, assume that wf1 > 0 without loss of generality. Hence, sgn( wf1) =
1. Now, having in mind that Wi1 = Wi1 - Wf1' we obtain
n
Ra = 2.:e Xi S;(X)Ui(Wi1 - wt1)
i=1
Case 1: Wi1 = C
Now Ra becomes
n
Ra = 2.:e xi si(x)Ui(C - Wf1)
i=1
but C - wf1 < 0 and eXiSi(X)Ui > 0 hold by definition. Hence, Ra < o.
Case 2: Wil = w m
In this case we have
n
Ra = :LeXiS;(X)Ui(W m - wtl)
i=1
but we have w m - wfl > 0 and eXisi(x)Ui < 0 so again Ra < O.

.c
Thus, is augmented in either case by a negative term.
Furthermore, to verify that Wil E W' Vt ~ 0, we examine the sign of Wi1
when Wil reaches the boundary of W'. With no loss of generality we assume
again that wfl > O.
Case 1: Wi1 = C
In this case we have
-exisHx)Ui if Wi1 E W' or Wi1 = C
Wil = { and eXisi(x)Ui ::; 0
o if Wil = C and eXisi(x)Ui >0
Hence, when Wil = we have Wil 2 0, which implies that Wil is directed
towards the interior of W', provided that w7I E W' and Wil(O) E W'.
Case 2: Wil = w m
In this case

-';XiS:(X)Ui if WI E W' or Wil = w m
= and ';XiS:(X)Ui 2

Wil {
if Wil = w m and ';XiS:(X)U; <

Hence, when WI E W' we have Wil ::; 0, which again implies that Wil is
directed towards the interior of W', provided that w7I E W' and Wi I (0) E W' .
In principle, the projection modification does not alter Wil given by (4.15)
if Wil is in the interior Win of W' or if Wil is at the boundary 8(W/) of W'
and has the tendency to move inward. Otherwise, it subtracts a vector normal
to the boundary so that we get a smooth transformation from the original
vector field, to an inward or tangent to the boundary, vector field.
Remark 4.1.2. Observe that in order to apply the projection algorithm
(4.23) we need to know the sign of the unknown parameter wtl'
To continue, we need to state the following well known Lemma [19]:
Lemma 4.1.3. Let 'I] be a C I time function defined on [0, T) where (0 ::;
T ::; <Xl), satisfying
~ ::; -c'l] + a(t)'T] + {3(t) ,
where c is a strictly positive constant and a(t) and {3(t) are two positive time
functions belonging to L2 (0, T) that is
lT a 2(t)dt ::; MI < <Xl,

and
lT {32(t)dt ::; M2 < <Xl .

Under this assumption, 'I](t) is upper bounded on (0, T) and precisely
'I](t) ::;.;~ ['1](0) + jIJM;"], Vt E [0, T),
Moreover, if T is infinite then

limsup'I](t) ::; 0.
t-+oo
Observe that (4.22) with K = 1 becomes

( ::; -c( + (1 + c)I~I, (4.24)
however, from Lemma 4.1.1 we have that I~I E L2 so (1 + c)I~1 E L 2. Fur-

thermore, observe that obviously T can be extended to be infinite. Hence,
Lemma 4.1.3 can be applied in (4.24) with Ml = 0 to obtain
limsup(t) :S O. (4.25)
t->oo
Moreover, since h =( - ~ and h ;::: 0 we have
(t) ;::: ~(t),
or
(4.26)
but from Lemma 4.1.1 we have
lim ~(t) = O. (4.27)
t->oo
Hence, (4.26) together with (4.25) and (4.27) prove

lim (t)
t->oo
= O. (4.28)
Furthermore, since h =( - ~, (4.27) and (4.28) yield

lim h(x(t))
t->oo
= 0,
Which by the definition of hex) finally implies that
lim Ix(t)1 = 0;
t->oo
therefore, we have proven the following theorem
Theorem 4.1.1. The closed loop system
x = -Ax + W*S(x) + wts'(x)u,
( = -,,( + "h - xT Ax + xTWS(x) + XTW1S'(X)U,
U = _[W1S'(X)]-lWS(x)
~=(-h,
h = 211 xl 2 ,
" = 1,
together with the update laws
Wij = -~XiSj(X),
-~XiS~(X)Ui ifwil E W' or wilsgn(w71) = C
and ~XiS~(x)uisgn(w71) :S 0
o if wilsgn( W71) = c and
~XiS:(x)uisgn(w71) >0
-~XiS~(X)Ui ifwil E W' or wilsgn(w71) = w m
and ~XiS~(x)uisgn(w71) ;::: 0
o ifwilsgn(w71) = w m and
~xisi(x)uisgn(w71) <0
4.2 Robustness Analysis 61
faT' all i, j = 1,2, ... , n guarantee that

lim Ix(t)1 = o.
t-oo
From the aforementioned analysis it is obvious that different choices of h( x)

and "', lead to different adaptive regulators. As we will see in the sequel,
appropriate selection of "', especially attenuates the effects ofthe uncertainties
that may be present.
Table 4.1. Direct adaptive regulation (n = m), complete matching
Actual System: i:=f(x)+G(x)u, x,uE~n

Model: i: = -Ax + W*S(x) + W{S'(X)U,
Control Law: u = _[W1S'(X)]-lWS(X) ,
Update Laws: Wi) = -eXiS)(X) ,
-exisi(x)u. if Wil E W' or Wilsgn(wtl) = e
and eXisi(X)Uisgn(wtd ::; 0
o if Wil sgn( wtd = e and
eXisi(x)Uisgn(wtd > 0
-exisi(x)Ui if Wil E W' or wilsgn(wtd = w rn
and eXisi(x)Uisgn(wi'i) ~ 0
o if Wil sgn( wtd = w rn and
eXisi(x)uisgn(wtd < 0
Filter: , = -,,( + "h - xT Ax + xTWS(x) + xTW1S'(X)U,
e=(-h,
h = ~lxI2,
" = 1,
Properties: W{ E W', W,('e,u E L oo ,
lei E L 2 , limt_oo W(t) = 0,
limt_oo Ix(t)1 = 0,
Requirements: W{ E W', Wt{O) E W',
zero modeling error
4.2 Robustness Analysis
In the previous section we investigated the adaptive regulation problem in

the case where we have complete model matching, in other words, when the
modeling error term is zero. Such a situation is very difficult to be verified a
priori, however. Furthermore, in many practical applications only an approx-

imate description of the actual plant is available. Neglected nonlinearities
may be present in reality. For the practical implementation of such an ideal,
adaptive regulator therefore, it is important to analyze how sensitive the be-
havior of the closed loop is, with respect to such modeling imperfections.
Furthermore, modifications should be added to assure at least bounded ness
of all signals in the closed loop.
In what follows, certain stability properties of the adaptive regulator and
of its modified form are established for cases where (4.1) is only an imperfect
model of the true plant. These results provide theoretical arguments why our
adaptive regulator may well be able to work satisfactorily under real world
conditions.
4.2.1 Modeling Error Effects
Let us assume that the true plant is of known order n and can be modeled
exactly by the recurrent high-order neural network (4.1) plus a modeling
error term w(x, u).
x = -Ax + W*S(x) + W{S'(x)u +w(x, u). (4.29)
We follow the same procedure as in Section 4.1. Thus, we choose a function
h( x) of class C 2 from ~n to ~+ whose derivative with respect to time is now
ahT
h = 8x [-Ax + W*S(x) + W{S'(x)u + w(x, u)] . (4.30)
Equation 4.30 can also be written

8h T 8h T ah T ah T
h + 8x Ax - 8x w(x, u) = 8x W*S(x) + ax wts'(x)u. (4.31 )
Defined as in Section 4.1

~ ah T ah T , . 8h T
1I= ax WS(x) + ax W1S(x)u-h- ax Ax,
and the error-filtering method used
~+K~=lI,
. ahT
= -h + ax [-Ax + WS(x) + W1S'(x)u] . (4.32)
Again set
~
~=(-h. ( 4.33)
Hence,
ahT
(+ K( = Kh + ax [-Ax + WS(x) + W1S'(x)u] , (4.34)
with the state ( E ~ and Ii a strictly positive constant. Furthermore, if we

choose
1
h(x) = 21x12,
then (4.34) finally becomes
( = -li( + rh - x T Ax + xTWS(x) + XTW1S'(x)u. (4.35)
Now consider the Lyapunov-like function
. = ~e
2
+ ~tr{WTW}
2
+ ~tr{WiWd,
2
( 4.36)
taking the derivative of , with respect to time, (4.36) becomes
C = -lie + ~ [-h - xT Ax + xTWS(x) + XTW1S'(x)u]

+tr{WTW} + tr{WiWd , (4.37)
which becomes
C = -lie + ~ [-xTW*S(x) - xTwts'(x)u + xTWS(x)]
+~ [xTW1S'(x)u - xTw(x, u)] + tr{WTW} + tr{WiWd,
or equivalently
C= -lie+ ~xTWS(x) + ~xTWlS'(X)U - ~xT w(x, u)
+tr{WTW} + tr{WiWd (4.38)
after using (4.31). Furthermore, if we chose the same update laws as we did
in Section 4.1, namely (4.14) and (4.15) in element form or (4.16) and (4.17)
in matrix form we obtain
2 T
= -li~ - ~x w(x, u),
:S _lil~12 + 1~llxllw(x, u)l (4.39)
At this point we can distinguish two possible cases. The complete model
matching at zero case and the modeling error at zero case.
The Complete Model Matching at Zero Case. We make the following
assumption.
Assumption 4.2.1. The modeling error term satisfies
Iw(x, u)1 :S k~lxl + k~lul,

where k~ and k~ are known positive constants.
Employing Assumption 4.2.1, (4.39) becomes
C:S _x:1~12 + k~I~lIxI2 + k~I~llxllul ( 4.40)

To continue, we need the following lemma
Lemma 4.2.1. The control law
u =- [WIS'(X)r l [WS(x) + v] , (4.41 )

1
v=-K,x-Ax, (4.42)
2
where the synaptic weight estimates Wand WI, are adjusted according to
(4.14) and (4.23) respectively, guarantee the following
(( t) ::::; 0 "It 2: 0
limt-+oo((t) = 0 exponentially fast
provided that ((0) < o.
Proof: Observe that if we use the control law (4.41) and (4.42), (4.35) becomes
( = - K,(, "It 2: 0 ,
which is a homogeneous differential equation with solution
((t) = ((O)e-"t .
Hence, if ((0), which represents the initial value of ((t), is chosen negative,
we obtain
((t) ::::; 0 \It 2: O.
Moreover, ((t) converges to zero exponentially fast.
Now, let us make the following assumption

Assumption 4.2.2. The k~ constant in Assumption 4.2.1 satisfies
k~ = O.
Assumption 4.2.2 tell us that at zero we have no modeling error in the con-
trolled vector fields since then Iw(O, u)1 ::::; O. Furthermore, observe that be-
cause of Assumption 4.2.2
w(x, u) == w(x).
C : : ; _K,1~12 + k~I~llxI2, ( 4.43)
although it is true that
h=(-~.
Hence, since h 2: 0 we have that

((t) 2: ~(t)
but
((t) ::::; 0, "It 2: 0,
which implies
1((t)1 :::; le(t)1 , Vt 2 o. (4.44)

From the above analysis we have for (4.43)
:::; -Klel 2 + 2k~lel(( - e),
:::; -Klel 2 + 2k~lel(I(1 + lei),
:::; -Klel 2 + 4k~lel2 ,
:::; -(K - 4kDlel 2 ; (4.45)
therefore, if we choose
K 2 4k~, (4.46)
(4.45) becomes
:::;0. (4.47)
Hence, we can prove the following theorem
Theorem 4.2.1. The update laws (4.14) and (4.23) together with the con-
trallaw (4.41) and (4.42) guarantee the following properties
e, lxi, W, WI, (, eE LrJO' lei E L2
limt-+oo {(t) = 0, limt-+oo Ix(t)1 = 0
limt-+oo W(t) = 0, limt-+oo WI(t) = 0
provided that K > 4k' and Assumption 4.2.2 is satisfied.
Proof: From (4.47) we have that E Loo hence e, W, WI E Loo. Furthermore,
W = W + W* E Loo and WI = WI + wt E Loo. Since = ( - hand e
( :::; 0, Vt 20, we have (, hE Loo which in turn implies Ixl E Loo. Moreover,
since is a monotone decreasing function of time and bounded from below,
the limt-+oo (t) = 00 exists so by integrating from 0 to 00 we have
1 lel
00
o
2dt :::;
r- 4
1
k' [(0) - Cool <
I
00,
which implies that lei E L 2 We also have that

~ = -Ke + xTWS(x) + XTWIS'(x)u - x T w(x).
Hence and since u, Ixl,~ E Loo the sigmoidals are bounded by definition,
W, WI E Loo and Assumptions 4.2.1, 4.2.2 hold so since e E L2 Loo and n
~ E Loo, applying Barbalat's Lemma we conclude that limt-+ooe(t) = O. Now,
using the boundedness of u, S(x), S'(x), x and the convergence of e(t) to
zero, we obtain that W, WI also converge to zero. Furthermore, we know that
7](t) converges to zero exponentially fast. Hence and since e(t) also converges
to zero, we have that
lim h(x(t))
t~oo
= t-+oo
lim ((t) - lim e(t),
t--+oo
= o.
Thus,
lim Jx(t)J
t .... oo
= O.
Remark 4.2.1. Again, we cannot conclude anything about the conver-

gence of the synaptic weights Wand WI to their optimum values W* and
wt respectively from the above analysis. Remark 4.1.1 provides some details
concerning the problem.
Now let us return to the status before making Assumption 4.2.2. One can
easily verify that the same results can be derived, provided that
(4.48)
where ku is a known positive constant but by observing (4.41) and (4.42), one
can easily verify that (4.48) is valid, provided that W is uniformly bounded
by a known positive constant Wm so W(t) is confined to the set W = {W :
IIWII :s w m } through the use of a projection algorithm. In particular, the
standard update law (4.16) is modified to
if WE W or IIWII = Wm
l
-~XsT(x)
and tr{~xsT(x)W} ~ 0
. T
W = -~xs (x) (4.49)
+tr{~xsT(x)W}( I+JI~II )2W if IIWII = Wm
and tr{~xST(x)W} <0
therefore, we have the lemma:
Lemma 4.2.2. If the initial weights are chosen such that W(O) E Wand
W* E W then we have W E W for all t ~ O.
Proof: The above lemma can be readily established by noting that whenever
IIWII = Wm then
!(IIWII2):sO, (4.50)
which implies that the weights W, are directed towards the inside of the ball
{W: IIWII :s w m }. It is true that
d(IIW(t)1I2) dtr{WTW}
dt dt
= tr{WWT}.
Employing the modified adaptive law (4.49), we obtain
WTW = -eWT xST (x)
+tr{exST(x)W} c+~~Wllr WTW;
therefore,
tr{WTW} = tr{ -eWT XST (x)

+tr{exST(x)W} C+~~lIr WTW},
= tr{-eWTxST(X)}
+tr{tr{exST(x)W} C+~~Wllr WTW},
:S tr{-eWTxST(X)}
+tr{exST(x)W} C+~~WII) 2 tr{WTW}
but
Thus,
tr{WTW}:S tr{-exST(x)WT }
+tr{exST(x)W} C+~~WII) 211W112.

Now IIWII = W m . Hence,
tr{WTW} :S tr{ -exST (x)WT}
+tr{exST(x)W}(l + w m )2;
therefore and since tr{ ex sT (x) W} < 0 and (1 + w m )2 > 1 we finally obtain
d(IIW(t)112) < o.
dt -
Furthermore, we can prove the following lemma:

Lemma 4.2.3. Based on the adaptive law (4.49), the additional terms
introduced in the expression for.c, can only make.c
more negative.
Proof" Employing the modified update law (4.49), .c becomes augmented by
the term
(4.51)
where is an indicator function defined as lind

lind 1 if the conditions
IIWII = Wm and tr{~xsT(x)W} < 0 are satisfied and lind =
0 otherwise.
Furthermore, (4.51) is less than
( 4.52)
Moreover, observe that

tr{WTW} = tr{(WT + W*T)W} ,

= ~tr{WTW} + tr{ ~WTW + W*T W } ,
2 2
=2
~ttWtt2 + ~tr{WTW} - ~tr{W*TW*}
22
-~tr{WTW*} + ~tr{W*TW},
which finally becomes
tr{WTW} = ~ttWtt2 + ~ttWtt2 - ~ttW*tt2 , ( 4.53)
SInce
tr{W*T W } = tr{WTW*}.
Substituting (4.53) into (4.52) we obtain that the additional term is less than
(4.54)
but by definition, tr{~xST(x)W} < 0 hence ttr{~xsr(x)W}(1+JI,:-'1I)2 < o.

Furthermore,
ttwtt2 + ttwtt 2-ttW*tt 2= ttwtt 2+ w~ -ttW*tt 2> 0,
since ttW*tt < W m . Hence, (4.54) is negative, which implies that becomes
more negative.
Lemma 4.2.2 and Lemma 4.2.3 imply that the projection modification
(4.49) guarantees boundedness of the weights, without affecting the rest of
the stability properties established in the absence of projection.
Now that we have established the validity of (4.48), we observe that if the
design constant Ii is chosen such that
Ii > 4(k~ + k~ku), (4.55)
then (4.47) is true and the results of Theorem 4.2.1 are still valid.
Remark 4.2.2. The previous analysis reveals that the accuracy of the
recurrent high order neural network model should be restricted only at the
origin. In other words, when we have complete model matching at zero, our
modified adaptive regulator can guarantee that the stability properties of
the closed-loop system do not alter. Furthermore, if we don't have modeling
error in the controlled vector fields, that is k~ = 0, there is no need to bound
the Ws uniformly through the use of the projection algorithm (4.49), thus
simplifying the implementation issue.
Remark 4.2.3. Inequalities (4.46) and (4.55) show how the design con-
stant Ii should be selected, in order to guarantee convergence of the state x
to zero, even in the presence of modeling error terms which are not uniformly
bounded a priori, as Assumption 4.2.1 implies. The value of r., becomes large
as we allow for large model imperfections but r., is implemented as a gain in
the construction of ( and for practical reasons it cannot take arbitrarily large
values. this leads to a compromise between the value of r., and the maximum
allowable modeling error terms.
Table 4.2. Direct adaptive regulation (n == m), no modeling error at Ixl == 0

(w(O, O) == 0)
Actual System: i: == f(x) + G(x)u, x, u E ~n

Model: i: == -Ax + W*S(x) + wts'(x)u + w(x, u),
Control Law: u == _[WIS'(X)]-I[WS(X) + tll:X - Ax]
Update Laws: Wi} == -~XiS}(X),
-~XiS:(X)Ui if Wil E W' or wilsgn(Wtl) == e
and ~XiS:(x)uisgn(wtl) ::; 0
o if wilsgn(wtl) == e and
~XiS:(x)uisgn(wtd >0
-~XiS:(X)Ui if Wi I E W' or wiISgn(Wtl) == w m
and ~XiS:( x )Uisgn( Wtl) ;::: 0
o if Wil sgn( wtd == w m and
~XiS:(x)uisgn(wtd <0
Filter: ( == -II:( ,
~==(-h,
h == tlxl2,
Properties: WI E W', W,~,lxl,e,u E L oo , 1((t)l::; 1((0)1
I~I E L 2 , ((t)::; 0, limt_oo Ix(t)1 == 0,
Requirements: wt E W', WI (0) E W',
((0) ::; 0, Iw(x, u)1 ::; k~lxl + k~/lul,
II: > 4(k~ + k~/kl.l)'
The Modeling Error at Zero Case. In the previous subsection, we have

assumed that the modeling error term satisfies the following condition
Iw(x, u)1 ~ k~lxl + k~lul,
which implies that the modeling error becomes zero when Ixl = 0 and we have
proven convergence of the state x to zero, plus bounded ness of all signals in
the closed loop. In this subsection however, we examine the more general case
which is described by the following assumption.
Assumption 4.2.3. The modeling error term satisfies
Iw(x, u)1 ~ ko + k~lxl + k~lul

Having made this assumption, we now allow a not-necessarily-known model-
ing error ko ::f. 0 at zero. Furthermore, as stated previously, we can find an a
priori known constant ku > 0, such that
thus making
Iw(x, u)1 == Iw(x)1 ,
and Assumption 4.2.3 equivalent to
Iw(x)1 ~ ko + kllxl, (4.56)

where
kl = k~ + k~ ku , (4.57)
is a positive constant. Employing (4.56), (4.39) becomes
.c ~ -Klel 2 + lellxl[ko + k1lxl]'

~ -Klel 2 + k 1 lellxl 2 + kolellxl (4.58)
Again, using Lemma4.2.1 and the fact that 1((t)1 ~ le(t)1 when ((t) ~ 0 '<It ~
0, (4.58) becomes
.c ~ -Klel 2 + 4kdel 2 + kolellxl (4.59)

but
~lxl2 ~ lei + 1(1,

~ 21el
Hence,
Ixl ~ 2M (4.60)
Substituting (4.60) into (4.59) we obtain
.c ~ -(K - 4k )le1 2 + 2kolelM,

1
~ [-(K - 4klh/iZl + 2kollelM,

~ 0, (4.61)
provided that
M> 2ko ,
K - 4kl
or equivalently
( 4.62)
with /'C > 4k l . Inequality 4.62 together with (4.60) demonstrate that the
trajectories of ~(t) and x(t) are uniformly ultimately bounded with respect
to the arbitrarily small, (since /'C can be chosen sufficiently large), sets [; and
X shown below
and
X = {x(t) : Ix(t)l:S 4ko

/'C - 4kl
,/'C > 4kl > o}
Thus, we have proven the following theorem:
Theorem 4.2.2. Consider the system (4.29) with the modeling error term
satisfying (4.56). Then the control law (4.41), (4.42) together with the update
laws (4.23) and (4.49) guarantees the uniform ultimate boundedness with
respect to the sets
[; = { ~(t): 1~(t)1 :S
4k2
(,,-4t)2' /'C > 4kl > 0 }
X = {x(t) : Ix(t)1 :S ,,~~t, /'C > 4kl > 0 }
Furthermore,
e= -/'C~ + xTWS(x) + XTWlS'(x)u - x T w(x). (4.63)
Hence, since the boundedness of Wand Wl is assured by the use of the pro-
jection algorithm and w( x) owing to (4.56) and Theorem 4.2.2, we conclude
e
that E Loo.
Remark 4.2.4. The previous analysis reveals that in the case where we
have a modeling error different from zero at Ixl = 0, our adaptive regulator
can guarantee at least uniform ultimate boundedness of all signals in the
closed loop. In particular, Theorem 4.2.2 shows that if ko is sufficiently small,
or if the design constant /'C is chosen such that /'C 2:: 4kll then Ix (t) I can be
arbitrarily close to zero and in the limit as /'C ---+ 00, actually becomes zero
but as we have stated in Remark 4.2.3, implementati0n issues constrain the
maximum allowable value of /'C.
4.2.2 Model Order Problems
Let us assume that the true plant is of order N 2:: n and therefore, is described
by
x = -Ax + W*S(x) + wts'(x)u + (x, Xud) ,
Xud = B(x, Xud), (4.64)
Table 4.3. Direct adaptive regulation (n = m), (w(O, 0) f= 0)
Actual System: x= f(x) + G(x)u, x, u E R n

Model: X = -Ax + W*S(x) + Wi""S'(X)U + w(x, u),
Control Law: u = _[WIS'(X)]-I[WS(X) + tl\;X - Ax],
_~xST(x) if WE W or IIWII = Wm
and tr{~xST(x)W} 2': 0
Update Laws: w= _~xST(x)
+tr{~xST(x)W}(I+JI:II?W if IIWII = Wm
and tr{~xST(x)W} < 0
-~XiS:(X)Ui if Wil E W' or wilsgn(wtd =e
and ~XiS:(x)uisgn(wii) S; 0
o if wilsgn(wtl) = e and
Wil =
-~XiS:(X)Ui if Wil E W' or wilsgn(Wtl) = w m
and eXiS:(X)Uisgn(wtd 2': 0
o if wilsgn(wtd = w m and
~xisHx)uisgn(wtl) <0
Filter: (= -1\;(,
~=(-h,
h=tlxI2,
Properties: e
WI E W', u, E Lao, 1((t)1 S; 1((0)1,
WE W, ~, x are u.u.b. with respect to
= { ~(t) : 1~(t)1 S; (K.::L" I\; > 4kI > O} ,
X = {x(t): Ix(t)1 S; K~~~l' I\; > 4kI > O} ,
Requirements: Wi"" E W', WI (0) E W',
WI E W, W (0) E W,
((0) ::; 0, Iw(x, u)1 S; ko + kdxl
where Xud E iRP , is the state of the unmodeled dynamics and (.), B(.) are
unknown vector fields of x and Xud. Obviously p = N - n.
Proceeding as in the previous subsection, we choose a function h( x) of
class C 2 from iRP to iR+ whose derivative with respect to time is
. ahT
h = ax [-Ax + W*S(x) + wts'(x)u + (x, Xud)] , (4.65)
which can also be written as

. ah T ahT ahT ah T
h + ax Ax - ax (X,Xud) = ax W*S(x) + ax wts'(x)u. (4.66)
Define
I::;. ah T ah T , . ah T
v = ax WS(x) + ax W1S (x)u - h - ax Ax,
and use the error filtering method
~+IC~=v,
. ah T
= -h + ax [-Ax + WS(x) + W1S'(x)uJ . (4.67)
Set
I::;.
~=(-h; (4.68)
therefore,
. ah T
(+ IC( = ICh + ax [-Ax + WS(x) + W1S'(x)uJ , (4.69)
with the state ( E ~ and IC a strictly positive constant. Furthermore, if we

choose
h(x) = ~lxI2,
2
( = -IC( + ICh - xT Ax + xTWS(x) + XTW1S'(x)u. (4.70)
1 2 1 -T- 1 -T-
C = 2'~ + V(Xud) + 2'tr{W W} + 2'tr{W1 Wd, (4.71)
where V(xud) is a positive definite function of class C 1 from ~p to ~+. Taking

the time derivative of (4.71) we obtain
.c = -ICe + ~ [-h - xT Ax + xTWS(x) + XTW1S,(x)u]

. 'T- 'T-
+V(xud) +tr{W W} +tr{W1 Wd, (4.72)
which after using (4.66)and (4.67) finally takes the form
.c = -ICe + ~xTWS(x) + ~xTWlS'(X)U - ~xT (x, Xu d) + V(Xud)
+tr{WTW} + tr{W[Wd. (4.73)
Furthermore, if we choose the same update laws as we did in Sections 1.1,
1.2, namely (4.23) and (4.49) we obtain
C = -IC~ - ~x (x, Xud)
T .
2
+ V(Xud). (4.74)
To proceed further, we distinguish two different cases.
Uniform Asymptotic Stability in the Large Case. For completeness,

we introduce from [67] the following definitions that are crucial to our dis-
cussion.
Definition 4.2.1. The equilibrium point Xud = 0 is said to be uniformly
asymptotically stable in the large if
for every M > 0 and any to E ~+, there exists a ii(M) > 0 such that
IXud(t; Xud(O), to)1 < M for all t ~ to whenever IXud(O)1 < ii(M)
for every ii > 0, and any to E ~+ there exists a M(ii) > 0 such that
IXud(t; Xud(O), to)1 < M(ii) for all t ~ to whenever IXud(O)1 < ii
for any ii, any M > 0 and to E ~+ there exists T(M, ii) > 0, inde-
pendent of to such that, if IXud(O)1 < ii then IXud(t; Xud(O), to)1 < M for all
t ~ to +T(M,ii)
For the state of the unmodeled dynamics we make the following assumption
Assumption 4.2.4. The origin Xud = 0 of the unmodeled dynamics is
uniformly stable in the large. More specifically, there is a C 1 func-tion V( Xu d)
from ~p to ~+ and continuous, strictly increasing, scalar functions /i(lxudi)
from ~+ to ~+, i = 1,2,3, which satisfy
/i(O) = 0, i = 1,2,3
lim /i(S)
8-+00
= 00, i = 1,2
such that for Xud E ~p
and
(4.75)

BV
I:- = -x:lel 2 - T
ex </J(x, Xud) + UXud
-B(x, Xud) ,
-.(l
::; -x:leI 2 - exT </J(x, Xud) -'Y3(lxuai),

::; -x:leI 2 - exT </J(x, Xud) ,
::; -x:leI 2 + lellxII</J(x, xud)l (4.76)
To continue we consider the following cases:
Case 1: Assume that the unknown vector field </J( x, Xud) satisfies the condition
(4.77)
with </J'(Xud) an unknown vector field that depends only on Xud. We further
assume that </J'(Xud) is bounded uniformly by a constant e. Hence,
(4.78)
Thus, we have for C

C~ _KI~12 + k.pBI~llxI2 . (4.79)
Moreover, if we apply (4.60) then (4.79) becomes
C~ _KI~12 + 4k.pBI~12 ,
~ -(K - 4k.pB)I~12 ,
~ 0, (4.80)
provided that the design constant K is chosen such that
K > 4k.pB. (4.81)
Thus, we can prove the following theorem:
Theorem 4.2.3. Consider the closed loop system
x = -Ax + W*S(x) + wts'(x)u + J(x, Xud) ,
Xud = B(x, Xud),
(= -K(,
u = -[WIS'(x)]-I[WS(x) + v],
1
v= -KX - Ax,
2
1 2
h = 21xl ,
~=(-h.
The update laws (4.23) and (4.49) guarantee the following properties
.~, Ixl, IXudl,( E Loo , I~I E L2
limt-+oo ((t) = 0, limt-+oo Ix(t)1 = 0
limt-+oo W(t) = 0, limt-+oo WI(t) = 0
provided that (4. 81) holds.
Proof: From (4.80) we have that E Loo hence, e, Xud E Loo. Furthermore, W
and WI are kept bounded owing to the projection algorithm. Since ~ = ( - h
and ( ~ 0, \:It 2: 0 we have (, h E Loo which in turn implies Ixl E Loo.
Moreover, since is a monotone decreasing function of time and bounded
from below, the limt-+oo (t) = 00 exists so by integrating C from 0 to 00
we have
1 00
o
1
1~12dt ~
K -
k B[(0) - 001
4 .p
< 00 ,
which implies that I~I E L 2 We also have that
e= -K~ + xTWS(x) + XTWIS'(x)u - xT J(x, Xud).
Hence, eE Loo. Moreover, since e E L2 nLoo and eE L oo , applying Bar-
balat's Lemma we conclude that limt-+oo~(t) = o. Now, using the bounded-
ness of u, S(x), S'(x), x and the convergence of ~(t) to zero, we obtain that
W, W! also converge to zero. Furthermore, we know that ((t) converges to

zero exponentially fast. Hence and since ~(t) also converges to zero, we have
that
lim h(x(t))
t--+oo
= t-+oo
lim ((t) - lim
t--+oo
~(t) ,
= o.
Thus
lim Ix(t)1 = o.

t-+oo
Case 2: Now assume that the unknown vector field (x, Xud) satisfies the
condition
(4.82)
In other words, ( x, Xud) is assumed to be bounded uniformly by a constant.
Thus, (4.76) becomes
.c ~ _"1~12 + Ol~"xl. (4.83)
Again, if we apply Lemma 4.2.1, then (4.44) holds, which in turns gives
Ixl ~ 2M (4.84)
Substituting (4.84) into (4.83) we obtain
.c ~ _"1~12 + 201~IM, (4.85)

~ [-"M+20JI~IM,
~ 0, (4.86)
provided that
4iP
I~I> -2' (4.87)
"
with" > o. Inequality 4.87, together with (4.84), demonstrates that the
trajectories of ~(i) and xCi) are uniformly ultimately bounded with respect
to the arbitrarily small, (since" can be chosen sufficiently large), sets [; and
X shown below
[; = {~(t): 1~(i)1 ~ 402}

~' ,,> 0 ,
and
X = {x(t) : Ix(i)1 ~ ~, " > o} .

Theorem 4.2.4. Consider the system (4.64) with the unmodeled dynamics
satisfying Assumption 4.2.4. Assume also that (4.82) holds for the unknown
vector field (x, Xud). Then the contrallaw (4.41), (4.42) together with the
update laws (4.23) and (4.49), guarantees the uniform ultimate boundedness
with respect to the sets
{~(t) : 1~(t)1 ~ ~,
t' = K > O}
X = {x(t) : Ix(t)1 ~ 4f, K > O}
Furthermore,
~ = -K~ + xTWS(x) + XTWIS'(x)u - x T (x, Xud). (4.88)
Hence, since the boundedness of Wand WI is assured by the use of the
projection algorithm and ( x, Xud) is bounded uniformly, we conclude that
~ E Loo.
The above analysis clearly indicates the importance of the appropriate se-
lection of the design constant K. If it is chosen wisely, our adaptive regulator
can assure convergence of x to zero, or at least uniform ultimate bounded-
ness of x and all other signals in the closed loop. Again, the more informa-
tion we have about the unknown system, the better the control results. The
above statement is mathematically translated as the constants kep, (), B will
be smaller, thus overcoming any implementation issues that may appear ow-
ing to large value in K. Hence, it is desirable, for the RHONN to match the
input-output behavior of the true-but-unknown-nonlinear-dynamical system
as accurately as possible.
Finally, observe that in the case where the unknown vector field ( x, Xud) is
uniformly bounded by a constant, any positive value of K, suffices to guaran-
tee uniform boundedness of x.
Violation of the Uniform Asymptotic Stability in the Large Con-
dition. In the present subsection we examine the effect of the unmodeled
dynamics on the stability properties of the closed-loop system when they vi-
olate the uniform asymptotic stability in the large condition, namely (4.75).
Thus we assume that instead of (4.75) we have
f)f)V B(x, Xu d) ~ -'Y3(lx udJ) + plxl 2 , (4.89)

Xud
where p is a positive constant. Employing (4.89) within (4.74) we have
. 2 T f)V
= -KI~I - + -f)-B(x, Xud),
~X (X, Xud)
Xud
~ _KI~12 - ~xT (X, Xud) - 'Y3(lx udJ) + plxI2 ,
~ _KI~12 + 1~lIxll(x,xud)1 + plxl2. (4.90)
As previously, we shall consider the following cases:
Case 1: Assume that the unknown vector field (x, Xu d) satisfies (4.77),
(4.78). Then (4.90) becomes
s -Klel 2 + kcpBlellxl2 + plxl 2. (4.91)

If we apply Lemma 4.2.1 then (4.60) holds, which in turn gives
s -Klel 2 + 4kcpBlel 2 + 4plel ,
S -(K - 4kcpB)lel 2 + 4plel ,
S [-(K - 4kcpB)lel + 4p]lel,
SO, (4.92)
provided that
4p
lei> K - 4kcpB ' (4.93)
with K > 4kcpB > O. Inequality 4.93, together with (4.60) demonstrates that
the trajectories of e(t) and x(t) are uniformly ultimately bounded with re-
spect to the arbitrarily small, (since K can be chosen sufficiently large), sets
and X shown below
= {e(t): le(t)1 S K _4:kcpB ' K > 4kcpB > o} ,

and
X = {x(t): Ix(t)1 S 4) K -:kcpB' K > 4kcpB} ,

satisfying (4.89). Assume also that (4.77), (4.78) hold for the unknown vector
field tfJ(x, Xud). Then the control law (4.41), (4.42) together with the update
laws (4.23) and (4.49), guarantees the uniform ultimate boundedness with
respect to the sets
= {e(t) : le(t)1 S I<-!tll' K > 4kcpB > 0}

X = {x(t) : Ix(t)1 S 4J~, > 4kcpB > o}K
Furthermore,
e= -Ke + xTWS(x) + XTW1S'(x)u - xT tfJ(x, Xu d) . (4.94)
Hence and since the boundedness of Wand Wt is assured by the use of the
e
projection algorithm and tfJ(x, Xud) is bounded, we conclude that E Leo.
Case 2: Now, assuming that the unknown vector field tfJ(x, xud) satisfies
(4.82), (4.90) becomes
S -Klel 2 + Olellxl + plxl 2. (4.95)
After applying (4.60), (4.95) becomes
S -Klel 2 + 20lelvW + 4plel,

or equivalently
" :::; [-Klel + 20M + 4P] lei,

:::; 0, (4.96)
provided that
M > 0+ v'()2 + 4PK ,

K
or
M < 0- v'()2 + 4PK ,
K
or equivalently if
lei> (0 + v'()22+ 4pK)2 , (4.97)

K
or
lei < (0 - v'022+ 4pK)2 ,

K
and K > o. Inequality 4.97 together with (4.60), demonstrates that the tra-
jectories of e(t) and x(t) are uniformly ultimately bounded with respect to
the arbitrarily small, (since K can be chosen sufficiently large), sets [; and X
shown below
[; = { e(t) : le(t)1 :::; (0 + v'0:/ 4pK)2, K> o} ,

and
X={X(t):IX(t)l:::; 2(0+v'~2+4PK), K>O}.

satisfying (4.89). Assume further that (4.82) holds for the unknown vector
field (x, Xud). Then the control law (4.41), (4.42) together with the update
laws (4.23) and (4.49), guarantees the uniform ultimate boundedness with
respect to the sets
[; = { e(t) : le(t)1 :::; (8+1+4;;)2, K> 0 }

X = {X(t) : Ix(t)1 :::; 2(8+~), K > o}
Furthermore,
e= -x:e + xTWS(x) + XTWIS'(x)u - xT(x, Xud). (4.98)
Hence and since the boundedness of Wand WI is assured by the use of
e
the projection algorithm and (x, Xud) is bounded, we conclude that E Loo.
Theorems 4.2.5 and 4.2.6 demonstrate that even in the case where the uniform
asymptotic stability in the large condition is violated, our adaptive regulator
can still assure the uniform ultimate boundedness of the state x and of all
signals in the closed loop. Appropriate selection of the design constant x: leads
to better performance, provided it satisfies the implementation constraints.
Again, the accuracy of our model, (recurrent high order neural network), is
a performance index of our adaptive regulator.
4.2.3 Simulations
In this subsection we apply the direct adaptive neural controller to the regu-
lation the nonlinearly-operated DC motor described in Chapter 3. To achieve
such a goal, assuming that the DC motor dynamics are unknown, we first as-
sume that the following first-order RHONN model describes, within a degree
of accuracy, the unknown dynamics
i l = -alzl + wtls(zt) + wt2S(Z2) + wt3s'(Zt)UI,
i2 = -a2z2 + wil s(Z2) + Wi2S'(Z2)U2 , (4.99)
where Zl = faand Z2 = n. Observe that the second differential equation in
(4.99) depends only on Z2 and U2. The above choice is motivated by the fact
that the actual differential equation that describes the evolution of the mag-
netic flux qJ with time, depends only on qJ and Vi. In this way we employ all a
priori information, to develop a model that matches the dynamic behaviour
of the unknown plant as closely as possible. The parameters wII' WI2, wI3' w~l
and w~2 being unknown, estimates are provided by update laws of the form
(4.23) and (4.49) with
( = -x:(, (4.100)
e= (- 1 2 + Z22] .
"2[Zl (4.101)
The control laws are taken to be

1
UI = - ( ) [WllS(ZI) + WI2 S(Z2)] , (4.102)
W13S' Zl
1
U2 = - '() [W12 S(Z2)] , (4.103)
W22S Z2
where in (4.99) the parameters ab a2 were chosen to be
al = a2 = 0.2.
Table 4.4. Direct adaptive regulation (n = m), model order problems under a
uniform stability in the large condition
Actual System: x=!(x)+G(x)u+(X,xud), x,uE~n

Xud = B(X,xud), Xud E ~p
Model: x = -Ax + W*S(x) + wts'(x)u + (x, xud),
Xud = B( x, xud),
Control Law: u = _[W1S'(X)]-1[WS(X) + ~I\;X - Ax],
_~xST(x) if WE W or IIWI/ = Wm
and tr{~xST(x)W} ~ 0
Update Laws: vir = _~xST(x)
+tr{~xST(x)W}e+Jt~"?W if IIWI/ = Wm
and tr{~xST(x)W} <0

-~XiS:(X)Ui if Wil E Wi or wilsgn(Wtl) = C
and ~XiS:(x)uisgn(wtd ::; 0

o if wilsgn(wtd = c and
-~XiS:(X)Ui if Wil E W' or wilsgn(wtd = wm
and ~XiS:(x)uisgn(wtd ~ 0
~XiS:(x)uisgn(wtl) <0
Filter: ( = -I\;c,
~=(-h,
h=~lxI2,
Casel:
Properties: W 1 E W', u,i,~,lxl,lxudl E L oo ,

W E W, 1((t)l::; 1((0)1, I~I E L2 ,
limt_oo 1~(t)1 = 0, limt_oo Ix(t)1 = 0
Requirements: wt E W', Wl(O) E W',
W* E W, W(O) E W,
((0) ::; 0, I\; > 4kO.
Case2: I(x, xud)1 ::; 8

Properties: W 1 E W', v,i, IXudl E L oo ,
WE w, 1((t)l::; 1((0)1,
~, x are u. u. b. with respect to
[={~(t):I~(t)I::;~, I\;>O} ,
X = {x(t) : Ix(t)1 ::; ~, I\; > O} ,
W* E W, W(O) E W,
((0) ::; 0, .
Table 4.5. Direct adaptive regulation (n = m), model order problems. Violation
of the uniform stability in the large condition
Actual System: x= f(x) + G(x)u + cjJ(x, xud), X, u E R n

XUd = B(x, xud), Xud E RP
Model: x= -Ax + W* Sex) + Wi"S'(X)U + cjJ(x, xud),
XUd = B(X,xud),
Control Law: U= _[WlS'(X)]-l[WS(X) + ~"x - Ax],
Update Laws: W= _~xST(x)
+tr{~xST(x)W}e+JI:1I )2W if IIWII = Wm
and tr{~xST(x)W} <0
-~XiS:(X)Ui if Wil E W' or Wilsgn(wtd = c:
and ~XiS:(x)uisgn(wtd ::; 0
o if wiIsgn(wtd = c: and
-~XiS:(X)Ui if Wil E W' or wiIsgn(wtd = w m
o ifwilsgn(wtI) =wm and
Filter: ,=
~=(-h,
-t>(,
~XiS:(x)uisgn(wtd <0
h=~lxI2,
Casel:
Properties: e,
W l E W', u, IXudl E Loo,
W E W, 1((t)l::; 1((0)1,
~, x are u.u.b. with respect to
E= {~(t): IW)I ::; I<-!~.pe' " > 4kB > o} ,

X = { x(t) : Ix(t)1 ::; 4) I<-:k.pe, ,,> 4kB > o} ,
Case2: IcjJ(x, xud)1 ::; if
Properties: W l E W', u,e,

IXudl E Loo,
W E W, 1((t)l::; 1((0)1,
~, x are u.u.b. with respect to
E = {~(t) : 1~(t)1 ::; (tJ+~)2, ,,> o} ,
X = { x(t) : Ix(t)1 ::; 2(tJ+/f4?K), " > 0 } ,
Requirements: Wi" E W', WI (0) E W',
W E W, W(O) E W,
((0) ::; 0, .
4.3 Modeling Errors with Unknown Coefficients 83
We simulated the 1KW DC motor with parameter values as given in

Chapter 3. The parameters of the sigmoid functions were set to
k = 1.0,
1= 1.0.
The initial values of all variables can be seen in Table 4.6. Figures 4.1, 4.2
Table 4.6. Initial values of all system variables
Variable Initial value
fa 0.3
n 0.3
~ 0.98
Wll = W12 = W21 0.0
W13 = W22 1.0
give the evolution of the angular velocity and armature current respectively,
when the design constant K is chosen to be K = 80. It can be seen, both {l
and fa converge to zero very fast.
4.3 Modeling Errors with Unknown Coefficients
In this section the theory previously developed is extended to cover the more
general case where the modeling error term has polynomial growth of first
order with unknown growth magnitude. We shall prove that despite the above
fact, the proposed adaptive regulator still retains its robustness properties,
provided that certain modifications are made in the control and update laws.
exactly by a recurrent high order neural network plus a modeling error term
w(x, u).
x = -Ax + W*S(x) + wts'(x)u + w(x, u), (4.104)
with the modeling error term satisfying the following assumption:
Assumption 4.3.1. The modeling error term satisfies the relation
Iw(x, u)1 ~ ko + kl (Ixl + luI) ,
where ki i = 0,1 are unknown, except for ko, positive constants.
Observe that in this way the modeling error is not restricted to be a priori
bounded. The above together with the fact that the actual parameter kl is
simulation results
~ 35.0 =J
w
28.0
~ 21 .0
v
0
Q)
> 14.0
'-
0
:l
0'
C
7.0
0.0
- 7 .0 -t--.--r-;---r---r---.--.-,---,--r---.---.---.--r--:I---r--r---:--..,...-l
0.0 0.05 0.1 0. 15 0.2
t ime (in seconds)
Fig. 4.1. Evolution of motor speed with time, K, = 80
considered unknown, makes Assumption 4.3.1 quite general and therefore,

valid in many applications.
Let us again take a function h( x) of class C 2 from ~n to ~+ whose
derivative with respect to time is
. oh T
h = ox [-Ax + W*S(x) + wts'(x)u + w(x, u)] . (4.105)

. ah T ah T ah T ah T
h + ax Ax - ax w(x, u) = ax W*S(x) + ax wts'(x)u. (4.106)
Define
t:::. ah T ah T . ah T ah T
/I = ox WS(x) + ax WIS'(x)u - h - ax Ax - ax sgn(OkIx,
where W, WI and kl are estimates of the unknown parameters W*, W{ and

kl respectively. Moreover, the function sgn(~) is defined as follows:
simu lat ion resu lt s

0.15-r--------------------------------~
0.075
.....,
c:
(l)
t
:J
0.0
U
(l)
'-
.....,
:J
E-0.075
'-
-0.15
0.0 0.05 0.1 0.15 0.2

time ( in seconds)
Fig. 4.2. Evolution of the armature current with time, " = 80
1 if~>O
sgn(~) = { -1 otherwise
Observe that the signal 1/ cannot be measured since h is unknown. Thus we
use the following filtered version of 1/.
~+K~=I/,
. oh T
= -h + ax [-Ax + WS(x) + W1S'(x)u]
oh T
- ax sgn(e)klx, (4.107)
with K a strictly positive constant. To implement (4.107), we take

Do
~=(-h. (4.108)
. K(
(+ oh T
= Kh + ax [ ]
-Ax + WS(x) + W1S'(x)u - sgn(e)klx ,(4.109)
with the state ( E ~. Furthermore, again, we choose h(x) to be

1 2
h(x) = 2'lxl .
( = -K( + ~Klxl2 - xT Ax + xTWS(x)

+XTWIS'(x)u - sgn(e)k I lxI 2 . (4.110)
1 2 1 - T - 1 - T - 1-2
C= 2'e + 2'tr{W W} + 2'tr{WI Wd + 2'k l , (4.111)
where
W=W-W*,
WI = WI - wt,
kl = kl - kl .
C = d + tr{WTW} + tr{W[Wd + klkl . (4.112)

C = -Ke + e [-h - xT Ax + xTWS(x) + XTW1S'(x)u - sgn(e)kIlxI2]

'T- 'T--
+tr{W W} + tr{WI Wd + klkl ,
A
(4.113)
C = -Ke + e [-xTW*S(x) - xTWrS'(x)u + xTWS(x)]
+e [xTW1S'(x)u - xTw(x, u)]- e sgn(Ok I lxl 2 + tr{WTW}
. T - -
Wd + kIkl ,
A
+tr{Wl
or equivalently
C = -Ke + exTWS(x) + exTWIS'(x)u - xTw(x, u) -lelkllxl2
'T- 'T--
+tr{W W} + tr{Wl Wd + klkl .
A
Hence, if we choose
tr{WTW} = -exTWS(x), (4.114)
'T- T-
tr{WI Wd = -ex WIS'(x)u, (4.115)
C becomes
(4.116)
It can easily be verified that (4.114) and (4.115) can he written in terms of
elements as
Wij = -eXiSj(X), (4.117)
Wil = -exisHx)Ui, (4.118)
= 1,2, ... , n and in matrix form as
for all i, j
. T
W = -exS (x), (4.119)
WI = -ex'S'(x)U, (4.120)
where
x' = diag[xI, X2, .. , xn] ,
U = diag[uI, U2, ... , un].
To continue, observe that
.( :S -Ke + lellxllw(x, u)I-lelk1IxI + k1kl ,

2
:S -Ke + lellxllko + kl(lxl + lul)l-lel kllxl 2 + klki .

To proceed, we need the following lemma:
Lemma 4.3.1. The control law is such that the following inequality holds
for all x E ~n.
lui :S flxl
Proof" We have that
u = - [W1S'(X)r l [WS(x) + ~KX - Ax - sgn(Ok1x] ,
from which we get
but since WI E W' owing to the use of the projection modification and S' (x)
is bounded by definition, we have that
Furthermore,
I[WS(x) + ~KX - I
Ax - sgn(e)k1x] :S IWS(x)1 + ~rlxl
+IIAlllxl + k1lxl (4.121)
Since W, ki are bounded owing to use of the projection modification and
IS(x)1 :S klxl by definition, finally, we obtain that
I[WS(x) + ~KX - Ax - Sgn(e)kIx] I ~ kwmlxl + ~Klxl

+IIAlllxl + kulxl,
= (kwm + ~K + IIAII + ku) lxi,
= Xlxl,
where X= kw m + tK+IIAII+ku. From the aforementioned analysis we finally
conclude that
lui koXlxl ,
~
= klxl,
where k = foX.
Using Lemma 4.3.1, becomes
~ -Ke + lellxllko+ kIlxll-lelhlxl 2 + kIkl . (4.122)
Furthermore, (4.122) becomes
~ -Ke + kolellxl + kIlellxl 2 -lel kllxl 2 + kIkl ,

= -Ke + kolellxl-lelkllxl 2 + kIkl ; (4.123)
therefore, if we choose
kl = lell xl2 , (4.124)

(4.123) becomes
~ -Klel 2 + kolellxl (4.125)
To continue, we also need the following lemma:
u = -[WIS'(X)rl[WS(x) + v], (4.126)
1 .
v = "2KX - Ax - sgn(e)kIx , (4.127)
where the synaptic weight estimates Wand WI are adjusted according to

(4.119) and (4.120) respectively and kl according to (4.124), guarantees the
following
t) ~ 0 Vt ~ 0
limt ...... oot) = 0 exponentially fast
provided that 0) < 0, where 0) the initial value of t).

Proof" Observe that if we use the control law (4.126) and (4.127), (4.110)
becomes

((i) = ((O)e-"t .
Hence, if ((0), which represents the initial value of ((i), is chosen negative,
we obtain
((i) ::; 0, Vi ~ O.
Moreover, ((i) converges to zero exponentially fast with rate of convergence
"'.
In order to apply the control law (4.126), (4.127), however, we have to
assure the existence of [W1S'(X)r 1. Such a condition is guaranteed through
the use of the projection algorithm already presented in (4.23).
Moreover, it is true that
h =(-C
Hence, since h(x) ~ 0, we have that
(( i) ~ ~ (i) , Vi ~ 0 (4.128)
but
(( i) ::; 0 , Vi ~ 0,
which implies that
1((i)1 ::; 1~(i)1 , Vi ~ O. (4.129)
Furthermore,
Ih(x)1 ::; I((i) - ~(i)l,
::; l((i)1 + 1~(i)l,
::; 21~(i)l;
therefore,
(4.130)
or
(4.131)
Remark 4.3.1. Observe that, although the proposed control law (4.126),
(4.127) is discontinuous at first sight since it includes the function sgn(~),
from Lemma 4.3.2 and (4.128) we conclude that ~(t) ::; 0 , "It ~ 0; there-
fore, sgn(~) does not switch from -1 to +1 as time passes and admits the
constant value of -1 Vt 2: o. In this way problems concerning the exis-

tence and uniqueness of solutions, chattering phenomena and/or excitation
of unmodeled dynamics which may be present, are avoided.
Substituting (4.131) into (4.125), becomes

:S _1\:1~12 + 2kol~lv1ZL
= - [I\:M- 2k o] I~IM (4.132)
From (4.132) we have that :S 0 as long as

M> 2ko ,
I\:
or equivalently when
I~I > 4k26 ,

I\:
with I\: > O. The above analysis, together with (4.131), demonstrates that the
trajectories of ~(t) and x(t) are uniformly ultimately bounded with respect
to the arbitrarily small, (since r can be chosen sufficiently large), sets , X
shown below.
4k2}
= { ~(t): 1~(t)l:S 1\:20 ,
and
X = { x(t): 4ko}
Ix(t)1 :S 7" .
Thus we have the following theorem.
Theorem 4.3.1. Consider the system
x = -Ax + w*S(x) + wts'(x)u + w(x, u),
( = -r.(,
u = -[W1S'(X)t1[WS(x) + v],
1
= ZI\:X - Ax - sgn(Ok1x,
A
~=(-h,
1 2
h = Zlxl ,
I\: > 0,
kl = 1~lIxI2,
Wij = -~XiSj(X),
-~Xisi(X)Ui ifwil E W' or Wilsgn(wtl) =

and ~XiS:(x)uisgn(wtl) :::; 0
o if wilsgn( wtl) = and
~xisHx)uisgn(wtl) > 0
-~XiS:(X)Ui if W l E W' or wilsgn(wtl) = wm
and ~XiSi(x)uisgn(wtl) ~ 0
o if wilsgn( wtd = w m and
~xisHx)uisgn(wtd <0
for all i, j = 1,2, ... , n and the modeling error term to satisfy 4.3.1, guarantee
the uniform ultimate boundedness of ~(t), x(t) with respect to the sets
&= {~(t) : 1~(t)1 :::; ~}

X = {x(t) : Ix(t)1 :::; ~ }
Boundedness of W l is achieved through the use of the projection modification
but Theorem 4.3.1 does not tell us anything about the bounded ness of W
and kl . To achieve such a goal W(t) and kl(t) are confined through the use
of a projection algorithm to the sets Wand K respectively, where
W = {W(t) : IIW(t)1I :::; wm} ,
and
K = {kl(t) : 0 :::; h(t) :::; ku} .
!
In particular, the standard update laws (4.119) and (4.124) are modified to
and tr{~xsT(x)W} ~ 0
W= -~xsT(x) (4.133)
+tr{~xsT(x)W}(1+JI:VII)2W if IIWII = Wm
and tr{~xsT(x)W} < 0
and
k {O kl
if = ku (4.134)
1 = 1~llxl2 otherwise
so we have the following lemmas the proof of which can be found in Section
4.2 and are thus omitted here.
Lemma 4.3.3. If the initial weights are chosen such that IIW(O)II :::; w m ,
then we have IIWII :::; Wm for all t ~ 0, provided that IIW*II :::; wm .
Lemma 4-3.4. Based on the adaptive law (4.133), the additional terms
introduced in the expression for.c, can only make .c
more negative, provided
that W(O), W* E W.
Lemmas 4.3.3 and 4.3.4 imply that the projection modification (4.133) guar-
antee boundedness of the weights, without affecting the rest of the stability
properties established in the absence of projection. Similarly, we can prove
the following lemma:
Lemma 4.3.5. The update law (4.124) with the projection modification
(4.134) can only make 1:. more negative and, in addition, guarantee that kl E
K, Vt 2: 0, provided that k1(0), kl E K.
Proof: Observe that the update law (4.134) has the same form as the one
without the projection except for the additional term
Ik = {_I~llxI2 if kl = ku
o otherwise
Thus, is augmented by
Ik = -1~llxI2kl'
= _1~llxI2 (kl - kl)
Now, since the above holds when kl = ku and by assumption we have that
kl < ku, we finally obtain that kl - kl > o. Hence Ik :S 0 so 1:. is augmented
by a negative term. .
Furthermore, from (4.134) we have that whenever kl = ku, then kl becomes
zero, thus establishing ku as the upper bound of k1(t), which concludes the
proof of Lemma 4.3.4.
A
Now that the boundedness of W(t) and kl(t) is guaranteed Vt 2: 0, we

can proceed and prove the following lemma:
Lemma 4.3.6. The closed loop system mentioned in Theorem 4.3.1 to-
!J.ether with the update laws (4.133) and (4.134), further guarantees that
~ E Loo.
Proof: We have that
~ = -K~ + xTWS(x) + XTW1S'(x)u - xTw(x, u) - sgn(~)kllxI2

but W = W - W* is bounded and so is WI =
WI - W{. S(x), S'(x) are
bounded by definition, ~, x are bounded owing to Theorem 4.3.1 and w(x, u)
is also bounded owing to Assumption 4.3.1 and Theorem 4.3.1. Moreover,
u is a function of bounded signals, so u is also bounded and kl is bounded
owing to the projection modification. Thus, we conclude that ~ E Loo.

Theorem 4.3.1 reveals the role that parameters k o, K play in the perfor-
mance of the closed loop system. High values of ko, which is the modeling
error at Ixl = 0, lead to large deviations from the origin. The above draw-
back, which is the cause of imperfect modeling of the original system by the
recurrent high-order neural network, can be compensated by choosing the

parameter/\' appropriately. In fact, it is easily seen, that as /\, --> 00, Ix(t)1
actually becomes zero but as already mentioned, even though it may seems
that large values of the design constant Ii are desirable, the truth is quite
the opposite. Implementation issues constrain its maximum value, since/\' is
implemented as a gain both in the construction of ((t) and in the control
signal u(t). Thus, a compromise between the value of/\, and the maximum
allowable modeling error at Ixl = 0, is unavoidable. The better the modeling of
the original system by the RHONN, the better the performance of the closed
loop system will be with, at the same time, easier implementation.
4.3.1 Complete Model Matching at Ixl = 0

In this subsection we will examine the case where ko = 0 in Assumption 4.3.1;
in other words, when we have complete model matching at Ixl = O. Hence,
Assumption 4.3.1 becomes
Iw(x, u)1 :::; k~(lxl + luI),
Having in mind Lemma 4.3.1, we obtain
Iw(x,u)l:::; k1lxl (4.135)
Substituting (4.135) into the expression for , we obtain
:::; _/\'1~12,
< O. (4.136)
The main result of this subsection goes as follows:
x = -Ax + W*S(x) + wts'(x)u + w(x, u),
(= -/\,(,
u = -[W1S'(x)tl[WS(x) + v],
1 -
v = '2/\'x - Ax - sgn(Oklx,
~=(-h,
1 2
h = '2lxl ,
/\, > 0,
kl = 1~lIxI2,
Wij = -~XiSj(X),
-~XiS~(X)Ui if Wi I E W' or wilsgn(wfl) = C

and ~xisi(x)uisgn(wfl) :S 0
o if wilsgn( wfl) = c and
~XiS~(x)uisgn(wfd >0
Wil = -~XiS~(X)Ui if WI E W' or wilsgn(wfl) = w m
and ~XiSi(x)uisgn(wfl) 2: 0
o if Wil sgn( Wfl) = w m and
~XiS~(x)uisgn(wfl) <0
for all i, j = 1,2, ... , n and the modeling error term to satisfy Assumption
4.3.1, guarantee the following properties:
~, lxi, W, WI, kl, (
E Loo, I~I E L2
limt-+oo {(t) = 0, limt-+oo Ix(t)1 = 0
limt-+oo ~(t) = 0, limt-+oo WI(t) = 0
limt-+oo kl(t) =0
Proof: From (4.136) we have that .c E Loo, hence, ~, W, WI, ki E Loo- Fur-
thermore, W = W+W* E Loo, WI = WI+wt E Loo and ki = kl+kl E Loo.
Since ~ = ( - hand ((t) :S 0, Vt 2: 0 we also have that (, h E Loo which
further implies that Ixl E Loo. Moreover, since .c is a monotone decreasing
function of time and bounded from below, the limt-+oo .c(t) = .coo exists so
by integrating C from 0 to 00 we have
o
1 00
1~12dt
"-
1
= -[.c(0) -
.cool < 00,
which implies that I~I E L 2 Moreover, we also have that

~ = -"-~ + xTWS(x) + XTWIS'(x)u - x T w(x, u) - sgn(~)kIlxI2.
. .
Since ~ is a function of bounded signals we conclude that ~ E Loo; therefore
n
and since ~ E L2 Loo and ~ E Loo applying Barbalat's Lemma [84], we con-
clude that limt-+oo~(t) = o. Now, using the boundedness ofu,. Sex), S'(x), x
and the convergence of ~(t) to zero, we have that W, WI and ki also converge
to zero. Furthermore, we know that ((t) converges to zero exponentially fast.
Hence and since ~(t) also converges to zero, we have that
lim h(x(t))
t~oo
= lim ((t) - lim
t~oo t-+oo
~(t)
=0
Thus,
lim Ix(t)1
t-+oo
=0
Remark 4.3.2. We cannot conclude anything about the convergence of

W, WI and ki to their optimal values W*, wt and ki respectively, from
Theorem 4.3.2. In order to guarantee convergence, Sex), S'(x)u and Ixl 2
need to satisfy a persistence-of-excitation condition.
4.4 Tracking Problems 95
Remark 4.3.3. The previous analysis reveals that the accuracy of the
recurrent neural network model should be restricted only at the origin. If we
have perfect model matching at Ixl = 0 there is no need uniformly to bound
both the W's and kl through the use of the projection algorithms (4.133)
and (4.134) respectively, thus simplifying the implementation issue.
For comparison purposes, the same DC motor model employed for simulation
in the previous sections, is also used herein, to verify the theory developed.
The RHONN model is also the same. The control laws are however,
Ul = - Wln+lS'(ZI)
1 [WllS(Zt} + W12 S(Z2) + -21 I\: Zl]
1 [-0.2z 1 - klSgn(~)Zl] ,
Wln+lS'(ZI)
U2 = - 1
W2n+l S'(Z2)
[W12 S(Z2) + -21 rZ2 - 0.2Z2 - klSgn(~)Z2]
The initial values of all variables can be seen in Table 4.8. Figures 4.3
and 4.4 give the evolution of the angular velocity and armature current
respectively, when the design constant I\: is chosen to be I\: = 10. It can be
seen that as desired, both [2 and Ia converge to zero very fast (in less than
one second). The control action needed to achieve such a goal is presented
in Figures 4.5 and 4.6 again for I\: = 10. The control action can become
smoother by choosing smaller values of the design constant 1\:.
Comparing the simulations with the ones obtained for the known coef-
ficients case, we observe that similar results are provided with significantly
smaller values of the design gain 1\:. In this way we may prevent the appear-
ance of the high-gain instability mechanism [46].
4.4 Tracking Problems
In this section, the recurrent high-order-neural-network-based adaptive con-

trol algorithms developed, are extended to cover tracking problems also.
Tracking is known to force the state of the actual system to follow the output
of a given stable-dynamical system.
When the RHONN model matches the unknown plant, we provide a com-
prehensive and rigorous analysis, of the stability properties of the closed-loop
system. Convergence of the control error to zero plus boundedness of all sig-
nals in the closed loop is guaranteed, without the need of parameter, (weight),
convergence, which is assured only if a sufficiency-of-excitation condition is
satisfied but when we depart from ideal situations, and modeling errors ap-
pear, the proposed adaptive controller can guarantee at least the uniform
Table 4.7. Direct adaptive regulation (n = m). Modeling errors with unknown
coefficients
Actual System: x=f(x)+G(x)u, x,uER n

Model: X = -Ax + W* Sex) + wts'(x)u + w(x, u),
Control Law: u = _[W1 S'(X))-I[WS(X) + v),
v = ~II:X - Ax - sgn(~)hx,
_~xST(x) if W E W or IIWII = Wm
+tr{~xST(x)W}(I+JI:1I )2W if IIWII = Wm
and tr{~xST(x)W} <0
-~XiS:(X)Ui if Wil E W' or wilsgn(wtd =c
o if wilsgn(wtd = c and
~XiS:(x)uisgn(wtl) >0
-~XiS:(X)Ui if Wil E W' or wilsgn(wtd = wm
<0
: {o
~XiS:(x)uisgn(wtd
kl =
if 1.1 = k"
1~llxl2 otherwise
Filter: (=-11:(,11:>0,
~=(-h,
h=tl x I 2 ,
Properties: WI E W', u,~, IX"dl E L oo ,

WE W, 1.1 E 1(, 1((t)1 ~ 1((0)1.
~, x are u. u. h. with respect to
E= { ~(t) : IW)I ~ 4:f} ,
X = {x(t) : Ix(t)1 ~ ~} ,
W* E W, W(O) E W,
kl' hEI(, Iw(x, u)1 ~ ko + kllxl
((0) ~ o.
fa 0.07
[} 0.3
<P 0.90
Wll = W12 = W2l 0.0
W13 = W22 2.5
0. 3r-------~------~------~------~------~------~
0.25
0.2
...
'C
0.
~ 0.15
o
~
0.1
0.05
O~----~~~~~~----~------~------~-------J
0.1 0.2 0.3 0.4 0.5 0.6
Time in seconds
Fig. 4.3. The evolution of motor speed with time when K = 10
ultimate boundedness of the tracking error with respect to an arbitrarily

small set.
4.4.1 Complete Matching Case
In this subsection we investigate the adaptive model reference control prob-

lem when the modeling error term is zero, or in other words, when we have
0.05
-0.05
o 0.05 0.1 0.15 0.2 0.25

Tim 8 in seconds
Fig. 4.4. The evolution of armature current with time when K, = 10
complete model matching. Under this assumption the unknown system can
be written as
x = -Ax + W*S(x) + wts'(x)u. (4.137)
Further, assume that we want the unknown system states to follow the states
of a stable linear model 2
(4.138)
From (4.137) and (4.138) we obtain the error equation
e= -Ax + W*S(x) + wts'(x)u + Amxm - Bmum , (4.139)
where we have defined
6-
e = x - xm .
If we add and subtract the term Ax m , (4.139) becomes

e= -Ax + W*S(x) + wts'(x)u + Amxm - Bmum + AXm - Ax m ,
-Ae + W*S(x) + wts'(x)u + AX m - Bmu m , (4.140)
2 The same analysis holds for the case were we want to follow a stable nonlinear
reference model xm = fm(xm, Um).
O.4,-----..-----....----~---_.__---__r_---__,
0.2~
o --------------------------------------~
.0.2
.0.6
.0.8
.1. 201 . - - - - . l - . - - - . J . - - - - . J . - - - - - ' - - - - . . . . . t . . . . - - - - . J

0.1 0.2 0.3 0.4 0.5 0.6
lima in sEICOnds
Fig. 4.5. The evolution of control action Ul with time when" = 10
where
A = Am -A.
Let us take a function h( e) from ~n to ~+ of class C 2 , whose derivative with
respect to time is
h= ~:T [-Ae+W*S(x)+wts'(x)u+Axm-Bmum].
The above equation which is linear with respect to W* and wt can be written
. ahT ahT _ ahT ahT
h + ae Ae - ae AXm + ae Bmum = ae W*S(x)
+ ah T
ae WI
*S'()
xu. (4.141)
Define
I:;. ah ah, . ah ah T _ ah T
l/ = ax WS(x) + ax WIS (x)u - h - ax Ax + ae AXm - ae Bmum ,
where Wand WI are estimates of W* and wt
respectively. This signal
cannot be measured since h is unknown. To circumvent this problem, we use
O~------~~=---~------~------r-------r-----~
-1
_1.21.....------.l....-------"---------L....------...JL....---__---1
o 0.1 0.2 0.3 0.4 0.5 0.6
Time in seconds
Fig. 4.6. The evolution of control action U2 with time when K = 10
the error filtering method again, according to which

~+x:e=v,
oh T oh T
= oe WS(x) + oe W1S'(x)u - h
oh T oh T _ oh T
oe Ae + oe AX m - oe Bmu m ,
. OhT
= -h + oe [-Ae + WS(x) + W1S'(x)ul
oh T [ _ ]
+ oe AXm - Bm U m , (4.142)
with x: a strictly positive constant. To implement (4.142), we take

t:;.
e=(-h. (4.143)
. oh T _
(+ x:( = x:h + oe [-Ae + WS(x) + W1S'(x)u +Axm - Bmuml ,(4.144)
with the state ( E ?R. Furthermore, if we choose h( e) to be
1
h(e) = "2 1e12 ,
(4.144) becomes
(+ x:( = x:h - eT Ae + eTWS(x) + eTWIS'(x)u
T- T
+e AX m - e Bm U m . (4.145)
If we consider the Lyapunov-like function
1 2 1 -T- 1 -T-
C = 2"~ + 2"tr{W W} + 2"tr{WI Wt}, (4.146)
where
W=W-W*,
WI = W 1 - wt,
and use the same procedure as in Section 4.1, we finally derive the following
results:
Lemma 4-4.1. Consider the system
x= -Ax + W*S(x) + wts'(x)u,
:em = -Amxm + Bmum ,
( = -x:( + x:h - eT Ae + eTWS(x) + eTW1S'(x)u
T- T
+e Axm-e Bmu m ,
~=(-h,
h=~leI2,
e= X - xm ,
The update laws

W = -~esT (x),
WI = -~e'S'(x)U,
with U = diag[ul, U2, ... , un], guarantee the following properties
~,Iel, W, WI, ( E Loo
I~I E L2
limt-+oo ~(t) = 0, limt-+oo Wet) = 0, limt-+oo WI(t) =
provided that U E Loo.

Theorem 4.4.1. The closed-loop system
x = -Ax + W*S(x) + wts'(x)u,
:em = -Amxm + Bmum ,
( = -x:( + x:h - eT Ae + eTWS(x)
+eTWIS'(x)u + eT AX m - eT Bmu m ,
U = -[W1S'(x)]-I[WS(x) + AX m - Bmum] ,
~=(-h,
1 2
h= 2iel ,
II: = 1,
'!Vij = -~eiSj(x),
-~eisHx)ui ifwil E W' or wilsgn(w71) = ~
and ~eis:(x)uisgn(w71) ::; 0
o if wilsgn( w71) = ~ and
~eis~(x)uisgn(w71) > 0
-~eis:(x)Ui ifwl E W' or wilsgn(w71) = w m
and ~eis:(x)uisgn(w71) ;::: 0
o if Wil sgn( w71) = w m and
~eis:(x)uisgn(w71) < 0
for all i, j = 1,2, ... , n guarantees that

lim le(t)1 = O.
t-+oo
4.4.2 Modeling Error Effects
In this subsection we investigate the adaptive model reference control prob-

lem when a modeling error term w(x, u) exists. Under this assumption the
unknown system can be written as
x = -Ax + W*S(x) + wts'(x)u + w(x, u). (4.147)
Further, assume that we want the unknown system states to follow the states
of the model
(4.148)
as stated in the previous subsection. From (4.147) and (4.148) we obtain the
error equation
e = -Ax + w*S(x) + wts'(x)u + Amxm - Bmum + w(x, u), (4.149)
where we have defined
A
e = x - xm .
If we add and subtract the term Ax m , (4.149) becomes

e= -Ax + wts'(x)u + w(x, u)
+ W*S(x)
+Amxm - Bm Um + AXm - AXm
-Ae + W* S(x) + wts'(x)u
+w(x, u) + AXm - Bmum , (4.150)
Table 4.9. Direct adaptive tracking (n = m). Complete matching
Actual System: x=J(x)+G(x)u, x,uE~n
Reference System: xm = -Amxm + Bmum , Xm E ~n, Um E ~p
Model: x = -Ax + W*S(x) + Wi"S'(x)u,

Tracking Error: e = x - Xm,
Control Law: u = -[W1S'(X)]-l[WS(X) + AX m - e T BmumJ,
Update Laws: Wij = -~eisJ(x),
-~eis:(x)Ui if Wil E W' or wilsgn(wtd = e
and ~eis:(x)uisgn(wtd :::; 0
o if Wil sgn( w;D = e and
~eisi(x)uisgn(wtd >0
Wil =
-~eis:(x)Ui if Wl E W' or wilsgn(wtd = w m
and ~eisi(x)uisgn(wtd ~ 0
~eis:(x)uisgn(wtl) <0
Filter: (= -,.( +,.h - eT Ae + eTWS(x)
+eTW1S'(x)u + eT AX m - eT Bmum ,
Am =Am -A,
~ = ( - h, ,. = 1 ,
h = ~lel2 ,
Properties: u,~,lel,(,W E L oo , W 1 E W',
I~I E L 2 , limt_oo Wet) = 0,
limt_oo ~(t) = 0, limt_oo le(t)1 = 0,
Requirements: Wi" E W', Wl(O) E W',
where
A = Am -A.
Let us take a function h( e) from iR n to iR+ of class C 2 , whose derivative with
respect to time is
. oh T [ _ ]
h = oe -Ae + W*S(x) + wts'(x)u + w(x, u) + AXm - Bmum .
The above equation which is linear with respect to W* and wt can be written
. oh T oh T oh T _ oh T
h + oe Ae - oe w(x, u) = oe AXm - oe Bmum
+ oh T W* Sex)
oe
(4.151)
Define
A oh oh, . oh oh T _ oh T
v= OXWS(X) + OX W1S (x)u-h- OXAx+ oe Axm - oe Bmum.
To overcome the problem of unknown h, we use the following filtered version
of v.
e+l\:e=v,
oh T oh T . oh T
= oe WS(x) + oe W1S'(x)u - h - oe Ae
oh T _ oh T
+ oe AX m - oe Bm U m ,
. oh T
= -h + oe [-Ae + WS(x) + W1S'(x)u]
oh T [ _ ]
+ oe AXm - Bm U m , (4.152)
with I\: a strictly positive constant. To implement (4.152) we take

A
e=(-h. (4.153)
oh T
(+ 1\:( = I\:h + oe [-Ae + WS(x) + W1S'(x)u]
oh T [ _ ]
+ oe AXm - Bm U m , (4.154)
with the state ( E ~. Furthermore, we choose h( e) to be
h(e) = ~leI2.
(+ 1\:( = I\:h - eT Ae + eTWS(x) + eT W 1S'(x)u
T - T
+e AX m - e Bm U m . (4.155)
1 2 1 -T- 1 -T-
, = 2e + 2 tr {W W} + 2tr{W1 Wd, (4.156)
where
W=W-W*,
W1- Wi'.
W1 =
.c = ee + tr{WTW} + tr{Wl'Wd. (4.157)

.c = -I'Ce 2 +e [-h - eT Ae + xTWS(x) + eTWiS'(x)u]

+e reT AXm - eT Bmum] + tr{WTW} + tr{wl'Wd, (4.158)

.c = -I'Ce + e[-eTW*S(x) -
2 eTWtS'(x)u - eT w(x, u)]
+e [eTWS(x) + eTWiS'(x)u] + tr{WTW}
'T-
+tr{Wi Wd,
or equivalently
.c = -rce + eeTWS(x) + eeTWiS'(x)u - ee T w(x, u)
+tr{WTW} + tr{Wl'Wd (4.159)
Hence, if we choose
tr{WTW} = -eeTWS(x) , (4.160)
tr{Wl'Wd = -eeTWiS'(x)u, (4.161)
.c becomes
.c = -I'Ce - ee T w(x, u),
::; -I'Clelrellellw(x, u)l (4.162)
To proceed further, we make the following assumption:
Assumption 4.4.1. The modeling error term satisfies:
Iw(x, u)1 ::; ko + k~lxl + k~lul,
where ko, k~ and k~ are known positive constants.
.c ::; -I'CleI 2+ kolellel + k~lellel2 + k~lellellul. (4.163)
To continue, we need the following lemma:
u = -[WiS'(X)ri[WS(x) + AX m - Bmum + v], (4.164)
1
v = -I'Ce - Ae, (4.165)
2
where the synaptic weight estimates Wand Wi, are adjusted according to the
update laws mentioned in 4.4.1, guarantee the following
(( t) ::; 0 Vt ~ 0
limt_oo((t) = 0 exponentially fast
provided that ((0) < o.
Proof: Observe that if we use the control law (4.164) and (4.165), (4.155)
becomes
( = - x;( , Vt 2: 0 ,
Hence, if ((0), which represents the initial value of ((t), is chosen negative,
we obtain
((t) ::; 0, Vt 2: 0,
Moreover, ((t) converges to zero exponentially fast.
Moreover, the above mentioned control law satisfies the property

lui::; ku + kulel (4.166)
It is true though that
h =(-C
Hence, since h 2: 0 we have that
((t) 2: ~(t)
but
((t) ::; 0, Vt 2: 0,
which implies
1((t)1 ::; 1~(t)1 , Vt 2: 0, ( 4.167)
however,
~lel2 ::; 1(1 + I~I,

::; 21el , (4.168)
or
Furthermore,
Ixi = Ie - xml,
::; lei + IXml ,
::; lei + DXm , ( 4.169)
since we assume that Xm is uniformly bounded by the constant 15xm Employ-
ing (4.166), (4.168) and (4.169), (4.163) becomes
.c :S _1\:1~12 + kol~llel + k~I~llel (lei + IxmD + k~I~llellul,

:S _1\:1~12 + 2kol~IM + 4k~I~12 + 2k~bxm I~IM
+k~I~llel (k" + k"lel) ,
:S _1\:1~12 + 2kol~IM + 4k~lel2 + 2k~bxm lelM
+2k~k" lelM + 4k~k"leI2 ,
= - (I\: - 4k~ - 4k~k,,) lel 2+ 2(ko + k~ bXm + k~k,,) lelM,
= - [(I\: - 4k~ - 4k~k,,) M - 2(ko + k~bxm + k~k,,)] I~IM,
:S 0,
provided that
M 2 (k o + k~bxm + k~k,,) (4.170)
> (I\: - 4k~ - 4k~k,,) ,
and
I\: > 4 (k1, + k1"-)
k" ,
or equivalently
lei> 2(ko+k~bxm +k~k,,)2 (4.171)

(I\: - 4k~ - 4k~k,,)2 '
and
I\: > 4 (k~ + k~k,,) .
Inequality 4.171, together with (4.168) demonstrates that the trajectories of
e(t) and e(t) are uniformly ultimately bounded with respect to the arbitrarily
small, (since I\: can be chosen sufficiently large), sets [ and A shown below.
and
4 (k + k' 15 + k" k ) 2
,I\: > 4 (k' + k"k,,) .
}
A = { e(t): le(t)1 :S 0 1 Xm _1 "
(I\: _ 4k~ _ 4k~ k,,) 2 1 1

:i: = -Ax + W*S(x) + wts'(x)u + w(x, u),
Xm = -Amxm + Bmum ,
( = -1\:(,
u = -[W1 S'(X)t 1 [WS(x) + AX m - Bmum + v],
1
v = -K,e - Ae,
2
h= (-C
h=~leI2,
2
with the modeling error term satisfying Assumption 4.4.1. Then the update
laws
Wij = -eeiSj(x),
-eeis~(x)Uiif Wil E WI or wilsgn(w71) =
and eeis:(x)Uisgn(w71) ~ 0
o if wilsgn( W71) = and
eeis:(x)Uisgn(w71) > 0
-ee;sHx)ui if W 1 E WI or wil sgn(w71) = wm
and eeis:(x)Uisgn(w71) 2: 0
o if wilsgn( w71) = wm and
eeis:(x)Uisgn(w71) < 0
for all i,j = 1,2, ... , n guarantee the uniform ultimate boundedness with
respect to the sets
4 ko+k~ bXm +k~'k"
> 4 (kl + kl k,,)
I"- }
[= { e(t): le(t)1 ~ "-4k;-4k;'k" ' K,
A = {'(t) : 1'(t)1 < 4(ko+k~bxm +k?,,), K, > 4 (k 1 + k"k

l )}
., ., - (,,-4k' -4k" k )2 ,
1 1 u
1"
!
but in order to guarantee the boundedness of W(t) and the validity of (4.166),
we modify the standard update law (4.147) to
-eeST(x) ifW E W or IIWII = Wm
and tr{eeST(x)W} 2: 0
W= -eeST(x) (4.172)
+tr{eeST(x)W}( l+JI~" )2W if IIWII = Wm
and tr{eeST(x)W} <0
As stated in previous sections, the projection modification (4.172) guarantees
the boundedness of the weights W without affecting the rest of the stability
properties established in the absence of projection.
4.5 Extension to General Affine Systems
So far we have dealt with affine nonlinear systems the number of control
in pu ts (m) to which equals the number of measurable states (n). In this and
in all subsequent sections of this chapter we shall consider the more general
case of m f:. n.
4.5 Extension to General Affine Systems 109
Table 4.10. Direct adaptive tracking (n = m). Modeling error effects
Actual System: x = f(x) + G(x)u, x,u E ~n

Reference System: xm = -Amxm + Bmum, Xm E ~n, Um E ~p
Model: x = -Ax + W*S(x) + wts'(x)u,

Tracking Error: e = x - xm ,
Control Law: u = _[WIS'(X)]-I[WS(X) + AX m - eT Bmum + vJ,

v = ~I\;e - Ae,
_~xST(x) if WE Wor IIWI! = Wm
and tT{~xST(x)W} ~ 0
+tT{~xST(x)W}(I+JI:1I )2W if IIWI! = Wm
and tT{~xST(x)W} <0
-~eis:(x)Ui if Wil E W' or wilsgn(wtd = e
and ~eis:(x)uisgn(wtd ::; 0
o if Wil sgn( Wtl) = e and
~eis:(x)uisgn(wtd >0
-~eis:(x)Ui if WI E W' or wilsgn(wtd = w m
and ~eis:(x)uisgn(wtd ~ 0
o ifwilsgn(wtd = w m and
~eis:(x)uisgn(wtd <0
Filter: (= -1\;(,
Am =Am -A,
~=(-h,
h = ~lel2 ,
Properties: WI E W', u,e,E L oo ,
WE W, 1((t)1 ::; 1((0)1.
~, e are u.u.b. with respect to
f = {e(t): le(t)l::; 4 ko+k;~"~~/kp'"
K-4k -4k k"
, I\; > 4 (k~ + k~/k,,)} ,
1 1
A = {I:(t):
,
II:(t)1 < 4(ko+k;6,,~+k;'k,,)2
, - (K-4k~ -4k~/ku)2 ,
I\; > 4 (k'
1
+ kllk
1
u )}
,
Requirements: wt E W', WI (0) E W',

W* E W, W(O) E W,
Iw(x, u)1 ::; ko + k~lxl + k~/lul. ((0) ::; O.
4.5.1 Adaptive Regulation

exactly by the recurrent neural network plus a modeling error term wo(x, u).
x = -Ax + W*S(x) + S'(x)wtu + wo(x, u), (4.173)
where x E ~n, the inputs u E ~m, W*, W[, are n x Land n x Lo matrices of
synaptic weights respectively and A is a n x n matrix with positive eigenvalues
which for simplicity can be taken diagonally. S( x) is a L-dimensional vector
with elements, Si(X), i = 1,2, ... , L of the form
Si(X) = II [s(Xj)]dj(i) , (4.174)
jEi,
where Ii, i = 1, 2, ... , L are a collection of L not-ordered subsets of {I, 2, ... , n}

and dj are non-negative integers. Similarly, S'(x) is a Lo x m matrix with
elements sfk (x) of the form
sfk(x) = II [s(Xj)]dj(l,k) , (4.175)
jEilk
=
for alII, 1,2, ... , Lo and k =
1,2, ... , m, where I,k are collections of mLo
not-ordered subsets of {1,2, ... ,n} and dj(l,k) are non-negative integers.
In all above relationships, s( x j) is a smooth, monotone increasing function,
which is usually represented by sigmoidals of the form
J1.
. + A,
s(Xj) = 1 + e- Iox,
for all j = 1,2, ... , n, with the parameters j.L, 10 to represent the bound and
slope of sigmoid's curvature and A a bias constant.
Choose a function h(x, u, W) of class C 2 from ~n x ~m X ~nxL to ~+
whose derivative with respect to time is
. 8h T ,
h = 8x [-Ax + W*S(x) + wts (x)u + wo(x, u)]
8h T 8h T .
+8u it+ 8W W. (4.176)
The above equation which is linear with respect to W* and W[ can also be
written
. 8h T 8h T 8h T . 8h T *,
h + 8x Ax - 8x wo(x, u) = 8x W*S(x) + 8x WI S (x)u
8h T 8h T .
+ 8u it + 8W W. (4.177)
Define
l!. 8h T 8h T . 8h T
v= - WS(x) + - WIS'(x)u - h - - Ax
8x 8x 8x
8h T 8h T . T
+ 8u it + 8W W - Intr{A W},
where In is an indicator function to be defined later, which is also the case

for the A term. Wand WI are the estimates of W* and wt respectively,
obtained by update laws which are to be designed in the sequel. This signal
cannot be measured since h is unknown. To circumvent this problem, we use
the following filtered version of v
e+ Ke = v,
. ah T ah T ah T .
= -h + ax [-Ax + WS(x) + WIS'(x)u] + au u+ oW W
-Intr{ATW}. (4.178)
To implement (4.178), we further define
6
e=(-h. (4.179)
. MT MT
(+ K( = Kh + ax [-Ax + WS(x) + WIS'(x)u] + au u
ah T . T
+ oW W - Intr{A W}, (4.180)
with the state ( E ~ and K a strictly positive constant. Furthermore, if we

choose h(x, u, W) to be
1
h(x, u, W) = 2"(lxI2 + lul 2+ tr{WTW}),
( = -K( + Kh - x T Ax + xTWS(x) + XTW1S'(x)u + u T u
+tr{WTW} - Intr{ATW},
1
= -K( + 2"K(lxI2 + lul 2+ tr{WTW}) - x T Ax + xTWS(x)
+XTWIS'(x)u + uT U + tr{(WT - InAT)W}. (4.181)
If we choose
uT = _~KuT - XTWIS'(x),
1
2"KI = aI,
-tKW - xsT(x) if WE W or IIWII = Wm
W - { and tr{(tKW + xsT(x))WT } ~ 0
- -tKW - xsT(x) + A if IIWII = Wm
and tr{(tKW + xsT(x))WT } < 0
( 4.182)
where I is the identity matrix, aI =
A with a > 0, A = tr{(tKW +
xsT (x)) WT}( I+JI~II)2 Wand W is the convex set
W = {W(t) : IIW(t)II ~ Wm } .
Furthermore, if we define In = =
1 ifllWil Wm and tr{(~II:W +xsr (x))WT} <
o and In =
0 if WE Wor IIWII =
Wm and tr{(!II:W + xsr(x))WT} ~ 0,
then ( becomes
(= -11:(;
therefore, we have the following lemma:
Lemma .4-5.1. The contra I law
v7 = _~lI:uT - xTW1S'(x) , (4.183)

2
together with the update law
-!II:W - xsr(x) ifW E W or IIWII = Wm
W_ { and tr{(~II:W + xsr(x))WT} ~ 0
- -!II:W-xsr(x)+AifllWII=wm
and tr{(!II:W + xsr(x))WT} < 0
and
1
211:1 = aI = A,
guarantees that
((t) ~ 0 'Vt ~ 0
limt-+oo((t) = 0 exponentially fast
provided that
((0) < o.
Moreover,
IIWII ~ Wm , 'Vt ~ 0
provided that W(O) E Wand W* E W.
Proof: The proof of the first part comes from ( = -11:( 'Vi ~ O. The solution
of which is
((t) = ((O)e- lCt , 'Vt ~ O.
Thus and since II: > 0, if we choose ((0) < 0 we have ((t) ~ 0 'Vt ~ 0
and moreover the convergence of ((t) to zero exponentially fast with rate of
convergence 11:.
To prove the second part, we only need to show that -it(IIW(t)11)2 ~ 0
whenever IIW(t)II = W m . The proof of such an argument, follows the lines of
the proof of 4.2.2 and thus it is omitted.
(4.184)
where
WI = W1 - W{.
Taking the derivative of , with respect to time, (4.184) becomes
. . . T -
, = ~~ + tr{WI Wd,
or after using (4.178)
= -lie + ~ [-h - xT Ax + xTWS(x) + XTWIS'(x)u + uT u]

+~ [tr{(W T - InAT)W}] + tr{W[Wd, ( 4.185)
which, after using (4.177), becomes

= -lie + ~ [-xTW*S(x) - xTW{S'(x)u + xTWS(x)]
+~ [XTWIS'(x)u - xTwo(x, u) - tr{InATW}] + tr{W[Wd,

or equivalently
= -lie + ~xTWS(x) + <XTWIS'(x)u - ~xT wo(x, u)
. T - T
+tr{WI Wd - ~tr{InA W}, ( 4.186)
where
Furthermore, if we choose the update laws

WI = -~x(S'(x)uf ,
we obtain
= -lie + ~xTWS(x) - ~xT wo(x, u) - ~tr{InATW}.
To continue, we need the following lemmas:
Lemma 4.5.2. The update law (4.182) guarantee the inequality:
tr{InATW} ~ o.
Proof: It is true that
InATW = Intr{(~IiW +xST(x))WT} C+~~") 2 WTW.
Hence,
since when In = 1, we have that tr{(~KW + xsT(x))WT } < 0 while

(1+~I:lIr IIWI1 2 ~ 0, "It ~ O.
Lemma 4.5.3. The following properties hold for our system:
~(t) ::; 0, "It ~ 0

Ix(t) 12 ::; 41~(t) I, "It ~ 0
lu(tW::; 41~(t)l, "It ~ 0
Proof: It is true that h = ( -~. Since h ~ 0, "It ~ 0, we have that ((t) ~
~(t), "It ~ 0 but from Lemma 4.5.1 we have that ((t) ::; 0, "It ~ O. Hence,
~(t) ::; 0, "It ~ O. Moreover, it is obvious that since the above inequalities
hold, the following must be true
1((t)1 ::; 1~(t)L "It ~ o.
Furthermore,
1
h = 2'(lxI2 + lul 2 + tr{WTW}) ,
~ ~lxI2, (4.187)
and
h::; 1((t)1 + 1~(t)l,
~ 21~(t)l (4.188)
From Inequalities 4.187 and 4.188 we finally, have
Ix(tW ::; 41~(t)l, "It ~ O.
Moreover, since h ~ ~lul2 and following the same analysis, we conclude that
lu(tW ::; 41~(t)l, "It ~ o.
Employing Lemmas 4.5.2 and 4.5.3, we conclude that In~tr{ATW} >

0, "It ~ O. Hence, becomes
= -Ke + ~xTWS(x) - ~xT wo(x, u).
The following assumption is also needed:
Assumption 4.5.1. The modeling error term wo( x, u) satisfies
Iwo(x, u)1 ::; kl + k2lxl,
where kl and k2 are known positive constants.
Observe that Assumption 4.5.1 underlines that the modeling error term
wo( x, u) satisfies a Lipschitz condition, a property necessary to guarantee
the existence and uniqueness of solutions of (4.173). The above is needed
since Theorem 2.1.1 implies that such a property holds for the actual sys-
tem (4.173). Moreover, by choosing larger values of kl, k2 we can enlarge
the admissible modeling errors but it is not expected for k 1 , k2 to admit too
large values, since in Theorem 2.1.1, the approximation error c: can be cho-
sen arbitrarily small. Moreover, notice that in the nonlinear adaptive control
literature, the problem of designing adaptive controllers with certain robust-
ness properties with respect to modeling errors and/or external disturbances,
is still in its infancy and only results that deal with a special class of sys-
tems, namely single-in put-single-output systems, transformable via a global
state-space diffeomorfism, into a nonlinear system the nonlinearities of which
depend only on the output, have appeared and these only recently [65]. In the
above paper, the authors require the uncertainties to be bounded by known
functions of the output, which are not necessarily Lipschitz. In the nonlinear
control literature, the problem of developing controllers for uncertain sys-
tems, has a history of over a decade starting from [13] in which results were
obtained under the very restrictive condition that only input-matched uncer-
tainties are present. See also the works of Barmish and Leitmann [5], that
allow small mismatched uncertainties, and Chen [11] which is along the lines
of [13] but in an adaptive environment. An attempt to relax the very hard
matching assumptions, was provided in [25] where n-dimensional single-input
systems in the form of a perturbed chain of integrators were considered. The
uncertainties 'lj;i (x) were assumed to be bounded by known functions Pi as
follows
In other words, they allow each bounding function to depend only on the first
i states, also known as the strict feedback condition, described in [53]. From
the above discussion it is apparent that generalizing the robustness results to
tolerate more general classes of modeling errors/uncertainties is still an open
problem.
Remark 4.5.1. It is important to note that if we assume that the mod-
eling error term wo( x, u) satisfies
wo(x, u) = wox(x) + wou(x)u,
and
Iwox(x)1 ::; kl + k;lxl,
Iwou(x)1 ::; k~ ,
for some known positive constants k 1 , k~, k~, then
Iwo(x, u)1 ::; Iwox(x)1 + Iwou(x)l,
::; kl + k~ Ixl + k~lul ,
and the results that follow also hold since Lemma 4.5.3 is valid, but now with
k2 = k~ + k~.
Moreover, we have
IIWI! ::; Wm , (4.189)
IS(x)1 ::; ko, ( 4.190)
where ko, wm are known positive constants. Inequality 4.190 holds by the
definition of S(x). On the other hand (4.189) is valid owing to the use of the
projection algorithm (4.182). Employing (4.189),(4.190).c becomes
.c ::; _1\:1~12 + 1~llxlwmko - ~xT wo(x, u).
From Assumption 4.5.1 and Lemma 4.5.3, .c becomes
.c ::; _1\:1~12 + 2kowml~IM + 1~llxllwo(x, u)l,
::; _1\:1~12 + 2kowml~IM + 2kll~IM + 4k21~12,
::; -[(I\: - 4k 2 )M - 2kl - 2kowmll~IM,
::; 0 , (4.191)
provided that
M> 2(kl + kowm ) ,
I\: - 4k2
and
I\: > 4k2 > 0,
or
and
I\: > 4k2 > O.
The above together with Lemma 4.5.3 demonstrates that the trajectories of
~(t) and x(t) are uniformly ultimately bounded with respect to the arbitrarily
small (since I\: can be chosen sufficiently large) sets E and X shown bellow
E = {~(t) : 1~(t)1 ::; 4(~~ ~ :~~~)2} ,

and
X = {x(t) : Jx(t)1 ::; 4(kl + k~wm)} ,
1\:-42
since if we start for example from inside E then ~(t) is bounded by 4(~~~~~~),;)2
owing to the definition of E but if we start from outside the set, then .c ::;
0;
hence, the solution ~(t) is forced towards the boundary of E. Thus, we have
the following theorem:
Theorem 4.5.1. Consider the system (4.173) with the modeling error
term satisfying Assumption 4.5.1. Then the dynamic compensator
iJ? = _~II:UT - XTWIS'(x) ,

2
together with
(= -11:(,
h =(-C
1
h = 2"(lxI2 + luI 2 + tr{WTW}),
Jl = tr{(~II:W + xST(x))WT}(1 + IIWII)2W,
2 Wm
and the update laws
. T
WI = -~x(S'(x)u) ,
-~II:W - xsT(x) ifW E W or IIWII = Wm
W_ { and tr{(~II:W + xsT(x))WT } 2: 0
- -~II:W-xsT(x)+JlifIIWII=wm
and tr{(~II:W + xsT(x))W T } < 0
guarantee the uniform ultimate boundedness of the trajectories of ~(t) and
x(t) with respect to the arbitrarily small sets E and X
E = {~(t) : 1~(t)1 :::; 4(1~~:~~1;,)2}
X = {x(t) : Ix(t)1 :::; 4(k~~:~~m)}
provided that II: > 4k2 > O.
From the above analysis, however, nothing could be said of the bounded-
ness of the weight estimates WI. Boundedness of the weight estimates is also
needed to prevent a well-known problem in adaptive control literature re-
ferred to as parameter drift [45]. Parameter drift has also been encountered
in empirical studies of neural network learning, where it is usually referred
to as weight saturation.
For the reasons given above, in order to avoid the appearance of the
parameter drift phenomenon, the weight estimates WI are confined to the
set WI = {WI : IIWIII :::; tV}, through the use of a projection algorithm
[73],[32],[45],[46] with tV a known positive constant. In particular, the stan-
dard update law is modified to
-~x(S'(x)u)T if WI E WI or IIWIII = tV
w = { ~nd tr{~x(S'(x)ufWl} 2: 0
(4.192)
1 -~x(S'(x)uf + P If IIWIII = tV
and tr{~x(S'(x)ufWl} < 0
where P = tr{~x(S'(x)u)TW1}( I+I!:"I! )2WI . Hence, by choosing the initial

weight estimates to satisfy IIWI(O)II :::; tV, we ensure that IIWIII :::; tV for all
t ~ 0, provided that wt E WI. The proof of the above statement is along

the lines of the proof of Lemma 4.2.2 so it is omitted here. The proof of the
following lemma resembles that of Lemma 4.2.3 and thus, it is also omitted.
Lemma 4.5.4. The additional terms introduced in (4.191) by the use of
the projection algorithm (4.192), can only make more negative.
Lemma 4.5.4 dictates that the stability properties obtained by the Theorem
4.5.1 are not affected.
In principle, the projection modification does not alter WI if WI(t) is in
the interior of WI or if WI(t) is at the boundary of WI and has the tendency
to move inward. Otherwise, it subtracts a vector normal to the boundary
so that we get a smooth transformation from the original vector field, to an
inward or tangent-to-the-boundary, vector field.
Furthermore,
uT = --r.:u
1 T - xTW1 S'( x,
)
2
is a bounded-input-bounded-output ordinary differential equation. Hence,
since the term XTWIS'(x) is bounded, we conclude that u E Loo. Moreover,
~ = -r.:e + xTWS(x) + XTWIS'(x)u - xT wo(x, u) - Intr{A?W}
but W, WI are bounded owing to (4.182) and (4.192) respectively, S(x), S'(x)
are bounded by definition, e,x are bounded owing to Theorem 4.5.1 and
wo(x, u) is also bounded owing to Assumption 4.5.1 and Theorem 4.5.1. Thus,
~ is a function of bounded signals. Hence, ~ E Loo. Thus, we have the lemma:
Lemma 4.5.5. The closed loop system mentioned in Theorem 4.5.1 with
the modified update law (4.192) further guarantees
1! E Loo
eE Loo
Remark 4.5.2. To ensure robustness of the adaptive regulation scheme
with respect to modeling errors, we have considered a projection algorithm
which prevents the weight estimates from drifting to infinity. The stability of
the proposed control scheme in the presence of modeling errors can also be
achieved by other modifications to the standard update laws, such as fixed,
or switching O"-modification [43],[44], E-modification [72] and the dead-zone
[101]. Use of the particular modification is due to its smoothness property
and to the fact that when external disturbances are affecting the system
dynamics, an a priori known bound of IIWIII is needed. The presence of
external disturbances and the problems that they create, is the theme of the
next subsection.
Remark 4.5.3. The dynamic compensator described in (4.183) can be
viewed as a special-purpose RHONN with W = 0, input the state of the true
plant and output the control action. Hence, RHONNs can be used not only
to model the unknown plant, but also to realize dynamic compensators.
Remark 4.5.4. Theorem 4.5.1 shows how the design constant I'C should be
selected, in order to guarantee the uniform utimate boundedness of the state
x, even in the presence of modeling error terms which are not uniformly
bounded a priori, as Assumption 4.5.1 implies. Generally, the value of I'C
becomes large as we allow for large model imperfections but I'C is implemented
as a gain in the construction of ( and for practical reasons it cannot take
arbitrarily large values, leading to a compromise between the value of I'C and
the maximum allowable modeling error. Observe though, that owing to the
approximation Theorem 2.1.1 the values of kl and k2 can be confined to be
small enough.
In what follows we shall examine two special cases of the one presented
above, which are of particular interest, since we can use them further to prove
the convergence of the state to zero.
The No Modeling Error at Zero Case. The analysis herein is dominated
by the following assumption.
Assumption 4.5.2. The modeling error term wo(x, u) satisfies
Iwo(x, u)1 ::; k2lxl
Observe that Assumption 4.5.2, which is actually Assumption 4.5.1 with kl =
0, dictates that both the unknown system and the RHONN model should
have the same origin. Employing Assumption 4.5.2 and following the same
analysis as we did in the previous section, C becomes:
C::; -I'CleI 2+ 4kowmlel2 + 4k 21e1 2,
::; - [(I'C - 4k2 - 4koWmh/iZl] lel 2,
::; 0 , (4.193)
provided that the design constant I'C is chosen such that
I'C > 4(k2 + kowm) ,
and the following inequality hold:
IS(x)1 ::; kolxl (4.194)
Hence, we can prove the following theorem:
Theorem 4.5.2. Consider the system:
x = -Ax + W*S(x) + wts'(x)u + wo(x, u),
iJ7 = _~I'CUT - xTW1S'(x) ,
2
(= -I'C(,
h =(-C
1
h =2(lx12 + lul 2+ tr{WTW}) ,
A= tr{(~I'CW + xST(x))WT}(l + IIWII)2W,
2 Wm
Table 4.11. Direct adaptive regulation under modeling errors (n t= m)

Actual System: i; = f(x) + G(x)u, xE ~n, uE ~m
Model: i; = -Ax + W*S(x) + wtsl(x)u + wo(x, u),

Control Law: iI,T = _!KU T - XTWISI(X),
-~KW - xST(x) IIWII = Wm
if WE W or
W= { and tr{(~KW + xST(x))WT} ~ 0
Update Laws:
-!KW - xST(x) + A if IIWII = Wm
and tr{(~KW + xST(x))WT} < 0
-~x(SI(x)u)T if WI E Win or II WI II = ill
. and tr{~x(S'(x)ufWn ~ 0
WI ={
_~X(SI(X)u)T + P if II WI II = ill
and tr{~x(S'(x)u)TWn <0
A = tr{(~KW + xS T (x))W T }C+1: 11 )2W,
P = tr{~x(SI(x)ufwt}(l+1':1II?WI'
Filter: , = -K(,
~=(-h,
h = hlxl2 + lul 2 + tr{WTW}) ,

Results: WI E WI, u,e,E Loo,
WE W, 1((t)l::; 1((0)1,
~, x are u.u.h. with respect to
.5" = {~(t) : 1~(t)1 ::; 4(~~+~~~,;,)2} ,
X = {x(t) : Ix(t)1 ::; 4(ki<::'~~:m)} ,
Requirements: wt E WI, WI (0) E WI,
W* E W, W( 0) E W,
Iw(x,u)l::; kl +k2lxl, ((0)::; O.
and the update laws

W1 = -~x(S'(x)uf,
-tKW - xsT(x) ifW E W or IIWII = Wm
W- { and tr{(t KW + xsT(x))W T } 2: 0
- -tKW - xsT(x) + 11 if I!WII = Wm
and tr{(tKW + xsT(x))WT } <0
guarantee the following properties:
~,~, Ixl, luI, W, W1,e E Loo, lei E L2

limt--+oo e(t) = 0, limt--+oo Ix(t)1 = 0
limt--+oo IIW(t)11 = 0, limt--+oo lu(t)1 = 0
limt--+oo Wl(t) = 0
provided that", > 4(k2 + kow m ) and (4.194) holds.

Proof: From (4.193) we have that E L o ,,, hence, ~, WI E Loo. Furthermore,
WI = WI + Wi*" E Loo. Since ~ = ( - hand ((t) :::; 0 Vt 2: 0, we also
have (, h E Loo which in turn implies lxi, lui, IIWII E Loo- Moreover, since
is a monotone decreasing function of time and bounded from below, the
.c
limt_oo (t) = 00 exists so by integrating from 0 to 00 we have
1
00
o
1~12dt :::; (k
'" - 4
1
k
2
) [(0) - 00] < 00,
+ OWm
which implies that I~I E L 2 Furthermore,

~ = -"'~ + xTWS(x) + XTWIS'(x)u - xT wo(x, u) - Intr{ilTW}.
Hence, ~ E Loo and since u,lxl E Loo the sigmoidals are bounded by
definition, W, WI E Loo and Assumption 4.5.2 hold; therefore and since
n
~ E L2 Loo and ~ E L oo , applying Barbalat's Lemma [84] we conclude
that limt_oo~(t) = O. Now, using the boundedness of u, S(x), S'(x), x and
the convergence of ~(t) to zero, we obtain that WI also converges to zero.
Furthermore, since ~(t), ((t) converges to zero, we also have
lim h(x(t), u(t), W(t))
t-oo
= t-oo
lim ((t) - lim ~(t),
t--HX>
= o.
Thus,
lim Ix(t)1 = 0,
t-oo
lim lu(t)1 = 0,
t-oo
lim IIW(t)1I = O.
t-+oo
Remark 4.5.5. The aforementioned analysis reveals that the accuracy of

the dynamic neural network model should be restricted only at the origin. In
other words when we have complete model matching at zero, our modified
adaptive regulator can guarantee that the stability properties of the closed-
loop system do not alter. Furthermore, as Theorem 4.5.2 clearly states, that
there is no need uniformly to bound the weight estimates WI through the use
of the projection algorithm (4.192), thus simplifying the implementation is-
sue. Moreover, since (4.194) should hold, the analysis and the results obtained
in this section are confined to a smaller class of systems.
No Modeling Error Case. The analysis is now based on the following
assumption:
Assumption 4.5.3. The modeling error term wo(x, u) satisfies
Iwo(x, u)1 = O.
Employing Assumption 4.5.3 and following the same analysis as we did in

the previous section, becomes:
.c : :; -1dcI 2 + 4kowm Icl 2 ,
:::; - [(K-4kowm )vW] Ic1 2 ,
:::; 0, (4.195)
provided that the design constant K is chosen such that
K > 4kowm,
and (4.194) holds. Hence, we can prove the following theorem:
Theorem 4.5.3. Consider the system:
x = -Ax + W*S(x) + wts'(x)u,
uT = 1
--KU T - X TWI S'( x,
)
2
(= -K(,
h = (-C
1
h= 2(lx12 + lul 2 + tr{WTW}) ,
!l = tr{(~KW + xST(x))WT}(1 + IIWII)2W,
2 Wm
and the update laws
WI = -cx(S'(x)uf ,
-tKW - xsT(x) ifW E W or IIWII = Wm
W _ { and tr{(t KW + xsT(x))WT } ~ 0
- -tKW-xsT(x)+!lifIlWII=wm
and tr{(tKW + xsT(x))WT } < 0
c,~,lxl,luI,W,WI,(ELoo, IclEL2
=
limt-->oo c(t) 0, limt-->oo Ix(t)1 0 =
limt-->oo IIW(t)1I = 0, limt-->oo lu(t)1 = 0
limt-->oo WI(t) = 0
provided that K > 4k owm and (4.194) holds.
Proof: From (4.195) we have that E Loo, hence, WI E Loo. Furthermore, e,
e
WI = WI + W{ E LOO' Since = ( - hand ((t) :::; 0 Vt ~ 0, we also
have (, h E Loo which in turn implies lxi, lui, IIWII E Loo. Moreover, since
is a monotone decreasing function of time and bounded from below, the
limt-->oo (t) = 00 exists so by integrating from 0 to 00 we have .c
1 o
00
lel 2dt :::; K - 4
(k
1
2
- [(0) -
+ kOWm 00] < 00,
which implies that I~I E L 2 Furthermore,

i = -"'~ + xTWS(x) + XTWiS'(x)u - Intr{ATW}.
Hence, i E Loo and since u, Ixl E Loo and the sigmoidals are bounded by
definition, W, Wi E Loo; therefore and since ~ E L2 Loo and ~ E Loo, n
applying Barbalat's lemma we conclude that limt->oo~(t) = O. Now, using
the bounded ness of u, S(x), S'(x), x and the convergence of ~(t) to zero, we
obtain that Wi also converge to zero. Furthermore, since ~(t), ((t) converges
to zero, we also have
lim h(x(t), u(t), W(t))
t-+oo
= t-oo
lim ((t) - lim
t--+oo
~(t),
= O.
Thus,
lim Ix(t)1
t->oo
= 0,
lim lu(t)1 = 0,
t->oo
lim
t->oo
IIW(t)11 = O.

4.5.2 Disturbance Effects
Let us assume that the true plant is of known order n and is described by
x= -Ax + W*S(x) + wts'(x)(u + udm(t) + da(t)) + wo(x, u) .(4.196)
In other words, the state equations are subject to external disturbances,
da(t), dm(t), which appear in both an additive and a multiplicative way and
are not contained in the original recurrent neural network model, of the actual
plant.
Let us further assume that the disturbances, da(t), dm(t), are unknown
and cannot be measured and the only information about them is that they
are piecewise continuous functions of time and satisfy
Ida(t)1 ::; 8a , (4.197)
Idm(t)1 ::; 8m , (4.198)
for all t ;:::: 0 and some positive constants 8a , 8m . Obviously, from the above
presentation d a and dm are the additive and the multiplicative plant pertur-
bation respectively.
Similar to the previous subsection, we choose a function h(x, u, W) of
class C 2 from ~n x ~m X ~nxL to ~+ whose derivative with respect to time
IS
. 8h T
h = 8x [-Ax + W*S(x) + wts'(x)(u + udm(t) + da(t)) + wo(x, u)]
8h T 8h T .
+ 8u it + 8W W. (4.199)

. fJh T fJh T . oh T . oh T
h +-
ax
Ax - -
au
u- -
oW
W = fJx wts'(x)(u + udm(t) + da(t))
fJh T fJh T
ax w*S(x) + fJx WO(x, u).
(4.200)
Define
f:> fJh T oh T , . oh T oh T .
v= fJx WS(x) + fJx W1S(x)u-h- ax Ax+ au u
fJh T . T
+ fJW W - Intr{A W},
and use the error filtering method

e+K~=V,
. ohT
= -h + fJx [-Ax + WS(x) + W1S'(x)u]
fJh T oh T . T
+ au u + oW W - In tr{ A W}. (4.201)
Again set
f:>
~=(-h. (4.202)
Hence,
. oh T oh T
(+ K( = Kh + ax [-Ax + WS(x) + W1S'(x)u] + au u
oh T . T
+ow W-Intr{A W}, (4.203)
with the state ( E ~ and K a strictly positive constant. Furthermore, if we

choose
1
h(x, u, W) = 2"(lxI2 + lul 2+ tr{WTW}) ,
then finally (4.203) becomes
( = -K( + Kh - xT Ax + xTWS(x)
+XTW1S'(x)u + u u + tr{(WT - InAT)W} ,
T
= -K( + ~K(lxI2 + lul 2 + tr{WTW})

2
_xT Ax + xTWS(x) + XTW1S'(x)u
+uT u + tr{(WT - InAT)W} , (4.204)
which finally, after using Lemma 4.5.1, becomes
( = -K(.

12 1
C = 2~ + 2tr {Win Win} .
-T-
(4.205)
Taking the derivative of C with respect to time, (4.205) becomes
.c = -I\,e + ~ [-h - xT Ax + xTWS(x) + XTW1S'(x)u]

+~ [u T u + tr{(W T - InAT)W}] + tr{W!Win }, (4.206)
which after using (4.200), becomes
.c = -I\,e + ~ [-xTW*S(x) - xTwts'(x)(u + udm(t) + da(t))]
+~ [xTWS(x) + XTW1S'(x)u - x Two(x, u)]
T . T-
-In~tr{A W} + tr{Wl Win},
or equivalently
.c = -I\,e + ~xTWS(x) + ~XTWinS'(X)U - ~xTwtS'(x)udm(t)
-~xTwtS'(x)da(t) - ~xTwO(x, u)
-In~tr{A
T
W} + tr{Wl
'T-
Win}. (4.207)
Furthermore, if we choose the update law
.
WI = -~X(S, (X)u) T , (4.208)
again and Lemma 4.5.2 we obtain
.c = -I\,e + ~xTWS(x) - ~xTwo(x, u) - ~xTwtS'(x)udm(t)
-~xTwtS'(x)da(t) (4.209)
but
IIWII ::; wm ,
Ilwtll ::; w,
IIS'(x)11 ::; 81
Hence, after using Assumption 4.5.1 and Lemma 4.5.3, .c becomes
C ~ _1\,1~12 + 1~llxIIIWIIIS(x)1 + 1~llxllwo(x, u)1
+lxIIS'(x)lllwtlllulldm(t)1 + IxIIS'(x)IIIWtlllda(t)l,
~ _1\,1~12 + 2kowml~IM + 4S1W6ml~12 + 281W6al~IM
+2(kl + k2Ixl)l~IM,
~ -[(I\, - 4s1w6m - 4k2)M - 2(SlW6a + kl + kowm)ll~IM,
~O, (4.210)
provided that
M> 2(81W6a + kl + kowm) ,
I\, - 4(SlW6m + k2)
and
or
II 4(SIWOa + kl + kowm)Z
~ > [1\:-4(SI WOm+ kZ)]2 ,
and
I\: > 4( SI WO m + kz) > 0 ,
The above, together with Lemma 4.5.3, demonstrate that the trajectories of
and x(t) are uniformly ultimately bounded with respect to the arbitrarily
~(t)
small (since I\: can be chosen sufficiently large) sets 3 and X shown bellow
3 = {~(t) : 1~(t)1 :s; 4(SIWO a + ~1 + koWm)Z} ,

[I\: - 4(SIWOm + kz)]Z
and
X = {x(t) : Ix(t)1 :s; 4(SIWOa + ~1 + kow m )} ,
I\: - 4(SI wO m + kz)
since if we start, for example, from inside 3, then ~(t) is bounded by
4 .,w6 a +k,+k O W m 2
I<-4(.,w6 m +k 2 )j2 owmg
d fi ill't'lOn 0f' :-: . 0 t h erwIse,
t 0 the e . .c' :S;,
0 wh'IC h
forces the solution ~(t) towards the boundary of 3. Hence, we have the the-
orem:
Theorem 4.5.4. Consider the closed-loop system with the modeling error
term, the additive and the multiplicative disturbances satisfying Assumption
4.5.1, (4.197) and (4.198) respectively
x = -Ax + W*S(x) + wr- S'(x)(u + udm(t) + da(t)) + wo(x, u),
it? = _~I\:UT - xTWl S'(x) ,
2
(= -1\:(,
h =(-C
1
h= Z(lx lZ + lul z+ tr{WTW}) ,
A = tr{( ~I\:W + xST(x))WT}( 1 + IIWII )zW,
2 Wm
p = tr{~x(S'(x)ufw!}( 1 + IIWl l1 )ZWI ,
W
the update laws

-!I\:W - xsT(x) ifW E W or IIWII = Wm
fir - { and tr{(!I\:W + xsT(x))WT } 2: 0
- -!I\:W-xsT(x)+AifIIWII=wm
and tr{(!I\:W + xsT(x))WT } < 0
-~x(S'(x)u)T if WI E Win or IIWIII = w

vir _ { and tr{~x(S'(x)ufWr} ~ 0
I- -~x(S'(x)u)T + P if IIWIII = w
and tr{~x(S'(x)u)TWl} < 0
guarantee the uniform ultimate boundedness of the trajectories of ~(t) and
x(t) with respect to the arbitrarily small sets S and X
S = {I:(t) : II:(t)1 < 4 Sltli6a+~1+kotli=
<" <" - [I<-4(SlW6=+k ) 2 2
2}
X = {x(t) : Ix(t)1 <
-
4(Sltli6a+kl+kotliml}
I<-4(SlW6 m +k 2 )
provided that If, > 4(SIWOm + k 2 ) > o.
Theorem 4.5.4 demonstrates practically that all signals in the closed loop are
at least uniformly ultimately bounded, even in the case that both additive
and multiplicative external disturbances affect the system dynamics, since
the bounded ness of the synaptic weights estimates W, WI, is guaranteed by
the use of the projection algorithm. Furthermore,
iJ7 = _~If,UT + xTWIS'(x) ,

is a BIBO ordinary differential equation. Hence, since the term XTWI S' (x) E
L oo , we conclude that u E Loo. Moreover,
e= -If,~ + xTWS(x) + XTWinS'(X)u - xTwo(x, u)
-xTWrS'(x)udm(t) - xTWrS'(x)da(t) - Intr{J1 T W}.
e
Hence, E L oo , since wo(x, u) is bounded owing to Theorem 4.5.4 and As-
sumption 4.5.1, da(t), dm(t) are uniformly bounded owing to (4.197) and
(4.198), W, WI are bounded owing to (4.182) and (4.192) respectively and
S(x), S'(x) are bounded dy definition. Thus, we have the lemma:
Lemma 4.5.6. The adaptive regulation scheme mentioned in Theorem
4.5.4 further guarantee:
u E Loo
e E Loo
In the sequel, we examine the effects of the multiplicative noise on the closed-
loop system when we have complete model matching at zero.
Complete Model Matching at Zero with Multiplicative Noise. We
now investigate the effects of the multiplicative noise on the closed-loop sys-
tem when we have complete model matching at zero. In other words, when
da(t) = 0 and Assumption 4.5.2 holds. Using similar arguments as previously,
finally becomes
.c ::; -(I\: - 4k2 - 40mSIW)I~1 ,
2
::; 0 , (4.211)
provided that x; > 4( k2 + 8m Sl ill) and (4.194) holds. Hence, we can prove the
following theorem that summarizes the stability properties of our adaptive
regulator, in this special case.
Theorem 4.5.5. Consider the closed loop system with the modeling error
term, and the multiplicative disturbances satisfying Assumption 4.5.2 and
(4.198) respectively
x = -Ax + W*S(x) + wts'(x)(u + udm(t)) + wo(x, u),
u. T I T +xTW1cJC"( x ) ,
=-"2X;U
( = -x;(,
h =(-C
h = ~(JxJ2 + JuJ2 + IIWI12),
A = tr{(~x;W + xST(x))WT}(l + IIWII)2W,
2 Wm
p = tr{~x(S'(x)u)TWr}( 1 + IIW111)2W1.
W
The update laws

-!x;W - xsr(x) ifW E W or IIWII = Wm
W_ { and tr{(!x;W + xsr(x))WT} ~ 0
- -!x;W - xsr(x) + A ifllWIl = Wm
and tr{(!x;W + xsr(x))WT} < 0
-~x(S'(x)uf if W 1 E Win or IIW111 = ill

w _{ and tr{~x(S'(x)ufWn ~ 0
1- -~x(S'(x)u)T + P if IIW111 = ill
and tr{~x(S'(x)ufWr} < 0

w,
~,e, lxi, lui, W1, ( E L oo , I~I E L2
limt--+oo ~(t) = 0, limt--+oo u(t) = 0
=
limt ..... oo IIW(t)11 0, limhoo W1(t) =
0
limt ..... oo Ix(t)1 = 0
provided that x; > 4(k2 + 8m s 1 ill) and (4.194) holds.
The proof of this theorem is similar to the proof of Theorem 4.5.2 so it is
omitted here.
Theorem 4.5.4 shows that the adaptive regulator system with disturbances
acting upon the plant, operates in a uniformly stable fashion, if the design
constant x; is selected appropriately. Practically, when we are given an upper
bound for the disturbances, one can always find a x; to guarantee the stability
of the closed loop system. Especially in the case where we have complete
model matching at zero and only multiplicative noise is affecting the actual
plant, the stability properties established in the ideal case are retained, as
Theorem 4.5.5 clearly demonstrates. Generally, the presence of disturbances
leads to larger values of the design constant K, which, on the other hand, is
restricted owing to hardware limitations. Hence, both the disturbances and
modeling error should be kept within a maximum admissible tolerance, thus
preventing K from exceeding a prespecified limit.
Table 4.12. Direct adaptive regulation disturbance effects (n # m)
Actual System: i: = f(x) + G(x)(u + udm(t) + da(t)), xE ~n, uE ~m
Model: i: = -Ax + W*S(x) + Wi" S'(x)(u + udm(t) + da(t)) + wo(x, u),

Control Law: itT = -~KuT - xTW1S'(x) ,
-~KW - xST(x) if WE Wor IIWII = Wm
Update Laws:
vir = { and tr{(~KW + xST(x))WT} ~ 0
-~KW - xST(x) +A if IIWII = Wm
and tr{(~KW + xST(x))WT} < 0
-ex(S'(x)u)T if W1 E Win or IIW1 11 = w
vir _ { and tr{ex(S'(x)ufw!} ~ 0
1 - -ex(S'(x)u)T + P if IIW1 11 = w
and tr{ex(S'(x)ufw!} <0
A = tr{(~KW + xST(x))WT}(1+J1,::r")2W,
P = tr{ex(S'(x)u)TWr}( 1+ 1\:111 )2W1'
Filter: (= -K(,
e=(-h,
h= Hlxl2 + lul 2+ tr{WTW}) ,
Results: W1 E W', u,t,E L oo ,
WE W, 1((t)1 ~ 1((0)1,
e, x are u.u.b. with respect to
:3 = t~(t) : 1~(t)1 < 4(Slwba+kl+koWm)2}
<, <, [K 4(81Wbm+k2)]2
- ,
.:t' = x(t): Ix(t)1 -< 4(81
K
wb a+k 1+koW m)}
4(SIWbm+k2) ,
Requirements: Wi" E W', W1(0) E W',
W* E W, W(O) E W,
Iwo(x, u)1 ~ k1 + k21xl, ((0) ~ 0, ,
Idal ~ Oa, Idm(t)1 ~ Om,
K > 4( 81 wOrn + k2) > O.
The DC motor model presented in Chapter 3 is also used to test the per-
formance of the neuroadaptive control algorithm developed to cover the case
were the number of control inputs is not equal to the number of measurable
states (n =F m).
The problem is to find a state feedback to force the angular velocity n
and the armature current la, to go to zero, while the magnetic flux <P, varies.
To achieve such a goal, assuming that the motor dynamics are unknown, we
first assume that the following first-order RHONN model describes, within a
degree of accuracy, the unknown dynamics.
il = -alzl + wtlS(Zt) + wt2s(Z2) + [wlltls~l(Z)Ul + [Wdt2s~2(Z)U2'
i2 = -a2z2 + W~lS(Zt) + W~2S(Z2) + [wll~lS'(Z)Ul + [wll~2S'(Z)U2'
(4.212)
where Zl = la and Z2 = n. Estimates are provided by update laws of the
form (4.182) and (4.192) with
( = -K( , (4.213)
2 2
~ = (- ~[Z~ + Z~ + U~ + U~ + L L wl;-J, (4.214)
2 i=lj=l
because the parameters wtl' wt2' w~l' W~2 and [wlltl' [WIltz, [wll~l' [wll52 are
unknown. The dynamic feedback is taken to be
'11 1 = -KUl - Zl[[WllllS~l(Z) + [WllnS~2(Z)l
-Z2[[WllllS~1(Z) + [wll21S~2(Z)l, (4.215)
'11 2 = - KU 2 - Zl[[Wlh2S~1(Z) + [wlhzS~2(z)l
-Z2[[Wlh2S~1(Z) + [wlh2S~2(Z)l, (4.216)
We simulated a 1KW DC motor with parameter values that can be seen
in Chapter 3. The parameters of the sigmoid functions were set to
k = 1.0,
1= 1.0,
A = -0.5.
Furthermore, we have chosen
S~l(Z) = S~l(Z) = S(Zt) ,
S~2(Z) = S~2(Z) = S(Z2)'
The above selection was motivated by the fact that it leads to a simpler
model structure, which in turn gives simpler adaptive laws, thus simplifying
the implementation issue. The initial values of all variables can be seen in
Table 4.13.
Ia 0.3
n 0.3
qi 0.98
WII = W12 0.0
W21 = Wn 0.0
[Wdll = [Wd12 0.0
[Wd21 = [Wd22 0.0
simulation results
0.35 ~------------------------------------------~
0.28
"0
~ 0.21
Q.
(/)
'-
....o
~ 0.14
0.07
0.0 0.05 0.1 0.15 0.2 0.25 0.3

time (in seconds)
Fig. 4.7. The evolution of motor speed with time, " = 550
simulation resu lts

N 27.0 - . - - - - - - - - - - - - - - - - - - - ,
I
w
18.0
--c
(l)
'-
'-
:J
U
(l) 9 .0
'-
:J
....-
0
E
I....
<l::
0.0
-9.0 -r
l ~I_,~_.-.-.._,-~~~_,~~_._r._~
0.0 0.05 0.1 0.15 0.2
time (in seconds)
Fig. 4.8. The evolution of armature current with time, K, = 550
Figures 4.7 and 4.8 give the evolution of the angular velocity and arma-
ture current respectively, when the design constant K, is chosen to be K, = 550.
It can be seen, that as desired, both Q and Ia converge to zero very fast.
We further assume the existence of both an additive and a multiplicative
external disturbance on the actual plant when Q and Ia have reached a steady
state. Thus the DC motor dynamics become:
Ta d~a = -Ia - rJ>Q + Va + dm(t)Va + da(t) ,
Tm dd~ = rJ>Ia - J{o[l- mL , (4.217)

drJ>
Tf dt = -If + Vf ,
rJ> = alf
1 + filf '
where da(t) and dm (t) are an additive and a multiplicative plant perturbation
respectively. It is well known [21] that a DC motor can be viewed as low pass
filters. Thus, if the input is affected by a deterministic signal of sufficient
frequency, then it will be suppressed not because of the control effort, but
owing to the system properties; therefore, in order to test the sensitivity of
our regulator to external disturbances we use step functions and in particular
da(t) = dm(t) = 0.1.
simulation res ults

0.16
1
0.12
J 550
u
Q)
Q)
0..
j
~
VI
0.08
'-
0 750
.-
0
~
0.04 950
0.0 -+~~~~~~~~~~~~~~~~~~~
0.0 0.2 0.3 0.4 0.6

time ( in seconds)
Fig. 4.9. The evolution of motor speed with time, various K; both additive and
multiplicative disturbances present
Figure 4.9 presents the evolution of motor speed for various K" when
both the additive and the multiplicative external disturbances are present.
Obviously, boundedness and attenuation are achieved as theory predicts. Fur-
thermore, we keep only the multiplicative disturbances. Figure 4.10 shows
simula ti on res ults

N 16.0 - , - - - - - - - - - - - - - - - - - - - - - - ,
I
w
12.0
"t)
il)
il)
8 .0
a.
If)
'-
0
-'
0
~
4.0
0.0
1000
750
-4.0 I
0 .0 0.045 0 .09 0.135 0.18 0.225 0.27
time (in seconds)
Fig. 4.10. The evolution of motor speed with time, various 11:; only multiplicative
disturbances present.
the motor speed for various K. We observe that regulation of Q to zero can
still be achieved.
Summary
This chapter was devoted to the design of direct adaptive controllers for affine
in the control of nonlinear-dynamical systems with unknown nonlinearities.
To overcome the great amount of uncertainty present, RHONNs are used to
model the actual system dynamics. In the complete matching case we showed
that convergence to zero can be achieved while all other signals in the closed
loop remain bounded. Several instability mechanisms including modeling er-
rors, unmodeled dynamics and external disturbances were also considered.
Certain modifications to the control and update laws were provided to guar-
antee at least a uniform ultimate boundedness property. Both regulation
Summary 135
and tracking problems were treated. Even though most of the results of this
chapter were developed for systems having the number of measurable sys-
tem states (n) equal to the number of control inputs (m), it can be easily
verified that they also hold when n f- m. Simulation results were provided
throughout the chapter to visualize certain performance issues.
CHAPTERS
MANUFACTURING SYSTEMS SCHEDULING
Efficient management of the production process, in manufacturing systems

is a vast subject with decisive impact on major economic entities, like pro-
ductivity, competitiveness and viability. Production scheduling (i.e., organi-
zation and control of production operations in manufacturing systems), is
essential for the efficient operation of a production environment. Production
scheduling actually concerns the efficient allocation over time of resources for
the manufacture of goods, involving decisions such as part release, routing,
machine scheduling, set up times etc., with the objective of producing cus-
tomers' demands in a timely and economic fashion. Consequently, scheduling
attempts to find a way to assign and sequence the use of these shared re-
sources, such that the production constraints are satisfied and production
costs are minimized.
Competition has motivated the introduction of sophisticated and capital-
intensive new manufacturing systems and this was made possible by the de-
clining cost and the increasing power of industrial computers and robots.
Most notable among these manufacturing technologies are systems for au-
tomated, flexible and computer-integrated manufacturing. Current market,
technological and theoretical developments, provide solutions to both long-
standing and newly emerging scheduling problems, which are the subject of
intense applied and theoretical research by a variety of different research com-
munities. Traditionally, scheduling problems have been handled by manage-
ment science and operations research in industrial engineering. The relevant
theory was dominated by attempts to design constructive solutions and op-
timization algorithms for highly simplified machine scheduling problems. As
the traditional boundaries of the problem have been cut across, a number
of alternative approaches have emerged, including control theory, artificial
intelligence, system simulation, man-machine interaction, large-scale systems
and other branches of engineering.
In this chapter one such alternative approach to production scheduling is
presented. It was developed in the context of ESPRIT Basic Research Project
8141, named HIMAC (Hierarchical Management and Control in Manufac-
turing Systems), in which the goal was to address the scheduling problem
in complex manufacturing systems, by means of hierarchical decomposition.
The main ideas in the project were simplified:

138 5. Manufacturing Systems Scheduling
scheduling by organization of machines and production routes to distinct

manufacturing cells (subplants);
production control in each individual cell and synchronization of the
scheduling operations by means of a hierarchical control structure.
Scheduling within a manufacturing cell and research into alternative ways of
utilizing neural networks for the solution of this problem is the task accom-
plished in this chapter, the results in which are based on [93].
Flexible Manufacturing Systems
Advanced technologies have inevitably introduced major changes in numer-

ous manufacturing organizations, by making feasible innovative restructuring
of the production process, owing to increased levels of factory automation.
The whole area of advanced manufacturing technology has been developed
around the basic idea of data processing, information retrieval and commu-
nication integration. Computer-Integrated Manufacture (CIM), has emerged
from this communication integration for planning, coordinating, monitoring
and controlling the totality of manufacturing activities, within manufacturing
organizations. The CIM approach, is characterized by the following promi-
nent features:
manufacturing information is integrated by communication systems,
manufacturing activities are integrated physically through computer net-
works,
manufacturing functions are integrated to achieve a coherent approach to
coordination and control of the total operation.
The attempt to apply computer control to production scheduling and the
control of machines and the flow of materials in a discrete manufacturing en-
vironment, has led to the concept of Flexible Manufacturing Systems (FMS).
FMS can be defined as a system [26] of general purpose manufacturing
machines, quite versatile and capable of performing different types of oper-
ations, linked together by material handling systems. To be more specific,
in a FMS machine, tools are linked together by conveyor systems with rel-
atively small buffers, not allowing accumulation of work-in-process (WIP),
i.e., parts of semi-finished products in intermediate buffers of the production
system. Machine tools are equipped with multiple toolholders, in order to
minimize setup time between machining different components, thus minimiz-
ing the total time in the manufacturing facility. Machines within a FMS cell,
are equipped with tooling storage magazines, which contain all the tooling
needed by the machine, to process the various part types flowing through
the cell. As parts flow through a FMS machine changeover is performed au-
tomatically with a minimum of setup. In FMS, new machine-center concept
emerged, by having several machine tools arranged around a single robot,
where machining is limited to handling a specific family or group of parts.
5. Manufacturing Systems Scheduling 139
The very low setup times experienced in FMS, allow unit production to ap-
proach the cost of batch production, with the secondary effect of reducing
the amount of finished component inventory to be carried.
Introduction of the aforementioned manufacturing system type, which
is naturally embedded in a CIM system, was also motivated by two main
objectives:
providing full capability for random order production, i.e., permitting the
machining of any desired mixed of parts in a given period of time
reduction of the work-in-process and the low utilization of machines expe-
rienced in small lot production.
For the reasons just cited, scheduling of components through the system
in a way as to avoids excessive queuing, while maintaining high machine
utilization, is a critical component in the overall success of system operation.
Consequently, research in production scheduling for FMS, has been ori-
ented towards the basis of units rather than to lots. Thus, it is obvious that
our approach is primarily involved in the scheduling problem of FMS, as it
provides a means of outputting discrete-part dispatching commands, in order
to achieve a certain production goal. The target of scheduling in manufac-
turing cells is identical to scheduling in FMS, as the latter manufacturing
systems paradigm is actually a building block of the cell. The proposed ap-
proach can be extended to cover the problem of scheduling a job shop, in
which the plant manufactures a variety of items, with different machinery
and routing requirements, using a process layout which requires machines of
the same functionality to be placed in close proximity in the plant.
Before proceeding to the detailed description of the proposed scheduling
methodology, a brief overview of current industrial practice in coping with the
scheduling problem and an enumeration of the various existing approaches,
traditional and new, is presented.
Scheduling Approaches
Industrial Practice. Despite the comparatively recent introduction of a

number of industrial databases and software tools for automated scheduling,
actual production schedules are still largely generated by hand, using paper,
pencil and graphical aids (Gantt charts). Knowledge and intuition gained
through years of experience, are the practical tools employed by the sched-
uler, in generating and maintaining satisfactory production schedules. Assis-
tance of the human scheduler and improvement in quality and consistency
of the production schedules, led major manufacturers to develop or purchase
database systems which track raw materials and work-in-process inventories.
The majority of these database systems incorporate software tools automat-
ing some aspects of schedule generation to certain degree. Classification of
these commercial tools is based on the scheduling philosophies employed, a
number of which are listed below:
Manufacturing Resource Planning (MRP) systems [23], are the most

widely installed systems in industry today. They operate on fixed planning
horizons and determine: 1) the quantities of each item that will be used in
the production of the desired volume of the end products and 2) the times
at which each of these items has to be purchased or manufactured to meet
the predefined due dates for the end products. Working backwards in time
and using processing times, MRP determines the latest time at which com-
ponents and subassemblies should be made and ordered. Then, more detailed
capacity-requirements analysis follows and operating sequences and produc-
tion lots are determined. MRP has many weak points though, since actual
installed capacities are ignored and achievement of feasible schedules requires
manual adjustments.
Just In Time (JIT) production [103], is a scheduling philosophy that dic-
tates reduced material and minimum WIP inventories, in order to achieve
process improvement and reduce process variability. The most significant re-
sult of this approach is the reduced cost associated with holding of inventories.
In addition, it improves process flow and floor control mainly by detecting
early failures in the production system. The only actual disadvantage of the
method is that it is purely descriptive with no available theory to assist in
derivation of JIT methods.
Optimized Production Timetables (OPT) [48], consist of proprietary soft-
ware and hardware packages and are an alternative to comprehensive MRP
for production planning and resource scheduling. OPT operates sequentially,
examining at fixed intervals of time, how production resources should be used
to meet the requirements. First, it selects a candidate schedule, simulates it,
and determines the bottleneck machines (critical resources). Next, operations
are scheduled from the critical resources using heuristic procedures to feed the
bottleneck during its periods of starvation. By repetition of this basic proce-
dure, an optimal solution results. Although generally, satisfactory schedules
are derived, large expense owing to their proprietary nature constitutes a
major drawback.
Machine Sequencing & Scheduling Theory. The formal job shop or
machine scheduling problem may be stated as follows: N jobs need to be
processed on M machines, where each job consists of a set of P operations
uniquely associated with the number of machines. The processing times for
the operation cannot be split and the constraint that operations for a job must
be performed in a unique order, applies. Scheduling, involves determining the
sequencing and timing of each operation on each machine, such that some
given performance index is optimised. Minimum makespan and minimization
of maximum tardiness are typical criteria.
The machine scheduling problem represents a highly simplified formalism
of the production scheduling problem encountered in practice. This problem
is of interest since it captures the fundamental computational complexity of
sequencing jobs in machines, where the computation of the optimal solution
is a NP-hard problem. Experience with the problem, demonstrated the need

for heuristic (nonoptimizing) approaches, which select the next operation to
be processed based upon some easily computed parameter of the jobs, oper-
ations, machines, which may include processing times, due dates, operation
counts, costs, setup times, arrival times and machine loadings. Typical ex-
amples are SPT (shortest processing time first), LPT (longest processing
time first), FIFO, LPR (longest processing time remaining), EDD (earlier
due date) and random selection (Monte Carlo) [33], [104], [79]. Commer-
cially viable strategies that employ a heuristic to find a good seed or starting
schedule, and then perform the rest of the scheduling have emerged.
Resource-Constrained Project Scheduling. Having as a special case the
aforementioned machine scheduling problem, resource-constrained project
scheduling is a more realistic scheduling paradigm, that takes into account
various complexities of the scheduling environment. Similarity of the two
problems is emphasized when both are viewed as networks so determining
the best resource-constraint project schedule consists of a combinatorial op-
timization problem. Among the approaches most commonly employed are
branch and bound methods, dynamic programming, zero-one integer pro-
gramming and Lagrange multiplier methods [20].
Control Theory. Control theory seeks scheduling methods that either re-
flect the uncertain nature of the available information explicitly, or give some
guarantee regarding the insensitivity of the schedule to future information.
Practically, schedules are sought that are robust to disruptions, in the ab-
sence or inaccuracy of status information and are flexible to change. Control
theory views scheduling as a dynamic activity, where the system under con-
trol operates on a sequence of discrete-event inputs u(t), yielding a sequence
of discrete-event outputs y(t) [27]. Defining release and dispatching times,
setup times and maintainance as control inputs and levels of inventory and
machine status as system states, the scheduling problem is then to drive the
state vector to some desired value (production requirement), or follow some
trajectory (varying demand) distributed over time, where both are the regu-
lation and tracking problems respectively, well-known in control theory. The
control must also satisfy the constraints imposed by the limited manufactur-
ing resources and the performance measure.
Discrete Event Simulation. Simulation models are built out of complex
manufacturing systems enabling their numeric evaluation for time periods
of interest. Discrete-event simulation has been used primarily as a means of
testing fixed scheduling heuristics and dispatching rules. Since heuristics plus
priority rules give better performance than plain priority rules, it is clear that
flexible, interactive simulation tools are useful to scheduling in manufacturing
systems. The user of a simulation tool can stop or interact with the system
model and try alternative scheduling approaches by reviewing the status of
the job-shop model, using his knowledge and practical experience.
Related to dynamic system linearization, is the recent development of

perturbation analysis [37], that offers efficient optimization of parameters in
simulation. Observing only one sample path, the technique enables the es-
timation of the gradient vector with respect to a variety of parameters. In
many situations, perturbation analysis outperforms repeated simulations.
Many advantages of the simulation approach steam from the fact that it
enables the modeling of the effect of a policy change, which cannot occur in
an analytic model, while the user can also perform exploratory tests upon
the schedules to be used. The experimental nature of simulation, constitutes
the principal disadvantage of this scheduling paradigm, as simulation studies
are difficult to generalize.
Stochastic Optimization. A number of types of manufacturing system
problems are addressed by application of the research practices of queuing
theory, reliability theory, lot-sizing techniques and inventory theory. The main
weakness of stochastic optimization, is that highly simplified models are re-
quired to pursue analytical solutions.
Queuing theory pictures each machine center as a set of servers with each
machine a server and each job a customer. The wide variety and the com-
plex routings of jobs are represented by assuming that processing times and
arrival of jobs at a center, are described by probability distributions [49].
Queuing models, tend to give reasonable estimates of system performance,
while demanding relatively little input data and not requiring high computa-
tional cost so they can be used interactively to arrive quickly at preliminary
decisions. These queuing-network models have the disadvantage of failing to
represent features such as limited buffer space. Further, in various situations,
queuing models are mathematically intractable.
Fluid networks which describe the asymptotic behaviour of queuing net-
works, may be utilized to solve the dynamic scheduling problem. They con-
stitute useful models for systems that process continuous flows. Also, fluid
models have been developed to face the scheduling problem for systems with
random disruptions (stochastic sequences of up and down times). They model
manufacturing environments when the arriving and processing of jobs is de-
terministic but typically the environment remains random.
Reliability Theory is primarily concerned with maintaining the stability of
a schedule in the face of machine failures, as reliability stands for the prob-
ability that a machine behaves adequately over an interval,. Knowledge of
machine reliability can be utilized to limit the disruptions caused by machine
failures. It also requires knowledge of the time-varying failure rate.
Lot-sizing techniques attempt to determine lot sizes which will maximize
production throughput and assure that due dates are met, while minimizing
production and inventory costs. Most techniques either cannot guarantee the
generation of a feasible solution, or are computationally prohibitive.
Finally, inventory theory, models and analyzes the effects of uncertainty,
to derive optimal stock policies. Inventory-stocking policies assign to each
stock item an exogenous demand, modeled by a stochastic process and at-

tempt to find the best stocking policy for each item. Many gaps exists in
the theory, as many complex inventory situations must still be taken into
account.
Artificial Intelligence. In the artificial intelligence (AI) approach [17], [59],
the scheduling problem is viewed as the determination and satisfaction of a
large number and variety of hard and soft constraints, encountered in the
scheduling domain. AI extends the knowledge representation techniques in
order to capture these constraints, to integrate constraints into a search
process and relax them when conflicts occur. AI search methods that are
performed by heuristic rules guiding the search, thus finding a solution to
computationally complex problems are of great merit in scheduling applica-
tions. Similarly, current research on genetic algorithms, simulated annealing
and learning systems, holds potential for the improvement of speed and ac-
curacy of various production scheduling approaches. Production scheduling
is bound to benefit from other areas within AI, specifically, rule-based and
knowledge-based expert systems. Expert systems though, support diagnosis
only within slowly evolving knowledge domains and moreover, their develop-
ment is costly and time-consuming. AI and conventional operations research
techniques may be used to alleviate some of the aforementioned difficulties.
Approach Outline
As a brief introduction to some of the most important scheduling approaches
has been made, this subsection is devoted to the presentation of a neural-
network-based solution to the problem. Two of the most significant motives
that underline the basic idea of the proposed scheduling methodology are the
following:
the need to perform scheduling in real time (i.e., the desire for a computa-
tionallyefficient method),
the capability of coping with a manufacturing environment undergoing
various changes like alterations in production demands and fluctuations in
machine statuses, such as machine failures.
For the reasons cited above, in our approach to production scheduling, the
control aspect of the scheduling problem in manufacturing cells is addressed.
In this chapter, production scheduling is interpreted as a control regulation
problem. The goal is to achieve the driving of every buffer in the manu-
facturing system to a desired value. The derived solution realizes real time
scheduling in manufacturing cells by outputting the appropriate dispatch-
ing commands in real time, while respecting all manufacturing system con-
straints. The scheduling strategy is actually a non-myopic one, that achieves
production demands for short time horizons so job-shop scheduling, where a
predefined finished-products demand over some relatively small time horizon
is to be reached, is considered.
The manufacturing system is assumed to consist of several machines, each

one capable of processing a number of different part types, while multiple ma-
chine visits are also allowed in a specific part type route. Each machine is
subdivided into a number of submachines equal to the number of parts it pro-
cesses, fed by one or more buffers and outputting products to a single buffer.
Thus assembly processes are allowed. To the output of every submachine is
attached a buffer that collects the parts that it produces. An adaptive con-
troller which is actually responsible for the regulation of the output buffer
state, (buffer contents) exists. The controller consists of a neural-network-
based direct adaptive architecture, that derives its control law by assuming
the existence of a model, the derivation of which is based upon the approx-
imation capabilities of the recurrent high-order neural networks. The exact
form of the control law is obtained through the use of the Lyapunov stability
theory. The control output is interpreted as the submachine operating fre-
quency. Since the adaptive controller's output is inevitably a continuous-time
signal which has to be applied to a discrete-event system, a translation into
a dispatching series through a sampling procedure is necessary. Moreover,
since the controllers of all submachines belonging to the same machine may
conflict, a decision for the actual part to be processed by the flexible machine
is taken using some criterion based on the several-route work-in-process. Ma-
chine processing times are assumed to be deterministic.
The adaptive properties inherent in the proposed scheduling approach, al-
low for a fast reaction to a potential change in the finished product demand.
The control procedure is immediately redirected to achieve the updated con-
trol objective.
Why adaptive control? It still remains to justify the choice of nonlin-

ear adaptive control and the rejection of alternative control schemes, like
optimal control. Here, an informative comparison will be made of the merits
of both adaptive and optimal control for the scheduling problem with respect
to various aspects inherent in manufacturing systems, in order to clarify the
motives that led to the choice of adaptive control.
Adaptive control approaches enjoy the property of on-line control, while
optimal control methods are essentially off-line techniques, owing to the large
dimensionality that scheduling problems require for even moderate cell sizes
and the precalculation of the entire schedule they perform. Similarly, the
computational complexity becomes extremely high when considering optimal
control approaches. Problem dimension is always a drawback when trying to
apply optimal control.
Both approaches, are capable of solving the scheduling problem (i.e., share
the capability of precise achievement of the production goal), while optimal
control offers optimal solutions compared with adaptive control that seeks
only suboptimal ones.
5.1 Problem Formulation 145
Precise model knowledge is essential for the application of an optimal

control scheme, while adaptive control techniques can handle model uncer-
tainties efficiently. Derivation of a relatively simple model capable of cap-
turing all details inherent in the behavior of manufacturing cells is hardly
feasible. Furthermore, any derived model would be rather be nonlinear, lead-
ing to extremely complicated, often infeasible optimal control formulations.
Model nonlinearities are more easily handled by the various adaptive control
methodologies.
Modifications in the definition of a scheduling problem, such as alterna-
tions in the production demands, are unacceptable in an optimal control set-
ting, since a recalculation of the optimal solution is required. On the contrary,
such variations can be immediately incorporated by an adaptive controller.
Finally, system disturbances can be tolerated by appropriate adaptive
control structures. The same is quite difficult in optimal control schemes.
5.1 Problem Formulation

We consider manufacturing systems consisting of M machines and producing
a multiple of P part types. Each part type requires a number of operations
to be performed in a given sequence, defined by its route. Multiple routing is
possible and parts may visit some machines several times. Each machine m is
assumed to consist of a number N (m) of submachines equal to the number of
different part types it is able to process. Each submachine actually represents
an operating mode of machine m. Let O(m) be a set such that S E O(m)
implies that submachine s is a submachine of the actual machine m. The
cardinality of O(m) is equal to N(m). Also, let NM = L:~~ N(m;). Only
one submachine is allowed to be working at a time, processing parts of a single
type. Machine operation times are supposed to be constant, where different
operation times for every submachine are allowed. Set-up times are assumed
to be insignificant. We also consider B buffers, one for each operation of a part
type at the machine that performs that operation. Let B+ (s) be a set such
that b E B+(s) implies that buffer b is an input buffer for submachine s. The
level of buffer b E B+ (s) will be denoted by xo b Similarly, b E B_ (s) implies
that buffer b is the output buffer of submachine s. An input buffer is defined as
a buffer b for which bn{ B_ (sl)UB_ (S2)U .. .UB_ (SNM)) = {0}. Similarilyan
output buffer is the one for which bn{B+(sr)UB+(S2)U ...UB+(SNM)} = {0}.
We assume that the input buffers are of infinite capacity. We further define
the set M_(b), where s E M_(b) implies that sub machine s is fed by buffer b.
NB(S) and NM(b) are the cardinalities of sets B+(s) and M_(b) respectively,
while the cardinality of B_ (s) is always equal to one, since we do not allow
for disassembly operations. On the contrary, assembly procedures can be
described if for some submachine s the cardinality of B+ (s) is greater than
one. It is not difficult to see that sets O(m), B+(s) and B_(s) can describe
the interconnectivity of the manufacturing system fully. A machine will be
called "idle with respect to a part type", if it is idling or is processing some

part type different from the specified one.
The objective is to steer (through appropriate scheduling) the manufac-
turing system output buffers to reach a small neighborhood of predefined
finished products, while keeping all intermediate buffers within their accept-
able values.
Example. As a first example consider the cell configuration shown in Figure
5.1. Two part types (P = 2) corresponding to two different routes are pro-
cessed by three different machines (M = 3). Two different operating modes
are assigned to machines A and C and one more to machine B, giving a total
of five submachines. Buffers 1, 2 and 8 are the raw material (input) buffers
for both part types. Route 1 uses the buffers 1, 3 and 5 while the second
route uses buffers 2, 4, 8, 6 and 7. Buffers 5 and 7 are the final product (out-
put) buffers, leading to a total of B = 8 buffers. According to the definitions
~ ~ ~ ~
{~ 3
2 5 4
Machine A Machine B Machine C

0
Fig. 5.1. Sample configuration
given above, the number of submachines per machine station are N(A) = 2,
N(B) = 1 and N(C) = 2. Similarily, the O(A), O(B) and O(C) sets denoting
machine subdivision equal {I, 2}, {5} and {3,4} respectively. The intercon-
nectivity sets B+(.) and B_(.) are equal to B+(I) ={I}, B+(2) =
{2},
= = = =
B+(3) {3}, B+(4) {6}, B+(5) {4,8} and B_(I) {3}, B_(2) {4}, =
= = =
B_ (3) {5}, B_ (4) {7}, B_ (5) {6}. The cardinallity of B+(5) equals 2
=
since an assembly operation takes place. Finally, M_(1) {I}, M_(2) =
{2},
M_(3) = {3}, M_(4) = M_(8) = {5}, M_(5) =M_(7) =
{0} and
M_(6) = {4}. The cardinallity NM(b) equals to one for all but the output
buffers 5 and 7. In this case NM(5) = NM(7) = O.
5.1.1 Continuous Control Input Definition
We assume that an equivalent machine-operation frequency is used as a con-

trol input to the system, defined as the inverse of the time between two
successive machine-starts. As demonstrated in Figure 5.2, the original dis-
patching schedule of a submachine, (dispatching times are shown by pulses),
is transformed to a piece-wise constant frequency function, fixed for a time
interval equal to the submachine operation time, (indicated by the working
tim.
r I"'
I---t
Worki ng
Idling Idling
Equlv.lent
wo rtc 1118
frequency
I I
Um.
Fig. 5.2. Equivalent frequency definition
clause), plus the time it remains idle with respect to the processed part type
(indicated by the idling clause). It can be easily seen from Figure 5.2 that the
time interval between the first and second dispatching times is longer than
that between the second and the third command, thus leading to a lower
operating frequency for the first case.
Using this definition, frequencies range between zero and a fixed finite
value. The lower bound equal to zero, corresponds to an infinite period time.
The upper bound U max , corresponds to a minimum period time and is equal
to the reciprocal of machine operation time. Owing to the machine flexibility,
frequency is expected to be smaller than the upper bound, thus allowing time
for other submachines of the same machine to produce parts. Observe that
this definition is a crucial point in our work, since it enables the transition
from a discrete-event system to a continuous-time one.
5.1.2 The Manufacturing Cell Dynamic Model
Consider buffer i collecting output products from submachine SOi and feeding
submachines Slj,j E M_(i). Machine SOi is fed by buffers XOb : b E B+(SOi).
This topology is depicted in Figure 5.3. The i index will be dropped for the
sake of simplicity. Assuming that submachines Slj are idling, the increasing
rate of buffer i is a function of the feeding submachine frequency U' O and the
operation time T. o ' Thus, buffer i evolution can be described by a differential
equation of the form
(5.1)
Machine mo
D c
Machine m,
0
0
B.(m o) 0
0
D
~, ... ~ e "
\ xo, } 1\
'''--"''
f--
I ~' S'l
U S11
,,r--,\
(~~
0
0
0
nJ=
r-;
So
us o
0
(
~ r /1
i. :"',
--'\
) 'L
0
c
5'2
1
II
0
US12
'~-\ , \
( XON / - _/
M.(i)
D
'---./ -"
0
C
0
Fig. 5.3. Sample Topology

D
where fi(.) is some unknown nonlinear function of its arguments.
The operation time T30 is a known constant and can be embedded as a
function of the frequency U 30 ' thus simplifying (5.1) to
(5.2)
Without loss of generality, we can extract a linear factor u(so) to obtain
(5.3)
Observe that fi(.) and g;(.) are, in general, different functions.
To relax a future control law, we add restrictions on the rate of increase
of buffer i, owing to machine So starvation. This can be accomplished by
multiplying gi(') by appropriate centered functions fij(XO j ), for all XOj E
B+(so), sharing the property
Observe that under this modification, buffer i will not be fed if some of
the preceding buffers Xij are empty, no matter the value of Uso. A possible
selection for the fij(XOJ functions is the centered sigmoid given by
(5.5)
where
k
A= -"2. (5.6)
The dynamic equation becomes
x = fi , (XoJfi 2 (X0 2 ) .fiNB('O/XONB('O))9i(UsO)U80. (5.7)

Another restriction concerns the inability of buffer i to be fed, when its
content has reached the capacity bound. To embed such a constraint, we
assume that gi(U so ) is extended by including Xi as an argument and becomes
a new function hi (Xi, U80 , c;), where Ci represents buffer i capacity, assumed to
be known and constant. Embedding Ci in Xi, collecting XOj for all j E B+(so),
together with Xi in a single vector Xi, and aggregating fij(XOJ together with
hi(xi, uso) in fOi(X), (5.7) simplifies to
Xi =fOi(Xi,Ui)Ui. (5.8)
To proceed, assume that submachine So is idling while some or all of the
dependent submachines Slj' j E M_(i),j = 1,2, .. . ,NM(i) are working.
The decreasing rate of buffer i level is a nonlinear function of the operating
frequencies of submachines Slj' that is
(5.9)
Collecting all operating frequencies arguments of (5.9) in a new vector Ui,

(5.8) is generalized and takes the form
Xi = fli(Ui). (5.10)
Aggregating (5.8) and (5.10), we obtain the dynamic model of the evolu-
tion of buffer i, given by
Xi = fOi(Xi, Ui)Ui + fli(Ui). (5.11)
Example( continued). The dynamic equations governing the evolution of
buffers 1 to 8 are constructed by taking into account the cell interconnectivity
which is given in Table 5.l.
The dynamic equations are given below :
Table 5.1. Cell interconnectivity
Buffer Preceding buffers Input submachine Output submachines
1 1
2 2
3 1 1 3
4 2 2 5
5 3 3
6 4,8 4 4
7 6 5
8 5
Xl = fu(ut),
X2 = f12(U2),
X3 = f03(Xl, X3, Ul)Ul + f13(U3),
X4 = f04(X2, X4, U2)U2 + f14(U5),
X5 = f05(X3,X5,U3)U3,
X6 = f06(X4, X6, xs, U5)U5 + f16( U4),
X7 = f07(X6,X7,U4)U4,
xs = !ts( U5) .
Since both fOi(.) and fli(.) are unknown functions of their arguments,
neural networks of the form described in Chapter 2 are employed to obtain
an accurate model for the manufacturing cell. More precisely, there exist
weight values Wti and W[i such that the system (5.11) can be written as
Xi = wt/ Si(Xi, Ui)Ui+W;/ Sli(Ui)+Ei(Xi, Ui, Ui), i = 1,2, ... , B(5.12)
Where all symbols in (5.12) are as defined in Chapter 2. Observe that
fOi(Xi, Ui)Ui in (5.11) is positive since it represents a buffer increase rate,
thus rendering wt? Si(Xi, Ui) positive also. Now, owing to the positiveness of
Si(Xi, Ui) elements, we can assume that the elements of Wti are also positive.
For the modeling error term Ei (Xi, Ui, Ui) we pose the following assumption
that is common in the literature.
Assumption 5.1.1. There exists an arbitrarily small positive constant fi
such that IEi(Xi, Ui, ui)1 :::; fi for all i = 1,2, ... , B and for all Xi E [2, where [2
is the compact in which the approximation capabilities of the RHONN model
hold.
Example( continued). Assuming that up to fourth-order terms of each ar-
gument are required with no coupling terms of the form X xf' 1i ,
the dynamic
equations for the buffers 1 to 8 are given by
5.2 Continuous-time Control Law 151
Xl = wt/ S(Ul) + S(Xl, Ul),

X2 = Wr2 T S(U2) + S(X2, U2)'
X3 = wt/ [S(xlf,S(X3f,S(uI)T( Ul
+wt3 T S( U3) + S(Xb X3, Ul, U3),
X4 = wtl [S(x2f,S(X4f,S(U2f( U2
+Wt4 T S(U5) + S(X2, X4, U2, U5),
X5 = wt/ [S(x3f, S(x5f, S(u3f( U3 + S(X3, X5, U3),
X6 = W~/ [S(x4f,S(X6f,S(xs),S(U5fl T U5
+Wt6T S(U4) + S(X4, X6, XS, U5, U4),
X7 = wt/ [S(x6f, S(x7f, S(u4f( U4 + S(X6, X7, U4, U5),
XS = wt/ S(U5) + S(XS, U5),
with
S(Xd = [sex;) s2(Xi) S3(Xi) s\x;)(
S(U;) = [S(Ui) s2(Ui) s3(Ui) s4(u;)(
while Wti E ~12 for i E {1, 2, ... , 8} - {6}, Wt6 E ~16 for i E {6} and
Wii E ~4 for i E {1, 2, ... , 8} - {5, 7}.
5.2 Continuous-time Control Law
The purpose of this section is to develop a continuous-time control law based

on the continuous-time model of the manufacturing cell whose validity was
the main theme of the previous section. The continuous-time control signal
will act as a basis upon which an adaptive scheduler will be built. From the
previous section, recall that a continuous-time model for the manufacturing
cell is
Xi = Wt/SOi(Xi,Ui)Ui + Wt"/Sli(Ui) +Si(Xj,Ui,Uj), (5.13)
for all i = 1,2, ... , B. Let us first consider the ideal case in which
and derive a basic control law. Afterwards, modeling errors will be allowed
and we shall examine their effect in the closed-loop system. Whenever neces-
sary modifications on the basic control law will be provided.
5.2.1 The Ideal Case
After adding and subtracting the terms w6;Soi(xi, Ui)Ui and WESli(ud,
(5.13) becomes
Xi = -WJ;SOi(Xi, UdUi - WlS1i(Ui)
+WJ;SOi(Xi, Ui)Ui + WlSli(Ui), (5.14)
where WOi = WOi - Wti' Wli = Wli - Wii, and WOi, Wli, i = 1,2, .. . ,B
- L:. - L:.
are weight estimates of the unknown weight values Wti, Wti' i = 1,2, ... , B
respectively. Let Xip i = 1,2, ... , B denote the target value for each buffer,
which by definition is a positive constant. Define the control error e Ci , i =
1,2, ... , B as eCi ~ Xi - Xit, i = 1,2, ... , B. Differentiating ec, with respect
to time, we get the error equation:
fCi = -WJ;SOi(Xi, Ui)Ui - WlS1i(ud
+Wl;SOi(Xi, Ui)Ui + WlS1i(ud. (5.15)
To derive stable control and update laws, Lyapunov stability theory is em-
ployed. Thus, take the following Lyapunov function candidate .c = L,~=1 e~i + !
2 L,i=l WOi WOi + 2 L,i=l Wli Wli. Differentiating .c along the solution of
1 B -T- 1 B -T-
(5.15), we obtain
.c. = ""'[ -T - Ui)Ui -

b
L...J -eCi WOiSOi(Xi, -T-]
ec, WliSli(Ui)
i=l
b
+ L [eCi Wl;SOi(Xi, Ui)Ui + eCi WlS1i (Ui)] (5.16)

i=l
If we choose
WOi = eciSoi(X,Ui)Ui, i = 1,2, ... ,B (5.17)
Wli=eciSli(ud, i=1,2, ... ,B (5.18)
then .c becomes
B
.c = Lee; [WJ;SOi(Xi,Ui)Ui + wlSli(Ui)] (5.19)
i=l
To proceed, choose the continuous-time control law
Ui = -qisgn (WJ;SOi(Xi, Ui)) sgn(e c.}, (5.20)
qi = 1 [T
-=- -
IWliIISli(Ui)I+,lec,1 1, (5.21)
Wi
W;TS .(- . . )) - { 1 ifWl;Soi(xi,ui) ~ 0

sgn ( Oi 0, X" U, - -1 otherwise (5.22)
-life c <O
sgn(e .) - { , - (5.23)
c, - 1 otherwise
5.2 Continuous-time Control Law 153
for all i = 1,2, ... , B, with I a strictly positive constant and wi as defined
in (5.24). Then (5.19) becomes = - L~=l ,e~; ::; 0 although in order for
(5.21) to be valid, the following must hold
(5.24)
By the definition of SOi(Xi, Ui), i = 1,2, ... , B we can select its entries to be
bounded from bellow, by a positive constant. Furthermore, since the actual
weight values Wti' i = 1,2, ... , B are positive, it is reasonable to employ
projection modification to guarantee that the weight estimates WOi, i =
1,2, ... , B remain positive also Vi ~ o. After these changes are made, (5.24)
is guaranteed. Observe further that (5.17) becomes Ui = -qis9n( ee;) since
WJ;SOi(Xi,Ui) ~ o. Thus, the update law (5.17) is modified to
if WOi E WOi or
WOi E 8WOi and
ee;s6;(Xi,Ui)UiWOi ~ 0
WOi = ee,SOi(Xi, Ui)Ui (5.25)
ee; s6;( Xi, Ui)Uj WOi
Wo ;II)2 WOi if W Oi E 8WOi and
x (1+IIwo,
ee; s6;(Xi, Ui)Ui WOi <0
where WOi is a set that guarantees the positiveness of the entries of WOi and
8WOi is its boundary. More information regarding projection modification
can be found in the preceding chapters.
Theorem 5.2.1. Consider the system {5.13}. The control law {5.20}-
{5.23} together with the update laws {5.18}, {5.25} guarantees the following
properties:
ee;, WOi, Wli,Xi,ee; E Loo
limt ..... oo ef , (t) = 0, limt ..... oo WOi(t) = 0
limt ..... oo W1i(i) = 0
Proof: Since ::; 0 we have that .c E .coo, hence ee;, WOi, Wli E .coo.
Since ee; = Xi - Xi, and Xi, is a constant we have that Xi E .coo. More-
over, since .c is a monotone decreasing function of time and bounded from
bellow, the limt ..... oo .c(t) = .coo exists so by integrating from 0 to 00 we
have fooo lee;J2 dt ::; ~ [.c(0) - .cool < 00, which implies that leci I E .c 2 From
the above analysis and (5.15) we also have that ee; E .coo. Moreover, since
ee; E .c 2 n.c oo and ee; E L oo , applying Barbalat's Lemma we conclude that
limt ..... oo ee; (i) = o. Now, using the boundedness of S9 i (Xi,. Ui), S1i( Ui), Ui, WOi
and the convergence of ee; to zero, we obtain that WOi, Wli also converge to
~o.
5.2.2 The Modeling Error Case

In this subsection we investigate the more general case in which the continuous-
time representation of the manufacturing cell by the dynamic neural network,
also incorporates modeling errors. In other words, the full model given by
(5.13) is employed in the analysis that follows. Hence, (5.19) now becomes:
B
.t = L ec; [WJ;SOi(X;, Ui)U; + W[;Sli(Ui) + Ei(Xi, Ui, Ui)] (5.26)
i=1
Again, if we choose (5.18), (5.20)-(5.23) and (5.25), .t becomes

B B
.t = L [-Ie~; + ec;Ei(xi,Ui,Ui)] ::; L [-/le c;12 + fil e c;1l ::; 0,
;=1 ;=1
provided that lec; I > !.L. The above implies that the control error e c; possesses
a uniform ultimate bo~ndedness property with respect to the arbitrarily small
set fc; = {ecJt): lec;l::; ~'I > o}.
Theorem 5.2.2. Consider the system {5.13} with the modeling error term
fi(Xi, Ui, Uj) satisfying Assumption 5.1.1. The control law {5.20}-{5.23} to-
gether with the update laws {5.18}, {5.25} guarantees the uniform ultimate
boundedness of the control error with respect to the arbitrary small set
fc; = {ec;(t) : lec; I ::; ~, I> o} for all i = 1,2, ... , B.
Theorem 5.2.2 states nothing about the boundedness of the weights Wli, i =
1,2, ... , B. Again, in a projection modification can be used to satisfy Wlis are
confined within a set Wli = {Wli : IIWlil1 < Wli}. Hence, (5.22) is modified
to
if W1i E W 1;
or IIWlil1 = Wli and
ec;s[;(u)W1i 2: 0
ec;Sli(Ui) (5.27)
-eC;s[;(Ui)Wlie+l~lilfWli ifWli E W1i
and ec;s[;(u)Wli < 0
for all i = 1, 2, ... , B.
Remark 5.2.1. Theorem 5.2.2 practically states that if we start from in-
side the set f c ;, i = 1,2, ... , B then ec ; is bounded uniformly by ~. Other-
wise, there exists a finite time in which ec ; reaches the boundary of fc; and
remains therein for all time thereafter. It is obvious that the performance of
the continuous-time control law is controlled by fi and I' The first term fj is
strongly related to the continuous-time model developed in Section 5.2, while
the second one, 1 is a design constant which can be chosen arbitrarily.
5.3 Real-time Scheduling 155
5.3 Real-time Scheduling
5.3.1 Determining the Actual Discrete Dispatching Decision
The control input signal obtained by the control law (5.20)-(5.23), (5.25),
(5.27) is a continuous-frequency signal so some dispatching algorithm has to
be employed to determine the actual parts-dispatching times. The dispatching
algorithm used proceeds as follows:
Algorithm 5.3.0 (Discrete Dispatching Decision-Single Pan Case).
Step 1 The controller output is sampled at a certain time instant,

and the control input frequency is transformed to a time interval
by taking its reciprocal.
Step 2 At the same time instant a new command is sent to the spe-
cific submachine.
Step 3 As soon as the time interval is completed, a new sample of
the control input is taken and the process is repeated.
Remark 5.3.1. The controller equations (5.20-5.23, 5.25 and 5.27) are
allowed to evolve in time, while the submachine state is left untouched until
the precalculated time interval is completed. This interval is greater than or
equal to the machine operation time since the control input frequency is less
than or equal to u max Clearly, this control input discretization introduces
some modeling error, to be discussed in the following subsection.
For the case of FMS systems containing multiproduct machines, in order to

determine which buffer is to be served, we propose the use of some criterion
representing a measure of the priority to be given. For a certain machine m
containing N(m) submachines, the criterion is calculated for each route which
employes any submachine s E O(m). The derivation of the criterion value is
based on the semifinished product availability, (work-in-process, WIP), as
well as on the control error of the route output buffer. According to the pro-
posed policy, the route to be served is the one with the largest criterion value.
The proposed criterion value for each submachine s of a certain multiproduct
machine m with s E O(m) is given by
J3 = Al !(x1J + Ai"" !(xi"") + ... + ANB (3/(x NB (3)
2
+Atg(xt) + Atg(xt) + ... + Xt~2 ' (5.28)
l+e
where xi, i = 1,2, ... , NB(S) are submachine s preceding buffer levels and
xt, i = 1,2, ... , N are the levels of the buffers that follow on the same route,
while N denotes the number of submachines that follow submachine s along
its route including the output buffer. The level of the latter is denoted by
e. The parameteres Ai and At are weighting factors to be determined. The

dependence of each argument on s has been omitted for the sake of simplicity.
PAl WorldllB
ES9 Idling
- Continuous signal
g - - Applied input
~ t Sampling point
...
.:::
i..
!
Fig. 5.4. Continuous control input discretization
In the definition above, f(.) is a positive monotonically increasing non-

linear function, with f(O) = 0 and f(c) = 1, where c stands for the corre-
sponding buffer capacity. Obviously, this function outputs the largest value
for the case of a large product accumulation on the feeding buffer. Similarly,
g(.) is a mirrored version of f(.), with g(O) = 1 and g(c) = O. This function
outputs the maximum value if the following buffers are almost empty. In this
way conditions like the feeding buffer being almost empty and/or the follow-
ing buffers being in the vicinity of their maximum capacity, lead to small J.
which in turn means that the corresponding route will not be served.
Functions f(.) and g(.) can be closely approximated by shifted sigmoids,
that is f(y) = l+e ,,~(y+vd and g(y) = 1 - He "!(Y+V2) ' where constants J.li
and Vi are selected such that f(c/2) = g(c/2) = 0.5.
Remark 5.3.2. Criterion J. gives a measure of the current route neces-
sity for priority to be given. The weights Ai, At can all be selected to be
equal to unity, thus allowing all semifinished product inventory to be equally
taken into account. A series of decreasing weights, leads to a criterion with
a local scope, taking into account the buffers in a small neighborhood near
submachine s.
Remark 5.3.3. An alternative definition of J. can be achieved by setting

f(y) =? g(y) 1 - = ? =
taking AN 0 and Ai, At equal to the total WIP
5.3 Real-time Scheduling 157
cost when the corresponding buffer has reached its capacity. In this case,
giving priority to the route possessing the largest value of J. causes the WIP
cost reduction of the most burdened route.
10.-----------------~~~
---.. .......
' .0 ,.,....__:::--- - - - - - - - - - - -- - - - - - - - ,
o.g
D. 0.8
0.7 0,7
...
! 0.6 ~ 06
;
~ D.' ~ 05
I D' ~ Q.4
U
03 0.3
02 0,2
01 0,1
DD ~-=~----------------~
0.0 L -_________________ --=-=-......>I
aun., Le.,.1(IHItts)
Fig. 5.5. Alternative definitions of a) f(.), b) g(.)
The discrete dispatching decision algorithm in the multiproduct case is sum-

marized in Algorithm 5.3.1.
Example (continued). Suppose that a dispatching command is to be sent
to machine A. The criterion values hand h are given by h = At g(X3) +
At 1~~~ and h = At g(X4) + At g(X6) + At 1~:~' Observe that no preceding
buffers are taken into account since buffers 1 and 2 are raw material buffers
and are supposed to have infinite capacity.
5.3.2 Discretization Effects
From the discretization procedure mentioned previously, becomes apparent,

that an additional error is introduced in the control input signal. More pre-
cisely, the continuous-time controller previously developed, contains the ac-
=
tual scheduling frequency as follows Ui Udi +Wi(Xi, iii, Ui), i 1,2, ... , B, =
where Uj is the continuous-time control law, Udi is the actual scheduling
frequency and Wi(Xj , iii , Uj) is the difference between the above-mentioned
signals.
Algorithm 5.3.1 (Discrete Dispatching Decision-Multi Product Case).
input U E ~B, x E ~B, ee E ~B

if machine is not FMS then
execute Algorithm 5.3.0.
else
for all submachines that conflict.
calculate J s as in (5.28).
select the submachine with the largest J s
execute steps 1,2 of Algorithm 5.3.0.

when the time interval is completed.
repeat
Obviously, Wi(Xi, Ui, Ui) is bounded since Ui is bounded owing to (5.20)-

(5.23), (5.25), (5.27) and Theorem 5.2.2, and Ud, is bounded by definition. In
the present subsection we examine the effects of Wi( Xi, Ui, Ui) on the stability
of the closed-loop manufacturing system.
Since the act.ual control law implemented is Ud" we substitute Ui in (5.25)
by Ud.' Hence, . finally becomes
B
.c = Lee. [Wl;SOi(Xi, Ui)Ui - Wl;SOi( UdWi(Xi, Ui, Ui)]
i=1
B
+ Lee. [W[;Sli(Ui) + Ci(Xi, Ui, Ui)] . (5.29)
i=1
After choosing (5.20)-(5.23), (5.25) and (5.27), .c becomes

B
.c = L [-ie~. - ee. Wl;SOi(Xi, Ui)Wi(Xi, Ui, Ui) + ee;Ci(Xi, Ui, Ui)] ,
i=1
B
: :; L [-il eciI 2 + lee.1 (wciilwi(xi, Ui, ui)1 + ti)] :::; 0, (5.30)
i=1
provided that lee, I > wpl;IW'(X'~il.,U;)I+f'. The above implies that the control
error ee. possesses a uniform ultimate boundedness property with respect
to the arbitrarily small (since i can be chosen arbitrarily large) set [c.
{ee. (t) : lee.1 :::; w;;Jw,(x'~il.,u;)l+f', i > 0 }.
Theorem 5.3.1. Consider the system {5.13}. The scheduling frequency

Ud. developed from the continuous control law {5.20}-{5.23}, {5.25} and {5.27}
guarantees the uniform ultimate boundedness of ee, with respect to the arbi-
trarily small set [ C . = {e c,.(t) : Ie c,.1 <
J -
wpl;IWi(Xi,il"u,)I+f'
"y '
"V>
I
o}.
Theorem 5.3.1 demonstrates practically that the actual scheduling frequency

also possesses the boundedness property obtained by the continuous-time
con t ro 11 er. The t erm'
W;;IWi(Xi,fii
'Y
,,,,;)I+<i serves as a per .m d ex wh'ICh
ormance
can be improved mostly by allowing the design constant 'Y to admit larger
values. Moreover, better approximation of the continuous control law by the
actual scheduling will lead to smaller values on Wi(Xi, 11i, Ui) which in turn
will further improve the overall performance.
lid x
FMS
Continuous u Discrete
xI Dispatching ~
Controller
Decision
xI
Fig. 5.6. The overall control scheme
5.4 Simulation Results
In this section the design methodology developed herein, is illustrated through

simulations performed on a realistic example of a manufacturing cell. Com-
parison with existing scheduling policies, specifically, Clear A Fraction (CAF) ,
and Clear Largest Buffer (CLB) is also provided.
The example considered consists of five machines mi, i = 1,2, ... ,5
and produces five different part types. Owing to the single assembly process,
six routes, given in Table 5.2, are defined. Of them, routes 2, 3, 4, and 5
lead directly to finished products. Routes 1 and 6 lead to part types which
are assembled into a new product. The assembly operation is performed on
machine 5. The elements of Table 5.2 show the order in which every product
visits the machines. The problem posed requires a number of products to be
Table 5.2. Part types routes
Route ffil ffi2 ffia ffi4 ffis
1 2,4 3 1 5
2 1 2
3 1 2 3
4 2 5 1,4 3
5 3 2 1
6 2 1 3 4
produced according to the predefined routes. In detail, all buffers are assumed
to be empty initially. A production demand of20 parts for each of the five-part
types, has to be achieved. Additionally, for all intermediate buffers a desired
state of one-part has been set. From Table 5.2, a new table is constructed,
describing the number of different products each machine may operate on, or
equivalently, the required number of submachines. Also, the operation times
of all submachines are included where, for simplicity, the operation of all
submachines of the same machine is considered of equal duration. The number
Table 5.3. Number of submachines & Operation Times
Machine Routes served No. of submachines Time required
1 2,4,5 4 5
2 1, 2, 3, 4, 5, 6 7 6
3 1,3,4,6 5 5
4 1,4,5,6 4 4
5 1,3,6 2 3
of submachines for the second machine equals seven instead of six, since it
serves part type 1 twice. The same holds for machine 3, where part type 4 is
also served twice, leading to a total of four submachines. However part types
1 and 6 share a common submachine on machine 5, owing to the assembly
nature of the operation, thus leading to a total of two submachines required.
After all, a total of 21 submachines and an equal number of controllers is
required. Furthermore, we consider that raw materials arrive in the cell at a
rate of 0.03 parts per time unit, implying that for each route a raw material
arrives every 33.33 sec, and is stored in the raw material buffer. All buffers
in the cell, both those storing raw materials and semifinished products, are
considered to have 15 parts capacity. Raw materials arrive at the system at
the specified rate, for so long as the production of the respective product
is not complete, and the buffer storing raw materials has not reached its
capacity. For a buffer that has reached its capacity limit, a raw material
can be brought into it again, only after the succeeding submachine issues a
dispatching command and thus decrements its state.
For comparison purposes, we have simulated the application of two con-
ventional scheduling policies performed on the same cell structure. The poli-
cies used are described next:
Clear Largest Buffer (CLB): Select at time t, submachine s* to work for which
the following holds:
x;(t) 2: xs(t)V submachine s E O(m), where xs(t) is the level of the input
buffer of submachine s of machine m at time t.
Clear A Fraction(CAF): Select at time t, submachines s* to work that sat-
isfies x;(t) 2: fE~J;n) x.(t) where N(m) is the number of submachines of
machine m. The constant f has been selected equal to 1/6.
Note that the policies actually employed are variations of the above def-
initions, which incorporate the following modifications for stability reasons:
1) Selection is restricted to the set of submachines that participate in part
type routes for which production has not been fulfilled. 2) All dispatching
commands for submachines whose output buffers have reached their capacity
are ignored.
Before proceeding to the presentation of the simulation results, some
comments on the employed controller strucure are made: Total of eight and
four high-order terms for the SOi(Xi, Ui) and Sli(Ui) terms respectively are
used for each of the 21 controllers. All sigmoids share the same properties
jJ= = =
I A 1; thus, they are positive for any positive argument. The initial
weight values selected are 14 and -10 for the elements of the WOi and W1i
vectors respectively. The WOi and Wli projection-ball radius has been set
equal to 3. Concerning the estimation of wit w
and li so that (3.12) holds, we
require that 0 < wo; ~ Iwl;sOil ~wit. Since both SOi and WOi are positive,
the maximum (minimum) of Iwl;sOil is achieved only ifthe elements of both
vector arguments are maximum (minimum). The boundary values of the sig-
moidal vector are given by 0'[1,0',0'2,0'3] with 0' = J.l + A and 0' = J.l/2 + A
respectively. Furthermore, by continuity we conclude that the boundary val-
ues of WOi are obtained on 8WOi. Hence, the minimum and maximum values
of wl; SOi (Xi, Ui) can be easily computed as the solution of the quadratic op-
timization problems wo; ~ minwoi {Wl; [0',0'2, 0'3f :0' = (I + A)} subject
to WOi E 8WOi and wit 2: maxWOi {Wl; h 0'2, 0'3, 0'4f :0' = (jJ + A)}, sub-
ject to WOi E 8WOi. All Ai, At in (5.28) have been taken as equal to unity.
In Tables 5.4 5.5 & 5.6, a number of simulation results are summarized.
Employment of the RHONN scheduler fulfills the production demand in the
Table 5.4. Simulation results
Methodology Makespan L max Xi L max x 0
RHONN 963 16 100

CAF 1044 80 107
CLB 1044 85 106
Table 5.5. Simulation results (continued)
Methodology Average Average Average Average

WIP Inventory Backlog Lead time
RHONN 487.56 7867.5 4.485 215.960

CAF 1406.3 11325 2.735 142.763
CLB 1244.5 12420 2.315 125.720
shortest time interval (963 time units) compared with that required by the
conventional policies CLB and CAF which take 1083 and 1044 time units
respectively. Deviation of the achieved results from the unknown optimal so-
lution can be illustrated by deriving a lower bound for the achievement of
the production demand. This bound is closely related to the largest work
that has to be done by the cell bottleneck machine, i.e., machine 2. Specifi-
cally, its seven submachines must process 20 parts requiring 7 x 20 x 6=840
time units. As at time 1/0.03 the first raw material arrives in the cell and
a lower bound of 873.33 time units is derived. Thus, the RHONN method-
ology achieves the production goal with a deviation of 10.31 % from the
lower bound. Simulations with higher raw material arrival rates which pro-
vide higher raw material availability result in a makespan deviating less than
6% from a calculated lower bound.
An overview of the peak buffer states experienced in the intermediate
buffers is presented by means of the sum of the maximum state of every
intermediate buffer. For the case of the RHONN controller this sum equals
the number of intermediate buffers in the cell since the algorithm assigns a
target value of 1 for all intermediate buffers that cannot be exceeded owing
to the control law formulation. In the case of the conventional policies this
sum is five times higher, thus resulting in an increase of WIP cost.
The next column gives the sum of output buffer states which again shows
a precise achievement ofthe production goal for the RHONN scheduler, while
in the case of policies an excess of parts have been produced owing to the
clearing nature of the policies, thus resulting in increased inventory costs.
The next two columns represent cost measures of storing parts in buffers.
The cost for holding Xj parts at a buffer for T time units can be considered
equal to kjxjT, where kj is the cost of storing a single part in buffer i, per time
Summary 163
unit. Integrating the above quantity over the entire interval that production
lasts, yields the cost paid for storing parts in a single buffer. Hence, assuming
for simplicity, k j = 1, the cost measure evolves into a plain integral of the
respective buffer states. Specifically, measures for the Work In Process and
Inventory costs experienced in the cell, are provided by means of the average
integral of the intermediate and output buffer states l . Thus, a more efficient
WIP cost control is achieved by the RHONN algorithm that manages to keep
the intermediate buffer state as small as desired while guaranteeing system
stability. Inventory cost is also comparatively smaller than those deriving
from the employment of the conventional scheduling policies owing to the
overall faster fulfillment of the production demand.
The results concerning the achieved average lead times and average back-
logging costs over the entire cell are seemingly contradictory to the most
promising results thus far presented, since the RHONN algorithm yields av-
erage values notably worse than the conventional schedulers. Actually, this
result originates in the great variance of the lead times (i.e., the time elapsing
between the input of a raw material in the cell and the output of the appro-
priate final product), which is clearly demonstrated in Table 5.5. Concluding,
it must also be mentioned that considering its implementation, the RHONN
scheduler constitutes a real-time algorithm. Simulation of almost 1000 min-
utes cell operation took about 15 min on an Ultra 1 Sun WorkStation, while a
hardware implementation of the algorithm is expected to be characterized by
negligible computational delays, yielding a distributed real-time non-myopic
robust controller structure.
Table 5.6. Average product lead times
Methodology Prod 1 Prod 2 Prods Prod 4 Prods
RHONN 77.2735 429.8435 18.9520 485.2107 68.5207

CAF 313.8690 69.6500 74.1600 135.6500 120.4840
CLB 183.2500 183.2500 103.7630 147.9500 108.0860
Summary
In this chapter, the manufacturing cell scheduling problem is considered to

be a control regulation problem, where system states (buffer levels) have to
be driven to some prespecified values (production requirements) by means
1 The values ir, Lx,JoT xidt and iro Lxo

JoT xodt where Ni = 16, No = 5 are the
number of intermediate and output buffers in the cell and T is the production
makespan yielded by each scheduling methodology.
of control input commands. A nonlinear-neural network controller for on-

line scheduling of small manufacturing cell processes is developed. Stable
adaptive laws are derived, guaranteeing convergence of the control error to
within an arbitrarily small neighborhood of the origin, as well as stability and
boundedness properties. Comparison, through simulations, with well known
scheduling policies, verified the promise of the method.
CHAPTER 6
SCHEDULING USING RHONNS: A TEST CASE
In Chapter 5, the non-acyclic FMS scheduling problem was considered to be a

control regulation problem, where system states (buffer levels), have to reach
some prespecified production requirements, by means of control input com-
mands. Based on a recurrent high-order neural-network model of the buffer
states, an adaptive continuous-time neural-network controller was developed.
Stable control and update laws guaranteeing system stability, boundedness
of all signals in the closed-loop system and a uniform ultimate boundedness
property of the control error were derived. Dispatching commands were issued
by means of a discretization process of the continous control input. Further-
more, modeling errors and discretization effects were taken into account, thus
rendering the controller robust and capable of driving system production to
the required demand.
In this chapter we are seeking to evaluate the neural network scheduler,
developed in the previous chapter, further by applying it on real data de-
rived from a challenging manufacturing system. The selected test case, the
mechanical workshop of a German company, constitutes an existing manu-
facturing cell and can be viewed as a highly complex non-acyclic FMS with
extremely heterogenous part-processing times, where 18 product types that
may visit 18 different machines have to be produced. The performance of the
algorithm is compared with modified versions of the well-established conven-
tional scheduling policies, First In First Out(FIFO), Clear A Fraction (CAF)
and Clear Largest Buffer (CLB).
All schedulers are compared over a range of raw material arrival rates
and their performance evaluated by means of the observed makespan, work
in process (WIP), Inventory, backlogging costs and average lead times. For
extremely low rates, all schedulers obtain approximately the same perfor-
mance owing to the phenomenon of extremely long cell idling. Moreover,
for higher rates, the RHONN scheduler achieves production in considerably
shorter time than all employed policies and presents superb performance in
issues of manufacturing system stability and low WIP. It also yields minimum
backlogging and Inventory costs and considerably lower average lead times.
Constant increase of arrival rate leads the RHONN scheduler to a state of
minimum achievable makespan and low costs while simultaneously, the per-
formance of all conventional schedulers deteriorates, yielding an increase in

166 6. Scheduling Test Case
the selected cost measures. Thus, the derived characteristic simulation results
establish the proposed scheduler's applicability on the control of non-trivial
manufacturing cells.
6.1 Test Case Description
To investigate, assess and establish the applicability and value of the pro-
posed approach to the scheduling problem, it has been considered necessary
to examine data taken from existing manufacturing systems, with all the
constraints, and complexity inherent in real-world cases. Thus an existing
industry has been chosen, and the algorithm's performance has been tested
on production processes crucial for the industrial operation.
6.1.1 General Description
The test case is derived from one of the oldest industrial enterprises in Ger-
many, SHW, of turnover of 400 Million DM per year, subdivided into 17
profit centers and employing more than 2000 men in 1994. Specifically, the
manufacturing system paradigm examined is part of the Machine Tool Divi-
sion, employing 120 people. Products are tools ranging from simple milling
machines to universal milling, boring and turning centers.
The production program of this division consists of 16 basic types of
products and although certain types of milling machines are produced regu-
larly, the majority of them are produced on demand so production must be
organized flexibly, to meet a number of customer requirements. Hence, con-
structed machining tools are made up of modules of a standardized building
block system, such as base, guideway, tool changer, control system, etc.
The layout of the machine tool division is pictured in Figure 6.1. It consists
of a warehouse for storing raw materials, a sawing section where material
preparation begins, a mechanical workshop where in-house preparation of
parts takes place, a painting department, a preassembly section for assembling
the parts of the machine tool modules and a final assembly and inspection
workshop where the whole machine tool is finally assembled.
The Mechanical Workshop of the machine tool division was chosen as the
test case upon which the applicability of the RHONN scheduler is investi-
gated.
6.1.2 Production Planning & Layout in SHW
Since in machine tool division, production consists of regularly manufacturing

simple milling machines in small lots while the majority of products are
constructed on demand, production organization in the entire division and
consequently in the mechanical workshop, depends strongly on the orders
6.1 Test Case Description 167
0-;'11111
....... g:::qpw: =::::a ..;..;: : .... _ .......;. #--;; _ % = 444gb eeA
= = = = = =
Fig. 6.1. Layout of the Machine Tool Division
arriving in the industry and the processing strategies adopted. Hence, before
proceeding to the detailed scheduling problem presentation, in order to obtain
a more thorough view of the industrial reality and the production planning
techniques employed and therefore, to evaluate and assess the vitality of the
scheduling task better, the basic steps of order processing are presented.
Order Definition: Initial cost estimation, long term schedule and deadline
determination for the various manufacturing steps, take place.
Design Phase: Product's main modules are defined, (i.e., it is decided which
parts will be supplied by external vendors and which will be built in-house).
The latter, are designed in the Mechanical Design Department and for each
part a work plan that defines all the necessary manufacturing operations is
generated by listing for every operation its name, the machine identification
number where it will be performed and the estimated production time.
Similar definitions are made for electrical tool components, that are to be
handled at the electrical design section. Hence, a number of documents
such as drawings, list of modules, work plans, etc., are outputted at this
order-processing phase.
Disposal f3 Purchasing: Ordering and quality inspection of raw materials
and parts supplied externally takes place.
Part Manufacturing performed in the mechanical workshop. The scheduling
problem that is to be solved by the RHONN scheduler is exactly this
one. In current industrial practice, the master of the mechanical workshop
determines medium-and short-term scheduling, when part drawings reach

the workshop, without any computer support, while the work plans are
stored in an information system.
Assembly: Main modules are assembled from the complete parts, essentially
in a manual manner, strongly dependent on human skills.
Electrical Installations: Since electrical components are manufactured in
parallel to the mechanical ones, they are added to the assembled machine
tool.
Quality Control fj Delivery: Final inspection is performed.
The above discussion naturally establishes the critical role and vitality
of the mechanical workshop for the economic success of the Machine Tool
Division.
Considering the workshop in greater detail, it consists of a total of 22 ma-
chines of 20 different types. The sequential numbering, identification numbers
and brief descriptive labels of the workshop machines are given in Table 6.1.
Part manufacturing takes place when the order documents describing the
machine tools manufacturing steps, which are outputted by the design de-
partment of the company, reach the workshop. Thus, scheduling policies are
employed to execute the numerous work plans corresponding to the pending
orders that reach the workshop.
6.1.3 Problem Definition
Every machine can be considered as a large flexible machine, capable of pro-

cessing as many different part types, as order operations are executed in it. As
hundreds of orders and multiple order steps are pending in the manufacturing
environment at any time, the mechanical workshop appears to be a FMS of
extremely high complexity and dimension. Any scheduling algorithm should
handle an excessive number of orders, with varying processing requirements
and operations complexity, manufacturing and delivery constraints, in an en-
vironment not fully automated, where the human factor imposes additional
difficulties. Also, different order numbers may correspond either to different
or to the same desired final product type.
Since the original scheduling problem is one of overwhelming complexity,
viewing each machine as a FMS, yields flexible machines capable of operating
on tens of different orders. Thus, some means of simplifying the original
problem in order to obtain a more tractable and to a large extent equivalent
one, has to be employed.
Examination of the order set showed that a major subset of the available
orders actually demands a number of the same final products. Hence, the
following technique was adopted for defining the scheduling problem.
First, a subset of the entire set of orders that was susceptible to classi-
fication according to the desired final product type was selected. Then the
chosen orders are grouped by the type of the final product demanded. Each
Table 6.1. The mechanical workshop machines
No Machine ID Name
1 704 Parts inspection

2 707 Tool storage and pre-adjustment
3 708 Sawing
4 712 Marking
5 718 Large planing machine
6 722 Jig drill machine DIXI
7 727 Milling machine UFZ4
8 728 Milling machine UFDZ1200
9 729 Milling machine UF21
10 731 Milling machine UFZ6 (old)
11 731 Milling machine UFZ6 (new)
12 732 Manual drilling machine RABOMA
13 736 Circular grinding machine SCHAUDT
14 737 Surface grinding machine BLOHM
15 739 Large surface grinding machine WALDRICH
16 752 Manual turning machine WEPERT
17 753 CNC turning machine NILES
18 768 Small planing machine
19 769 Broaching
20 773 CNC boring center AXA
21 774 Tool storage and pre-assembly for NILES(16)
22 775 Deburring
such group of orders with identical processing steps, is replaced by a unique

order, demanding a number of finished products equal to the sum of the
lot-sizes of the orders in the group from which it is derived. Consequently, a
number of work plan entries, defining the processing steps necessary for the
production of the group part type and requesting the production of a number
of parts equal to sum of lots, emerges from each such group of orders.
This process finally led to the following formulation of the scheduling
problem. Achievement of production of a number of parts for each of the 18
different final product types.
6.1.4 Manufacturing Cell Topology
The different product-type routes are shown in Table 6.2. All numbers under
the label "Processing Route" are identification numbers of mechanical work-
170 6, Scheduling Test Case
shop machines and the assembly points external to the workshop visited.
In the tabulated set of part type routes, 18 different types of products are
produced in this cell of 18 processing locations, following non-acyclic routes,
while no assembly operations are involved. Each entry of the above table
represents a processing step of a specific duration. In Table 6.1, the (i, j)
entry being equal to m, means that the i-th product in its j-th processing
step, visits machine with label m. The specific submachine will be referred
to as Bi,j and controls the state of a buffer attached to its output which will
be noted as Xi,j' The output signal of the neural controller regulating the
state of the Xi,j buffer (i.e., the frequency signal controlling the operation
of submachine Bi,j), will be denoted as Ui,j' Also, the first submachine of
every product route, Bi,1> i = 1,2"",18 is preceeded by a buffer that stores
raw materials of the respective i-th product temporarily and which will be
referred to as Xi,O. Furthermore, L( i), i = 1,2, ' , " 18 will denote the number
of processing steps in route i, thus Xi,L(i) will be the output buffer collecting
the finished products of route i. Also, Xt(i,j) will stand for the target state of
buffer xi,j, so Xt(i,L(i)) will denote the demand for product type i. Any further
information necessary for the complete definition of the scheduling problem
will be presented with direct reference to the cell topology, as provided in
Table 6.2. Observe that a number of machine labels that are not contained
in the Table 6.1 are recorded in the production routes listed. These labels,
namely 777, 902, 966, 999 stand for processing steps performed outside the
workshop, at other departments of the Machine Tool Division and usually
involve assembly operations that occur at some intermediate steps of a part
process in the workshop. The duration in minutes of all processing steps un-
derlying the presented cell topology is tabulated next, where the value T( i, j)
of its (i, j) element stands for the duration of the (i, j) processing step.
The scheduling problem is completely defined by specifying the desired
target states, the capacity and the initial state values, for all buffers in the
system, as well as the rates under which raw materials for the different part
types arrive in the system. Specifically, all intermediate buffers' target states
have been set equal to one and their capacity values equal to nine-parts, with
the exception of buffers X8,1, X12,1, X13,1 where three-parts capacity is allowed.
All buffers, are considered to be empty initially. Raw material buffers have
two-part capacity, and when this capacity is reached, arrival of raw materials
is ceased until the raw material buffer level is decreased.
The formulated problem constitutes a reasonably complicated one, mainly
owing to the diversities and variability of part-processing times in the various
submachines, the inequalities in machines' workloads and their dimensions
as it involves 18 different final product types, visiting 18 machines, implying
the existence of 95 submachines, in the neural-network-scheduling algorithm.
The most extreme differences exist for the twelfth product type 2783, where
the processing times in the 11 submachines visited range from 8.4 to 1539
minutes.
Table 6.2. Manufacturing Cell Topology
Order No Order Type Processing Route
1 1331 737,753,773,704,737,704
2 1414 753,773,704
3 1623 753,773,704
4 2409 753,999,773,704
5 2420 753,999,999,736,704
6 1953 731,704,999,773,739,737,704,775
7 2422 773,783,704
8 2685 783,704
9 2766 708,731,774,739,783,783,774,704
10 3057 999,753,999,773,775,999,737,999,999,704
11 3061 753,966,773,736,704
12 2694 728,732,722,783,704,783
13 2783 732,752,731,777,999,777,731,722,783,783,704
14 2900 773,704
15 2904 966,966,704
16 1312 704,775,732,783,704
17 2916 753,773,704
18 3207 999,999,753,773,999,999,999,704
6.1.5 RHONN Model Derivation
In order to gain further understanding of the underlying neural network con-

troller structure, a subset of the interconnections of the inherent sub machine
topology is described next. Following the notation introduced in Chapter 5,
let O(m) be the set of submachines Si,j of machine m and N(m) its cardi-
nality. Let B+(s) be the set of input buffers to submachine s, B_(s) be the
set of output buffers of submachine s, and M_ (b) the set of submachines fed
by buffer b. For the specific test case we obtain:
B+(Si,j) = {Xi,j-1}, 'v'i,j,
B_(Si,j) = {Xi,j} 'v'i,j
For example, close examination of the cell definition yields the set of subma-
chines related to machine 773,
0(773) = {Sl,3, S2,2, S3,2, S4,3, S6,4, S7,1, SlO,4, Sl1,3, S14,l, S17,2, S18,4}.
Moreover, the sets that define the route of the eleventh product type com-
pletely are given below:
Sets of input buffers:
Table 6.3. Processing Steps Duration
Order Part Processing Time (in minutes)
1331 31.0, 25.2, 30.6, 20.0, 9.4, 20.0

1414 23.4, 15.6, 20.0
1623 21.0, 15.6, 20.0
2409 43.2, 3.0, 27.0, 20.0
2420 34.2, 0.5, 3.6, 34.2, 20.0
2422 16.8, 1.2, 20.0
2685 2.4, 20.0
2766 7.7, 1179, 244.2, 348.0, 46.8, 25.2, 111.6, 20.0
3057 20.0, 37.2, 20.0, 46.0, 20.0, 20.0, 14.4, 61.2, 90.0, 20.0
1953 162, 20.0, 20.0, 119.0, 72.0, 65.0, 20.0, 20.0
3061 22.8, 153, 18.2, 45.0, 20.0
2694 876, 43.4, 990.0, 36.0, 20.0, 1.8
2783 14.2, 351, 444.0, 8.4, 20.0, 8.4, 1260, 1539, 10.8, 12.6
2900 24.0, 20.0
2904 1.2, 6.2, 20.0
1312 20.0, 20.0, 11.0, 1.2, 20.0
2916 22.8, 25.8, 20.0
3207 20.0, 20.0, 22.8, 19.8, 3.6, 4.8, 24.6, 20.0
B+(Sl1,1) = {X11,a}, B+(Sl1,2) = {x11,d, B+(Sl1,3) = {X11,2},

B+(Sl1,4) = {Xl1,3}, B+(Sl1,5) = {Xl1,4},
Sets of output buffers:
B_ (Sl1,1) = {xl1,d, B-(Sl1,2) = {X11,2}, B_ (Sl1,3) = {Xl1,3},
B-(Sl1,4) = {Xl1,4},B_(Sl1,5) = {Xl1,5}.
Finally, the sets of submachines fed by each buffer in route 11 are:
M-(X11,a) = {sl1,d,M_(xl1,1) = {sll,d,M_(X11,2) = {Sll,3},
M-(Xl1,3) = {Sll,4}, M-(Xl1,4) = {sl1,d, M-(Xl1,5) = {0}.
Taking up to fourth-order terms for each argument, the dynamic equations
employed for the buffers of the eleventh product route, are:
Xl1,1 = Wt,l1/ [S(Xl1,1f, S( Ul1,1f( Ul1,1

+Wt,11,1T S(U11,2) + C:(X11,l. Ul1,1, U11,2) ,
Xl1,2 = Wt,l1,/ [S(Xl1,2f,S(Ul1,2f( Ul1,2
+ Wi': 11 ,2 T S( U11,3) + (Xl1,2' Ul1,2, U11,3) ,
Xl1,3 = Wt,11,3 T [S(Xl1,3)T, S( Ul1,3)Tf U11,3
+Wi':11,3T S(U11,4) + (X11,3, U11,3, U11,4) ,
Xl1,4 = Wt,11,4T [S(X11,4f,S(U11,4ff U11,4

+ Wi': 11 ,l S( U11,5) + ( Xl1,4, Ul1,4, Ul1,5) ,
X11,5 = wt,l1,l [S(Xl1,5f,S(Ul1,5f( Ul1,5
+(X11,5, Ul1,5) ,
with
S(Xi,j) = [S(Xi,j) S2(Xi,j) S3(Xi,j) S4(Xi,j )]T ,
S(Ui,j) = [S(Ui,j) S2(Ui,j) S3(Ui,j) S4(Ui,j)f ,
Sl(Ui,j) = [S(Ui,j) S2(Ui,j) S3(Ui,j) S4(Ui,j)]T ,
where Wt,j E lR8 Vj E {l, 2, ... , 5} and Wt,j E lR4 Vj E {l, 2, ... , 4}.
The above neural network model is used further in the development of
the RHONN scheduler, as indicated in Chapter 5.
6.1.6 Other Scheduling Policies
A set of conventional scheduling methodologies has been selected for com-

parison purposes. The methods employed are the following:
First In First Out (FIFO): Select submachine Si,j whose input buffer Xi,j-I,
contains the part which arrived earlier than all parts contained in the input
buffer of any other submachine in O(m).
Clear Largest Buffer (CLB): Select si,j such that Xi,j_l(t) 2: Xi,j-l(t)
VSi,j E O(m).
Clear a fraction (CAF): Select si,j such that
N(m)
Xi,j-l(t) 2: f L Xi,j-l
j=l
Clearing methods process all parts of the selected buffer until it is empty.
It should be noted that the policies actually employed are variations of the
above definitions, which incorporate the following modifications, for stability
reasons:
All dispatching commands for submachines whose output buffers have
reached their capacity are ignored.
Selection is restricted to the set of submachines that participates in part-
type routes for which production has not been fulfilled.
The first modification concerns canceling the application of the current

dispatching command for a specific submachine, whenever the correspond-
ing buffer reaches its capacity limit. To illustrate and clarify the purpose of
this modification, we have considered the case where the cell operates under
the original scheduling methodologies (FIFO, CAF, CLB) while various raw
material arrival rates are also allowed.
The results are presented in Table 6.4 in which, under column Ii is reg-
istered the time at which the violation of a buffer's capacity when applying
policy t, together with the number of finished products thus far produced
(i.e., the sum of output buffer states L::i
Xi,L(i)' The above violation obvi-
ously occurs whenever the raw material arrival rates are higher than 0.0008.
Application of only the first modification though, does not provide solu-
tion to another serious issue caused by the phenomenon of overproduction
of specific product types. The above leads to a surplus of completed prod-
ucts for certain product types and simultaneously, to non achievement of the
targets for others. Furthermore, the cell often reaches a state where inappro-
priate dispatching commands are issued and production is driven to complete
paralysis. These phenomena are demonstrated in Tables 6.5 and 6.6 where
the sum of all items of all product types produced, the corresponding pro-
duction time and the sum of all items left to be produced for all product
types for which production demand has not been fulfilled are presented, for
all scheduling policies (FIFO, CAF, CLB), under the first modification, and
with various raw material arrival rates. In this table Ep m stands for the
sum of all items of all product types produced when employing policy m
and is equal to L::;!1
Xi,L(i) and EU m stands for the sum of all items left
to be produced for all types when policy m is employed, which is equal to
L::;!1 f(Xi,L(i) where
f( x. .) -
"L(,) -
{Xt(i,L(i - Xi,L(i) if Xt(i,L(i
otherwise
< Xi,L(i)
The 00 in the production time column is used to denote the transition of the
cell in a state of overall production paralysis or of continuous processing of
specific routes
From the above discussion, it becomes apparent that the second modifica-
tion has been employed to overcome such problems, by restricting production
scheduling to only those product types for which production has not been ful-
filled.
6.2 Results & Comparisons
In what follows, we shall discuss the results obtained when demanding 15, 25,
20, 20, 25, 20, 20, 3, 10, 8, 15, 5, 2, 20, 20, 15, 10, 20 parts for the 18 product
types respectively, beginning from clear intermediate and output buffers.
6.2 Results & Comparisons 175
Table 6.4. Simulation Results for Schedulers Free from Modifications 1 & 2
Rate Tfifo L i Xi,L(i) Tclb LiXi,L(i) Teaf LiXi,L(i)
0008 30115 273 30115 273 30112 273

0010 6007 180 10533 161 10533 161
0015 3340 75 6365 147 6365 148
0020 2507 73 2507 73 2507 73
0025 1607 55 2007 69 2007 71
0100 407 15 307 13 307 12
0300 107 3 107 4 107 3
Table 6.5. Simulation Results for Policies Incorporating Modification 1
Rate EPcaJ Teaf EUcaJ Epclb Tclb EUclb
0.0008 283 30113 0 283 30116 0

0.001 278 00 7 285 43756 0
0.0015 283 00 7 283 00 7
0.002 286 00 7 274 00 9
0.0025 282 00 7 276 00 9
0.01 356 00 8 348 00 11
0.03 289 00 73 101 00 199
Table 6.6. Simulation Results for Policies Incorporating Modification 1 (Contin-

ued)
Rate EPJiJo Tfifo EUJiJO
0.0008 287 30115 0

0.001 284 24116 0
0.0015 289 32100 0
0.002 292 00 1
0.0025 292 00 1
0.01 270 00 54
0.03 248 00 77
More precicely, in Figure 6.2 the makespan (i.e., the time period elapsing
until the production goal is achieved for all product types) is plotted for
various raw material arrival rates.
r
x 10'
G----l FIFO
4
~I NN
!1'- ------
(I_ . _ . Q CLB
)too - - )( CAF
- - -- - - - - -- -
-x_ - - ___
3.5 'f
----J+-----
+f
II
I
"'
!!
"
c::
~2.5
3
t
....,"'os cj
:;
2
1.5
t1
.. . . ..
o 0.005 0.01 0.015 0.02 0.025 0. 03 0.035

Raw material arrival rate
Fig. 6.2. Production makespan
Examination of this set of curves reveals that generally for all raw mate-
rial arrival rates the proposed neural network scheduling methodology results
in significant speedup of the achievement of production demand. Considering
the conventional but modified schedulers, the production goal achievement
requires longer time intervals. Owing to the potential instability of the se-
lected policies, the inability of CLB to terminate is observed for rates greater
than or equal to 0.002 parts per minute. Thus, in this and in all subsequent
figures, CLB values are plotted only for the range of rates for which the pro-
duction goal is achieved, (i.e., for rates less than 0.002). For low raw material
arrival rates 1 we observe similar performance for all the schedulers that were
compared. This originates from the fact that for lower raw material arrival
rates, makespan is dominated by the time required for the entrance to the
cell of all raw materials for the product types with the highest demand. It
has been assumed that initially, all raw material buffers contain a single part.
Since a maximum demand of 25 parts has been defined, and one part arrives
1 Rates lower than 0.0027 are defined as low, while greater than 0.0027 as high
every r;te minutes, it is easily derived that 24 x r;te

minutes must elapse
before all necessary raw materials have entered the system.
For the case of high rates, it is necessary, to extract a lower bound for
the corresponding makespan for comparison purposes. An obvious one should
be the maximum over all machines of the sum of the duration of the mini-
mum work that should be done by all submachines of each machine, plus the
minimum time required for a single part to reach machine input. The above
maximum is calculated under the assumption that no machine is idling. The
minimum work that any submachine in the cell should do is equal to the
corresponding product demand. Thus, the time required for the minimum
work of machine m is
Tm = L Ti,j x Xt(i,L(i))'
si,jEO(m)
In the specific test case the lower bound has been found equal to 8947.3 min.
The neural network methodology achieves the production goal in 9778
minutes, thus obtaining a deviation from the lower bound equal to 9.29 % for
raw material arrival rates equal to or greater than 0.0027. Optimal perfor-
mance for the conventional schedulers FIFO, CAF and CLB occur at much
lower rates, specifically for 0.0018, 0.0009 and 0.0009 parts per minute, where
the lower bounds are provided by the formula 24 x ~ with corresponding devi-
ations 0.867%, 1.865%, 1.87% and makespans 13449, 27164 and 27165 minutes
respecti vely.
Simulations with higher raw material arrival rates which provide higher
raw material availability, resulted in no change of the makespan for either the
neural network or FIFO schedulers, while analogous performance is yielded
by CAF. Figure 6.2 also reveals the superior robustness of the neural network
algorithm with respect to variations in the raw material arrival rates.
An overview of the peak buffer states experienced by the intermediate
buffers is presented by means of the sum of the maximum state of all inter-
mediate buffers, which is plotted in Figure 6.3. For the case of the neural
network scheduler, this sum is equal to the number of intermediate buffers in
the cell (i.e., 77) since the algorithm assigns a target value of one for all in-
termediate buffers, which cannot be exceeded as the control law formulation
enforces the control signal to become zero whenever a buffer state reaches its
target value.
When the conventional policies are used, this sum becomes considerably
larger and as can be seen from Figure 6.3, it can be more than five times
higher in the case of CAF. Specifically, for very low rates, this sum slightly
exceeds the limit of 77 but a slight increase in the rate results in a consid-
erable increase in the sum of the maximum states that intermediate buffers
acquire, with the most extreme case that of CAF at 0.01 parts per minute.
The neural network scheduler provides a sufficient control for the maximum
possible states of the buffers. This is not the case for all conventional sched-
ulers studied, which at higher rates can even force the buffers to reach their
capacity limits. Thus, the neural network scheduler ensures the cell's stable
operation while requiring minimum buffer capacities.
FI FO
--
0--------()
400 I
~ NN
0- - (I CLB
I
I
,..--x CAF
I -,... -- - -
350 I
l!?
'"
"S I
S> I
~ 300
'6

.5'": 250 I
E I
~
'x I
'"
E 200 I
15
E
"
(/) I
,
I
150
f.}
o o.oos 0.01 0.015 0.02 0.025 0.03 0.Q35

Fig. 6.3. Sum of maximum intermediate buffer states
Figure 6.4 presents the sum of maximum output buffers states. The
achieved results for both FIFO and NN are identical and equal to the total
demand of 273 parts for all product types. This sum acquires slightly larger
values in the cases of the other schedulers, thus denoting the production of a
surplus of parts. This excess of parts originates in the clearing nature of the
employed policies (i.e., CAF, since CLB is not functioning for rates greater
than 0.002).
At low rates, all schedulers result in precise achievement of the production
goal, owing to the fact that all operations, even the clearing ones, empty
buffers containing very few parts, causing submachines that output finished
products, to process buffers which contain a single part only.
Both FIFO & NN emerge as schedulers guaranteeing accurate achieve-
ment of production.
The next two figures, represent cost measures for storing parts in inter-
mediate and output buffers respectively. The cost for holding Xi,j parts at
a buffer for T time units can be considered equal to ki,jXi,jT, where ki,j is
the cost for storing a single part in the output buffer of sub machine Si,j, per
time unit. Integrating the above quantity over the entire makespan, yields
G------El FIFO
320 NN
0- - 0 CLB
,... - -x CAF
310
_...K-- ---
.,
~
---
'S
~300 I
,..--
il.
8 I
I
.1290 I
I
"0
E
" 280
(J)
I
I
270
2600L-- --0....LOOS---O--'.OL1---0-.0.Ll-S-- -0-'.0-2- - -0-

.0L2S---0.- '-03------'0
.03S
Raw male rial arrival rale
Fig. 6.4. Sum of maximum output buffer states
the cost for storing parts in a single buffer. Thus, assuming for simplicity
that for all buffers in the system ki,j = 1, the cost measure evolves into a
plain integral of the respective buffer states. Specifically, measures for the
Work In Process and Inventory costs, are provided by means of the aver-
age integral of the intermediate and output buffer states, (i.e., the values
~i L:~l Lfiil- J::
1 xj,jdt and ~o L:~l
are the number of intermediate and output buffers in the cell and T is the
I::
Xi,L(i)dt where Ni = 77, No = 18
production makespan yielded by each scheduling methodology). Owing to the

low intermediate buffer states guaranteed by the neural network algorithm, a
lower WIP is observed. Considerably larger costs are introduced by the rest
of the schedulers.
Thus, the neural network algorithm achieves the most efficient WIP cost
control, by keeping the intermediate buffer state as small as desired, while
guaranteeing system stability. Inventory costs are also considerably smaller
than those occuring when employing the conventional scheduling policies ow-
ing to the faster overall fulfillment of the production demand.
For lower rates, when total production is achieved almost simultaneously
by all schedulers, deviations between the Inventory costs yielded by the poli-
cies considered, and the one introduced by the NN are not really significant
with the exception of the far worst performance of CLB which is much the
worse but the increase in rates causes the occurrence of a considerably larger
peak in intermediate and output buffer states as well as significant differences
in makespan, which are sufficient justification of the very much greater, WIP
& Inventory costs in the cases of the conventional schedulers. For higher rates
the NN scheduler presents a far more robust-stable behavior when compared
with FIFO and CAF. Thus, NN superiority with respect to these popular
cost measures is obvious.
x 10'
5 G------O FIFO
NN
1)- _ . (z CLB
4.5 ,... - -x CAF
4 ,- - - - -- - --- -
_ __ _ __ __ __ >f----
3.5
I
.'"
Q.
3 I
I
I
25 f
~
Q
2
'ji
*
I
I
1.5
(~
t
~
I
0.005 0.01 0.015 0.02 0025 0.03 0.Q35

Fig. 6.5. Average WIP cost
Considering backlogging costs (i.e., the cost resulting from the delay in
achieving production targets) the NN scheduler reaches the production goal
with the minimum cost when compared with the rest of the policies exam-
ined. Specifically, the backlogging cost employed has been defined as fol-
J J
lows: o Lp 1f;p(t) = o Lp (3p(r)dr where (3p(t) = Ip(t) - Op(t) with f;
Ip (t), Op (t) the number of parts of type p that entered and exit the cell until
time t. As Figure 6.7 demonstrates, backlogging costs remain small for rela-
tively low rates, since buffers in the system contain few parts, less conflicts
among submachines occur and thus finished products are outputed with re-
duced delays. On the other hand, increased raw material arrival rates, leads
to overall increased buffer levels and conflicts, thus making the output of
finished products occur with a considerably larger delay. For any rate, the
NN is considerably less expensive (with respect to backlogging cost) than all
other policies examined.
x 10'
6 G------l FIFO
NN
0- - 0 CLB
5.5 I -"--
)I- --x CAF
'I'
5 I - - - -x- - - - -
. ",'"
4.5 *
+
1.5
o 0.005 0.01 0.015 0.02 0.025 0.03 0.035

Fig. 6.6. Average Inventory Cost
Finally, Figure 6.8 gives the average lead time (i.e., the average of the time
elapsing between the input of a raw material in the cell and the output of
the respective finished product) for all product types and all parts produced
in the cell. Comparing this plot with backlogging cost curves, the impressive
similarity in slope and shape is immediately observed. Actually, all reasoning
about the causes of backlogging cost also applies in the case of product lead
times, where the superiority of the NN algorithm is justified once again by
the smaller production makespan.
Considering implementation issues, the NN scheduler constitutes a real
time algorithm. Simulation of almost 10000 minutes of cell operation lasted
about 20 minutes on a Solaris workstation, featuring a Pentium II/266 MHz
processor. Moreover a hardware implementation of the algorithm is expected
to be characterized by negligible computational delays, yielding a distributed
real-time non-myopic robust-controller structure.
The results presented reveal the capability of the proposed NN scheduler
of achieving scheduling in inherently complex manufacturing cells, proving
that the sound mathematical formulations developed in Chapter 5, constitute
an applicable and worthwhile scheduling methodology. The NN controller
guarantees stability of the manufacturing cells while production is achieved
in a time quite close to the optimal in the minimal makespan sense. Critical
x 10'
4.5~---.-----r----'---"'---r--;:==::::::::::::====~
FIFO
NN
4
1)- -- -* CLB
,)4. ______ _ )II- --x CAF
3.5
,, --- ----...c:-- - --
,,
,
2 ,, ...... ... . " .. ............ . . . ... ... . . .. . , .- . .. . , . . . . . . . .. . .. . . . . . . .
0.01 0.015 0_02 0.025 0.03 0_035

Fig. 6.7. Average Backlogging cost
G-----E> FIFO
NN
2500
,r-- - - - - ________ _ 1)- ' - '*
,..--x
CLB
, ---.----
- - -!-=- -- - - - '
CAF
2000
,,
~
!, 1500 ,, ........................ ............. ,. -..... -
<
0.01 0_015 0,02 0_025 0,03 0.035

Fig. 6.S. Average Lead Time

Summary 183
costs like WIP and Inventory can be efficiently controlled, while backlogging
costs are also kept quite suppressed.
Summary
In this chapter, an evaluation of the proposed neural-network-scheduling al-

gorithm developed in Chapter 5, has been performed. Its application to a
challenging real-world manufacturing cell has been considered and for a wide
range of raw material arrival rates it presented superb performance when
compared with a set of conventional schedulers. With the exception of ex-
tremely low rates, where all schedulers converge to an almost identical perfor-
mance, owing to long idling periods, the algorithm proposed features superb
stability and robustness as well as efficient control of critical costs such as
WIP, Inventory and backlogging, outperforming all conventional schedulers
discussed.
The study has also shown that the uniform ultimate bounded ness prop-
erty that the RHONN scheduler possesses, corresponds to notably low
makespan and that the stable control laws constitute an efficient and robust
control scheme which results in low intermediate and output buffers. The
above characteristics justify the observed minimal cost values and are to be
acclaimed for the invariance of the scheduler's performance to a continuous-
rate increase.
Thus, the derived results, when considered in conjunction with the real-
time property, the linearity of the scheduling problem dimension and the
adaptation to the variation in demand, establish the proposed technique as
a worthwhile and promising approach for the control of manufacturing cells.
REFERENCES
1. Special Issue on Neural Networks, (1988) IEEE Trans. on Acoustics, Speech,

and Signal Processing, vol. 36, no. 7.
2. Albus J.S., (1975) "A New Approach to Manipulator Control: The Cerebellar
Model Articulation Controller (CMAC)," in Transactions of the ASME, Journal
of Dynamic Systems, Measurement, and Control, vol. 97, series G, no. 3, pp.
220-227.
3. Astriim K.J., Wittenmark B., (1989) Adaptive Control, Reading, MA, Addison-
Wesley.
4. Baldi P., (1988) "Neural networks, orientations of the hypercube and algebraic
threshold functions," IEEE Transactions on Information Theory, vol. IT-34,
pp. 523-530.
5. Barmish B.R., and Leitmann G., (1982) "On Ultimate Boundedness Control
of Uncertain Systems in the Absence of Matching Condition", IEEE Trans.
Autom. Contr., vol. AC-27, no. 1, pp. 153-158.
6. Campion G., and Bastin G., (1990) "Indirect Adaptive State Feedback Con-
trol of Linearly Parametrized Nonlinear Systems", Int. J. Adap. Contr. Signal
Processing, vol. 4, pp. 345-358.
7. Chapman S.J., (1985) Electric Machinery Fundamentals, New York, McGraw-
Hill.
8. Chassiakos A.G., Pak A., Ioannou P.A., (1989) "A nonlinear robust adaptive
controller for manipulator tracking," Proc. of the American Control Conf., pp.
1469-1475.
9. Chen F.-C., Liu C.-C., (1994) "Adaptively controlling nonlinear continuous-
time systems using multilayer neural networks", IEEE Trans. Automat. Control,
vol. 39, no. 6, pp. 1306-1310.
10. Chen F .-C,Khalil H.K., (1995) "Adaptive control of a class of nonlinear discrete
time systems using neural networks", IEEE Trans. Automatic Control, vol. 40,
no. 5, pp. 791-801.
11. Chen Y.H., (1992) "Adaptive Robust Control of Uncertain Systems with Mea-
surement Noise", Automatica, vol. 29, no. 4, pp. 715-728.
12. Cohen M.A., Grossberg S., (1983) "Absolute stability of global pattern for-
mation and parallel memory storage by competitive neural networks," IEEE
Transactions on Systems, Man, and Cybernetics, vol. SMC-13, pp. 815-826,
1983.
13. Corless M.J., and Leitmann G., (1981) "Continuous State Feedback Guaran-
teeing Uniform Ultimate Boundedness for Uncertain Dynamic Systems", IEEE
Trans. Autom. Contr., vol. AC-26, no. 5, pp. 1139-1144.
14. Cotter N.E., (1990) "The Stone-Weierstrass theorem and its application to
neural networks," IEEE Trans. on Neural Networks, vol. 1, no. 4, pp. 290-295.
15. Craig J.J., (1989) Introduction to Robotics. Mechanic, and Control, Addison-
Wesley.
186 References
16. Cybenko G., (1989) "Approximations by superpositions of a sigmoidal func-

tion," Mathematics of Control, Signals, and Systems, vol. 2, pp. 303-314.
17. Davis L., (1985) "Job Shop Scheduling with genetic Algorithms", in Proc. Int.
Conf. Genetic Algorithms and Their Applications, Carnegie-Mellon University,
Pittsburg, PA, July 24-26.
18. Dempo A., Farotimi 0., Kailath T., (1991) "High-order absolutely stable neural
networks," IEEE Transactions on Circuits and Systems, vol. 38, no. 1.
19. Desoer C.A., and Vidyasagar M., (1975) Feedback Systems: Input-Output Prop-
erties, N ew-York: Academic.
20. Dolinska M., Besant C.B., (1995) "Dynamic Control of Flexible Manufacturing
Systems," The International Journal of Advanced Manufacturng Technology,
vol. 10, pp. 131-138.
21. Dubey G.K., (1989) Power Semiconductor Controlled Drives, Englewood Cliffs,
NJ, Prentice-Hall.
22. Feuer A., and Morse A.S., (1978) "Adaptive Control of Single-Input Single-
Output Linear Systems", IEEE Trans. on Automatic Control, vol. AC-23, pp.
557-569.
23. Fox K.A., (1984) "MRP-II providing a natural hub for computer integrated
manufacturing system", Ind. Eng., pp.44-50.
24. Funahashi K., (1989) "On the approximate realization of continuous mappings
by neural networks," Neural Networks, vol. 2, pp.183-192.
25. Freeman R.A., and Kokotovic P.V., (1993) "Design of Softer Robust Nonlinear
Control Laws", Automatica, vol. 29, no. 6, pp. 1425-1437.
26. Gershwin S.B., Hildebrant R.R., Suri R. and Mitter S.K., (1986) "A Control
Perspective on Recent Trends in Manufacturing Systems," IEEE Contr. Syst.
Mag., pp. 3-15.
27. Gershwin S.B., (1994) Manufacturing Systems Engineering, Englewood Cliffs,
New Jersey, Prentice Hall.
28. Giles C.L., Chen D., Miller C.B., Chen H.H., Sun G.Z., and Lee Y.C., (1991)
"Second-order recurrent neural networks for grammatical inference," Proc. of
Inter. Joint Conf. on Neural Networks, IJCNN91 , Vol. II, pp. 273-281.
29. Golub G.H., and Van Loan C.F., (1989) Matrix Computations, The John Hop-
kins Univ. Press, 2nd Edition.
30. Goodwin G.C., Ramadge P.J., and Caines P.E., (1980) "Discrete-time Mul-
tivariable Adaptive Control", IEEE Trans. Automat. Contr., vol. AC-25, pp.
449-456.
31. Goodwin G.C., Sin K.S., (1984) Adaptive Filtering, Prediction, and Control,
Prentice Hall, New Jersey.
32. Goodwin G.C., and Mayne D.Q., (1987) "A parameter estimation perspective
of continuous time model reference adaptive control", Automatica, vol. 23, pp.
57-70.
33. Gupta Y.P., Gupta M.C., Bector C.R., (1991) "A review of scheduling rules
in flexible manufacturing systems", Int. J. Comput.Integ. Manuf, vol. 2, no. 6,
pp.356-377.
34. Hale J.K., (1969) Ordinary Differential Equations, New York, NY, Wiley-
InterScience.
35. Hartman E.J., Keeler J.D., Kowalski J.M., (1990) "Layered Neural Networks
with Gaussian Hidden Units as Universal Approximators", Neural Computation,
vol. 2, pp. 201-215.
36. Hertz J., Krogh A., and Palmer R.G., (1991) Introduction to the Theory of
Neural Computation. Lecture Notes Volume I, Santa Fe Institute Studies in
the Sciences of Complexity. Redwood City, CA: Addison-Wesley Publishing
Company.
References 187
37. Ho Y.C., (1984) "Perturbation Analysis of Discrete Event Systems," Proc. 1st
ORSA/TIMS Conf. FMS, Ann Arbor, Michigan, USA.
38. Hopfield J.J., (1982) "Neural Networks and Physical Systems with Emergent
Collective Computational abilities," in Proceedings of the National Academy of
Science, U.S.A., vol. 79, pp. 2554-255~.
39. Hopfield J.J., (1984) "Neurons with graded response have collective computa-
tional properties like those of two-state neurons" Proc. Natl. Acad. Sci., vol. 81,
pp. 3088-3092.
40. Hornik KM., Stinchombe M., White H., (1989) "Multilayer feedforward net-
works are universal approximators", Neural Networks, vol. 2, pp. 359-366.
41. Hou Z., (1992) "Analysis of Auto Powertrain Dynamics and Modelling Using
Neural Networks," M.Sc. Thesis, Department of Electrical Engineering, Univer-
sity of Notre Dame.
42. Hunt KJ., Sbarbaro D., Zbikowski R., and Gawthrop P.J., (1992) " Neural
Networks for Control Systems - A Survey", Automatica, vol. 28, no. 6, pp.
1083-1112.
43. Ioannou P.A., and Kokotovic P.V., (1983) Adaptive Systems with Reduced Mod-
els, Springer-Verlang, New-York, NY.
44. Ioannou P.A., and Tsakalis K.S., (1986) "A robust direct adaptive controller",
IEEE Trans. Automat. Contr., vol. AC-31, no. 11, pp. 1033-1043.
45. Ioannou P.A., Datta A., (1991) "Robust adaptive control: a unified approach,"
Proceedings of the IEEE, December.
46. Ioannou P.A., and Sun J., (1996) Robust Adaptive Control, Upper Saddle River,
NJ, Prentice Hall.
47. Isidori A., (1989) Nonlinear Control Systems, New York, NY, Springer-Verlag.
48. Jacobs F.R., (1983) "The OPT scheduling system: A Review of A New Pro-
duction Scheduling System", Product. Inventory Management, vol.24, pp.47-51.
49. Jackson J.R., (1963) "Jobshop-like Queuing Systems", Management Science,
Vo1.10, No.1, pp.131-142.
50. James D.J., (1971) "Stability of a model reference control system", AIAA Jour-
nal, vol. 9, no. 5.
51. Kamp Y., Hasler M., (1990) Recursive Neural Networks for Associative Mem-
ory, J. Wiley & Sons.
52. Kanellakopoulos I., Kokotovic P.V., and Marino R., (1991) "An Extended Di-
rect Scheme for Robust Adaptive Nonlinear Control", Automatica, vol. 27, no.
2, pp. 247-255.
53. Kanellakopoulos I., Kokotovic P.V., and Morse A.S., "Systematic Design of
Adaptive Controllers for Feedback Linearizable Systems", IEEE Trans. Au-
tomat. Contr., vol. 36, no. 11, pp. 1241-1253.
54. Kanellakopoulos I., Kokotovic P.V., and Morse A.S., (1992) " Adaptive Output
Feedback Control of Systems with Output Nonlinearities", IEEE Trans. on
Automatic Control, vol. 37, no. 11, pp. 1666-1682.
55. Kohonen T., (1980) Content-Addressable Memories. New York, NY: Springer-
Verlag.
56. Kokotovic P.V., Khalil H.K, and O'Reilly J., (1986) Singular Perturbation
Methods in Control: Analysis and Design, Academic Press, New York.
57. Kosmatopoulos E.B., Polycarpou M., Christodoulou M.A., Ioannou P.A.,
(1995) "High-order neural network structures for identification of dynamical
systems", IEEE Trans. Neural Networks, vol. 6, no. 2, pp. 422-431.
58. Krause P.C., (1987) Analysis of Electric Machinery, Singapore, McGraw-Hill.
59. Kusiak A., (1987) "Designing Expert Systems for Scheduling of Automated
Manufacturing," Ind. Eng. pp.42-46.
60. Leonhard W., (1985) Control of Electrical Drives, Berlin, Springer-Verlag.
188 References
61. Lewis F.L., Liu K., Yesildirek A., (1995) "Neural net robot controller with
guaranteed tracking performance", IEEE Trans. Neural Networks, vol. 6, no. 3.
pp. 703-715.
62. Lewis F.L., Jagannathan S., Yesildirek A., (1999) Neural Network Control of
Robot Manipulators and Nonlinear Systems, Taylor and Francis, London, UK.
63. Liou R., Azimi-Sadjadi M.R., Dent R., (1991) "Detection of dim targets in
high cluttered background using high order correlation neural network," Proc.
of Inter. Joint Conf. on Neural Networks, IJCNN91, Vol. I, pp. 701-706.
64. Marino R., and Tomei P., (1993) "Global Adaptive Output Feedback Control
of Nonlinear Systems, Part I: Linear Parameterization", IEEE Trans. Automat.
Contr., vol. 38, no. 1, pp. 17-32.
65. Marino R., and Tomei P., (1993) "Global Adaptive Output Feedback Control
of Nonlinear Systems, Part II: Nonlinear Parameterization", IEEE Trans. Au-
tomat. Contr., vol. 38, no. 1, pp. 33-48.
66. Margaris N., Goutas T., Doulgeri Z., and Paschali A., (1991) "Loss Minimiza-
tion in DC Drives", IEEE Trans. Ind. Electron., vol. 38, no. 5, pp. 328-336,
1991.
67. Michel A.N., and Miller R.K, (1977) Qualitative Analysis of Large Scale Dy-
namic Systems, Academic Press, New-York.
68. Morse A.S., (1980) "Global Stability of Parameter Adaptive Control Systems",
IEEE Trans. Automat. Contr., vol. AC-25, pp. 433-439.
69. Moody J., Darken D.J., (1989) "Fast Learning in Networks of Locally-Tuned
Processing Units," in Neural Computation, vol. 1, pp. 281-294.
70. Nam K, and Arapostathis A., (1988) "A Model Reference Adaptive Control
Scheme for Pure Feedback Nonlinear Systems", IEEE Trans. on Automatic
Control, vol. 33, pp. 803-811.
71. Narendra KS., Lin Y.H., and Valavani L.S., (1980) "Stable adaptive controller
design, Part II: Proof of stability", IEEE Trans. Automat. Contr., vol. AC-25,
pp. 440-448.
72. Narendra K.S., and Annaswamy A.M., (1987) "A new adaptive law for robust
adaptation with persistent excitation", IEEE Trans. Automat. Contr., vol. AC-
32, no. 2, pp. 134-145.
73. Narendra KS., Annaswamy A.M., (1989) Stable Adaptive Systems, Englewood
Cliffs, NJ, Prentice-Hall.
74. Narendra KS., Parthasarathy K., (1990) "Identification and control of dynam-
ical systems using neural networks," IEEE Trans. on Neural Networks, vol.l,
no.l, pp. 4-27.
75. Narendra KS. ,Parthasarathy K, (1991) "Gradient methods for the optimiza-
tion of dynamical systems containing neural networks," IEEE Transactions on
Neural Networks, vol. 2, no. 2, pp. 252-262.
76. Nijmeijer H., and van der Schaft A.J., (1989) Nonlinear Dynamical Control
Systems, New York, NY, Springer-Verlag.
77. Paretto P., Niez J.J., (1986) "Long term memory storage capacity of multicon-
nected neural networks," Bioi. Cybern., vol. 54, pp. 53-63.
78. Parks P.C., (1966) "Lyapunov redesign of model reference adaptive control
systems" ,IEEE Trans. Automat. Contr., vol. AC-ll, pp. 362-367.
79. Perkins J .R., Kumar P.R., (1990) "Stable ,Distributed, Real-Time Scheduling
of Manufacturing Systems", IEEE Transaction on Automatic Control, vol. 35,
no. 3, pp. 289-298.
80. Pineda F.J., (1988) "Generalization of back propagation to recurrent net-
works," Phys. Rev. Lett., vol. 59, no. 19, pp. 2229-2232.
References 189
81. Polycarpou M.M., Ioannou P.A., (1991) "Identification and control of nonlinear
systems using neural network models: design and stability analysis," Tech. Rep.
91-09-01, Univ. of Southern Cal., Los Angeles.
82. Polycarpou M.M., Ioannou P.A., (1994) "On the existence and uniqueness of
solutions in adaptive control systems," IEEE Trans. on Automatic Control, vol.
30, no. 3, pp. 474-479.
83. Pomet J.-B., and Praly L., (1992) " Adaptive Nonlinear Regulation: Estimation
from the Lyapunov Equation", IEEE Trans. Automat. Contr., vol. 37, no. 6,
pp.729-740.
84. Rouche N., Habets P., and Laloy M., (1997) Stability Theory by Liapunov's
Direct Method, New York, Springer-Verlag.
85. Rovithakis G.A., Christodoulou M.A., (1994) "Adaptive Control of Unknown
Plants using Dynamical Neural Networks", IEEE Trans. Systems Man Cyber-
netics, vol. 24, no. 3, pp. 400-412.
86. Rovithakis G.A., and Christodoulou M.A., (1995) "Direct Adaptive Regulation
of Unknown Nonlinear Dynamical Systems via Dynamic Neural Networks",
IEEE Transactions on Systems, Man and Cybernetics, vol. 25, no. 12, pp. 1578-
1594.
87. Rovithakis G.A., and Christodoulou M.A., (1997) "Neural Adaptive Regulation
of Unknown Nonlinear Dynamical Systems", IEEE Transactions on Systems,
Man and Cybernetics-Part B: Cybernetics, vol. 27, no. 5, pp. 810-822.
88. Rovithakis G.A., and Christodoulou M.A., (1994) "Model Reference Adaptive
Control of Unknown Plants using Dynamic Neural Networks", Journal of Ap-
plied Mathematics and Computer Science, vol. 4, no. 3,pp. 433-446.
89. Rovithakis G.A., and Christodoulou M.A., (1995) "Regulation of Unknown
Nonlinear Dynamical Systems via Dynamical Neural Networks", Journal of
Intelligent and Robotic Systems, vol. 12, pp. 259-275.
90. Rovithakis G.A., and Christodoulou M.A., (1996) "Neural Adaptive Track-
ing Control of Unknown Plants via Recurrent Neural Networks", International
Journal of Intelligent Control and Systems, vol. 1, no. 2, pp. 211-222.
91. Rovithakis G.A., and Christodoulou M.A., (1996) "Direct Adaptive Regulation
using Dynamic Neural Networks: Application to Dc Motor Speed Control",
IMACS International Journal: Mathematics and Computers in Simulation, vol.
41, pp. 53-62.
92. Rovithakis G.A., and Cristodoulou M.A., (1996) "On using Recurrent Neural
Network Models to Develop Direct Robust Adaptive Regulators for Unknown
Systems", Journal of Intelligent Automation and Soft Computing, vol. 2, no. 4,
pp. 321-338.
93. Rovithakis G.A., Gaganis V.I., Perrakis S.E. and Christodoulou M.A., (1999)
"Real Time Control of Manufacturing Cells using Dynamic Neural Networks",
Automatica, vol. 35, pp. 139-149.
94. Rovithakis G.A., (1999) "Robust Neural Adaptive Stabilization of Unknown
Systems with Measurement Noise", IEEE Transactions on Systems, Man and
Cybernetics, vol. 29, no. 3, pp.453-458.
95. Rovithakis G.A., (1999) "Robustifying Nonlinear Systems using High Order
Neural Network Controllers", IEEE Trans. Automatic. Control., vol. 44, no. 1,
pp. 102-108.
96. Rovithakis G.A., (1999) "Tracking Control of Multi Input Affine Nonlinear
Dynamical Systemswith Unknown Nonlinearities using Dynamical Neural Net-
works", IEEE Transactions on Systems, Man, & Cybernetic-Part B: Cybernet-
ics, vol. 29, no. 2, pp. 179-189.
190 References
97. Rumelhart D., Hinton D., Williams G., (1986) "Learning internal representa-
tions by error propagation," in Parallel Distributed Processing, D. Rumelhart
and F. McClelland, Eds, Vol. 1, Cambridge, MA, MIT Press.
98. Sadegh N., (1991) "Nonlinear identification and control via neural networks",
Control Systems with Inexact Dynamic Models, DSC-vol. 33, ASME Winter
Annual Meeting.
99. Sadegh N., (1993) "A percept ron network for functional identification using
radial gaussian networks", IEEE Trans. Neural Networks, vol. 4, no. 6, pp.
982-988.
100. Sanner R.M., Slotine J.-J.E., (1992) "Gaussian neural networks for direct
adaptive control", IEEE Trans. Neural Networks, vol. 3, no. 6, pp. 837-863.
101. Sastry S., and Bodson M., (1989) Adaptive Control: Stability, Convergence
and Robustness, Englewood Cliffs, NJ, Prentice-Hall.
102. Sastry S. and Isidori A., (1989) "Adaptive Control of Linearizable Systems",
IEEE Trans. Automat. Contr., vol. 34, no. 11, pp. 1123-1131.
103. Schonberger R.J., (1984) "Just-In-Time Production Systems: Replacing Com-
plexi.ty with Simplicity in Manufacturing Management", Ind. Eng., pp.52-63.
104. Sharifnia A., (1994) "Stability and Performance of Distributed Production
Control Methods based on Continuous-Flow Models", IEEE Transactions on
Automatic Control, vol. 39, no. 4, pp. 725-737.
105. Taylor D.G., Kokotovic P.V., Marino R., and Kanellakopoulos I., (1989)
"Adaptive Regulation of Nonlinear Systems with Unmodeled Dynamics", IEEE
Trans. Automat. Contr., vol. 34, no. 4, pp. 405-412.
106. Werbos P.J., (1990) "Backpropagation through time: what it does and how
to do it", Proc. IEEE, vol. 78, pp. 1550-1560.
107. Williams R.J., Zipser D., (1989) "A learning algorithm for continually running
fully recurrent neural networks," Neural Computation, vol. 1, pp. 270-280.
INDEX
associative memories 11 electric time constant, 45

activation function 2 mechanic time constant, 45
adaptive nonlinear control 6 field time constant, 45
additive disturbance 123 normalized viscous friction coeffi-
admissible inputs 30 cient,45
affine system 30 applied load torque, 45
approximation problem 13 nominal no load spped, 45
artificial neural networks 1 nominal flux, 45
potential benefits, 1 nominal armature voltage, 45
types, 1 extrapolated stalled rotor current at
assembly operation 159 nominal voltage, 45
extrapolated rotor torque, 45
Barbalat's Lemma 18, 20, 35, 39, 56, 94 nominal field current at nominal field
Bellman-Gronwall Lemma 15 voltage, 45
bias constant 12 rotor resistance, 45
bottleneck machine 140, 162 rotor self-inductance, 45
stator self-inductance, 45
capacity-requirements analysis 140 number of turns, stator winding, 45
capacity bound 149 moment of inertia of the motor-load
ceredellar model articulation controller system referred to the motor shaft,
(CMAC) 1 45
clear largest buffer (CLB) 161, 173 armature votage, 45
clear a fraction (CAF) 161, 173 field voltage, 45
Cohen-Grossberg neural network 18 armature current, 45
computer-integrated-manufacturing field current, 45
(CIM) 137, 138 viscous friction coefficient, 45
content-addressable memory 1 angular spped, 45
controlled vector fields 64 magnetic flux, 45
cost work-in-process (WIP) 162, 179 dead zone algorithm 118
cost average backlogging, 163 discontinuity hypersurface 24
cost backlogging drift term 32
cost inventory 179 dynamical system identification 5
covariance resetting 18 using RHONNs, 9, 29
covariance wind up 18 robustness owing to unmodeled
dynamics, 31-35
DC motors 44 dynamical neurons 10
separately excited, 44 dynamic extension 30
nonlinear model, 44 dynamic programming 141
in affine form, 44 dynamic uncertainties 31
linear model, 44
loss minimization problem, 44 earlier due date (EDD) 141
field excitation, 44 Euclidean norm 8
192 Index
error fitering method 54, 62, 100, 124 longest processing time first (LPT) 141
error equation 36 longest processing time remaining
e:-modification algorithm 118 (LPR) 141
existence of solution of differential logistic function 12
equations 30 Lyapunov function 18, 20, 22, 34, 36,
expert systems 143 54, 63, 73, 86, 101
knowledge-based, 143 Lyapunov equation 34, 36
rule-based, 143
machine starvation 148
feedback linearization tecniques 6 Machine Tool Division 166
feedforward multilayer neural network machine parts inspection 169
2 machine tool storage & pre-adjustment
first-in-first-out (FIFO) 141, 173 169
flexible manufacturing systems (FMS) machine sawing 169
138-139 machine marking 169
forward modeling 2 machine large planing 169
Frobenius matrix norm 8 machine jig drill 169
machine milling 169
Gantt charts 139 machine manual drilling 169
gradient method 17 machine circular grinding 169
gaussian node network 1 machine surface grinding 169
grammatical inference 10 machine manual turning 169
machine turning CNC 169
Hopfield neural network 2, 12 machine small planing 169
machine boring center CNC 169
identification problem 10 machine de burring 169
identification model 10 makespan 162, 163
idling clause 147 manufacturing systems 137
indicator function 23, 68 manufacturing cell dynamic model 147
industrial databases 139 Mechanical Workshop 166
integer programming 141 modeling error 15, 61
intermediate buffer 160 Monte Carlo 141
inververce dynamics modeling 3 multiplicative disturbance 123
job-shop 140 neurocontrol 7

negative semidefinite 35, 42
Lagrange multiplier method 141 non-acyclic routes 170
lead time 163
average, 163, 181 operating frequency 147
leakage term 22 output buffer 163
learning algorithms 9
backpropagation, 1 parameter drift 21
backpropagation-through-time, 9 parametric uncertainties 31
dynamic backpropagation, 9 part relese 137
recurrent backpropagation, 9 pattern recognition 3
real-time recurrent, 9 peak buffer state 162
based on Lyapunov stability theory, persistency of excitation 18, 56
9, 31 perturbation analysis 142
least-squares method 17 polynomial expansion 14
linear-in-the-weights property 6 positive definite 35, 42
linear parameterization 7 priority measure criterion 155
Lipschitz condition 13, 30 processing time 140
L2 8 production process 137
Leo 8 projection algorithm 37, 57, 66, 91, 117
Index 193
properties, 39, 58-59 gravitational forces, 26

joint torques, 26
quadratic cost functional 17 routing 137
quasi-steady-state 32
scheduling 137, 140
policies, 173
radial-basis gaussian function 2 continuous control input, 146-147
raw material 139 scheduling approaches 139
arrival rate, 162 industrial practice, 139
recurrent neural networks 5 manufacturing resource planning
recurrent high-order neural networks (MRP),140
(RHONNs) 10-12 just-in-time (JIT), 140
approximation properties, 13-15 optimized production timetable
approximation theorem,13 (OPT), 140
learning algorithms, 15 machine sequencing & scheduling
filtered-regressor, 16-19 theory, 140
filtered-error, 19-20 resource constrained project
robust learning algorithms, 20-25 scheduling 141
indirect adaptive control, 29, 35 control theory, 141
parametric uncertainty, 36-39 discrete event simulation, 141
plus dynamic uncertainty, 39-43 stochastic optimization, 142
the two-stage control algorithm, 29 queuing theory, 142
speed control of DC motors, 44-48 fluid networks, 142
direct adaptive control, 53 reliability theory, 142
adaptive regulation, 53 lot-sizing, 142
complete matching 53-61 inventory theory, 142
modeling error effects, 61 artificial intelligence, 143
complete model matching at zero, test case, 166
64 production planning & layout, 166
modeling error at zero, 69 order processing steps, 167
model order problems, 72-80 order definition, 167
DC motor speed control, 80-83, 95, design phase, 167
130-134 disposal & purchasing, 167
modeling error with unknown part manufacturing, 167
coefficients, 83-95 assembly, 168
tracking, 95 electrical instalations, 168
complete matching, 97-102 quality control & delivery, 168
modeling error effects, 102-108 RHONN model derivation, 171
general affine systems, 108-123 sensitivity function 9
disturbance effects 123-129 sensitivity model 9
manufacturing cell model, 150 set up time 137
continuous-time scheduling, 151 singular perturbation theory 31
real-time scheduling, 155 singularly perturbed model 34, 39
discrete dispatching decision, 155 equilibrium, 34
discrete dispatching decision single shortest processing time first (8PT) 141
part case algorithm, 155 sigmoid function 12
discretization effects, 157 sign function 85
discrete dispatching decision multi smooth vector field 13
part case algorithm, 158 square integrable 20
region of attraction 34 stable matrix 31
robotic manipulator model 26 8tone-Weierstrass Theorem 14
degrees of freedom, 26 switching O"-modification 22
inertia matrix, 26 smoothness, 24
coriolis forces, 26 synaptic weights 31
194 Index
estimates, 36 transient behavior 46

sum of the maximum state of all
intermediate buffers 177 weight estimation error 16
sum of maximum output buffers states working clause 147
178
subassemply 140
submachine 145 uniform asymptotic stability in the
frequency, 147 large 74
uniqueness of solution of differential
target detection 10 equations 30
trace of a matrix 8 unmodeled dynamics 31

(George A. Rovithakis Manolis A. Christodoulou) Adaptive Control With Recurrent High Order Neural Networks

Загружено:

Сведения о документе

Оригинальное название

Авторское право

Доступные форматы

Поделиться этим документом

Поделиться или встроить документ

Параметры публикации

Этот документ был вам полезен?

Это неприемлемый материал?

Авторское право:

Доступные форматы

(George A. Rovithakis Manolis A. Christodoulou) Adaptive Control With Recurrent High Order Neural Networks

Загружено:

Авторское право:

Доступные форматы

Advances in Industrial Control

British Library Cataloguing in Publication Data

Springer-Verlag London Limited 2000

69/3830-543210 Printed on acid-free paper SPIN 10728731

Professor Michael J. Grimble, Professor ofIndustrial Systems and Director

Series Advisory Board

Professor Dr-Ing J. Ackermann

Professor I.D. Landau

Professor D.W. Clarke

Professor A.J. Laub

Professor J.B. Moore

Professor Ton Backx

The series Advances in Industrial Control aims to report and encourage

M.J. Grimble and M.A. Johnson

Recent technological developments have forced control engineer~ to deal with

2. Identification of Dynamical Systems Using

3. Indirect Adaptive Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 29

4. Direct Adaptive Control. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 53

4.3 Modeling Errors with Unknown Coefficients. . . . . . . . . . . . . . .. 83

5. Manufacturing Systems Scheduling ....................... 137

6. Scheduling using RHONNs: A Test Case ................. 165

References .................................................... 184

Index ......................................................... 191

1.1 General Overview

G. A. Rovithakis et al., Adaptive Control with Recurrent High-order Neural Networks

invoke is an important decision and varies depending on the intended appli-

Fig. 1.1. Plant identification with a multilayer neural network

Fig. 1.2. A neural network controller

REFERENCE 1 reference model output

Once an identification neural model of the plant is available, this model

configurations. The training of a neural network as an open loop controller is

the mathematical representation that governs their evolution. This leads to

the literature with typical examples [70],[105),[102],[52], [53],(54),[6],[83], [64).

1.2 Book Goals & Outline

As the first results in neural control started to appear, it became increasingly

Chapter 2 introduces the RHONN structure and analyze its approxima-

G. A. Rovithakis et al., Adaptive Control with Recurrent High-order Neural Networks

2.1 The RHONN Model

Xi = -aiXi + bi L WijYj , (2.1)

where {II, h, ... , h} is a collection of L not-ordered subsets of {I, 2, ... , m+

Moreover, if we define the adjustable parameter vector as

then (2.6) becomes

2.1.1 Approximation Properties

Consider now the problem of approximating a general nonlinear dynamical

e(t) = lot eA(t-r) [W*T z(x(r), u(r)) - G(x(r), u(r))] dr,

= lot eA(t-r) [W*T z(x(r), u(r)) - W*T z(x(r), u(r))] dr

le(t)1 :::; lot lIeA(t-r)lIlw*T z(x(r), u(r)) - W*T z(x(r), u(r))1 dr

+ lot lIeA(t-r)lIlw*T z(x(r), u(r)) - I

:::; lot e-a(t-r) Lle( r)ldr + lot 8e- a(t-r)dr ,

:::; ~e-~ + L lot e-a(t-r)le(r)ldr.

Using the Bellman-Gronwall Lemma [34], we obtain

le(t)1 :::; ~e-~ + eL lot e-a(t-r)dT,

2.2 Learning Algorithms

2.2.1 Filtered Regressor RHONN

The following lemma is useful in the development of the adaptive identifica-

wtT (i + e-aitx? = e-aitx? + 1t e-ai(t-T)wt T z(X( r), u( r))dr . (2.23)

Xi=Wi.T(i+(j, i = 1,2, .. . ,n, (2.24)

This can be achieved by using well-known optimization techniques for mini-

Depending on the optimization method that is employed, different weight

where P(O) is a symmetric positive definite matrix. In the above formulation,

Using (2.28) and (2.26), the time derivative of V in (2.30) is expressed as

so for i = 1, ... n, e;(t) is square integrable. Furthermore, using (2.26)

= -ei(T riC; - ai<Pr (i + <pr Z - fi .

= lot eA(t-r) [WT z(x(r), u(r)) - WT z(x(r), u(r))] dr

le(t)1 :::; lot lIeA(t-r)lIlwT z(x(r), u(r)) - WT z(x(r), u(r))1 dr