Identification of MIMO Systems by Input-Output TS Fuzzy Models

R. BabuSka J.A. Roubos H.B. Verbruggen Control Engineering Laboratory, Faculty of Information Technology and Systems, Delft University of Technology, Mekelweg 4, P.O. Box 5031,2600 GA Delft, The Netherlands tel: +31 15 27851 17, fax: +31 15 2786679, e-mail: r.babuska@et.tudelft.nl

Abstract
A number of techniques have been introduced to construct

fuzzy models from measured data. Most attention has been

focused on multiple-input, single-output (MISO) systems. This article concentrates on the identiflcation o multiplef input, multiple-output(MIMO) systems by means ofproductspace fuzzy clustering with adaptive distance measure (the Gustafon-Kessel algorithm). The MIMO model is represented as a set of coupled input-output MISO models of the Takagi-Sugeno type. Knowledge o the physical structure f can easily be incorporated in the structure o the model. f Sojhare implementation in the form o a MATLAB toolbox f is briefi described. A simulation example offour cascaded tanks is given.

an illustrative example of a two-input, four-output nonlinear dynamic process is given. It is shown that the fuzzy identification approach leads to a significant improvement in comparison with standard linear identification.

2 Fuzzy Model Structure

Consider a MIMO system with nj inputs: U E U c EXn', and no outputs: y E Y C E". This system will be t. approximated by a collection of coupled MISO discretetime fuzzy models. Denote q-l the backward shift operator: def q-lg(A) = g(k - l),where y is a signal sampledat discrete time instants k. Denote by the capital letters A and B polynomials in q-l, e.g., A = a0 alq-'+ a ~ q - ~ . .. . Given two integers, m < n, define an ordered sequence of delayed samples of the signal y as:

Keywords: Fuzzy modeling, nonlinear identification, multivariable (MIMO) systems, fuzzy clustering, "I'LAB.

{()% yk}

def

=[y(k - m), ~ ( - m - l, . ..,y(k - m - n k )

+ I)].

The MISO models are of the input-output NARX type:

1 Introduction
Fuzzy modeling and identification from measured data are effective tools for the approximation of uncertain nonlinear systems. So far, most attention has been devoted to single-input,single-output (SISO) or multiple-input, singleoutput (MISO) systems. Recently, also methods to deal with multiple-input,multiple-output (MIMO) systems were investigated, mainly in connection with control [I, 2, 31. Most articles deal with various aspects of multivariablerelational models, such as the decomposition of fuzzy relations, simplification of the models to avoid memory overload, etc. Relatively little attention has been devoted to the identification of MIMO fuzzy models from input-output data. The contribution of this articleis three-fold First, it is shown that the identification technique based on product-space fuzzy clustering, which was developed in our previous articles [4,5] for MISO systems, can be extended to MIMO systems in a straightforward way. Second, a MATLAB toolbox is presented which implements the presented techniques. The toolbox can be obtained from the authors on request. Third, where the regression vector XI is given by: (k)

Here ny and n, are the number of delayed outputs and inputs, respectively, and n is the number of pure (transport) d delays from the input to the output. ny is an no x nomatrix, and nu,n are no x n, matrices. & are rule-based fuzzy d models of the TakagiSugeno (TS) type 161:

Rr.i:

Ifxl(k)is Rli then yii(k 1)= Aliy(k) Briu(k)

+q i
(2)

i = l , 2 ,..., Ki.
Here

i the antecedent fuzzy set of the ith rule, Ali = s [Aril,...,Arino],Bli = [&I,. ..,&in*] are vectors of polynomials, cri are offsets, and K1 is the number of rules in the Zth model.

0-7803-4863-X/98 $10.0001998 IEEE

657

The fuzzy sets R can be defined by multivariate membership functions w(x(k)):RP' -+ [0,1], where pi = n,lj n,lj is the dimension of the antecedent space. Alternatively, the antecedent of (2) can be given in the conjunctive form:

3.1

Regression D t aa

The regression matrix X and the regressand vector y are constructed from the input and output data sequences:

Rli:

Ifxll (IC) is R i and . . . and xzp1 is Qip1 ll (k) then yli(k 1) = Aliy(IC) Bliu(k) q i

X = !!x
x(N - 1)

] [ v" ]
,y =
Y (NI

(5)

i = l , 2 ,...,Kl.

(3)

Both these forms can be constructed from data, as shown in Section 3. The output of the TS model is computed by the weighted mean:

Here N >> p is the number of data samples available for identification. The data set Z to be clustered is formed by concatenating y to X:ZT = [X y].

3.2 Fuzzy Clustering

Given Z and the desired number of clustersc, the Gustafson13 Kessel (GK) algorithm 1 0 is applied. This algorithm computes the fuzzy partition matrix, U = [/.ii&X~, with pik E [O,1 ,the prototype matrix, V = [vl,. . ,v ] and the 1 . ,, set of cluster covariance matrices F = [F1,. . . ,F,] (Fi are positive definite matrices in IW(P+l) x(P+l)). The algorithm is given in the Appendix. Given the triplet, (U, V, F), the antecedent membership functions Ri and the consequent parameters Ai, Bi and ci can be computed. 3.3 Antecedent Membership Functions The multivariate membership functions Ri of eq. (8) is represented analytically by computing the distance of x ( k ) from the prototypes vi, and then computing the membership degree in an inverse proportion to this distance. Denote F" = [fjj], 1 5 i, j 5 p the partition of the cluster covariance matrix, which includes all but the last column and last row. The corresponding norm-inducing matrix is given by (see the Appendix):

is In the product-space form (8), w~((xl(IC)) simply the membership degree /.ini (xl(IC)) In the conjunctive form (3), uli(xl(k)) given by the product of the individual anteis cedent membership degrees.

3 Identification Method
The structure of the model, i.e., the matrices nyr and n d n, are determined by the user on the basis of prior knowledge and/or by comparing several candidate structures in terms of the prediction error or other suitable criteria [7]. Once the structure is fixed, the parameters of the noMISO models (2) are estimated. These parameters are the antecedent membership functions and the consequent polynomials. An additional structuralparameter is the number of rules, which can be specified by the user, or can be sought automatically, as discussed later on. The MISO models are estimated independently of each other, using the procedure described in this section. The identificationalgorithm proceeds in four steps: From the input-output sequences of N d measurey(k))}~~l, the nonlinear regresform ments, { (u(lc), sion problem of eq. (l), using the user-specified structural parameters nu,n, and nd. Partition the data into a set of local linear submodels by using fuzzy clustering in the product space X x Y for each model. Compute the antecedent membership function from the cluster parameters. Given the antecedent membership functions, estimate the consequent parameters by the least-squares method.

M = [det(F:)]"P(FT)-'. q

(6)

Let vs = [ Y I ~ ,.,v,i]* denote the projection of the cluster .. center onto X. The squared inner-product induced distance

d2(x(IC),v?) = ( ~ ( k )V~)~M?(X(IC) - v?)

is converted into the membership degree by:

(7)

This technique was introduced in our previous publications [4,5] and was successfully applied to modeling and control of MISO processes [8, 91. Here, only a brief overview is given. For the sake of brevity, the output index 1 will be omitted in the rest of the paper.
65 8

In order to obtain the membership functions for fuzzy sets Rij, 1 5 j 5 p of the conjunctive form (3), the multidimensional fuzzy sets defined point-wise in the ith row of the partition mamx U = [pzk]can be projected onto the regressors xj by:
PQ,, (xj(IC)) = projj (pik),

(9)

where proj is the point-wise projection operator [l 11. The point-wise defined fuzzy sets Rij are then approximated by a suitableparametric function, in order to be able to compute wi( j z (k)) also for values of z ( I C ) not contained in 2. j
3.4

Consequent Parameters

There are several possibilities to estimate the consequent parameters [ 121. Here, the weighted ordinary least square estimate is presented. Let 6: denote a vector containing the coefficient of the consequent polynomials Ai, B and the i offset ci. Let X, denote the matrix [X1 and let Wi denote 1 a diagonal matrix in R having the membership degree pik as its kth diagonal element. If the columns of X are , linearly independent and &k > 0 for 1 5 k L N , then

often less accurate models. m > 1 is the fuzziness exponent with the default value m = 2. Larger values imply fuzzier (more overlapping) clusters. The termination tolerance for the clustering algorithm can be given in to1 (default to1 = 0.01). The fuzzy partition matrix is initialized at random. In order to obtain reproducible results, the random generator may be seeded by supplying the seed parameter. Default value is sum(lOO*clock).The ny,nu andnd parameters are the delay matrices defined in Section 2. The output of fmclust is the FM matrix which contahs the parameters of the obtained fuzzy model (under MATLAB 5.1, FM is defined as a structure). The obtained fuzzy mode1 can be simulated by using function fmsim with the following synopsis: IYm,VAFl = fmsim(U,Y,FM)

[ : i , -l xwy x W x ] :i is the least squares solution of y = Xed+- E where the kth

e i

data p&

(Xk ,Yk)

is weighted by pik .

35 Number of Clusters .
Two main approaches to determine the appropriate number
of clusters in data can be distinguished:
0

Clustering data for differentvalues of c and using validity measures to assess the goodness of the obtained partitions [ 131. Starting with a sufficientlylarge number of clusters and successively merging clusters that are similar (compatible) with respect to some criteria. This approach is called compatible cluster merging [14, 151.

This function simulates the fuzzy model F from the input M data U and plots the simulated output Ym along with the true output Y. The first ny values of Y are used to initialize Ym. The output argument VAF is the performance index of the model, computed as the variance accountedfor ( " V) given by: VAF = 100% [l - (var(Y Ym)) /var(Y)] . As additional output arguments, the degrees of fulfillment and the local outputs of the individual rules can be obtained. A simple example of an approximation of a sinusoidal function by the TS fuzzy model with five rules is:

U = (0:0.02:1)'; y = sin(7*u); FM = fmclust(u,y,5) ; [ym,VAF,Ylocl = fmsim(u,y,FM);

4 Matlab Implementation
The identification algorithm presented in Section 3 was implemented in MATLAB (versions 4.2 and 5.1). The core of the identification routine is the GK clustering algorithm, whose implementation is given in the Appendix. The function to construct a MIMO fuzzy model from data, fmclust,has the following synopsis:
FM = fmclust (U,Y,c,FMtype,m, tol, . . seed,ny,nu,nd) .

The output obtained on the screen is shown in Fig. 1. The approximation accuracy is VAF = 99.67%.
1.5

The data sequences are given in columns of the matrices U and Y, respectively. The number of required clusters, c, is a scalar for MISO systems and a vector for MIMO systems (each MISO model may have a different number of clusters). The remaining parameters are optional. FMtype specifies whether the antecedent membership functions of the resulting model are computed analytically in the antecedent product space using (8), or are derived by projection (9). Product-space membership functions give faster but

-1.5' 0

0.1

"

0.2

0.3

"

0.4

0.5
U

"

0.6

0.7

0.8

"

0.9

'

Figure 1. Approximation of a static function (solid line) by a TS model (dashed-dotted line). The consequent model are plotted by dotted lines.

Section 5 shows how these functions are used to identify a MIMO dynamic process. In addition to the above two

659

functions, utilities are provided, e.g., for the automatic generation of LaTEX documentation of the obtained model (fm2tex).

5 SimulationExample
Consider a MIMO process consisting of four cascaded tanks as shown in Fig. 2. The inputs are the two flow rates U = [ & ~ , Q z ] * , the outputs are the four leveIs and

Y = fh, h,hIT. hz,

Time [ I d $1

Figure 3. Input data for identification.

The measured outputs are the levels in the four tanks. They are similar to signals given in Fig. 4. The number of samples available for identification is lo00 and the sample time is 10 s. The structure of the MIMO model is selected by using the insight in the physical structure of the system as follows:

Figure 2. Four cascaded tanks.

This system is described by the following nonlinear differential equations: h = A& BQ, with

-kfi S, 1l

-%A
0

~ 4 . 1 2 -

r3,2%*

A=[

0
0

-e&
0

r4,28d%

r o

-e&

0 1

The first row of the ny matrix states that level hl(k 1) depends on hl(lc),h3(k) and h4(k), but not on h2(k),see Fig. 2. Similarly, the third row of this matrix states that h (k 1) depends on h3 (k), 3 but not on the other variables. The meaning of n, and n d should be clear. To construct the fuzzy model in MATLAB,the data is first loaded and the structural parameters of the model are defined:

where h are the liquid levels, SI is the area of the tank m2), 52 is the area of the outlet m2), ri,j is the restriction parameter from tank i to tank j (731 = ~ 4 , 2= 0.8), Q is the inlet flow rate, and g is the gravity constant. This model was simulated in Simulink in order to obtain input-output data sequences for identification. The input signals are composed of a low-pass filtered normally distributed white noise to which white noise with a small amplitude is added, see Fig. 3. The low-frequency component signal drives the nonlinear system through the entire operating range, while the high-frequency component takes care for persistent local excitation.

load 114data c = [ 3 3 3 31; m = 2.2; to1 = 0.01; FMtype = 1; ny = [l 0 1 1; 0 1 1 1 ;

0 0 nu = [ O 0 0 1 0 ; 0 0 11; 0; 0; 1 0; 0 11; nd = [ O 0; 0 0; 1 0: 0 11;

% % % % % %

load input-output data number of clusters fuzziness parameter termination criterion product-space M F S output lags

% input lags

% transport delays

Now the fuzzy model can be constructed by using input-

660

output data U and y, and validated by simulation on a different data set (ue, ye):
FM = fmclust (u,y, FMtype,m, tol,O,ny,nu,nd) ; c, [ym,VAFl = fmsim(ue,ye,m) ;

The validation data set is obtained by interchanging the two input signals. A comparison of the outputs of the fuzzy model with the process data is shown in Fig. 4.

was presented. A simple simulation example is given to illustrate the definition of the various parameters and also the software implementation in MATLAB.The method was applied to various problems, including two FAMIMO benchmarks (engine power-train management and a waste-water treatment process) and an air-conditioning system [9]. Due to the lack of space, the fuzzy model could not be presented here. It appears, however, that the interpretation of the consequent parameters in relation to the antecedent operating regions is not as straightforward as with SISO or MIS0 models. This is one of the aspects for further research. Acknowledgement: This research was supported by Esprit LTR project FAMIMO (Fuzzy Algorithms for the Control of Multi-Input, Multi-Output Processes), no. 21911.

"

'

"

"
2 3 4 5

"
6

'

1 io

APPJWDIX: Gustafson-Kessel algorithm

This appendix presents the GK clustering algorithm and its implementation in MATLAB.The superscript 1 denotes the iteration number.
Given Z, choose 1 < c < N,m > 1 and E a random. t
Repeat for 1 = 1,2,. . .
Srep I : Compute cluster means:

> 0. Initialize U(O)

Step 2: Compute covariance matrices:

0 1 2 3

5
Time [ldSI

IO

Figure 4. Comparison o the process output (solid f line) with the fuzzy model output (dashed-dotted line).

For a comparison, a 4th-order linear state-space model was identified from the same data set by means of a subspace identification technique [16]. The VAF performance indices for the TS fuzzy model are: VAF = t99.41, 99.46, 99.47, 99.811, and for the linear state-space model: VAF = i95.37, 85.11, 96.88, 89.641. It is clear that the fuzzy model is considerablymore accurate than the linear one.

6 Conclusions
In this paper, a method to approximatenonlinear MIMO dynamic processes by means of input-output TS fuzzy models

66 1

function [U,V,Fl = gk(Z,UO,m,tol) % Clustering with fuzzy covariance matrix

8 (Gustafson-Kessel algorithm)
% % %

n Intelligence i Real Erne Control, pages 61-66, Valencia, Spain, October 1994.
[5] R. BabuSka and H.B. Verbruggen. Identification of composite linear models via fuzzy clustering. In Proceedings European Control Conference, pages 12071212, Rome, Italy, September 1995. [6] T. Takagi and M. Sugeno. Fuzzy identificationof systems and its application to modeling and control. ZEEE Trans. Systems, Man and Cybernetics, 15(1):116-132, 1985. [7] M. Sugeno and T. Yasukawa. A fuzzy-logic-based approach to qualitative modeling. IEEE Trans. Fuzzy Systems, 1:7-3 1, 1993. [8] R. BabuSka, H.A.B. te Braake A.J. Krijgsman, and H.B. Verbruggen. Comparison of intelligent control schemes for real-time pressure control. Control Engineering Practice, 4(11): 1585-1592,1996. [9] J.M. Sousa, R. BabuSka, and H.B. Verbruggen. Fuzzy predictive control applied to an air-conditioning system. ControlEngineering Practice, 5( 10):1395-1406, 1997. [lo] D.E. Gustafson and W.C. Kessel. Fuzzy clustering with a fuzzy covariance matrix. In Proc. IEEE CDC, pages 761-766, San Diego, CA, USA, 1979.
[ 113 R. Kruse, J. Gebhardt, and F. Klawonn. Foundations of Fuuy Systems. John Wiley and Sons, Chichester,

[U,V,Fl = GK(Z,UO,m,tol)

%-----------------------------------------------------

Input: Z % U0 % m % to1
% %
% %

...

...
...

N by n data matrix

%-----------------------------------------------------

initial fuzzy partition matrix fuzziness exponent (m > 1) . . _termination tolerance

Output: U

v
F

... ...

...

%----------------------------------------------------%

fuzzy partition matrix cluster means (centers) concatenatea cluster covariance matrices F = [Fl;FZ;...Fcl

(c) Robert Babuska, 1996

prepare matrices
%

%e----------------

------------------

[mz,nzl = size(Z); c = size(UO,2); mZ1 = ones (mz, ; 1) nZ1 = ones(nz,l); Vlc = ones(1,c); U = zeros(mz,c); d = U; F = zeros(c*nz,nz); $----------------iterate

%
%

8
% % %

...........................

data matrix size number of clusters auxiliary variable auxiliary variable auxiliary variable partition matrix distance matrix covariance matrix

while max(max(U0-U)) > to1 % U = UO; Um = U.^m; sumU = sum(Um); % V = (Um'*Z)./(nZl*sumU)'; % 8 for j = 1 : c, ZV = Z - mZl*V(j,:); % % f = nzl*Um(:,j)'.*zv'*Zv/s~u(j);

iteration loop aux. vars clust. centers for all clust. auxiliary var cov. matrix d(:,j)=sum((ZV*(det(f)^(l/nz)*inv(f)) .*ZV)')'; end; d = (dtle-lO).-(-l/(m-l)); % distances 8 part. matrix U0 = (d . / (sum(d')'*Vlc)); end
%-----------------

update final F and U

--------------

U = UO; Um = U.^m; sumu = nZl*sum(Um); for j = 1 : c, ZV = Z - mZl*V(j,:);

F((j-l)*nz+(l:nz),:)=nZ1*Um(:,j)'.*ZV'*ZV/sumU(l,j); end;

1994.
[ 121 R. Babulka. Fuzzy Modeling and ZdentiJication. PhD

a d of function

-------------------

dissertation, Delft University of Technology, Delft, The Netherlands, 1996. [13] I. Gath and A.B. Geva. Unsupervised optimal fuzzy clustering. IEEE Trans. Pattem Analysis and Machine Intelligence, 7:773-781,1989. [14] R. Krishnapuram and Chin-Pin Freg. Fitting an unknown number of lines and planes to image data through compatible cluster merging. Pattem Recognition, 25(4):385-400,1992. 151 U. Kaymak and R. BabuSka. Compatible cluster merging for fuzzy modeling. In Proceedings FUZZIEEE/FESS'95,pages 897-904, Yokohama, Japan, March 1995. 161 M. Verhaegen and P. Dewilde. Subspace model identification. Part I: the output-error state space model identification class of algorithms. Zntemational Journal o f Control, 56:1187-1210, 1992.

References
[ 13 I.M. Kouatli. A simplified fuzzy multivariable struc-

ture in a manufacturingenvironment. Joumal o Intelf ligent Manufacturing, 5:365-387, 1994. [2] Y.-2. Lu, M. He, and C.-W. Xu. Fuzzy modeling and expert optimization control for industrial processes. IEEE Trans. on Control Systems Technology, 5:2-12, 1997. [3] A. Gegov. Multilayer fuzzy control of multivariable systems by active decomposition. Znt. Joumal of Intelligent Systems, 12, 1997. [4] R. BabuSka and H.B. Verbruggen. Applied fuzzy modeling. In Proceedings IFAC Symposium on Artijicial

662