See

discussions, stats, and author profiles for this publication at: http://www.researchgate.net/publication/3335794

Reduction of fuzzy rule base via singular value

decomposition
ARTICLE in IEEE TRANSACTIONS ON FUZZY SYSTEMS MAY 1999
Impact Factor: 6.31 DOI: 10.1109/91.755394 Source: IEEE Xplore

CITATIONS

DOWNLOADS

VIEWS

160

61

152

3 AUTHORS, INCLUDING:
Peter Baranyi
A) Budapest University of Technology and Ec
271 PUBLICATIONS 2,244 CITATIONS
SEE PROFILE

Available from: Peter Baranyi

Retrieved on: 22 July 2015

120

IEEE TRANSACTIONS ON FUZZY SYSTEMS, VOL. 7, NO. 2, APRIL 1999

Reduction of Fuzzy Rule Base Via

Singular Value Decomposition
Yeung Yam, Member, IEEE, Peter Baranyi, and Chi-Tin Yang

AbstractThis paper introduces a singular value-based method

for reducing a given fuzzy rule set. The method conducts singular
value decomposition of the rule consequents and generates certain
linear combinations of the original membership functions to form
new ones for the reduced set. The present work characterizes
membership functions by the conditions of sum normalization
(SN), nonnegativeness (NN), and normality (NO). Algorithms
to preserve the SN and NN conditions in the new membership
functions are presented. Preservation of the NO condition relates
to a high-dimensional convex hull problem and is not always
feasible in which case a closed-to-NO solution may be sought. The
proposed method is applicable regardless of the adopted inference
paradigms. With product-sum-gravity inference and singleton
support fuzzy rule base, output errors between the full and reduced fuzzy set are bounded by the sum of the discarded singular
values. The present work discusses three specific applications of
fuzzy reduction: fuzzy rule base with singleton support, fuzzy
rule base with nonsingleton support (which includes the case
of missing rules), and the TakagiSugenoKang (TSK) model.
Numerical examples are presented to illustrate the reduction
process.
Index TermsFuzzy systems, reduced order systems, singular
value decomposition.

I. INTRODUCTION

HE advantage of fuzzy control lies in its ability to mimic

and implement the actions of expert operator(s) without
the need of accurate mathematical models. The drawback,
however, is that there is no standardized framework regarding
the design, optimality, reducibility, and partitioning of a fuzzy
rule set. A fuzzy rule base, be it generated from expert
operators or by some learning or identification schemes, may
contain redundant, weakly contributing, or outright inconsistent components. Moreover, in pursuit of good approximation,
one may be tempted to overly assign the number of antecedent
sets, thereby resulting in large fuzzy rule bases and problems
in computation time and storage space. A formal approach to
extracting the more pertinent elements of a given rule set is,
hence, highly desirable. The present work is an attempt in this
direction.
Manuscript received June 27, 1997; revised August 26, 1998. This work was
supported by the Hong Kong Research Grant Council (RGC) CUHK4138/97E,
the Hungarian Ministry of Culture and Education (MKM) FKFP 0422/1997,
and the Hungarian National Science Research Foundation (OTKA) T019671.
Y. Yam and C. T. Yang are with the Department of Mechanical and
Automation Engineering, The Chinese University of Hong Kong, Shatin, N.T.,
Hong Kong.
P. Baranyi is with the Computer and Automation Institute, Hungarian
Academy of Science and the Department of Automation, Technical University
of Budapest, Budafoki u. 8, H-1111 Hungary.
Publisher Item Identifier S 1063-6706(99)02800-3.

In this paper, a method to reducing a fuzzy rule set by

capturing a large extent of its input/output characteristics is
introduced. The method is based on conducting singular value
decomposition (SVD) of the rule consequents and generating
proper linear combinations of the original membership functions to form new ones for the reduced set. Instead of assuming
certain specific shapes for the membership functions, here we
characterize the membership functions by the conditions of
sum normalization (SN), nonnegativeness (NN), and normality
(NO). Algorithms to preserving the SN and NN conditions in
the new membership functions are presented. Preservation of
the NO condition, however, involves the solution to a convex
hull problem and is not always achievable.
The proposed approach can be applied regardless of the
inference paradigm adopted for the fuzzy rule base. For
product-sum-gravity inference and singleton support rule base,
output error bound between the original set and the reduced
set is readily expressible as the sum of the discarded singular
values. The method is applicable to fuzzy rule set with any
number of input variables, but is illustrated here mostly with
the example of a two input fuzzy system. Three specific cases
of fuzzy reduction are discussed. They are fuzzy rule base
with singleton support case, fuzzy rule base with nonsingleton
support case (which includes the case of missing rules), and
the TakagiSugenoKang (TSK) model [1]. These three cases
represent situations where the output of a fuzzy rule is defined
by one, two, or more parameters.
The present work constitutes a detailed investigation of the
preliminary approaches outlined in the works of [2][4], and
gives rigorous solutions to the problems posed thereof. The
algorithms to be utilized here are mostly developed in [5]
for fuzzy approximation of general functions. Application of
SVD to fuzzy reduction has been investigated recently by
Wang et al. [6]. Their approach utilizes SVD to reduce the
dimensionality of the input space and then performing system
reduction based on optimizing a certain objective functions.
Their work uses B-splines as the membership functions. The
present work, on the other hand, applies SVD directly to the
rule consequents of the given rule set, and works with the
membership functions constrained only by the conditions of
SN, NN, and NO.
This paper is organized as follows. Section II first defines
the various concepts to be used in the proposed method.
Section III presents the basic operations of SVD reduction
for the example of a two input fuzzy rule set. Section IV
extends the SVD operations to fuzzy rule set of three input
variables. Extension to fuzzy set with a general number of

10636706/99$10.00 1999 IEEE

YAM et al.: REDUCTION OF FUZZY RULE BASE VIA SINGULAR VALUE DECOMPOSITION

inputs then follows readily. Section V applies fuzzy reduction

to the cases of singleton support, nonsingleton support, and the
TSK model, with a numerical example for each. And finally,
Section VI presents the conclusions.

121

Theorem 1: Given a set of function

and a matrix
of dimension
by
as
functions

, one forms

new

(4)
II. DEFINITION

OF

BASIC CONCEPTS

This section introduces the basic concepts to be used in the

ensuing SVD operations. These concepts are first utilized in
,
[5] for fuzzy approximation of general functions. Let
be functions of variable
defined on some
. The following properties can
compact domain
.
be defined for the function set
Sum Normalization (SN): A set of function
is SN if for any value of
within the domain
of interest

Then
1) the set of
,
2) the set of
,
3) the set of
,
Proof: For

functions
and

is SN if
are SN;

functions
and

is NN if
are NN;

,
is NO if
and
are NO.
within the domain of interest, one has

functions
any

(1)
Without ambiguity, a matrix

is SN if

..
.

(2)

denotes the column vector obtained by sumwhere

ming over the rows of matrix .
Nonnegativeness (NN): A set of function
is NN if for any value of within the domain
of interest
(3)
. Likewise, a matrix is NN if every
for each
is greater than or equal to zero.
one of its elements
The concepts of SN and NN are consistent with the usual
definition of membership function. Note that
being SN and NN implies that
for all within the domain of interest. Similarly, matrix
being SN and NN implies
for all of its element.
Another property of the membership functions is normality.
Normality (NO): A set of function
is NO if it is SN and NN and that each of the functions
attains value one at some point within the domain of
. Correspondingly, a matrix is NO if it is SN and NN and
that each of its column contains the value one as an element.
For a set of membership functions, the NO condition
implies a certain localization property, i.e., when one of the
functions achieves membership degree of approximate unity,
the remaining ones would yield close to zero membership
degrees. Hence, the membership functions take turns dominating different regions within the domain of interest. As a
result, linguistic labelings are readily assigned to membership
functions satisfying the NO condition.
The above definitions enable the following result to be
stated.

which proves 1). The proof of 2) follows straightforwardly

constitutes the sums and products of positive
as each
and
. To prove 3) since
and are
quantities
is also
NO and, hence, SN and NN; then by 1) and 2),
within the domain of
SN and NN. This implies
. To show that
attains the value of one at some , one
is NO, there exists a value
notes the following. Since
within the domain of such that
. Similarly, since
is NO, there exists integer ,
, which gives the
attaining the
location of the element in the th column of
. As a result
value of one, i.e.,

and, at

,
(5)

Now using the fact that

NN, one has at

Since
,
,
This proves 3).

, this implies
, . Hence, (5) yields

is SN and

for

,
at

,
.

122

IEEE TRANSACTIONS ON FUZZY SYSTEMS, VOL. 7, NO. 2, APRIL 1999

III. SINGULAR VALUE-BASED REDUCTION

This section outlines the procedures to conduct singular
value-based reduction in terms of SN and NN, and possibly
NO, matrices. Details of the method can be found in [5]. The
procedures are presented here with a two-dimensional (2-D)
matrix and then extended to higher dimension in the next
section.
in SVD form
Consider a 2-D matrix

we desire
to be invertible matrix. We adopt the following
does not contain
scheme for constructing : if
is chosen as
zero elements, the matrix
(10)
contains zero element(s),

and if

is chosen as

(6)
by
and and are
by
and
by ,
where is
and
are orthogonal, i.e.,
respectively. Matrices
and
. Here,
denotes the by
identity matrix. For later reference, we also introduce
as
as the by matrix of
the by matrix of zeros, and
by
matrix contains the singular values of
ones. The
in decreasing magnitude. The maximum number of nonzero
. The singular values
singular values is
indicate the importance of the corresponding columns of
and
in the formation of . A close approximation to
can be obtained by keeping those components having large
singular values.
be the number of singular values to keep and,
Let
hence, an approximation of
(7)
and
contain the
columns of
and
where
corresponding to the retained singular values in
.
Approximation is exact if all singular values are kept,
. Matrices
and
are in general neither SN nor
NN. However, they can be converted into SN and NN matrices
by the following procedures. Take , as an example, and
consider the SN condition first; we have Theorem 2.
be an
by
orthogonal matrix
Theorem 2: Let
, where
and
have,
partitioned as
and
columns. Let
be
respectively,
by
matrix satisfying the constraint
any
. Then, if
, the
by
matrix
satisfies SN, where
(8)
and if
satisfies SN, where

, the

by

matrix

(11)
is
where one ensures that the th entry of
is chosen, one can proceed to compute
nonzero. After
For

For

(12)

,
, and
as results of
Similar expressions hold for
applying the above procedures to matrix . The approximation
(7) can then be written as

(13)
and
. Other cases corresponding
and/or
to
follow in a straightforward manner. The matrices
and
are SN. They have either
or
columns depending on whether
or
.
and
contain negative
It is possible, still, that
elements and, hence, not satisfying the NN condition. The
following gives a set of procedures to generate from a matrix
satisfying the SN condition another one of the same dimension
satisfying both SN and NN conditions. Let be a matrix of
by
satisfying the SN condition. Then
dimension
note the following.
, of . Set
1) Look for the minimum element
parameter
if

for the case when

(9)
Theorem 2 characterizes the extra column one needs, if at
so as to achieve the SN
all, to supplement the matrix
condition. The proof of the theorem can be found in [5] and
is omitted here. Note that the theorem does not require
to be invertible. In theory,
may contain zero
are not
elements, which implies that some columns of
necessary to satisfy the SN condition. If so, a noninvertible
can actually be used for (8) or (9). For the present work,
and
however, we prefer to retain all columns of
so as to facilitate a close approximation of
in (7). Hence,

(14)

otherwise.

2) Form a

by

..
.

matrix

..
.

..

..
.

(15)

YAM et al.: REDUCTION OF FUZZY RULE BASE VIA SINGULAR VALUE DECOMPOSITION

3) The matrix product

satisfies the SN and NN
conditions.
is adopted to stress the dependence of
The subscript in
on . To see that
is SN, one notes that
and, hence,
as
is SN to begin with. To see that
is also
is given by the
NN, one notes that the th column of
multiplying a column vector that is the
scalar
and
times the th column of , and
sum of
ensures proper scaling to result in positive
the parameter
values for all entries.
Applying these results to (13), one has
(16)
with

123

3) Check the convex hull. If the convex hull has exactly

vertices, successful incorporation of the NO conis possible. In this case the matrix
dition by
can be obtained as inverse of the matrix containing the
rows of associated with the convex hull. This is
called the tightest bounding case.
vertices, however,
4) If the convex hull has more than
to strictly satisfying the NO condetermination of
dition is generally not possible. In this case, we have
vertices not
to search for a relaxed bounding with
points of . The
all of which coming from the
is then determined according to these
corresponding
vertices. This leads in general to a close-to-NO
.
matrix product
A more detail discussions of these steps can be found in [5].
Carrying out the NO procedures for and , the approximation (20) now becomes

(17)
(21)

(18)
(19)
and
are
by
and
by
,
The matrices
respectively. They satisfy the SN and NN conditions. The same
procedures can be applied to other cases of
and/or
. The
and
will have
or
columns,
resulting
depending on different cases.
The NO condition is now considered. Following the approach for SN and NN, we desire to have invertible matrices
and
of appropriate dimensions such that (16) can be
expressed as
(20)
and
are SN, NN, and
and that the matrix products
NO. However, while it is always possible to tailor the matrices
involved to satisfying the SN and NN conditions, the same is
not true for the NO condition. Successful incorporation of the
NO condition depends very much on the specific matrix at
hand. The following gives a set of procedures which conducts
a tight bounding operation on the matrix and yields, if possible,
a NO matrix or otherwise, a close-to-NO matrix. Here, a closeto-NO matrix is taken to be one that is SN and NN but not all
of its columns containing the value one as element.
,
Take as an example and let its column dimension be
rows of
corresponds to
one first notes that each of the
-dimensional space. Furthermore, since
a point lying on a
is SN, the
points from the
rows of
actually lie
dimension. The procedures then
on a hyperplane of
include the following steps.
points in the
-dimensional space onto
1) Project the
-dimensional hyperplane satisfying the SN
the
condition.
points on the
2) Obtain the convex hull of the
dimensional hyperplane. Algorithms to treat convex hull
problem in a general dimensional space are discussed
in, e.g., [8] and [9].

where

(22)

and
and
are SN, NN, and NO, or close-to-NO.
Several points are worth noticing here. First, The NO
equals to two or .
condition is always possible when
, the hyperplane is a straight-line and one can
For
always find (in this case) a convex hull of two points. For
, the full set of the points constitute the vertices of
and
. This
the convex hull. As such,
corresponds to the trivial case where there is actually no SVD
reduction. In general, however, NO condition is not possible
and one has to adopt a relaxed bounding and settle with closeto-NO condition. Moreover, the choice of a relaxed bounding
is not unique and exact choosing will depend on the matrix at
hand. One may even consider a bounding convex hull which
slightly violates the NN condition at a few points in exchange
for readily bounding on a majority of the points. Note that
out
the scheme of testing various combinations of taking
rows of works only if a tightest bound of the points
of
do exist. The scheme would not be applicable for the relaxed
bounding case.
Second, the reduction , , and of (22) is not unique. In
fact, given that and are SN, NN, and NO, the choice of

is another valid reduction if the invertible matrices

and
of appropriate dimensions are SN, NN, and NO.
and
Third, one observes that the choice of constructing
to satisfy the SN and NN conditions is not unique. An
issue of interest is whether there exists and in what way a more
efficient method to choosing them. In this regards, choices with
better results do exist. The point to note, however, is that no

124

IEEE TRANSACTIONS ON FUZZY SYSTEMS, VOL. 7, NO. 2, APRIL 1999

Fig. 1. SVD reduction for 3-D matrix.

matter the initial choices of

and
, they could actually
yield the same final result after the bounding procedures,
as topology would dictate similar relative distribution of the
points and bounding convex hull. Interested readers may refer
to [5] for more detail discussion of the above issues.
As a final topic in this section, we derive an expression
for the error due to the SVD reduction. One notes here that
incorporating the SN and NN conditions and carrying out the
NO procedures do not actually change the approximation of
by
. This is because
(23)
Hence, the approximation error can be expressed as
(24)
in (24) as singular
Absolute value is not required for
and
values are always positive. Since the columns of
are having Euclidean norm of unity, absolute value of
their elements must be bounded by one. Thus

(25)
denotes the th singular value in . Hence, the error
where
using the approximation
introduced into each elements of
(21) is less than or equal to sum of discarded singular values.

IV. GENERALIZATION TO HYPERDIMENSIONAL MATRIX

The previous section gives the SVD-based procedures for
when is a 2-D
constructing an approximation
matrix, with and being SN, NN, and NO, or close-to-NO.
The approximation can be expressed in terms of the elements
of the various matrices
(26)
and
are the number of columns of the matrices
where
and . By previous results,
and
equal
or
,
depending on the specific case at hand.
The procedures developed above can be readily extended to
matrix of higher dimension. In [5], extension of the procedures
to a three-dimensional (3-D) matrix is given. The extended
procedures convert a 3-D matrix into a sequence of 2-D ones
where SVD can be applied. To provide a basic idea of the
process involved, a pictorial depiction of the procedures is
by
by
matrix
given in Fig. 1. Upon given a 3-D
[step (i)], the extended procedures call for spreading the matrix
by
matrix
in the -index direction to form a 2-D
[step (ii)], and then apply the SVD reduction of Section III to
[step (iii)]. This yields
(27)
is now SN, NN, and NO, or close-to-NO. The
where
dimension of is
by
where
or
,
by
matrix
denotes the
as the case may be. The

YAM et al.: REDUCTION OF FUZZY RULE BASE VIA SINGULAR VALUE DECOMPOSITION

resultant matrix to the right of . One can now restack

to become a 3-D
by
by
matrix [step (iv)], which can
then be spreaded, this time in the -index direction, to form
by
matrix
[step (v)]. Again, applying the
a 2-D
[step (vi)], one obtains
SVD reduction procedures to

125

where
and
,
,
, are
membership functions of the variable and , respectively,
is the rule consequent of the
th fuzzy rule. To
and
by
matrix
perform rule-base reduction, one forms a
with
as the
elements

(28)
is SN, NN, and NO, or close-to-NO. There are
or
columns in . The matrix
is
by
. Restacking
into a
by
by
matrix
[step (vii)] and spreading it in the -index direction to form
[step (viii)], one has, once more, after SVD reduction on
[step (ix)]

where

(29)
is SN, NN, and NO, or close-to-NO, and there
The matrix
or
columns in
. The
may be
is
by
. It can be restacked into a 3-D
matrix
by
by
matrix
[step (x)]. This completes the process
of generating all necessary quantities for the reduction of the
given matrix . The resulting approximation to the elements
, can be expressed as
of ,

..
.

..
.

..
.

..
.

and then applies the SVD procedures to obtain

(34)
where
by

, , and are, respectively,

by ,
by , and
matrices. A reduced rule base can then be obtained as
If

and

with
and
,
membership functions

. Here, the new

are given by
(35)
(36)

(30)
Equation (30) is a 3-D generalization of (26). One can also
,
work out an approximation error bound for
which is
(31)
,
, and
are sum of the discarded singular
Here,
, , and
.
values in the SVD reduction of matrices
They represent the errors induced in the process of obtaining
, , and
. Interested readers are referred to [5] for the
mathematical details of these results.
The above procedures structures a 3-D matrix so that
SVD reduction can be performed in stages. At each stage,
approximation and conditioning of the matrix is conducted
for one of the dimensions and a certain singular value error
is generated. With additional stages and proper indexing, the
procedures can be extended to matrix of a general number of
dimensions.
V. APPLICATION TO FUZZY RULE-BASE REDUCTION
The methodology developed above is now applied to the
actual reduction of fuzzy rule bases. For convenience, fuzzy
rule bases of two input variables are utilized for illustration
of the various cases. As mentioned before, application can be
readily extended to fuzzy rule base with a general number of
inputs.

If

and

and the new rule consequent

of ,

is given by the

element
(37)

fuzzy rules as compared

The reduced rule base has
of the original one. Since
is definitely SN and
to
,
NN, by Theorem 1 the membership function set
is SN and NN if the original set
,
is SN and NN. If
is NO as well, the new
membership functions are also NO if the original membership
,
functions are NO. If is only close-to-NO, however,
will only be SN and NN, but not NO in
general. The new membership functions and rule consequents
can be interpreted as the de facto fuzzy rule components
embedded in the operation of the original rule base. These
results apply to the nonsingleton support and the TSK model
cases to follow as well.
The above scheme can be applied to any fuzzy rule base
regardless of the adopted inference paradigm. Compact expression for the output error bound, however, can be obtained
if product-sum-gravity (PSG) inference is adopted. Given that
,
, the output error is given by

(38)

A. Fuzzy Rule Base with Singleton Support

Consider a fuzzy rule base with two inputs
single output ,

(33)

and

and a
(32)

,
,
,
where the fact that membership functions
satisfying the SN condition has been used. The first
and second term on the right-hand side of (38) correspond,

126

IEEE TRANSACTIONS ON FUZZY SYSTEMS, VOL. 7, NO. 2, APRIL 1999

TABLE I
FUZZY RULE CONSEQUENT ri; j OF THE ORIGINAL RULE BASE

respectively, to the PSG-inferred outputs of the original and

the reduced rule base. Substituting (35)(37) into (38)
(39)
and, with (25)
(40)
Hence, the sum of the discarded singular values constitutes an
upper bound to the output error incurred by the reduction. If
one is to include all nonzero singular values in the reduction,
the reduced rule base would yield the same output as the
original one with PSG inference. Error bounds for other
inference paradigms can also be derived but the final form
is not as pleasing.
Example 1: Consider one of the examples from [10] with
a fuzzy rule base of input variables and . The membership
functions of are overlapping isosceles triangles of basewidth
to
at separations
0.08 centering from
of 0.04. The membership function of are similar but with
to
basewidth of 0.6 and centering from
at separations of 0.3. The original example in [10]
utilizes minmax inference paradigm and assigns triangular
membership functions to the fuzzy outputs. In this example,
however, we adopt the PSG inference to illustrate the reduction
process. As a result, the fuzzy rules can be expressed as
If

and

where
s are the center points of the output membership
functions in the original example of [10] and are tabulated
on which the proposed
in Table I. They form the matrix
reduction method is performed.
are determined as 22.6767,
The singular values of
22.6767, 0.9835, 0.9835, 0.5523, 0.5523, 0.3433, 0.3433, 0 .
Keeping only the two largest singular values and going through
the SN and NN processes yield three membership functions
for and for . The convex hull problem then amounts to
finding a bounding triangle on a 2-D plane and can be readily
for
carried out. Fig. 2 shows the membership function
. The membership functions of are similar except

Fig. 2. Membership functions of a for the reduced rule base.

TABLE II
FUZZY RULE CONSEQUENT i; j OF THE REDUCED RULE BASE

for the domain of interest. In this case, only close-to-NO

condition can be achieved, leading to only two out of the three
membership functions for and attaining value of one within
are given in Table II.
their ranges. The rule consequents
The reduced rule base contains nine rules, as compared to
the original set of 81, in the form of
If

and

For comparison, Fig. 3 shows the PSG-inferred output using

the original fuzzy rule base of 81 rules (top plot) and the
reduced rule base of nine rules (bottom plot). To check
how close the reduced rule set duplicates the output of the
original set, the rule bases are applied to controlling an
inverted pendulum given in [10]. The dynamics equation of

YAM et al.: REDUCTION OF FUZZY RULE BASE VIA SINGULAR VALUE DECOMPOSITION

(a)

127

Fig. 4. Closed-loop responses due to original and reduced rule bases.

The SN and NN conditions, however, can always be achieved

regardless the number of membership functions.
B. Fuzzy Rule Base with Nonsingleton Support
In the nonsingleton support case, each of the fuzzy rules
, which can
in (32) is associated with a support factor
be interpreted as a firing strength or a reliability coefficient.
and
, and assuming PSG inference,
Given that
the inferred output is now generated as

(42)

(b)
Fig. 3. PSG-inferred control surfaces due to (a) original and (b) reduced
rule bases.

the pendulum is

Equation (42) indicates that reduction in this case involves a

numerator part and a denominator part, in contrast to just the
numerator part in the singleton support case. To proceed with
by
by two matrix
the reduction, one forms a 3-D
with elements
(43)

(41)
where is the angular position, is the angular velocity, and
is the control input. Fig. 4 shows the closed-loop responses
due to the original and reduced rule bases for the initial
and
conditions (0.08, 0) and (0.12, 0.8). Here,
are used as fuzzy variables in the inference of . Note that
have been scaled down by a factor of 0.1
the responses
in the figure. It can be observed that closed-loop responses
produced with the reduced rule base are quite close to that of
the original one.
The present numerical example yields three membership
functions for the reduced set. As a result, the bounding
procedures is readily executed in a 2-D plane. In the case
where the number of membership functions is larger than
three, the bounding procedures require a solution to a hyperdimensional convex hull problem, which can be quite tedious.

and then applies the extended procedures in Section IV as

follows. Referring to Fig. 1, one first carries out steps (i)(iii)
by
matrix for variable , and then steps
to obtain a
by
matrix
for variable . In the
(iv)(vi) to obtain a
remaining steps (vii)(ix), however, one keeps both singular
without any reduction, i.e., one has
.
values of
As such, (30) becomes in this case
(44)
where matrices
and
are SN, NN, and NO,
for
is
by
by two. One
or close-to-NO, and the matrix
can now write a reduced fuzzy rule base as
If

and

128

RULE CONSEQUENT i; j

IEEE TRANSACTIONS ON FUZZY SYSTEMS, VOL. 7, NO. 2, APRIL 1999

OF THE

TABLE III
GIVEN FUZZY RULE BASE WITH MISSING RULES

where
and
are the new membership functions. The new rule conseand support factor
are computed as
quent

Again, the reduced set has

fuzzy rules as compared to
rules.
the original of
Two special cases are worth noticing here. First, when
, the results
the supports are singleton, i.e., when
here reduce to that of the singleton support case. This can
can be
be observed from the fact that (44) with
decomposed into the following two parts

Fig. 5. Membership functions of a for the reduced rule base.

TABLE IV
RULE CONSEQUENT r {; | AND SUPPORT
FACTOR s{; | FOR THE REDUCED RULE BASE

(45)

(46)
where one recognizes that (45) is the reduction problem for the
singleton support case and, if one were to obtain the matrices
and
and values
from (45) only, then (46) would
. This is because
automatically be satisfied with
and are SN and NN so that the right-hand side of (46) would
become one. In effect, (46) indicates that singleton support rule
base would be reduced to another singleton support rule base
and the reduction process would be dictated solely by (45),
the same as for the singleton support case.
Second, the nonsingleton support case considered here includes fuzzy rule base with missing rules as special case. In
is assigned value zero for
this case, the support factor
the missing rules and one for the included rules. This case is
further explored by the use of following numerical example.
Example 2: Consider another example from [10]. The
are now isosceles triangles of
membership functions of
to
basewidth 0.16 with the centers located from
at separations of 0.08. The membership functions
for are also isosceles triangles but with basewidth of 0.12 and
to
at separations
the centers going from
are given in Table III, where
of 0.6. The rule consequents
the blank entries correspond to missing rules. The number of
fuzzy rules in this case is 17. The rule set can be expressed as
for the missing
a nonsingleton support system, with
for the rest.
rules and
Application of the SVD procedures yields the following
singular values: 13.0860, 12.7475, 2.0633, 2.0, 0 . Again,
keeping only the two largest singular values, one obtains

three membership functions for and for . Fig. 5 shows the

of ,
. Membership
membership function
functions of are again similar except for the different domain
of interest. In this case, the bounding procedures are able to
achieve the NO condition for the membership functions. The
and support factor
for the
resultant rule consequents
reduced set are given in Table IV.
The reduced rule set in this case has nine rules and is
expressed as
If

and

with support factor

. Note that in this example negative
support factors are obtained. To gain more insight, Fig. 6(a)
shows the inferred outputs for the original and Fig. 6(b) shows
the reduced rule base adopting the inference of (42). Because
of the missing rules, the output surface of the original rule
domain.
base is not defined in certain region on the
By comparison, the output surface for the reduced case is
defined on a larger region in the domain. Furthermore, because
of the negative support, the reduced rule base actually yields
. Fig. 6 has in fact
infinite outputs at certain values of
been constrained to show only the output surface with values
between 6. One may thus claim that the SVD procedures has
extrapolated somewhat the original surface to a wider region
than it is initially defined. There is, however, a limit to the
extent of this extrapolation; the negative supports may cause
the interpolated surface to go to infinity if one ventures too

YAM et al.: REDUCTION OF FUZZY RULE BASE VIA SINGULAR VALUE DECOMPOSITION

(a)

129

Fig. 7. Closed-loop responses due to original and reduced rule bases.

reduced rule base. The closed-loop responses hence exhibit

smoother trajectories.
C. TakagiSugenoKang (TSK) Model
The TSK model utilizes functions of the input variables as
fuzzy outputs. For a fuzzy system with input variables and
, the fuzzy rules are expressed as
If

and

To illustrate application of the reduction procedures, we here

to be consisting of only three terms, a
confine
constant term, and linear terms in and in , i.e.,
(47)
(b)
Fig. 6. PSG-inferred control surfaces due to (a) original and (b) reduced
rule bases.

far out. The extrapolation property of the SVD procedures

and how it relates to existing interpolation techniques, say,
e.g., [7], is a research topic of interest.
For further comparison, the two rule bases are again utilized
to controlling the inverted pendulum of (41). Fig. 7 plots the
closed-loop responses due to the original and reduced rule
bases for the initial conditions of (0.08, 0) and (0.12, 0.8).
The first set of initial conditions and its subsequent responses
lie inside the region where the original fuzzy rule base is well
defined. The second set of initial conditions, however, starts
at a region where the original output surface is not defined.
To carry out the simulation, we let the output be zero in
the undefined region. The idea here is to let the pendulum
dynamics return itself to where the original output is defined.
response at around
s shows a slight change
The
in the decay rate which is indicative of this transition. On
the other hand, both set of initial conditions and subsequent
trajectories remain in the extended well-defined region of the

In this case, one forms a

elements

by

by three matrix

with
(48)

and applies the extended SVD procedures of Fig. 1. Similar to

the nonsingleton case, here we apply the reduction procedures
by
and
by
matrices
and ,
to obtain the
but in step (ix) we keep all three singular values for , i.e.,
. As a result, (30) becomes
(49)
. The matrices and are SN, NN, and NO,
for
or close-to-NO. The reduced TSK model can be written as
If

and

where
and
are the new membership functions. The output functions
for the reduced set are given by
(50)

130

IEEE TRANSACTIONS ON FUZZY SYSTEMS, VOL. 7, NO. 2, APRIL 1999

(a)

(a)

(b)
Fig. 8. Membership functions of (a)

a and (b) b for the original rule base.

TABLE V

PARAMETERS

i; j , i; j , AND i; j

FOR

FUNCTION

(b)
Fig. 9. Membership functions of (a)
#2.

gi; j (a; b)

TABLE VI

PARAMETERS

where

The reduced set now has

fuzzy rules compared to the
.
original number of
Example 3: In their work [1], Takagi and Sugeno considered the following fuzzy rule base:
If

a for case study #1 and (b) case study

and

with
and
. The functions
are of the form (47) with parameters
,
, and
as
tabulated in Table V. The membership functions for and
are as shown in Fig. 8. They do not satisfy the SN condition.
Moreover, the example utilizes an inference paradigm, which,
and
, generates the output as
upon given

(51)

The reduction procedure of Fig. 1, however, can be applied

just the same. Two case studies are conducted here. From step
(iii), the singular values of the matrix are: 15.9065, 3.6559,
2.1562, 0 . The first case study keeps all three nonzero singular

{; | ,  {; | , AND {; |

FOR

FUNCTION

g{; | (a; b)

values of , as well as the two nonzero singular values of

in step (vi), which are 13.3853, 3.6433 . This yields three
membership functions for variable and two for . The top
for the
plot of Fig. 9 shows the membership functions
are the same
first case study. The membership functions
as the original ones in Fig. 8. The corresponding function
is given by:
, where
,
, and
are shown in Table VI.
the parameters
The top plot of Fig. 10 shows the output surface of the
reduced TSK model for the first case study according to the
inference of (51). The output surface turns out to be the same
as that due to the original rule set. Hence, the original TSK
model of eight rules can be replaced by the presently reduced
one of six rules without any output error.
Instead of retaining all nonzero singular values in the
reduction process, the second case study keeps only the largest
and
in steps (iii) and (vi). As a result,
singular value of
two membership functions each for and are obtained. The
bottom plot of Fig. 9 shows the membership functions
for this case. Again, the corresponding membership functions
are identical to
in Fig. 8. The reduced TSK model
is now comprised of four rules instead of the original eight.
,
, and
giving rise to functions
The parameters
in this case are tabulated in Table VII.
The bottom plot of Fig. 10 shows the output surface of
the reduced TSK model for the second case study using the

YAM et al.: REDUCTION OF FUZZY RULE BASE VIA SINGULAR VALUE DECOMPOSITION

(a)

(b)
Fig. 10. Control surfaces due to reduced rule bases in (a) case study #1 and
(b) case study #2.
TABLE VII

PARAMETERS

{; | ,  {; | , AND {; |

FOR

FUNCTION

g{; | (a; b)

inference of (51). Comparing the output surface of the first case

study, which is the same as the original one, it can be observed
that appreciable error is generated for the region
and
.
VI. CONCLUSIONS
This paper introduces a new reduction method for fuzzy
rule base. The method calls for forming a matrix with the rule
consequents and then applying singular value decomposition.
The resultant singular values and orthogonal matrices are then
tailored to form linear combinations upon which membership
functions and rule consequents of the reduced set are generated
from the original ones. Given that the original membership

131

functions are sum normalized and nonnegative, the procedures

guarantee that membership functions for the reduced set are
sum normalized and nonnegative as well. Normality or closedto-normality condition for the reduced set, however, depends
on the nature of the bounding solution to a convex hull
problem. The proposed method is applicable independent
of the inference paradigm. In the special case when PSG
inference is being used and that the fuzzy rule base has
singleton support, output error due to reduction is readily
bounded by the sum of the discarded singular values. The
method is presented here using a fuzzy system of two inputs
but is readily extended to rule base with a general number of
antecedent variables.
Three cases of fuzzy reduction are discussed in this paper.
They include the singleton support case, the nonsingleton
support case, and the TSK model case. These three cases
represent situations where the number of parameters defining
the fuzzy system is gradually increased. For the singleton
support case, the output of each rule is determined by just one
numerical value. For the nonsingleton support case two are
needed, namely, the rule consequent and the support factor.
As for the TSK model, the numerical example here utilizes
a fuzzy output function of three parameters, but, in general,
the function can include any number of parameters. From
this perspective, the proposed method can be applied to a
variety of other cases. Take the example where the output
membership functions are trapezoidal; for instance, one may
apply the reduction procedures using the characteristic points
of the trapezoids as the parameters.
The present work aims at establishing a methodology in
extracting the essential elements of a given fuzzy rule base.
This is important as there is yet no uniformly accepted
formulation for designing a fuzzy rule set efficiently and
effectively. As illustrated by the numerical examples, the
proposed approach manages to generate a reduced rule base
which quite reasonably approximates the operation of the
original one. This is the case especially in Example 1, when a
fuzzy rule base of 81 rules is effectively replaced by one with
just nine. Moreover, Example 2 demonstrates a certain ability
in the present approach to extrapolate the original output to
region where it is previously undefined. The third example in
this work demonstrates that the proposed method is capable
of eliminating certain redundancy in the fuzzy rule set. In
this case, the method reveals that the original rule base of
eight rules can actually be replaced by a rule base of six rules
without any output error.
REFERENCES
[1] T. Takagi and M. Sugeno, Fuzzy identification of systems and its
applications to modeling and control, IEEE Trans. Syst., Man, Cybern.,
vol. 15, pp. 116132, 1985.
[2] P. Baranyi and Y. Yam, Singular value-based fuzzy approximation
with nonsingleton support, in Proc. 7th Int. Fuzzy Syst. Assoc. World
Congress, Prague, Czech Republic, June 1997, pp. 127132.
[3]
, Singular value-based fuzzy approximation with Sugeno type
fuzzy rule base, in Proc. 6th IEEE Int. Conf. Fuzzy Syst. (FUZZIEEE97), Barcelona, Spain, July 1997, pp. 265270.
[4] Y. Yam and C. T. Yang, Singular value-based fuzzy approximator: A
case study, in Proc. Int. Panel Conf. Soft Intell. Comput., Budapest,
Hungary, Sept. 30Oct. 3, 1996, pp. 305312

132

[5] Y. Yam, Fuzzy approximation via grid point sampling and singular
value decomposition, IEEE Trans. Syst., Man, Cybern., vol. 27, pp.
933951, Dec. 1997.
[6] L. Wang, R. Langari, and J. Yen, Principal components, B-splines, and
fuzzy system reduction, in Fuzzy Logic for the Applications to Complex
Systems, W. Chiang and J. Lee, Eds. Singapore: World Scientific,
1996, pp. 255259.
[7] P. Baranyi, T. D. Gedeon, and L. T. Koczy, A general interpolation
technique in fuzzy rules bases with arbitrary membership functions,
in IEEE Int. Conf. Syst., Man, Cybern. (SMC96), Beijing, China, Oct.
1996, pp. 510515.
[8] J. ORourke, Computational Geometry in C. Cambridge, U.K.: Cambridge Univ. Press, 1994.
[9] H. Edelsbrunner, Algorithms in Combinatorial Geometry. Berlin, Germany: Springer-Verlag, 1987.
[10] F. Bouslama and A. Ichikawa, Application of limit fuzzy controllers
to stability analysis, Fuzzy Sets Syst., vol. 49, pp. 103120, 1992.

Yeung Yam (M92) received the B.S. degree from

the Chinese University of Hong Kong and the M.S.
degree from the University of Akron, OH, both in
physics, in 1975 and 1977, respectively, and the
M.S. and Sc.D. degrees in aeronautics and astronautics from the Massachusetts Institute of Technology,
Cambridge, in 1979 and 1983, respectively.
From 1985 to 1992, he was a member of the
Technical Staff in the Control Analysis Research
Group of the Guidance and Control Section at the
Jet Propulsion Laboratory, Pasadena, CA. He joined
the Chinese University of Hong Kong in 1992 and is currently an Associate
Professor in the Department of Mechanical and Automation Engineering.
His research interests include analysis, design, and identification of control
systems.

IEEE TRANSACTIONS ON FUZZY SYSTEMS, VOL. 7, NO. 2, APRIL 1999

Peter Baranyi was born in Hungary in 1970. He

received both the M.Sc. (electrical engineering) and
the M.Sc. degrees (education of engineering sciences) from the Technical University of Budapest,
Hungary, in 1994 and 1995, respectively. He is
currently working toward the Ph.D. degree at the
Technical University of Budapest, Hungary.
Since 1998, he has been a Research Assistant
at the Technical University of Budapest. He has
also conducted research work at the CNRS LAAS
Institute in Toulouse, France, in 1996, the Chinese
University of Hong Kong in 1996 and 1998, and the University of New South
Wales, Sydney, Australia, in 1997. His research interests include fuzzy and
neural network techniques.
Mr. Baranyi is a member of the Hungarian Society of IFSA (International
Fuzzy Systems Association), the Hungarian Neuman Janos Computer Science
Association, and the Hungarian Elektrotechnic Association.

Chi-Tin Yang was born in Hong Kong in 1970.

He received the B.S. and M.S. degrees in electrical engineering from the University of WisconsinMadison in 1992 and 1993, respectively.
Since 1995, he was a graduate student in the
Department of Mechanical and Automation Engineering at the Chinese University of Hong Kong.
His research interests include fuzzy control, identification, and modeling.