Hybrid systems I

Every intelligent technique has particular computational properties (e.g. ability to learn, explanation of decisions) that make them suited for particular problems and not for others. For example, while neural networks are good at recognizing patterns, they are not good at explaining how they reach their decisions. Fuzzy logic systems, which can reason with imprecise information, are good at explaining their decisions but they cannot automatically acquire the rules they use to make those decisions. These limitations have been a central driving force behind the creation of intelligent hybrid systems where two or more techniques are combined in a manner
1

that overcomes the limitations of individual techniques. Hybrid systems are also important when considering the varied nature of application domains. Many complex domains have many different component problems, each of which may require different types of processing. If there is a complex application which has two distinct sub-problems, say a signal processing task and a serial reasoning task, then a neural network and an expert system respectively can be used for solving these separate tasks. The use of intelligent hybrid systems is growing rapidly with successful applications in many areas including process control, engineering design, nancial trading, credit evaluation, medical diagnosis, and cognitive simulation.
2

Neural networks are used to tune membership functions of fuzzy systems that are employed as decisionmaking systems for controlling equipment. Although fuzzy logic can encode expert knowledge directly using rules with linguistic labels, it usually takes a lot of time to design and tune the membership functions which quantitatively dene these linquistic labels. Neural network learning techniques can automate this process and substantially reduce development time and cost while improving performance. In theory, neural networks, and fuzzy systems are equivalent in that they are convertible, yet in practice each has its own advantages and disadvantages. For neural networks, the knowledge is automatically acquired by the backpropagation algorithm, but the
3

learning process is relatively slow and analysis of the trained network is difcult (black box). Neither is it possible to extract structural knowledge (rules) from the trained neural network, nor can we integrate special information about the problem into the neural network in order to simplify the learning procedure. Fuzzy systems are more favorable in that their behavior can be explained based on fuzzy rules and thus their performance can be adjusted by tuning the rules. But since, in general, knowledge acquisition is difcult and also the universe of discourse of each input variable needs to be divided into several intervals, applications of fuzzy systems are restricted to the elds where expert knowledge is available and the number of input variables is small.
4

To overcome the problem of knowledge acquisition, neural networks are extended to automatically extract fuzzy rules from numerical data. Cooperative approaches use neural networks to optimize certain parameters of an ordinary fuzzy system, or to preprocess data and extract fuzzy (control) rules from data. First we present some methods for implementing fuzzy IF-THEN rules by trainable neural network architectures. Consider a block of fuzzy rules

: If x is Ai, then y is Bi

(1)

where Ai and Bi are fuzzy numbers, i = 1, . . . , n.

Each rule in (1) can be interpreted as a training pattern for a multilayer neural network, where the antecedent part of the rule is the input and the consequence part of the rule is the desired output of the neural net. The training set derived from (1) can be written in the form {(A1, B1), . . . , (An, Bn)} If we are given a two-input-single-output (MISO) fuzzy systems of the form
i

: If x is Ai and y is Bi, then z is Ci

where Ai, Bi and Ci are fuzzy numbers, i = 1, . . . , n. Then the input/output training pairs for the neural net are the following {(Ai, Bi), Ci}, 1 i n. If we are given a two-input-two-output (MIMO) fuzzy
6

systems of the form

: If x is Ai and y is Bi, then r is Ci and s is Di

where Ai, Bi, Ci and Di are fuzzy numbers, i = 1, . . . , n. Then the input/output training pairs for the neural net are the following

{(Ai, Bi), (Ci, Di)}, 1 i n. There are two main approaches to implement fuzzy IF-THEN rules (1) by standard error backpropagation network. In the method proposed by Umano and Ezawa a fuzzy set is represented by a nite number of its
7

membership values. Let [1, 2] contain the support of all the Ai, plus the support of all the A we might have as input to the system. Also, let [1, 2] contain the support of all the Bi, plus the support of all the B we can obtain as outputs from the system. i = 1, . . . , n. Let M 2 and N be positive integers. Let xj = 1 + (j 1)(2 1)/(N 1) yi = 1 + (i 1)(2 1)/(M 1) for 1 i M and 1 j N . A discrete version of the continuous training set is consists of the input/output pairs {(Ai(x1), . . . , Ai(xN )), (Bi(y1), . . . , Bi(yM ))},
8

for i = 1, . . . , n.

x1

xN

Representation of a fuzzy number by membership values. Using the notations

aij = Ai(xj ),

bij = Bi(yj )

our fuzzy neural network turns into an N input and M output crisp network, which can be trained by the generalized delta rule.

Bi
bij

y1

yj
Multilayer neural network

yM

Ai
aij x1

xi

xN

A network trained on membership values fuzzy numbers. Example 1. Assume our fuzzy rule base consists of three rules
1 2 3

: If x is small then y is negative, : If x is medium then y is about zero, : If x is big then y is positive,

where the membership functions of fuzzy terms are

10

dened by 1 2u if 0 u 1/2 0 otherwise 2u 1 if 1/2 u 1 0 otherwise 1 2|u 1/2| if 0 u 1 0 otherwise

medium

small (u) =

big (u) =

medium(u) =

1
small big

1/2

11

negative(u) =

u if 1 u 0 0 otherwise

about zero(u) =

1 2|u| if 1/2 u 1/2 0 otherwise u if 0 u 1 0 otherwise

1

positive(u) =

negative

about zero

positive

-1

The training set derived from this rule base can be written in the form
12

{(small, negative), (medium, about zero), (big, positive)} Let [0, 1] contain the support of all the fuzzy sets we might have as input to the system. Also, let [1, 1] contain the support of all the fuzzy sets we can obtain as outputs from the system. Let M = N = 5 and xj = (j 1)/4 for 1 j 5, and yi = 1 + (i 1)2/4 = 1 + (i 1)/2 = 3/2 + i/2 for 1 i M and 1 j N . Plugging into numerical values we get x1 = 0, x2 = 0.25, x3 = 0.5, x4 = 0.75 and x5 = 1; and y1 = 1, y2 = 0.5, y3 = 0, y4 = 0.5 and y5 = 1.
13

A discrete version of the continuous training set is consists of three input/output pairs {(a11, . . . , a15), (b11, . . . , b15)} {(a21, . . . , a25), (b21, . . . , b25)} {(a31, . . . , a35), (b31, . . . , b35)} where a1j = small (xj ), a2j = medium(xj ), a3j = big (xj ) for j = 1, . . . , 5, and b1i = negative(yi), b2i = about zero(yi), b3i = positive(yi) for i = 1, . . . , 5. Plugging into numerical values we obtain the following training set for a standard backpropagation network {(1, 0.5, 0, 0, 0), (1, 0.5, 0, 0, 0)} {(0, 0.5, 1, 0.5, 0), (0, 0, 1, 0, 0)}
14

{(0, 0, 0, 0.5, 1), (0, 0, 0, 0.5, 1)}. Uehara and Fujise use nite number of -level sets to represent fuzzy numbers.
= 1

Ai

j
1 = 0

aij

aR ij

Representation of a fuzzy number by -level sets. Consider a simple neural net in Figure 1. All signals and weights are real numbers. The two input neurons do not change the input signals so their output is the same as their input. The signal xi interacts with the weight wi to produce
15

the product pi = wixi, i = 1, 2. The input information pi is aggregated, by addition, to produce the input

net = p1 + p2 = w1x1 + w2x2 to the neuron. The neuron uses its transfer function f , which could be a sigmoidal function, 1 , f (x ) = 1 + ex to compute the output
16

y = f (net) = f (w1x1 + w2x2). This simple neural net, which employs multiplication, addition, and sigmoidal f , will be called as regular (or standard) neural net.
x1 w1 y = f(w1x1 +w2 x2 ) x2 w2

Simple neural net. If we employ other operations like a t-norm, or a tconorm, to combine the incoming data to a neuron we obtain what we call a hybrid neural net. These modications lead to a fuzzy neural architecture based on fuzzy arithmetic operations.

17

Let us express the inputs (which are usually membership degrees of a fuzzy concept) x1, x2 and the weigths w1, w2 over the unit interval [0, 1]. A hybrid neural net may not use multiplication, addition, or a sigmoidal function (because the results of these operations are not necesserily are in the unit interval). Denition 1. A hybrid neural net is a neural net with crisp signals and weights and crisp transfer function. However, we can combine xi and wi using a t-norm, tconorm, or some other continuous operation, we can aggregate p1 and p2 with a t-norm, tconorm, or any other continuous function f can be any continuous function from input to output We emphasize here that all inputs, outputs and the
18

weights of a hybrid neural net are real numbers taken from the unit interval [0, 1]. A processing element of a hybrid neural net is called fuzzy neuron. Denition 2. (AND fuzzy neuron The signal xi and wi are combined by a triangular conorm S to produce

pi = S (wi, xi), i = 1, 2. The input information pi is aggregated by a triangular norm T to produce the output y = AN D(p1, p2) = T (p1, p2) = T (S (w1, x1), S (w2, x2)), of the neuron.
19

So, if T = min, S = max then the AND neuron realizes the min-max composition

y = min{w1 x1, w2 x2}.

x1 w1 y = T(S(w1, x1), S(w 2, x2)) x2 w2

AND fuzzy neuron. Denition 3. (OR fuzzy neuron The signal xi and wi are combined by a triangular norm T to produce pi = T (wi, xi), i = 1, 2.
20

The input information pi is aggregated by a triangular conorm S to produce the output y = OR(p1, p2) = S (p1, p2) = S (T (w1, x1), T (w2, x2)) of the neuron.
x1 w1 y = S(T(w1, x1), T(w2, x2)) x2 w2

OR fuzzy neuron. So, if T = min, S = max then the OR neuron realizes the max-min composition y = max{w1 x1, w2 x2}.

21