i
T
(1)
where
k
= [1 x
k
1
x
k
2
. . . x
k
p
];
i
= [
i0
i1
. . .
ip
] is the coefcient vector of the local
linear model for the i th rule.
Each input pattern has a membership degree associated with each region of the in
put space partition. This is calculated through membership functions g
i
(x
k
) that vary
according to centers and covariance matrices related to the fuzzy partition, and are com
puted by:
g
i
(x
k
) = g
k
i
=
i
P[ i  x
k
]
M
q=1
q
P[ q  x
k
]
(2)
326 I. Luna et al.
x
k
x
k
x
k
Input space partition
R
1
R
2
.
.
.
. . .
R
M
y
k
1
y
k
2
y
k
M
g
k
1
g
k
2
g
k
M
g
k
i
Rule base
y
k
Fig. 1. A general FIS structure
where
i
are positive coefcients satisfying
M
i=1
i
= 1 and P[ i  x
k
] is dened
according to
P[ i  x
k
] =
1
(2)
p/2
det(V
i
)
1/2
exp
1
2
(x
k
c
i
)V
1
i
(x
k
c
i
)
T
(3)
where det() is the determinant function. The model output y(k) = y
k
, which repre
sents the predicted value for future time instant k is calculated by means of a nonlinear
weighted averaging of local outputs y
k
i
and its respective membership degrees g
k
i
, i.e.
y(x
k
) = y
k
=
M
i=1
g
k
i
y
k
i
(4)
2.2 Learning Algorithm
First, an initial structure composed by fuzzy rules is dened, and its parameters are
adjusted via the traditional Expectation Maximization (EM) algorithm, originally pro
posed in [5] for mixture of experts models.
Model structure is initialized using the unsupervised clustering algorithm called the
Subtractive Clustering Algorithm (SC), proposed in [2]. This algorithm provide a set
of M clusters from an specic training data set presented to the algorithm. Patterns
processed by the SC algorithm are composed by the inputoutput patterns used in a
second stage for model optimization.
These groups are associated to a set of fuzzy rules codied in the FIS structure.
Therefore, after the number of fuzzy rules is dened, we proceed to initialize the model
parameters, for i = 1, . . . , M, according to the following criteria:
c
0
i
=
0
i

1...p
, where
0
i

1...p
is composed by the rst p components of the ith
center found by the SC algorithm;
Fuzzy Inference System 327
0
i
= 1.0;
0
i
= [
0
i

p+1
0 . . . 0]
1p+1
, where
0
i

p+1
is the p +1th component of the ith
center found by the SC algorithm;
V
0
i
= 10
4
I, where I is a p p identity matrix;
0
i
= 1/M.
After this initialization, model parameters are readjusted based on the traditional ofine
EM algorithm, following an iterative sequence of EM steps, given incomplete data y
k
.
It means that, a complete data is composed by the output variable y
k
and a missing data.
The goal of the EM algorithm is to nd a set of model parameters, which will maximize
the loglikelihood L, of the observed values of y
k
at each M step of the learning process.
This objective function is dened by
L(D, ) =
N
k=1
ln
i=1
g
i
(x
k
, C) P(y
k
 x
k
,
i
)
(5)
where D = {x
k
, y
k
k = 1, . . . , N}, contains all model parameters and C contains
just the antecedents parameters (centers and covariance matrices). However, for maxi
mizing L(D, ), it is necessary to estimate the missing data h
k
i
(E step). This missing
data, according to mixture of experts theory, is known as the posterior probability of x
k
belong to the active region of the ith local model.
When the EM algorithm is adapted for adjusting fuzzy systems, h
k
i
may also be
interpreted as a posterior estimate of membership functions dened by Eq. (2). So, h
k
i
is calculated as
h
k
i
=
i
P(i  x
k
)P(y
k
 x
k
,
i
)
M
q=1
q
P(q  x
k
)P(y
k
 x
k
,
q
)
(6)
for i = 1, . . . , M. These estimates are called as posterior, because these are calculated
assuming y
k
, k = 1, . . . , N as known. Moreover, conditional probability P(y
k
x
k
,
i
)
is dened by:
P(y
k
 x
k
,
i
) =
1
2
2
i
exp
[y
k
y
k
i
]
2
2
2
i
(7)
with
2
i
estimated as:
2
i
=
k=1
h
k
i
[y
k
y
k
i
]
2
/
N
k=1
h
k
i
(8)
Hence, the EM algorithm for determining FIS parameters can be summarized as:
1. E step: Estimate h
k
i
via Eq. (6);
2. M step: Maximize Eq. (5) and update model parameters, with optimal values cal
culated as:
328 I. Luna et al.
i
=
1
N
N
k=1
h
k
i
(9)
c
i
=
k=1
h
k
i
x
k
/
N
k=1
h
k
i
(10)
V
i
=
k=1
h
k
i
(x
k
c
i
)
(x
k
c
i
)
/
N
k=1
h
k
i
(11)
for i = 1, . . . , M, where M is the size of the fuzzy rule base, N is the number of
inputoutput patterns at the training set. For all these equations, V
i
was considered
as a positive diagonal matrix, as an alternative to simplify the problem and avoid
infeasible solutions. An optimal solution for
i
is derived solving the following
equation:
N
k=1
h
k
i
2
i
y
k
k
= 0 (12)
where
i
is the standard deviation for each local output y
i
, i = 1, . . . , M, with
2
i
dened by Eq.(8). After parameters adjustment, calculate the new value for
L(D, ).
3. If convergence is achieved, then stop the process, else return to step 1.
There are some differences to consider if he FIS structure is compared to a basic one
given by the SC algorithm. First, The FIS structure has coefcients
i
as parameters,
which are not directly initialized by the SC algorithm. Secondly, consequents assumed
by the SC algorithm are singletons, whereas the FIS considers local linear models
(which are a function of the input vector). Therefore, even though the SC algorithm
can be used directly for modeling purposes, it is still necessary a global optimiza
tion, considering all the FIS parameters, which is performed using the EM algorithm in
this paper.
3 An Application to the Central Banks Reaction Function
In order to develop an econometric model of the Takagi and Sugenotype, we need to
determine the underlying structure of the fuzzy system and its parameters. The model
is identied by a fuzzy modelling method, using economic theory in combination with
a set of input  output data. The rst step in structure identication is to choose the rele
vant explanatory variables. As with any estimation procedure, this is the point at which
a careful consideration of economic theory is important, to ensure that only relevant in
puts to the model are used. In the present context, it is important because a fuzzy model
will always attempt to match the chosen inputs and outputs.
Probably, the most well known reaction function is the Taylor Rule, proposed in
[10], by which the Central Bank uses the nominal interest rate to minimize the total
Fuzzy Inference System 329
variance of ination and output. Taking the Taylor Rule, this section illustrates how
a fuzzy inference system of the rstorder TakagiSugeno type can be used to model
the relationship between the nominal interest rate, ination and output. Our principal
objective is to demonstrate the potential benets of the method, particularly its power
in handling nonlinear relationships. For this reason, we consider a relatively simple
version of the relationship which contains only three input variables.
We study whether there is evidence that the Brazilian nominal interest rate followed
a nonlinear process between March 2002 and September 2008. A brief description of
the data set used is made below.
3.1 The Data
The data sources are Banco Central do Brasil, IBGE (Instituto Brasileiro de Geograa
e Estatstica), and IPEA (Instituto de Pesquisa Econmica Aplicada). The nominal in
terest rate used is the annualized endofperiod Taxa Selic
1
, controlled by the Central
Bank. Output is measured by monthly industrial production and the output gap is mea
sured as the residual from a HodrickPrescott lter [4] applied to the monthly index
of industrial production. The ination rate is calculated by a monthly wholesale index
(IPCA). The index is computed between the 21st day of the previous month and 20th
day of the reference month [8].
We use the endofperiod interest rate in order to avoid endogeneity problems. The
endofperiod interest rate is the rate of the last day of the month. In that case, it is
clear that ination could be considered predetermined. Moreover, it also reasonable to
assume that the nominal interest rate of the last day of the month will not affect the
output of the same month.
Another important point to discuss is whether or not the series considered in this
paper have a nonstationary behavior. Although the nominal interest rate is a variable
controlled by the central bank and the hypothesis of a unitroot seems not to be a rea
sonable one, the usual unitroot tests did not reject the null hypothesis of a unitroot. In
[8], it is argued that this may happen because of the convergent behavior of the series
during the period analyzed in the paper and the relative small number of observations
(80). All the other series were considered nonstationary by the usual tests. Hence, the
models were constructed taking the series in rst difference.
3.2 Estimating Results
In this section, a fuzzy inference system and a linear model will be estimated and
then compared. Given the inherent exibility of the fuzzy modelling procedure, be
fore we present estimates of the nominal interest rate equation, it is appropriate to
say a few words about the criteria for model selection. In this paper we applied stan
dard model selection procedure BIC [9], which indicated the following inputs: rst
1
The Selic (Special Settlement and Custody System) rate is an overnight rate expressed per
year, which is obtained by an average weighted, taking the operations total in oneday with
federal public bond. These transactions are made between Brazilian Central Bank and autho
rized nancial institutions. Selic rate is the basic rate used as reference by the monetary policy.
330 I. Luna et al.
difference of the nominal interest rate i and output gap y in t 1, rst difference of the
deviation of the ination rate with respect to the target
, (
), in t and t 1
2
.
As our objective in this paper is not to estimate the best model as such, but rather to
illustrate the potential power of fuzzy modelling and how the structure of the model can
be adjusted to achieve the desired degree of generality. For this reason, our approach is
to use the full sample to estimate a series of models of increasing generality and then
to examine how the forms of the underlying relationships vary across the estimated
models. This provides a useful way of determining the robustness of the identied re
lationships. The performance of each model is shown with respect to the Root Mean
Square Error (RMSE) and the pattern of the associated residuals. We also show plots of
the actual and model outputs.
3.3 A Linear Estimate
As a point of reference, we begin by briey presenting the estimates of a singleequation
linear model. This is the equivalent of estimating a fuzzy model in which the entire data
set is effectively encompassed by a single cluster, so that the inputs are each described
by a single membership function and the behavior of system is shown by a single rule,
represented by a simple linear equation.
We estimate a linear model where the error are normally and independently dis
tributed. As the usual unitroot tests [3] did not reject the null hypothesis of a unit
root, we consider the rst difference of the historical database. We found the following
results:
i
t
= 0.720 i
t1
0.042 y
t1
+ 0.177
t
+ 0.178
t1
(0.063) (0.024) (0.083) (0.087)
where = (
) and the values between parentheses bellow the estimates are the
standard errors. We note that all coefcients are statistically signicant at 10% and have
the desired signs. The RMSE is 0.327 and Figures 2 (a) and (b) show, respectively, the
estimated interest rates and deviations for the period considered.
In Figures 2 (a) and (b) the very large positive errors in periods usually associated to
currency crises (specially during 2002 and early 2003) suggest that a nonlinear model
may be more adequate to represent the Brazilian nominal interest rate. Furthermore,
there are some evidence that the model may not be correctly specied, indicated by
JarqueBera test [6] where the hypothesis of normally distributed residuals is strongly
rejected with signicance level 5%.
3.4 Estimates of the Fuzzy Model
In this section we present the results for a fuzzy inference models. We have already
said that the initial step in the identication of the fuzzy model is to use the subtractive
2
Ination targeting (IT) is a monetary policy strategy, formulated by Central Banks, that makes
public a target for annual ination rate (
i=1
g
t
i
y
t
i
332 I. Luna et al.
where y denotes the rst difference of the nominal interest rate estimated, x
t
is the
input vector at instant t, g
t
i
is the membership degree and y
t
i
is the output of the local
linear models. The relative weights of the rules g
t
i
are determined by the positions of
the inputs in their respective membership functions and the parameters of the equations
are estimated via the EM method. The RMSE of this model is igual 0.121.
The increased exibility of this model leads to an improvement in performance,
judged in terms of the RMSE, and plots of the actual and estimated outputs shown
in Figure 3(a) conrm the increased explanatory power of the model. The associated
residual plot for the model, shown in Figure 3(b), indicates that the error structure
is much closer to white noise, although there is still some discernible pattern in the
residuals.
4 Conclusions
In this paper, we have demonstrated that a modelling procedure based on the application
of fuzzy logic has considerable potential as a complement to traditional linear and non
linear estimation methods. The method involves the identication and estimation of a
series of rules, described by local linear relationships, which are weighted according
to the position of input observations in their respective fuzzy membership functions.
The behaviour of any linear or nonlinear system is then approximated via a weighted
interpolation across the local regions of the model. We have seen that the fuzzy logic
approach is particularly wellsuited to the estimation of illdened systems in which
there is theoretical and quantitative uncertainty about the nature and range of key input
variables. Additional strengths of the approach are that it requires no prior knowledge
of the functional form of the underlying relationships, and is also robust with respect to
outlying observations or noisy data.
To illustrate the potential benets of the fuzzy modelling procedure, we used it
in conjunction with cluster identication techniques to estimate the Brazilian Central
Banks reaction function to determine the interest rate. We showed how the exibil
ity of the model, and its ability to track any underlying nonlinearity, can readily be
increased by expanding the number of operational rules in the system.
Acknowledgement
The last author thank the Brazilian National Research Council, CNPq, for grants 302407/
20081.
References
1. Angelov, P., Filev, D.P., Kasabov, N.: Evolving Intelligent Systems: Methodology and Ap
plications. WileyIEEE Press (March 2010)
2. Chiu, S.: A cluster estimation method with extension to fuzzy model identication. In: Pro
ceedings of The IEEE International Conference on Fuzzy Systems, June 1994, vol. 2, pp.
12401245 (1994)
Fuzzy Inference System 333
3. Dickey, D.A., Fuller, W.: Distribution of the estimators for autoregressive time series with a
unit root. Journal of the American Statistical Association 74, 427431 (1979)
4. Hodrick, R., Prescott, E.: Postwar U.S. Business Cycles: An Empirical Investigation. Journal
of Money, Credit and Banking 29, 116 (1997)
5. Jacobs, R., Jordan, M., Nowlan, S., Hinton, G.: Adaptive Mixture of Local Experts. Neural
Computation 3(1), 7987 (1991)
6. Jarque, C.M., Bera, A.K.: Efcient tests for normality, homoscedasticity and serial indepen
dence of regression residuals. Economics Letters 6(3), 255259 (1980)
7. Luna, I., Soares, S., Ballini, R.: A ConstructiveFuzzy System Modeling for Time Series
Forecasting. In: Proceedings of The International Joint Conference on Neural Networks
(2007)
8. Salgado, M.J.S., Garcia, M.G.P., Medeiros, M.C.: Monetary Policy During Brazils Real
Plan: Estimating The Central Banks Reaction Function. Technical Report 442, Department
of Economics, Pontical Catholic University of Rio de Janeiro (August 2004)
9. Schwarz, G.: Estimating the dimension of a model. Annals of Statistics 6(2), 461468 (1978)
10. Taylor, J.: Discretion Versus Policy Rules in Practice. In: CarnegieRochester Conference
Series on Public Policy, vol. 39, pp. 195214 (1993)