Академический Документы
Профессиональный Документы
Культура Документы
Abstract— A self-organized fuzzy neural network (SOFNN) to a probability policy which to determinate actions in
with a reinforcement learning algorithm called Stochastic Gra- the procedure of reinforcement learning and the error of
dient Ascent (SGA) is proposed to forecast a set of 11 time series. forecasting is as reward value (Subsection 2.4).
The proposed system is confirmed to predict chaotic time series
before, and is applied to predict each/every time series in NN3 A. Embedding
forecasting competition modifying parameters of threshold of
fuzzy neurons only. The training results are obviously effective According to the Takens embedding theorem, the inputs
and results of long-term prediction give convincible trend values of prediction system on time t, can be reconstructed as a n
in the future of time series. dimensions vector space X(t), which includes n observed
points with same intervals on a time series y(t).
I. I NTRODUCTION
Though many artificial neural networks (ANN) are suitable X(t) = (x1 (t), x2 (t), · · · , xn (t)) (1)
to forecast time series, radial basis function network (RBFN) = (y(t), y(t − τ ), · · · , y(t − (n − 1)τ ) (2)
is still recommend to be applied on chaotic time series [1],
[2], [3] and financial time series [4]. Meanwhile, how to de- where τ is time delay (interval of sampling) may be an
sign the structure of hidden-layer in RBFN is a puzzling and arbitrary value in theory but it effects prediction accuracy
confusing problem in practice. Leung and Wang proposed in practice, n is the embedding dimension, n > 1.
a technique called the cross-validated subspace method to B. Self-organized Fuzzy Neural Network (SOFNN)
estimate the optimum number of hidden units, and applied
Fig. 2 shows an architecture of self-organized fuzzy neu-
the method to prediction of noisy chaotic time series [3]
ral network (SOFNN) we proposed. The initial number of
but the method may fall its efficiency when the training
membership function and fuzzy rule is only 1, respectively.
sample data is not enough. To overcome the problem of
1) Membership Function: To each element xi (t) of the
RBFN, we proposed a self-organization fuzzy network and
input X(t), membership function Bij (xi (t)) is represented
its effectiveness on prediction of chaotic time series was
as ½ ¾
investigated [5], [6]. (xi (t) − mij )2
Furthermore, the learning algorithm is so important to any Bij (xi (t)) = exp − 2 (3)
2σij
artificial neural network. Reinforcement learning (RL), a kind
of goal-directed learning, is well known for an agent adapting where mij and σij are the parameters of mean and standard
unknown environments [7], [8]. We have proposed to apply deviation of the Gaussian membership function in jth node,
a kind of RL called stochastic gradient ascent (SGA) on respectively. Initially, j = 1, and with increasing of input
nonlinear predations [5], [6], [9]. The accurate of forecasting patterns, the new membership function will be added.
in experiments using Lorenz chaotic time series shown its
good efficiency whatever the type of ANN is either multi- Time Series
layer perception (MLP) or SOFNN.
y(t)
In this paper, we intend to use SOFNN and SGA to
forecast a set of 11 time series given by neural forecasting Reconstructed Inputs
Reinforcement learning
competition (NN3) [10]. The prediction system is introduced
in Section 1 in detail, and all of train forecasting results and X(t)
forecasting results are shown in Section 2.
Self-organized Fuzzy Neural Network
II. F ORECASTING S YSTEM
r: reward
Flow of forecasting is shown in Fig. 1 and self-organized Prediction under Stochastice Policy
fuzzy neural network, ANN predictor, is a RBF-like neural
network (Fig. 2). The input layer is given by data in history ^
y(t+1)
of time series (Subsection 2.1). The hidden layer consists
Gaussian membership functions which number is decided Evaluation of Prediction
by a threshold and the rule layer realizes fuzzy inference
(Subsection 2.2 and 2.3). Forecasting is executed according
Authors are with the Graduate School of Science and Systems Engineer-
ing, Yamaguchi University , Tokiwadai 2-16-1, Ube, Yamaguchi 755-8611, Fig. 1. Flow chart of training and forecasting.
Japan, Tel: +81-836-859520, Fax: +81-836-859501, Email: {wu, m.obayas,
koba}@yamaguchi-u.ac.jp
2) Fuzzy Inference: The fitness λk (X(t)) is an algebraic C. Reinforcement Learning Algorithm (SGA)
product of membership functions which connects to rule k.
Reinforcement learning has recently been well-known as
n
Y a kind of intelligent machine learning [7], [8]. It needs not
λk (X(t)) = Bio (xi ) (4) any model of operator but learns to approach to its goal
i=1 by observing sensing rewards from environments. Kimura
where o is the number of membership function, connects and Kobayashi suggested an algorithm called stochastic
with kth rule. o ∈ {1, 2, · · · , li }. gradient ascent (SGA), which respect to continuous action
and confirmed in control systems [8]. The error of forecasting
Inputs Fuzzy Membership Functions Fuzzy Rules Distributions of prediction Predicted Values
is as reward to adjust probability policy in training process.
SGA algorithm is shown under.
R1
R
x (t)= y(t)
1
wµ1
1) Accept an observation X(t) from environment.
wσ1
Average 2) Predict a future data ŷ(t + 1) under a probability
R
wµ2
R2
µ π(ŷ(t + 1), W, X(t)).
Stochastic Function
x2(t)= y(t-τ)
w σ2 3) Collate training samples of times series, take the error
π
as reward ri .
R
Deviation
wµΚ
σ 4) Calculate the degree of adaption ei (t), and its history
R
x n(t)= y(t-(n-1)τ)
for all elements ωi of internal variable W . where γ is
RK wσΚ
a discount (0 ≤ γ < 1).
R
∂ ¡ ¢
ei (t) = ln π(ŷ(t + 1), W, X(t)) (9)
∂ωi
R
σij
m ij
k=1 λk
"101"
"train_forecast"
5800 "forecast"
PK 5600
k=1 λk ωσk
σ(X(t), ωσk ) = PK (7) 5400
k=1 λk
5200
½ ¾ 4400
1 (ŷ(t + 1) − µ)2
π(ŷ(t + 1), W, X(t)) = √ exp − 4200
2πσ 2σ 2
(8) 4000
0 20 40 60 80 100 120 140 160
1.1
8
1
7
0.9
6
0.8
5
0.7
0 5000 10000 15000 20000 25000 30000
4 "fuzzy1.txt"
"fuzzy2.txt"
Fig. 4. Change of forecasting error in training. "fuzzy3.txt"
3
8000
8000
7000
7000
6000
6000
5000
5000
4000
4000
3000
3000
2000
2000 1000
1000 0
0 20 40 60 80 100 120 140 160 0 20 40 60 80 100 120 140
Fig. 7. Forecasting results of time series 102. Fig. 9. Forecasting results of time series 104.
60000 5400
"103" "105"
"train_forecast" "train_forecast"
"forecast" 5200 "forecast"
50000
5000
4800
40000
4600
30000 4400
4200
20000
4000
3800
10000
3600
0 3400
0 20 40 60 80 100 120 140 160 0 20 40 60 80 100 120 140 160
Fig. 8. Forecasting results of time series 103. Fig. 10. Forecasting results of time series 105.
IV. C ONCLUSION [4] V. Kodogiannis, A. Lolis, Forecasting Financial Time Series using Neu-
ral Network and Fuzzy System-based Techniques, Neural Computing
The robustness of proposed self-organization fuzzy neural and Applications, No.11, pp.90-102, 2002
[5] Kuremoto T., Obayashi M., Yamamoto A., and Kobayashi K., Neural
network with reinforcement learning algorithm is confirmed Prediction of Chaotic Time Series Using Stochastic Gradient Ascent
by a set of NN3 reduced data. The predictor is easy to use by Algorithm. Proceedings of the 35th ISCIE International Symposium
only tuning its threshold and initial deviation of membership on Stochastic Systems Theory and Its Applications (SSS’03), pp.17-22,
2003
function and reward parameter. Except normalization to [6] Kuremoto T., Obayashi M., Yamamoto A., and Kobayashi K., Predicting
sample data, no any additional operation to original data and Chaotic Time Series by Reinforcement Learning. Proceedings of the 2nd
forecasting result. All train steps were limited to 30,000 and International Conference on Computational Intelligence, Robotics and
Autonomous Systems (CIRAS2003), CD-ROM, 2003
average training time was not over 2 minutes on a personal [7] R.S.Sutton and A.G. Barto, Reinforcement Learning: An introduction,
computer (3.0GHz CPU). The MIT Press, 1998
[8] H. Kimura, S. Kobayashi, Reinforcement Learning for Continuous
ACKNOWLEDGMENTS Action using Stochastic Gradient Ascent , Intelligent Autonomous
Systems, Vol.5, pp.288-295, 1998
This work was partly supported by Grants-in-Aid for [9] Kuremoto T., Kobayashi K., Obayashi M., Nonlinear Prediction by
Reinforcement Learning. Lecture Notes in Computer Science (ICIC
Scientific Research of JSPS (No.18500230). 2005), Vol.3644, pp.1085-1094, 2005
[10] http://www.neural-forecasting-competition.com/
R EFERENCES
[1] M. Casdagli, Nonlinear prediction of chaotic time series, Physica D:
Nonlinear Phenomena, Vol. 35, Issue 3, pp.335-356, 1989
[2] K. A. de Oliveira, A. Vannucci, E. C. da Silva, Using artificial neural
networks to forecast chaotic time series, Physica A, No.284, pp.393-
404, 1996
[3] H. Leung, T. Lo, S. Wang, Prediction of Noisy Chaotic Time Series
Using an Optimal Radial Basis Function, IEEE Trans. on Neural
Networks, Vol. 12, No.5, pp.1163-1172, 2001
6500 6500
"106" "train_forecast"
"train_forecast" "109"
"forecast" "forecast"
6000
6000
5500
5500 5000
4500
5000
4000
4500 3500
3000
4000
2500
3500 2000
0 20 40 60 80 100 120 140 160 0 20 40 60 80 100 120 140 160
Fig. 11. Forecasting results of time series 106. Fig. 14. Forecasting results of time series 109.
4400 16000
"107" "110"
"train_forecast" "train_forecast"
4300 "forecast" "forecast"
14000
4200
12000
4100
4000 10000
3900
8000
3800
3700 6000
3600
4000
3500
2000
3400
3300 0
0 20 40 60 80 100 120 140 160 0 20 40 60 80 100 120 140 160
Fig. 12. Forecasting results of time series 107. Fig. 15. Forecasting results of time series 110.
9000 6500
"108" "111"
"train_forecast" "train_forecast"
"forecast" "forecast"
8000 6000
5500
7000
5000
6000
4500
5000
4000
4000
3500
3000
3000
2000 2500
1000 2000
0 20 40 60 80 100 120 140 0 20 40 60 80 100 120 140 160
Fig. 13. Forecasting results of time series 108. Fig. 16. Forecasting results of time series 111.