Financial Forecasting Using Pattern Modeling and Recognition System

Based on Kernel Regression

Defu Zhang1,2, Yubao Liu3 and Yi Jiang1
1
Department of Computer Science, Xiamen University, 361005, China
2
Longtop Group Post-doctoral Research Center, Xiamen, 361005, China
3
School of Information Science and Technology, Sun Yat-sen University, 510275, China
dfzhang@xmu.edu.cn http://xmuzdf.ik8.com

Abstract: - The increased popularity of financial time series forecasting in recent times lies to its great
importance in predicting the best stock market timing. In this paper, we develop the concept of a pattern
modeling and recognition system for predicting future behavior of time series using local approximation. In
order to improve the performance of this system, we propose a systematic and automatic approach to technical
pattern recognition using nonparametric kernel regression, and use this method for filtering the noise of the
time series. The computational results on the well-known stock market indices reveal that kernel regression is
an important tool for improving the performance of the proposed forecasting system, and the performance of
the improved method can outperform the performance of advanced methods such as neural networks.

Key-Words: - financial time series; forecasting; pattern modeling and recognition system; kernel regression

1 Introduction fluctuations to be ignored. Fortunately, there are a

Forecasting becomes more and more important in
many domains. It has been one of the greatest
challenges to predict the stock market. However,
determining stock market timing, that is when to
buy and sell, is a very difficult problem for humans
because of the complexity of the stock market. The
traders and investors in the stock market have come
to need powerful assistants in their decisions
making.
Throughout the literature, many techniques have
been implemented to perform time series forecasting.
Artificial intelligence methods such as neural
network [1], genetic algorithms [2], Markov models
[3] and fuzzy methods [2] have been frequently used.
Pattern modeling and recognition system (PMRS)[4]
which was developed recently has received special
interest. The reasons for its popularity are as follows
[4]: 1) its algorithm is easy; 2) such a tool requires
lesser number of parameters that need optimization;
3) PMRS works in real-time. However, there is a
fatal shortcoming in this technique. When used to
dealing with noise data, its performance usually
discourages people.
In this paper, we hope to realize the improvement
of this forecasting method. Our goal is to identify
regularities in the time series of prices by extracting
nonlinear pattern from noisy data. Implicit in this
goal is the recognition that some price movements
are significant, they contribute to the formation of a
specific pattern, and others are merely random
class of statistical estimators, called smoothing
estimators [5], are ideally suited to this task because
they extract nonlinear relations by “averaging out”
the noise. The kernel smoothing estimator has been
used to improve the performance of neural networks
in [6]. Therefore, we propose to use these estimators
to improve the performance of PMRS. We then
study the usefulness of noise-filtering technique for
improving the performance of the tool. The
computational results show that PMRS based on
kernel regression are highly accurate and
outperform the performance of advanced methods
such as neural networks.
2 PMRS
From statistics to artificial intelligence, there are
myriad choices of techniques for financial time
series forecasting. One of the simplest techniques is
to search a data series for similar past events and use
the matches to make a forecast. PMRS is an
example of this technique. This pattern recognition
method is based on the philosophy that it is easier to
approximate the chaotic behavior of time series at
the local than global level [7].
The algorithm of PMRS is penetrable and easy to
program into a computer as well. However, the
prediction results are not always satisfying. People
who would like to profit from time series
forecasting can often be frustrated by the low
direction % success. In a sense, it’s somewhat like a
voodoo. One explanation for this frustrating
situation is that the stock prices are arbitrary, so the
time series may contain random fluctuations which
are not components of a specific pattern. In the next
chapter, we will propose to apply a statistical
method to filter noises.
3 Noise-filtering—Kernel Regression
Although PMRS fails to identify regularities in the
time series of prices by extracting nonlinear patterns
from noisy data, a class of smoothing estimators is
fit to this task. By “averaging out” the noise, they
can extract nonlinear relations from time series. We
choose one kind of smoothing estimators, namely
nonparametric kernel regression to filter the noise in
the time series.
Kernel regression is one class of modeling
methods that belongs to the smoothing methods
family. It is a part of the nonparametric regression
methods. Kernel regression allows you to base the
prediction of a value on passed observations, and to
weight the impact of passed observations depending
on how similar they are compared with the current
values of the explanatory variables.
We will use the following formula [8] to filter
noise.
∑
T of a time series may be extended to include more
1 T K h ( x − X t )Yt
m ( x) = ∑ ω t ,h ( x)Yt =
' t =1
∑
h T
T t =1
t =1
Kh (x − X t )
2
x
1 −
K h ( x) = e 2h2
h 2π
where h is the bandwidth that is an important aspect
of any local-averaging technique, { Yt } is a time
series, X t denotes the distance far from x ,
mh′ (x) is the smooth value of { Yt } at time x .
4 PMRS Based on Kernel Regression
As we discuss before, noise seriously influences the
performance of PMRS. Now we apply kernel
regression to reduce the impact of noise.
To illustrate the noise-filtering procedure, we
need to describe the forecasting process at the same
time. First we set a fixed length of time series,
called time window. We will search for the similar
past states to match the current states in the time
window. As the current states move forward by one
data point, the time window is also slide forward by
one data point. To forecast the next data point of
current states, we need to smooth the time series
before the current data point in the time window.
When the current states move one data point
forward, the smoothing process should be repeated,
all data points in the time window besides the
current data point should be treated with again, for
some treated data points will be modified with the
sliding of the time window.
To deal with one data point yi , or filter the noise,
we consider 80 data points ahead and at the back for
reference. According to the weighted average
principle, the closer the data point is to yi , the
larger the weights it would have, we neglect the
effect of other data points beyond
[ y i − 80, y i + 80] within the time widow.
In order to program the principles of PMRS into a
computer, we need to define the mathematical
representation of time series data for our purpose.
Providing a time series is represented as a vector,
y = { y1 , y 2 ,L , y n } , then the state of the time
series represents its current value y n . The
prediction process can be described as follows: first
identifying the closest neighbor of y n in the past
data, say y j , then predict yˆ n +1 on the basis of y j +1 .
This approach may be extended, i.e. the current state
than one value. For example, take values of size
three, the current state may be defined as
{ y n − 2 , y n −1 , y n } . For such a current state, the
prediction will depend on the past state
{ y j − 2 , y j −1 , y j } . The optimal state size must be
determined experimentally on the basis of achieving
minimal errors on standard measures.
A segment in the series may be defined as a
different vector δ = (δ 1 , δ 2 , L, δ n −1 ) where
δ i = yi +1 − yi . A pattern includes one or more
segments and may be visualized as
p = (δ i , δ i +1 , L , δ h ) for given values of i and h ,
1 ≤ i, h ≤ n − 1 , provided that h > i . The technique
of matching structural primitives is based on the
premise that the past repeats itself [4]. In detail, the
algorithm of PMRS based on kernel smoothing can
be described as follows:
Step 0: Process data using kernel smoothing.
Step 1: Define a pattern of size as k , here k =2 as
the beginning value of k .
Step 2: Choose a pattern of size k : Composite Index (SSEC) and S&P 500 Index (SPX),
p = (δ n − k , δ n − ( k −1) , L , δ n −1 ) at the end of the time HANG SENG Index (HSI) and NASDAQ
Combined Composite Index (CCMP) from January
series. 1993 to December 2003. We have tried to do
Step 3: A nearest neighbor experiment with as long time series as possible in
p ′ = (δ j − k , δ j − ( k −1) , L , δ j −1 ) of pattern p is order to eliminate the likelihood of luck.
determined from historical data on the basis of The overall procedure consists of dividing the
smallest offset ▽. Here, j is the marker position. data into two parts: estimation data and test data.
The estimation data is to determine the optimal k.
Step 4: If δ j > 0 , it denotes to predict high, then
16% of the total data, about 500 data is the
predict value yˆ n +1 can be calculated as follows: estimation data, and the remaining 84% is test data.
The evaluation criteria are the absolute error and the
1 k δ n −i
yˆ n +1 = y n + βδ j , here, β = ∑ δ j −i ; If
k i =1
direction of series change: positive or negative
relative to the current position. The latter is one of
δ j < 0 , it denotes to predict low, then predict value the most important measures in financial markets.
The computational results are reported in Table 1~4.
yˆ n +1 can be calculated as follows: yˆ n +1 = y n − βδ j , The forecasting results of PMRS and BPNN really
1 k δ n −i make us frustrate, the direction % success of them
here, β = ∑ δ j −i ; If δ j = 0 , it denotes to
k i =1 are less than 52% from Table 1~4.

predict same, then predict value yˆ n +1 = y n . Table 1. The prediction performance of three
Step 5: We use the standard error measurement methods for SSEC stock market.
PMSE for calculating the accuracy of the forecast Method Optimal k % success Absolute
and record the error to select optimal k . error
Step 6: Increase k by one, and repeat the process PMRSKR 4 68% 15.65
from step 2 to step 5 until it reaches a predefined PMRS 5 49% 20.84
maximum allowed for the experiment (here is 5). BPNN / 51% 26.96
Step 7: Choose a pattern size k which yields
minimal error measurements. Table 2. The prediction performance of three
Step 8: Forecasting. Then apply this pattern size methods for SPX stock market.
k to forecasting according to the principle of Method Optimal k % success Absolute
error
moving time widow.
PMRSKR 3 67% 55.53
PMRS 3 50% 24.86
BPNN / 49% 229.31
5 Experimental Results
In this section, the performance of the pattern Table 3. The prediction performance of three
modeling and recognition system based on kernel methods for HSI stock market.
regression (PMRSKR) is compared with PMRS,
Method Optimal k % success Absolute
BPNN. BPNN is a Feedforward Backpropagation
error
Network with three layer structure. In this paper,
PMRSKR 3 70% 159.63
BPNN has 20 neurons in the input layer, 5 neurons
PMRS 4 52% 668.12
in the hidden layer and one neuron in the output
BPNN / 51% 163.47
layer, f is typically taken to be a sigmoidal
function, such as the logistic function Table 4. The prediction performance of three
1
f ( x) = . The backpropagation algorithm methods for CCMP stock market.
1 + e−x Method Optimal k % success Absolute
with momentum factor is adopted to perform error
steepest descent on the total mean squared error. PMRSKR 3 69% 26.67
The more detailed description about BPNN can be PMRS 2 52% 6.93
found in [9]. BPNN / 50% 45.34
PMRSKR, PMRS and BPNN are implemented
through an object oriented VC programming tool. It is known that if the hit ratio of the direction is
The test data sets used for prediction are Shanghai highly above 51%, the technique for forecasting can
then be regarded as useful. Obviously, the developed and realized by integrating the
performance of PMRS could not achieve this goal in trading strategy for decision-making.
dealing with these indices in the well-known stock
markets. Also, the performance of neural networks
(BPNN) is not good with our test data.
Acknowledgments
This work has been supported by academician start-
By using kernel regression for filtering noise, the
up fund (Grant No. X01109), Xiamen University
performance of PMRS is greatly improved and
985 information technology fund (Grant No. 0000-
reaches 67~70% in % success. It means that
X07204) and fund (Grant No. Y07025).
investment according to this technique can result in
a good profit. Note that h is an important parameter
References:
of kernel regression. If h is very small, the
[1] Azoff, M. E, Neural network time series
averaging will be done with respect to a rather small
forecasting of financial markets, John Wiley
neighborhood around each data point. If h is too
and Sons, 1994.
large, the averaging will be over larger
[2] Chorafas, D. N, Chaos theory in the financial
neighborhoods of a data point. Therefore,
markets: Applying fractals, fuzzy logic, ge-netic
controlling the degree of averaging amounts to
algorithms, Swarm simulation & the Monte
adjusting the smoothing parameter h, also known as
Carlo method to manage markets, Probus
the bandwidth. In our experiment, the appropriate h
Publishing Company,1994.
is 2.7 for all indices.
[3] MacDonald, I. L. and Zucchini, W, Hidden
Although the absolute error of PMRSKR is not
Markov and other models for discrete valued
minimal, its direction % success is highest. In fact,
time series, London: Chapman and Hall, 1997.
the latter is most important for make a good profit.
[4] Singh, S, A Long Memory Pattern Modeling
Therefore, throughout the ten years of test data, the
and Recognition System for Financial Fore-
usefulness of this improved technique is proven. In
casting, Pattern Analysis and Applications, vol.
addition, the comparisons of the predict value and
2, issue 3 ,1999, 264–273.
the real value about twenty days among PMRS,
[5] Hardle, Wolfgang, Applied Nonparametric
PMRSKR, BPNN are reported in Fig. 1. From Fig.1,
Regression, Cambridge University Press, Cam-
we can observe the predict value drops behind real
bridge, UK, 1990.
value.
[6] Defu Zhang, Qingshan Jiang, and Xin Li, A
Hybrid Mining Model Based on Neural Net-
work and Kernel Smoothing Technique,
Lecture Notes in Computer Science, 3516, 2005,
801–805.
[7] Farmer, J. D. and Sidorowich, J. J, Predicting
chaotic dynamics, in Dynamic patterns in
complex systems, J.A.S. Kelso. A. J. Mandell
and M. F. Shlesinger(Eds.), Singapore: World
Scientific, 1988, 265–292.
[8] Andrew W. Lo, Harry Mamaysky, Jiang Wang,
Foundations of Technical Analysis: Compu-
tational Algorithms, Statistical Inference, and
Empirical Implementation. http://www.nber.
6 Conclusion org/papers/w7613. National Bureau of
In this study, we develop PMRS based on Economic Research, 2000.
kernel regression for predicting future behavior [9] Rao, V. and Rao H, C++ Neural Network and
of dynamic time series using local Fuzzy Logic, MIS press, New York, 1995.
approximation. The experimental results are
highly encouraging. Thus, we could come to the
conclusion that the kernel regression is a very
useful preprocessing stage in making accurate
forecasts, and this improved PMRS can be an
efficient tool for stock market timing. We
expect that this technology will be further