State Estimation of Nonlinear System through
Particle Filter based Recurrent Neural Networks
N. Yadaiah SMIEEE
Department of Electrical and Electronics Engineering
JNTUH College of Engineering
Hyderabad, Andhra Pradesh, INDIA.
svpnarri@yahoo.com
Raju S. Bapi SMIEEE
Dept. of Computer and Information Sciences, University of
Hyderabad, Hyderabad, Andhra Pradesh, INDIA
bapics@uohyd.ernet.in
Abstract-This paper presents a Hybrid Particle Filter based
RNN method for state estimation of non-linear dynamical
system with knowledge of its input and output measurements.
Particle filters are sequential Monte Carlo methods based on
point mass (or particle) representations of probability
densities, which is used to train Recurrent Neural Networks for
estimation problems. The performance this method is
compared with EKF based estimation and RNN based
estimation. An Induction motor is considered as typical non-
linear system and is implemented in MATLAB environment.
Keywords- State Estimation, Nonlinear system, Hybrid
Particle filter, Recurrent Neural Networks
I. INTRODUCTION
State estimation is an important problem in systems
engineering and it has been achieving through Kalman
Filtering with some limitations. It is well known that for
linear systems subject to Gaussian measurement or process
noise the Kalman Filter (KF) is a good tool for an optimal
state estimation, since it results in minimization of the trace
of the estimation errors in the form of covariance matrix. For
nonlinear systems, subject to Gaussian noise one could use
the generalization of KF as formulated in terms of the
Extended Kalman Filter (EKF). The EKF is based on a
linearization of the system dynamics using a first order
Taylor expansion, and thus there is neither a proof of its
convergence, nor a proof that the estimation produced by the
EKF satisfies optimality criteria.
To overcome the limitations of KF and of EKF the
Particle Filter (PF) has been proposed [1, 2]. It has been
shown that PF based state estimation is suitable for industrial
systems. Moreover, PF-based state estimation has been
proposed for control and fault diagnosis tasks in
mechanical/robotic systems [3]. The particle filtering
algorithm reminds one of the genetic algorithms where a
number of N particles is subject to a mutation mechanism
which corresponds to the prediction stage, and to selection
mechanism which corresponds to the correction stage. The
A. Suresh Kumar
Department of Electrical and Electronics Engineering
JNTUH College of Engineering
Hyderabad, Andhra Pradesh, INDIA
sureshpck4u@gmail.com
M. Roopchandan
Department of Electrical and Electronics Engineering
JNTUA College of Engineering
Anantpur, Andhra Pradesh, INDIA
roopchandan@gmail.com
main features of particle filtering are summarized in the
following:
(i) It is a nonparametric state estimator since it is not
dependant on assumptions about the probability density
functions of the process and measurement noises and can
function equally well for Gaussian and non-Gaussian noise
distributions.
(ii) It has improved performance over the Extended Kalman
Filter, since it can provide optimal estimation in non-
Gaussian state-space models, as well as in estimation of
linear and nonlinear models [4, 5].
(iii) It is not based on any linearization of the system
dynamics and can be very efficient in state estimation in
nonlinear dynamical systems,
(iv) When applied to linear system with Gaussian noise, the
Particle Filter asymptotically approaches the Kalman filter if
the number of particles becomes large.
This paper proposes a Hybrid Particle Filter based
training algorithm to train the Recurrent Neural Networks
(RNN) for the problems of state estimation. Hybrid Particle
Filter is developed using Particle Filter in combination with
EKF.
II. PARICLE FILTERING BASED ESTIMATION
Application of Particle filtering for state estimation
involves various stages as discussed in the following sub-
sections.
A. The particle approximation of probability density
functions
Particle filtering is a method for state estimation that is
not dependent on the probability density function of the
measurements. In the general case the equations of the
optimal filter used for the calculation of the state-vector of a
dynamical system do not have an explicit solution. In that
case approximation through Monte-Carlo methods can be
used [6,7]. A sample of size N is assumed, that is, N
independent identically distributed (iid) variables, ξ1,ξ 2 ,· · ·
ξ N. This sampling follows the probability density functions N
(p.d.f.) p(x) i:e:ξ1: N ~ p(x). Instead of p(x) the function p(x(k)| Y ) = ∑ wki −1δ ξ i ( x(k )) (4)
1 N i=0 k
p(x) ≈ pN(x) =
N
∑ δ ξ i ( x)
i =0
can be used. It is assumed that
with ξ i ~ p(x(k) | x(k - 1)) = ξ ki −1 )
all points ξi have an equal weighted contribution to the k
The Eq. (4) describes the state equation of the system is
approximation of p(x).
executed N times, starting from the N previous values of the
A more general approach would be if weight factors were
assigned to the points ξi, which would also satisfy the state vectors x(k - 1) = ξ i
N k −1
normality condition ∑q i =1 xˆ ( k + 1) = A( xˆ ( k )) + B ( k )u ( k ) + w( k ) (5)
i =1 y (k ) = C ( xˆ (k )) + u (k ) (6)
In the latter case Thus estimations of the current value of the state vector are
N
1 obtained, xˆ ( k ) and consequently the mean value of the
p(x) ≈ pN(x) =
N
∑ q δ ξ i ( x)
i =0
i (1)
state vector will be given from Eq. (4).This means that the
If p(ξi) is known then the probability P(x) can be value of the state vector which is calculated in the prediction
approximated using the discrete values of the p.d.f. p(ξi) = stage is the result of the weighted averaging of the state
wi. If sampling over the p.d.f. p(x) is unavailable, then one vectors which were calculated after running the state
equation, starting from the N previous values of the state
can use a p.d.f. p (x) with a similar support set, i.e. p(x) = 0 vectors ξ i .
k −1
⇒ p (x) = 0. Then it holds that
p(x)
E(g(x)) =∫g(x)p(x)dx = ∫ g(x)p(x) dx .
p(x)
If the N samples of p (x) are available at the points
~1 ~N ~ i
ξ ….. ξ i.e. p (ξ ) = δ ~ i ( x ) and the weight
ξ
~
p(ξ i )
coefficients qi are defined as qi = ~
p(ξ i ) Fig 1: Schematic diagram of the Particle Filter Loop
Then it is easily shown that C. The correction stage
N ξ~1: N ≈ p ( x)
i ~
E ( g ( x )) ≈ ∑ q g (ξ i) where  (2) The a-posteriori probability density is found using Eq.
 i p( ~ xi) (4). Now a new position measurement y(k) is obtained and
i =0 q = ~ i
 p( x ) the objective is to calculate the corrected probability density
Eq. (2) is important in the sense that assuming pdf p(x) is p(x(k)|Y ), where Y = y(1),y(2),· · · ,y(k). From Bayes law it
unknown (target distribution) but the pdf p (x ) (importance holds that p(x(k)|Y ) = p (Y ) | x(k )) p ( x(k ))
p(Y )
law) is available, then, it is sufficient to sample on p (x ) and
which can be also written as
find the associated weight coefficients qi so as to calculate
E(g(x)). p ( y (k ) | x (k )) p ( x(k ) | Y (7)
p(x(k) | Y ) =
B. The prediction stage ∫ p ( y (k ) | x(k ), Y ) p( x(k ) | Y
Substituting Eq. (4) into Eq.(7) and after intermediate
As in the case of the Kalman Filter or the Extended calculations one finally obtains
Kalman Filter, the Particle Filter consists of the N
measurement update (correction stage) and the time update p (x(k)| Y ) = ∑ qki δ ξ i ( x(k ))
i =0 k
(prediction stage). The prediction stage calculates p(x(k)| Y )
where qi p ( y ( k ) | x( k ) = ξ i )
where q i = k k (8)
Y = y(1); y(2); · · · ; y(k − 1) according to Eq. (1). k N
It holds that: ∑ qi p( y (k ) | x(k ) =ξi
N k k
p(x(k − 1)| Y ) = ∑ w i δ (3) j =1
k −1 ξ i ( x( k − 1)) Eq. (8) denotes the corrected value for the state vector. The
i =0 k −1
while from the Bayes formula it holds that recursion of the Particle Filter proceeds in a way similar to
the update of the Kalman Filter or the Extended Kalman
p(x(k)| Y ) =∫p(x(k)|x(k − 1))p(x(k − 1)| Y )dx. Filter, i.e.:
Using also Eq. (3) one finally obtains
• Measurement update: Acquire y(k) and compute new value network with zero value, the error covariance matrices with
of the state vector identity matrices and the state and measurement noise
N covariance matrices with suitable values :
p(x(k)| Y ) = i
∑ q k δ ξ i ( x(k )) For i = 1,2,………,N
i=0 k • Draw the weights w(i)0 from the input layer of the
with corrected weights recurrent neural network, p(w0)
• Evaluate the importance ratios:
q i p ( y (k ) | x ( k ) = ξ i )
i
qk = k k (9) q0(i) = p(y0|w0(i )
N (i )
∑ q k p( y (k ) | x(k ) = ξ ki ) i
i
~ (i ) =
• Normalize the importance ratios: q
q0
j =1 0 N
( j)
and ξ ki = ξ i ∑ q0
k j =1
• Time update: Compute state vector x(k +1) according to 2. SAMPLING STAGE: For i = 1,· · · ,N
the pdf • Predict via the dynamic equation:
N
p(x(k + 1)|Y ) = i
∑ q k δ ξ i ( x (k )) (10) wˆ k(i+) 1 = wk(i ) + d k(i )
i =0 k (i )
where d k is a sample from p(dk) (N(0,Q)in our case).
where
k
ξi k
)
~ p(x(k) | x(k - 1)) = ξ i The stages of state
• Evaluate the important ratios:
vector estimation with the use of the particle filtering
algorithm are depicted in Fig.1. q k(i+) 1 = q k(i ) p ( y k +1 | wˆ k(i+) 1
• Normalise the important ratios:
III. HYSIR BASED RNN FOR STATE ESTIMATION q k(i+) 1
RNN can be used to represent the behaviour of complex q~k(i+)1 =
N
( j)
dynamical systems and this is not possible through normal ∑ q k +1
feed-forward networks. In this work Hybrid Particle Filter j =1
(HySIR) based training is considered for RNNs that solve 3. RESAMPLING STAGE: For i = 1, · · · ,N
complex estimation problems. HySIR was originally If Neff ≥ Threshold:
proposed for training Multi-layer Feedforward Networks [2]. (i )
• wk +1 = w ˆ k(i+) 1 Else
A. Recurrent neural networks for state estimation
• Resample new index J from the discrete
The main advantage associated with the use of RNN for
the estimation problem is that it does not require the set w ˆ k(i+) 1 , qˆ k(i+) 1
mathematical model of the system.
(i )
The state space model of the recurrent neural networks • wk +1 = w ˆ k(i+) 1
considered here is of the form:
W (k + 1) = W (k) + d(k) (i ) 1
• qˆ k +1 =
y(k) = g(W (k), u(k), V (k)) + v(k) (11) N
The process equation specifies that the state of the ideal In neural networks framework, these limitations are
neural network is characterized as a stationary process overcome by proposing a new algorithm by combining EKF
corrupted by process noise d(k) which is assumed to be a algorithm with the particle filter. That is, the covariance
zero mean Guassian process with covariance Q and matrix of the network weights for each sampling trajectory
uncorrelated with network weights, where the state of the is propagated and updated with an extended Kalman filter in
system is given by the networks weight parameter values sampling stage of particle filter structure. This leads to an
W(k). The observation or measurement equation represents improved annealed sampling algorithm, whereby the
the networks desired response vector g(k) as a nonlinear variance of each sampling trajectory decreases with time.
function of the input vector u(k), the weight parameter vector That is, we start searching over a large region of the error
W(k), the recurrent node activations V(k) and the random surface and as time progresses, we concentrate on the
measurement noise v(k) which is also assumed to be a zero regions of lower error. This technique is described in the
mean Guassian process with covariance R and uncorrelated following section.
with network weights
C. Proposed algorithm
B. Particle filter for Training RNN
A Proposed algorithm is framed by using the concept of
The essential structure of a sequential monte carlo EKF and particle filter based RNN .In this section describes
(Particle filter) algorithm to train recurrent neural networks the hybrid particle filter algorithm is referred as Hybrid
involves the following steps: Sequential Sampling Importance Resampling (HySIR)
1. INITIALISE NETWORK WEIGHTS (k=0) with algorithm. This is train the RNN based algorithm. A suitable
uniform distribution values, the feedback state of the structure is designed for state estimation of a typical non-
linear system such as induction motor which is considered q k( i+) 1
above in a Bayesian framework with knowledge of input- q~k( i+) 1 =
N
output data for training the network. ( j)
∑ q k +1
j =1
The extended Kalman filter equations are given by: • RESAMPLING STAGE: For i = 1; · · · ;N
wk+1|k= wk If Neff ≥ Threshold:
Pk+1|k = PTk + QImm
(i )
Kk+1 = Pk+1|kGk+1[RIoo + GTk+1Pk+1Gk+1]-1 - wk +1 = w ˆ k(i+) 1
wk+1 = wk+1|k + Kk+1(yk+1 − g(xk;wk+1|k))
(i ) (i )
Pk+1 = Pk+1|k− Kk+1GTk+1Pk+1|k - Pk +1 = Pˆk +1 Else
where kk+1 is known as the Kalman gain matrix, Imm - Resample new index J from the discrete
denotes the identity matrix of size m×m, and R and Q are
two tuning parameters, Here, it suffices to say that they set wˆ k(i+) 1 , qˆ k(i+) 1
control the rate of convergence of the EKF algorithm and
(i )
the generalisation performance of the neural network. In the - wk +1 = w ˆ k( j+)1 and Pk(i+)1 = Pˆk( +j )1
general multiple input, multiple output (MIMO) case,g ЄR0
is a vector function and G represents the Jacobian matrix: (i ) 1
- qˆ k +1 =
 ∂g 1 ∂g 2
.. .. ..
∂g 0 
T
(13) N
 ∂w ∂w1 ∂w1  The HySIR with EKF updates may be interpreted as a
 1
 ∂g 1  dynamical mixture of EKFs. That is, the estimates depend
 ∂w2 
G=
∂dg 
| w − wˆ =  .
 on a few modes of the posterior density function. In addition

∂w  .  to having to compute the Normalized importance ratios

 .


(O(N(oS21+ d2S1))) operations, where S1 is the number of
 ∂g 1 ∂g 0  hidden neurons, N the number of samples, d the input
 .. .. .. ..
 ∂wm ∂wm  dimension and o the number of outputs and resampling
Since the EKF is a suboptimal estimator based on
(O(N)operations), the HySIR has to perform the extended
linearisation of a nonlinear mapping, Pk is an approximation
Kalman filter updates (O(Nom2) operations,where m is the
to the covariance matrix. In mathematical terms:
number of network weights). That is, the computational
[
Pk ≈ E (wk − wˆ k )T (wk − wˆ k ) | Yk ] complexity of the HySIR increases approximately linearly
with the number of EKFs in the mixture. The HySIR
The Kalman filter step, before the update stage, allows us to
incorporate the latest measurements into the prior. As a provides accurate and computationally efficient solutions.
result, we can sample from the prior p(wk|wk-1; yk) before the The HySIR algorithm will only give a representation of a
update stage. Another advantage of this method is that the narrow region of the posterior density function. Typically,
covariance of the weights changes over time. Because the the algorithm will converge to a few modes of the posterior
extended Kalman filter is a minimum variance estimator, density function.
the covariance of the weights decreases with time.
IV. SIMULATION RESULTS
Consequently, as the tracking improves, the variation of the
network weights is reduced. This annealing process In order to illustrate the proposed method let us consider
improves the efficiency of the search for global minima in an induction motor, which is typical nonlinear system and
parameter space and reduces the variance of the estimates. It its parameters selected are given in the Table1.
should be pointed out that the weights need to be resampled
Table 1: Induction Motor Parameters
in conjunction with their respective covariances.
The pseudo-code for the HySIR (Hybrid Sequential Rs=0.85 ohms Rr=0.65 ohms
Importance Resampling) algorithm with EKF updating is as Ls=20.8e-3H Lr=11.9e-3H
follows Lm=11.5e-3H P=2 pair of poles
1. INITIALISE NETWORK WEIGHTS (k = 0): J=0.09Kg.m2 F=0.005N.m.s
2. For k = 1,· · · ,L Wrmax=375(r/s) Tmax=110N.m
• SAMPLING STAGE: For i = 1,· · · ,N Vrms=440V F=50Hz
- Predict via the dynamics equation:
The state noise covariance Q and measurement noise
wˆ k(i+) 1 = wk(i ) + d k(i ) covariance R for Particle filter are set to:
10 0 0 0 0 
(i ) 0
where d k is a sample from p(dk) (N(0,Q)in our  10 . 8 0 0 0 
case). Q = 0 0 0 . 0001 0 0 
 
- Update samples with the EKF equations. 0 0 0 0 . 0001 0 
 0 0 0 0 0 . 01 
- Evaluate the important ratios:
 0 . 01 0 
q k(i+) 1 = q k(i ) p ( y k +1 | wˆ k(i+) 1 R = 
 0 0 . 01 
The inputs are the voltages vqs and vds. The initial state
- Normalise the important ratios: vector is assumed to be zero and initial state error
covariance is taken as identity matrix. The RMS values for
different no: of particle are shown in Table 2
Table 2: RMS values of Particle filter for different no:of particles

Number of particles RMS value

100 4.29996
10000 3.00096
20000 1.8666
30000 0.14091

Fig 6: Estimated Torque through PF and Error in estimation with load

The results comparing the actual states and the estimated

Fig 2: Estimated currents through PF and Error in estimation with no-load
states through PF for 100 particles are shown in Fig 2,3.
The Fig 2. plots the estimated currents The estimated speed
and its error are plotted by the Fig 3is at no-load case and
Fig 5 is at load case. Fig 4 & 6 shows the accuracy of the
estimation method is tested by no-load, varying load torque
conditions of the induction motor. The load torque
variations are applied at the 0.37 sec, 0.55 sec and 0.68 sec.

Fig 3: Estimated speed through PF and Error in estimation with no-load

Fig 7: Comparison of d-q axis rotor flux through EKF and HySIR based
RNN with no-load

Fig 4: Estimated Torque through PF and Error in estimation with no-load

Fig 8: Comparison of d-q axis rotor flux through EKF and HySIR based
RNN with load

Fig 5: Estimated speed through PF and Error in estimation with load Fig 9: Comparison of rotor speed and torque through EKF and HySIR
based RNN with no-load

Fig 10: Comparison of rotor speed and torque through EKF and HySIR
based RNN with load
The simulation based implementation of RNN based
state estimator in the first stage is structured with 4 input
nodes, which are per unit d-q stator currents and voltages.
The numbers of nodes in the hidden layer are optimally
selected to 40 and the out put neurons are set to 2 which are
the required state estimates, i.e., the rotor d-q fluxes for the
first stage.The network is trained with 300 sample points
and 10 particles using the HySIR learning algorithm. The
network weights are initialized with random numbers with
normal distribution.
For HySIR algorithm the state error covariance matrices
are set to small positive valued, diagonal matrices. The state
noise covariance matrices are selected from range of 10− 2 to
10− 8 valued, diagonal matrices. Similarly the measurement
noise covariance matrices are selected from range of 100 to
10− 3valued, diagonal matrices. For fast learning process the
state and measurement noise matrices are to be varied as the
training proceeds. The load torque variations are applied at
0.37 sec, 0.55 sec and 0.68 sec. For RNN in the second
stage, the number of inputs are 4, i.e., the rotor d-q fluxes
and the stator d-q currents, the number of hidden nodes are
40 and the number of outputs are 2 which corresponds to the
rotor speed and the torque. The network is trained with
similar conditions as used for first stage RNN. For testing
the trained network the rotor d-q fluxes estimated in the first
stage and the stator d-q currents are given as inputs and with
no load and varying load torque conditions. The Fig.5,6
plots the speed and torque estimated by cascaded RNN state
estimator. Here also the load torque variations are applied
at 0.37 sec, 0.55 sec and 0.68 sec .The comparison between
EKF based RNN and HySIR based RNN estimator are
plotted in Fig. 7 to 10 for both type of operating conditions..
The speed and torque estimations is performed in a better
way than that of the EKF based RNN estimators shown in
Fig. 10 with the varying load torque conditions.
V. CONCLUSIONS
For state estimation of Induction motor using recurrent
neural networks, a sequential Monte Carlo approach i.e.
particle filter is presented for training neural networks in a
Bayesian setting. A particular contribution is the adoption of
HySIR algorithm, which combines gradient information of
the error function with probabilistic sampling information,
for the RNN case. Simulations show that this combination
performs better than EKF and conventional SIR approaches.
HySIR can be interpreted as a Gaussian mixture filter, in
that only a few sampling trajectories need to be employed.
Yet as the number of trajectories increases, the
computational requirement increases only linearly.
Therefore, the method is also suitable as a sampling strategy
for multimodal optimization. Further avenues of research
include the design of algorithms for adapting the noise
covariances R and Q, studying the effect of different noise
models for the network weights, and improving the
computational efficiency of the algorithms.
REFERENCES
[1] Fichou J, LeGland F, Mevel L. "Particle-based methods for parameter
estimation and tracking: Numerical experiments”. Rennes, France:
Publication InterneIRISA; 2004. 1604p.
[2] De Freitas JFG, Niranjan M, Gee AH, Doucet A. “Sequential Monte
Carlo Methods for Optimisation of Neural Network Models,”
Cambridge University Engineering Department,
CUED/FINFENG/TR 328, November 18, 1998.
[3] Volkan Cevher, “Importance Sampling”, November 17,
2008 Rice University,STAT631 / ELEC 639: Graphical Models.
[4] Rigatos GG. “Particle Filtering for state estimation in industrial
robotic ystems”. IMeche Journal of Systems and Control Engineering
2008;222:437-55
[5] KadirkamanathanV, Jaward MH, Fabri SG, Kadirkamanathan
“Particle filters for recursive model selection in linear and nonlinear
system identification”. In: Proc. 39th IEEE CDC conference. 2000.pp.
2391-96.
[6] Gustafsson F, Gunnarsson F, Bergman N, Forssel U. “article filters
for positioning, navigation and tracking. IEEE Transactions on Signal
Processing” Special Issue on Monte Carlo Methods for Statistical
Signal Processing 2005; vol50,Pp:425-37.
[7] Brmaud P. “rkov-chains, Gibbs fields, Monte-Carlo simulation and
queues”. In: Text in applied mathematics. Springer; 1999.
[8] Doucet A, De Freitas N, Gordon N, editors. “Sequential Monte Carlo
methods in practice”. Springer; 2001.
[9] Gerasimos G. Rigatos. “Particle and Kalman filtering for state
estimation and control of DC motors”.Unit of Industrial Automation,
Industrial Systems Institute, 26504 Rion Patras, Greece. ISA
Transactions 48 (2009),Pp: 62-72
[10] R. E. Kalman, “A new approach Linear Filtering and Prediction
Problems”, Trans. of the ASME, J.of Basic Engg., March 1960,
Pages: 35-45.
[11] Manya Afonso, “Particle Filter and Extended Kalman Filter for
Nonlinear Estimation:A Comparative Study”, IEEE Transaction, 4th
March 2008.
[12] I. Loukil, M. Gossa, A. Chaari, M. Jemli and K. Jarray,
“Induction Machine Paprameters and State Estimation using Kalman
Filter approach”, IEEE Electrotechnical Conference, Vol. 1, 1996,
Pp:265-268.
[13] K.S Narendra and K.Parthasarathy,”Identification and control of
dynamical systems using neural networks”, IEEE Transactions on
Neural Networks, Vol.I, 1990, Pp: 4-27.