Embedded Neural Network

MANOJKUMAR.R*1, GOKULA KRISHNAN.D*2 AND DR.SENTHIL,HOD*3.

ELECTRONICS AND INSTRUMENTATION DEPARTMENT, PANIMALAR ENGINEERING COLLEGE, ANNA UNIVERSITY
*1

manojkumarr161@gmail.com

*2gokulakannankrishnan@gmail.com
*3

sen.annauniv@gmail.com

ABSTRACT-:
This paper describes a method of linearizing the nonlinear characteristics of CO2 sensor using an embedded neural network .Generally the efficiency of sensors will reduce by usage. This Neural network comprises of many algorithms such as Multilayer Preceptron(MLP), Bridged Multilayer Preceptron(BMLP), Fully Connected Cascade(FCC), Neuron by Neuron(NBN) etc. In MLP the training becomes more difficult because the networks become less transparent for error propagation. Unfortunately, most of available learning software is not able to train FCC or BMLP architectures. In order to implement more powerful neural network architectures on microcontrollers, the special NBN (neuron-by-neuron) process was developed in assembly language to allow the fastest and shortest code on the embedded system. The embedded neural network also requires an accurate approximation for hyperbolic tangent to be used as the neuron activation function. The proposed method allows for complex neural networks with very powerful architectures to be embedded on an inexpensive ARM 7 microcontroller. 1.

INTRODUCTION

1.1. EXISTING SYSTEM: The power of neural networks strongly depends on the neural network architectures. Popular MLP architectures are not only one of the least powerful architectures but also have other disadvantages. With an increased number of layers, the training of such networks becomes more difficult because the networks become less transparent for error propagation. In other words, the effect of weight changes on the first layers are disturbed by weight changes of the subsequent Even for FCC neural networks with connections across layers, their architectures layers. The second disadvantage of multilayer networks without connections across layers is that inaccuracies in the first layers are magnified by inaccuracies of subsequent layers. Due to crosslayer connections, the network is much more transparent for training and is also less sensitive to error accumulations from layer to layer.

should have as small number of layers as possible. with seven hidden layers, where there are only single neurons in each layer, it would be more desirable to introduce several neurons in each layer and to reduce the number of hidden layers. Such architectures, known as bridged multilayer

many applications. This particular application method has been implemented using assembly language on an 8-b microcontroller. This method, however, can be easily implemented using C and integer math in the same manner. The key components are the implementation of the superior types of neural network architectures the concept of using an exponent for each neuron, custom multiply and accumulate method, and a better activation function for embedded neural networks. The combination of these four components
creates a functionally practical and efficient embedded neural network solution for a large variety of

perceptron (BMLP), are a good compromise between MLP and FCC architectures. For example, the BMLP architecture N = 4 = 4 = 4 = 1 with only three hidden layers and four neurons in each layer is capable to solve problems up to parity-249. In order to be able to implement the neural networks with arbitrary connected architecture, a special method of computation scheme had to be developed. The computation the process on

applications. This implementation, being hardware

independent, allows the user to customize the requirements for his application. These parameters are completely adjustable for the users application. These concepts have been presented to be implemented on any embedded system without floating-point hardware to optimize the balance of speed and accuracy. It has been shown that, with the correct neural network architectures, even very difficult problems can be solved with just a few neurons. When using the NBN training method, these networks can be easily trained. Then, by using the NBN forward calculation method, networks with any architecture can be used at the embedded level. The second-order approximation of tanh in conjunction with the pseudo floatingpoint routines allows almost any neural network to

microcontroller algorithm.

follows

neuron-by-neuron requires special

This

method

modifications due to the fact that assembly language is used with very limited memory resources. Unfortunately, most of available learning software is not able to train FCC or BMLP architectures. Only very recently, adequate and very efficient algorithms for arbitrarily connected feed forward neural networks have been developed. In order to implement more powerful neural network architectures on microcontrollers, the special NBN computing routine should be used. 1.2 PROPOSED SYSTEM:

be embedded in a simple low-cost microcontroller. This paper has presented a method for implementing an very inexpensive and low-end arbitrary neural network on an embedded system. The ThisFor process balances speed with accuracy for systems that do microcontrollers, a floating-point algorithm not have floating-point hardware. The operational goal was has been developed and optimized for neural to create a neural network that was as fast as possible while introducing an error that is not greater than the error alreadynetworks. introduced by using a neural network to approximate the original function. When these very similar errors are combined, they fail to produce a final product that is closer to the original function than that of the ideal double precision neural network. This goal was met and has been a successful compromise between speed and accuracy for

2.

Problem Description:

under 4bit mode, only four data lines are connected to ADC pin of processor. Other four data lines are grounded 4.SOLUTION TO THE PROBLEM: 4.1 NEURAL NETWORK: Neural networks can be classified into dynamic and static categories. Static (feed forward) networks have no feedback elements and contain no delays; the output is calculated directly from the input through feedforward connections. The

Generally the efficiency of sensors will reduce by usage. This non linearity can be reduced by using the concept of neural network and it comprises algorithms such as MLP,BMLP, FCC, NBN etc. With an increased number of layers in MLP the training of such networks becomes more difficult because the networks become less transparent for error propagation. Unfortunately, most of available learning software is not able to train FCC or BMLP architectures. 3. APPROACH TO THE PROBLEM:

training of static networks was discussed in Back propagation. In dynamic networks, the output depends not only on the current input to the network, but also on the current or previous inputs, outputs, or states of the network. You saw some linear dynamic networks in Linear Filters.Dynamic networks can also be divided into two categories: those that have only feedforward connections, and those that have feedback, or recurrent, connections. Probabilistic neural networks can be used for classification problems. When an input is

In order to implement more powerful neural network architectures on microcontrollers, the special NBN computing routine should be used.The neuron-by-neuron process was developed in

assembly language to allow the fastest and shortest code on the embedded system. The embedded neural network also requires an accurate

approximation for hyperbolic tangent to be used as the neuron activation function. The proposed method allows for complex neural networks with very powerful architectures to be embedded on an inexpensive ARM 7 microcontroller. 3.1BLOCK DIAGRAM: ARM7 SENSOR Controller LCD Display

presented, the first layer computes distances from the input vector to the training input vectors and produces a vector whose elements indicate how close the input is to a training input. The second layer sums these contributions for each class of inputs to produce as its net output a vector of probabilities. Finally, a compete transfer function on the output of the second layer picks the maximum of these probabilities, and produces a 1

PROJECT DESCRIPTION:

(LPC The gas sensor sense the co2 gas present in the 2103) 4.2 NETWORK ARCHITECTURE:
place where it need to be measure. It has nickel chromium membrane, it has the ability to sense co2 upto 3000 PPM. present in the

for that class and a 0 for the other classes.

It is assumed that there are Q input

The sensor measures co2 in terms of PPM, and the ADCvector/target vector pairs. Each target vector has K ARM processor converts that intoelements. One of these elements is 1 and the rest corresponding digital value. The operating voltage requiredare 0. Thus, each input vector is associated with for LCD display is 3.3v and 5v. The LCD display usedone of K classes.

The

first-layer

input

weights,

IW1,1

The transfer function net input n, again a scalar, is the sum of the weighted input wp and the bias b. This sum is the argument of the transfer function f. (Radial Basis Networks, discusses a different way to form the net input n.) Here f is a transfer function, typically a step function or a sigmoid function, that takes the argument n and produces the output a. Examples of various transfer functions are in Transfer Functions. Note that w and b are both adjustable scalar parameters of the neuron. The central idea of neural networks is that such parameters can be adjusted so that the network exhibits some desired or interesting behavior. Thus, you can train the network to do a

(net.IW{1,1}), are set to the transpose of the matrix formed from the Q training pairs, P'. When an input is presented, the dist box produces a vector whose elements indicate how close the input is to the vectors of the training set. These elements are multiplied, element by element, by the bias and sent to the radbas transfer function. An input vector close to a training vector is represented by a number close to 1 in the output vector a1. If an input is close to several training vectors of a single class, it is represented by several elements of a1 that are close to 1 The second-layer weights, LW1,2

particular job by adjusting the weight or bias parameters, or perhaps the network itself will adjust these parameters to achieve some desired end. All the neurons in the Neural Network Toolbox software have provision for a bias, and a bias is used in many of the examples and is assumed in most of this toolbox. However, you can omit a bias in a neuron if you want. The approximate value of tanh(net) is found in several steps. 1) Using the higher bits of the net value, the proper piecewise segment is selected between xA and xB.

(net.LW{2,1}), are set to the matrix T of target vectors. Each vector has a 1 only in the row associated with that particular class of input, and 0's elsewhere. (Use function ind2vec to create the proper vectors.) The multiplication Ta1 sums the elements of a1 due to each of the K input classes. Finally, the second-layer transfer function,

compete, produces a 1 corresponding to the largest element of n2, and 0's elsewhere. Thus, the network classifies the input vector into a specific K class because that class has the maximum probability of being correct. 4.3 SIMPLE NEURON: A neuron with a single scalar input and no bias appears on the left below. The scalar input p is transmitted through a connection that multiplies its strength by the scalar weight w to form the product wp, again a scalar. Here the weighted input wp is the only argument of the transfer function f, which produces the scalar output a. The neuron on the right has a scalar bias, b. You can view the bias as simply being added to the product wp as shown by the summing junction or as shifting the function f to the left by an amount b. The bias is much like a weight, except that it has a constant input of 1.

Then, the corresponding values of yA and yB are found from memory. In our implementation, four most significant bits were used to retrieve data for 16 segments. 2) The x value is obtained from lower bits of net value. 3) Using Xa , xB, yA, and yB, the first-order linear approximationis computed at first y1(x) =(yA+(yB yA) x)/x

4) Then, the quadratic component is found as y2(x) =(4 yA x (x x))/ x2 4.4 TRAINING ARBITRARILY CONNECTED NEURAL NETWORKS: The NBN algorithm introduced in this chapter is developed for training arbitrarily connected neural networks using LevenbergMarquardt

sequence has to follow the concept of feed forward signal propagation. If a signal reaches the inputs of several neurons at the same time, then these neurons can be processed in any sequence.

update rule. Instead of layer-by-layer computation , the NBN algorithm does the forward and backward computation based on NBN routings, which makes it suitable for arbitrarily connected neural networks 4.4.1IMPORTANCE ARBITRARLY NETWORKS The traditional implementation of Levenberg Marquardt algorithm [TM94], like MATLAB neural network toolbox (MNNT), was adopted only for standard MLP (multilayer perceptron) OF TRAINING NEURAL Figure 4.1: Structure Of Neurons With Hidden Layers

CONNECTED

networks, it turns out that the MLP networks are not efficient. The standard MLP network with one hidden layer needs at least eight neurons to find the solution. The BMLP (bridged multiplayer Figure 4.2: Input Weight Calculation The two procedures will lead to different computing processes but with exactly the same results. When the forward pass is concluded, the following two temporary vectors are stored: the first vector y with the values of the signals on the neuron output nodes and the second vector s with the values of the slopes of the neuron activation functions, which are signal dependent.

perceptron) network can solve the problem with four neurons. The FCC (fully connected cascade) network is the most powerful one, and it only

requires three neurons to get the solutions. One may notice that the last two types of networks are better choices for efficient training, but they also require more challenging computation
4.4.2 FORWARD COMPUTATION:

In the forward computation, the neurons connected to the network inputs are first processed so that their outputs can be used as inputs to the subsequent neurons. The following neurons are then processed as their input values become available. In other words, the selected computing 4.4.3 BACKWARD COMPUTATION: The sequence of the backward computation is opposite to the forward computation sequence. The process starts with the last neuron and continues

toward the input. In the case of the network, the following are two possible sequences (back propagation paths): n5n4n3n2n1 or n5n4n3n1n2, and also they will have the same results. To demonstrate the case, let us use the n5n4n3n2n1 sequence. The vector represents signal

has been a successful compromise between speed and accuracy for many applications. The example showing a method of linearizing nonlinear sensor data for nonlinear control

problems using neural networks at the embedded level has been introduced. It has been shown that, with the correct neural network architectures, even very difficult problems can be solved with just a few neurons. When using the NBN training method, these networks can be easily trained. Then, by using the NBN forward calculation method, networks with any architecture can be used at the embedded level. 6.REFERENCES:
1.Bashyal S; Venayagamoorthy G.K; Paudel B.,(2008) Embedded neural network for fire classification using an array of gas sensors, in Proc. IEEE SAS,pp. 146148. 2. Bose, B.K., (2007) Neural network applications in power electronics and motor drivesAn introduction and perspective, IEEE Trans. Ind. Electron., vol. 54, no. 1, pp. 1433. 3. Dempsey G.L; Alt N.L; Olson B.A; Alig J.S., Control sensor linearization using a microcontroller-based neural network, in Proc. IEEE Int. Syst., Man, Cybern (1997) Computational Cybernetics and Simulation Conf., pp. 30783083. 4. Ge, S.S; Wang, C.,(2004) Adaptive neural control of uncertain MIMO nonlinear systems,IEEE Trans. Neural Netw., vol. 15, no. 3, pp. 674 692. 5. Kosmatopoulos E.B; Polycarpou M.M; Christodoulou M.A; Ioannou P.A.,(1995) High-order neural network structures for identification of dynamical systems, IEEE Trans. Neural Netw., vol. 6, no. 2, pp. 422431. 6.Lewis F.A; Yegildirek A; Kai L.,(1996) Multilayer neural-net robot controller with guaranteed tracking performance, IEEE Trans. Neural Netw., vol. 7, no. 2, pp. 388399.

propagation from a network output to the inputs of all other neurons. The size of this vector is equal to the number of neurons. For the output neuron n5, its sensitivity is initialed using its slope 1,5 = s5. For neuron n4, the delta at n5 will be propagated by w45the weight between n4 and n5, then by the slope of neuron n4. So the delta parameter of n4 is presented as 1,4 = 1,5w45s4. For neuron n3, the delta parameters of n4 and n5 will be propagated to the output of neuron n3 and summed, then multiplied by the slope of neuron n3, as 1,3 = (1,5w35 + 1,4w34)s3.For the same procedure, it could be obtained that 1,2 = (1,3w23 + 1,4w24)s2 and 1,1 = (1,3w13 + 1,5w15)s1.After the backpropagation process is done at neuron N1, all the elements of array are obtained. 5.CONCLUSION: This project has presented a method for implementing an arbitrary neural network on an embedded system. This process balances speed with accuracy for systems that do not have floating-point hardware. The operational goal was to create a neural network that was as fast as possible while introducing an error that is not greater than the error already introduced by using a neural network to approximate the original function. When these very similar errors are combined, they fail to produce a final product that is closer to the original function than that of the ideal double precision neural network. This goal was met and