NPC simulation using artificial neural networks

HUMAN or COMPUTER?

What are ANNs?

An artificial neural network (ANN), usually

called neural network (NN), is a mathematical model or computational model that is inspired by the structure and/or functional aspects of biological neural networks. A neural network consists of an interconnected group of artificial neurons, and it processes information using a connectionist approach to computation

Biological Artificial

Neurons
http://faculty.washington.edu/chudler/color/pic1an.gif

http://research.yale.edu/ysm/images/78.2/articles-neuralneuron.jpg

How do ANNs work?

An ANN is typically defined by three types of

parameters: The interconnection pattern between different layers of neurons The learning process for updating the weights of the interconnections The activation function that converts a neuron's weighted input to its output activation.

Network for Calculating XOR Function

Hopfield pattern recognition

-PATTERN RECOGNITION AND ERROR CORRECTION USING DISTANCE METHOD -HOW?

Basic Perceptron Example

Cash Register Game A contestant in The Price is Right is sometimes asked to

play the Cash Register Game. A few products are described, their prices are unknown to the contestant, and the contestant has to declare how many units of each item he or she would like to (pretend to) buy. If the total purchase does not exceed the amount specified, the contestant wins a special prize. After the contestant announces how many items of a particular product he or she wants, the price of that product is revealed, and it is rung up on the cash register.

Perceptron
Consider each product on the shelf to be a neuron in the input layer, with

its input being the unit price of that product The cash register is the single neuron in the output layer. The only connections in the network are between each of the neurons (products displayed on the shelf) in the input layer and the output neuron (the cash register). The contestant actually determines these connections, because when the contestant says he or she wants, say five, of a specific product, the contestant is thereby assigning a weight of 5 to the connection between that product and the cash register. The total bill for the purchases by the contestant is nothing but the weighted sum of the unit prices of the different products offered In a Perceptron, the way the threshold works is that an output neuron is supposed to fire if its activation value exceeds thethreshold value.

Perceptron

TABLE 3.2 Summary of the Perceptron Convergence Algorithm Variables and Parameters: x(n) = (m +1)-by-1 input vector = [+l,xl(n),xin), ...,xm(nW w(n) = (m + 1)-by-1 weight vector = [b(n), wl(n), wz(n), ... , wm(n)V b(n) = bias y(n) = actual response (quantized) d(n) = desired response 'Tj = learning-rate parameter, a positive constant less than unity 1. Initialization. Set w(O) = O. Then perform the following computations for time step n = 1,2, .... 2. Activation. At time step n, activate the perceptron by applying continuousvalued input vector x(n) and desired response d(n). 3. Computation ofActual Response. Compute the actual response of the perceptron: y(n) = sgn[wT(n)x(n)] where sgn(-) is the signum function. 4. Adaptation ofWeight Vector. Update the weight vector of the perceptron: w(n + 1) = w(n) + 'Tj[d(n) - y(n)]x(n) where + 1 if x(n) belongs to class 'fb1 -1 ifx(n) belongs to class 'fbz 5. Continuation. Increment time step n by one and go back to step 2.

Special NPCs
Goalie and the defender

The goalie A goalie can be best positioned in such a way that the area which needs to be covered is small.
If a goalie only moves on the goal line it will

have to cover the entire goal. But if the goalie moves a bit forward to the ball, the cover-area becomes smaller.
The best spots for the goalie can be found by

drawing a line through the ball and the goal poles. The best spot for the goalie is somewhere on the bisector of this Angle The defender
Defending robot can use the same approach

with a larger diameter.

Other NPCs-Passing
Be receptible for a pass. Players which dont have ball possession should move to positions where they can

receive a pass. The NPC should position itself on the start of the path it needs to travel to the point where it will hit the ball for a shot or a dribble. The NPC providing the pass needs to recognize these points when it attempts a pass. Make passes playable for the receiving robot. Move towards a passed ball. Skill Meaning Move Moves the robot to a place on the field. Dribble Drives the ball to a point. Shoot Shoot the ball to a point. Intercept Reverse the direction of the ball. Catch Defend the goal from shots by diving to the right spot. Kick Directly hits the ball. (skills provided by motion control)

Design Goals for NPCs

ENGINE WAS TO MAKE IT EXTENSIBLE, MODULAR AND ADAPTABLE. THIS IS NEEDED FOR SEVERAL REASONS. F I R S T O F A L L T H E M O T I O N C O N T R O L I S E V O L V I N G ALL THE TIME. THE MODEL MUST MAKE IT E A S Y T O A D D A N D R E M O V E S K I L L S . FURTHERMORE THE SKILLS ITSELF CHANGE TOO. THE CONTEXT W H E N T H E Y S H O U L D B E E X E C U T E D M U S T B E MADE ADAPTABLE. AND FINALLY IT MUST BE EASY .
O N E O F T H E D E S I G N G O A L S O F T H E D E C I S I O N

Design framework

The network has three fields of neurons: 1)one for input neurons, 2)one for hidden processing elements 3)onefor the output neurons. Connections are for feed forward activity.

There are connections

from every neuron in field A to every one in field B, and, in turn, from every neuron in field B to every neuron in field C. Thus, there are two sets of weights, those figuring in the activations of hidden layer neurons, and those that help determine the output neuron activations. In training, all of these weights are adjusted by considering what can be called a cost function in terms of the error in the computed output pattern and the desired output pattern.

Training-Computed Vs Actual pattern

Training The feedforward backpropagation network undergoes supervised training, with a finite number of pattern pairs consisting of an input pattern and a desired or target output pattern.

An input pattern is presented at the input layer. The neurons here pass the pattern activations to the next layer neurons, which are in a hidden layer. The outputs of the hidden layer neurons are obtained by using perhaps a bias, and also a threshold function with the activations determined by the weights and the inputs. These hidden layer outputs become inputs to the output neurons, which process the inputs using an optional bias and a threshold function. The

final output of the network is determined by the activations from the output layer. The computed pattern and the input pattern are compared, a function of this error for each component of the pattern is determined, and adjustment to weights of connections between the hidden layer and the output layer is computed. A similar computation, still based on the error in the output, is made for the connection weights between the input and hidden layers. The procedure is repeated with each pattern pair assigned for training the network.

Training
WEIGHTS THE WEIGHTS USED ON THE CONNECTIONS BETWEEN DIFFERENT LAYERS HAVE MUCH SIGNIFICANCE IN THE WORKING OF THE NEURAL NETWORK AND THE CHARACTERIZATION OF A NETWORK. THE FOLLOWING ACTIONS ARE POSSIBLE IN A NEURAL NETWORK: 1. START WITH ONE SET OF WEIGHTS AND RUN THE NETWORK. (NO TRAINING) 2. START WITH ONE SET OF WEIGHTS, RUN THE NETWORK, AND MODIFY SOME OR ALL THE WEIGHTS, AND RUN THE NETWORK AGAIN WITH THE NEW SET OF WEIGHTS. REPEAT THIS PROCESS UNTIL SOME PREDETERMINED GOAL IS MET. (TRAINING)

Specific details
Input and Output layers

-(x,y) of player -(Vx,Vy) of player(indicating the direction) -(x,y) of Ball -(Vx,Vy) of ball Hidden layer -Pass -Dribble -Shoot to goal -Receive ball -Positioning to receive the ball Based on inference fron Hidden layer,Output layer changes values of parameters This change is linked to the Actions of the players in interface.

Now partial graphics and interface implementation

Tools : C , Allegro
C : NATIVE LANGUAGE , FAST IMPLEMENTATION ALLEGRO : BITMAP HANDLING

Learning !!!!
Supervised learning

Unsupervised learning