The McCulloch-Pitts Neuron.

In 1943, McCulloch and Pitts demonstrated that because of the all-or-none

character of nervous activity, neural events and the relations among them can be treated
by means of the propositional logic. They modelled the neuron as a binary discrete-time
element with excitatory and inhibitory inputs and an excitation threshold. The network of
such elements was the first model to tie the study of neural networks to the idea of
computation in its modern sense.
The basic idea was to divide time into units comparable to a refractory period
(assumed to be the same for each neuron) so that in each time period at most one spike
can be initiated in the axon of a given neuron. Thus, the McCulloch-Pitts neuron operates
on a discrete time scale, t = 0,1,2,3, (Fig.2.1).

Fig.2.1. The McCulloch-Pitts neuron.

The input values ait from the i-th presynaptic neuron at any instant t may be equal either
to 0 or 1 only, while the weights of connections wi are +1 for excitatory type connection
and -1 for inhibitory type connection. There is an excitation threshold associated with
each neuron. Output X t 1 of the neuron at the following instant t+1 is defined according
to the rule

X t 1 1 if and only if S t wi ait , while wi 0 ai 0.

i

The latter means that activity of a single inhibitory input, i.e. input via a connection with
negative weight wi = -1, would absolutely prevents excitation of the neuron at that instant.
Note also that the state S t of the McCulloch-Pitts neuron does not depend on the
previous state of the neuron itself, but is simply

S t f (t ) i wi ait ,

thus, the output may be written as function of discrete time

X (t ) g ( f (t )),
where g is the threshold activation function

1, f
g( f ) H ( f ) .
0, f

1, x 0;
Here H is the Heviside (unit step) function: H ( x)
0, x 0.
Simple logical functions can be implemented directly with a single McCulloch-
Pitts unit. The output value 1 can be associated with the logical value true and 0 with the
logical value false. Now, let us demonstrate how weights and thresholds can be set to
yield neurons which realise the logical functions AND, OR and NOT.

1) AND (the output fires if a1 and a2 both fire):

2) OR (the output fires if a1 or a2 or both fire):

3) NOT (the output fires if a1 does NOT fire):

 A single McCulloch-Pitts neuron can compute the conjunction or disjunction of
n arguments, as it is shown below, while two conventional logic units are needed to
perform the conjunction of just three arguments:

OR AND

The same kind of computation requires several convention logic gates with two inputs.
From that simple example, it is clear that threshold logic elements can reduce the
complexity of the circuit used to implement a given logical function.
Although the McCulloch-Pitts neuron model was very simplistic, it had
substantial computing potential. As it can perform the basic logic operations AND, OR
and NOT, it can implement any multivariable combinational function, provided the
thresholds and weights are appropriately selected. The unity delay property of the model
makes it possible to build sequential digital circuitry.
For that note that a single neuron with a single input a and with the weight and
threshold values both of unity, computes the output Xk+1=ak. Such a simple network thus
behaves as a single register cell able to retain the input for one period elapsing between
two instants. Consequently, once a feedback loop is closed around the neuron, as it is
shown in the Fig.2.2, we obtain a memory cell. An excitatory input of 1 initialises the
firing in this memory cell, and an inhibitory input of 1 initialises a nonfiring state.
 Fig.2.2. Memory cell.

The output value, in the absence of inputs, is then sustained indefinitely. This is because
of the output of 0 fed back to the input does not cause firing at the next instant, while the
output of 1 does.

Conclusion.

The McCulloch and Pitts 1943 paper had a huge influence on the thoughts and
studies that led to modern digital computer design, as it outlined the first formal model of
an elementary computing neuron. The model included all necessary elements to perform
logic operations, and thus it could function as an arithmetic-logic computing element to
compute any computable function. This was a very significant result and with it, it is
generally agreed that the disciplines of neural networks and artificial intelligence were
born.
Though the threshold elements are, from the combinatorial point of view, more
versatile than conventional logic gates, there was a problem with assumed unlimited fan-
in: the implementation of the compact electronic model was not feasible in the days of
bulky vacuum tubes. The formal neuron model was not widely adopted for the vacuum
tube hardware description, and the model never became technically significant. Now-
days a possible way of circumventing the hardware difficulties could be the use of optical
computing elements capable of providing unlimited fan-in.
The main ideological problems of the McCulloch-Pitts model were that the
network must be completely specified before its using and there were no free parameters
to suit different problems. Learning can only be implemented by modifying the
connection pattern of the network and the thresholds of the units, which is necessarily
more complex than just adjusting numerical parameters.