Assignment 2

(Machine Intelligence)

Ashish Kushwaha
Research Scholar,
Deptt. Of EE, SOE,
Shiv Nadar University

1
Ashish Kushwaha, Ph.D Research Scholar (EE), Deptt. Of EE, SOE, Shiv Nadar
University

Answer 1.1 The function is

(v )=

1
1+eav

The derivative is
d d
1
=
dv dv 1+eav

d
=a(1+ eav )2 eav
dv
d
1
=a
av
dv
1+e

)(

av

e
av
1+ e

Or we can write
d
=a ( v )(1 ( v ))
dv
Since we need to find the value of this derivative at the origin, therefore by putting v=0 in the
derivative, we get
d
a 0 2 a 0
=a(1+ e ) e
dv v=0

Or
d
2
=a(1+1) 1
dv v=0

i.e.
d
a
=
dv v=0 4

2
Ashish Kushwaha, Ph.D Research Scholar (EE), Deptt. Of EE, SOE, Shiv Nadar
University

Answer 1.2
(v )=

1eav
av
=tanh
av
2
1+ e

The derivative of the above function will be

av

d d 1e
=
dv dv 1+eav

2
d
1
=( 1eav ) a ( 1+eav ) eav +
a eav
av
dv
1+ e

The above equation can be written as

av

d
ae
=
dv 1+eav

)(

av

1e
+1
av
1+ e

av
( v )= 2 e av
1
As
1+ e

So,
Derivative can be written as

av

d a
1e
= 1
av
dv 2
1+e

)(

av

1e
+1
av
1+ e

d a
= ( 1 ( v ) ) ( ( v )+1 )
dv 2
d a
2
= ( 1 (v) )
dv 2
On putting the v=0, the value of derivative at origin is

3
Ashish Kushwaha, Ph.D Research Scholar (EE), Deptt. Of EE, SOE, Shiv Nadar
University

a 0

d a
1e
= 1
a 0
dv 2
1+e

a 0

)(

1e
+1
a 0
1+e

d a
=
dv 2
Now if the slope parameter a is made infinitely large, the odd sigmoid function will become the
hard limiter threshold function.

Answer 1.3 - Algebraic odd sigmoid function is given by

(v )=

v
1+ v2

The derivative of the given function is given by

d d
=
dv dv
d
=
dv

v
1+v 2

1
1
v (1+v 2 ) 2 2 v
2
2
1+v

( )(

d
=
dv

1+v

d
=
dv

1
1+v 2

v
2
1+ v

1
1+ v2

)( )

As
3 ( v ) =

v3
(1+ v 2 ) 1+v 2

So
4
Ashish Kushwaha, Ph.D Research Scholar (EE), Deptt. Of EE, SOE, Shiv Nadar
University

3
d ( v )
=
dv
v3

( )

On putting the v=0, the value of derivative at origin is

d
=
dv

1
1+02

1
1+02

)( )

d
=1
dv
Answer- 1.4(i)
v

1
(v )=
e
2

x
2

dx

So, we know that

v

c x
2

ec x dx=

erf
4c

(v) includes error function and error function is the part of sigmoid function. So, it fits into the
requirements of a sigmoid function.

(ii)
2 1
( v ) = tan (v )

It is the arc tangent function. Sigmoid function also contains arc tangent function. So we can say
that is a sigmoid function.
Difference between two functions: Both the functions are sigmoid but they have different
varieties means first one is the error function and second one is the arc tangent function.
Answer 1.5
5
Ashish Kushwaha, Ph.D Research Scholar (EE), Deptt. Of EE, SOE, Shiv Nadar
University

Function in Q. 1.4
v

1
(v )=
e
2

x
2

dx

Is cumulative (probability) distribution function (CDF) because,

v

ec x dx= 12 (1+erf x2

( ))

CDF describe probability of a random variable falling in interval ( ,v).

Answer 1.6 (a)

The function is linear between -0.5a to 0.5a. So, the function is given by

0 , for v 0.5 a

( v )= b v +0.5 b , for0.5 a<v < 0.5 a

a
b , for v 0.5 a
(b) If a is allowed to be zero, the slope will become infinite and the function will become
unipolar hard limiter threshold function.
Answer -1.7

6
Ashish Kushwaha, Ph.D Research Scholar (EE), Deptt. Of EE, SOE, Shiv Nadar
University

The function is given by

b , for v a
b
( v )= v , fora< v <a
a
b , for v 0.5 a

if a is allowed to be zero, the slope will become infinite and the function will become bipolar
hard limiter threshold function.

Answer 1.8. If we consider the first function i.e.

(v )=

1
av
1+e

We can write it as
(v )=

1
a (w1 x1 +w 2 x 2+ w3 x3 ++w N x N )

1+e

Similarily the function given in this problem could be written as

(v )=

1
(w1 x1 +w 2 x 2+w 3 x3 ++w N x N )

1+e

Equating these above two equations i.e.

7
Ashish Kushwaha, Ph.D Research Scholar (EE), Deptt. Of EE, SOE, Shiv Nadar
University

 (v )=

1
a ( w 1 x 1+w 2 x 2+ w3 x 3++w N x N )

1+e

(v )=

1
(w1 x1 +w 2 x 2+w 3 x3 ++w N x N )

1+e

Taking log on both sides i.e.

( v ) =log e

( v ) =log e (

1
1+ e

a ( w 1 x 1+ w2 x2 +w 3 x 3+ + w N x N )

1
( w 1 x 1 ' +w 2 x 2 '+w 3 x 3 '++w N x N ' )

1+ e

And now solving and equating we get

x '1=ax 1 , x '2=ax 2 , x '3=ax3 . x 'N =ax N
So we conclude that the the weights should be multiplied by a to get the same inputs as before.

Answer-1.9
In this case
And

x1
w1

x2

=10,
=0.8,

w2

=-20,
=0.2,

x3
w3

=4 amd

x4

=-2

=-1.0 w4=-0.9

So in this case
Output of neuron will ber given by

w 1 x 1+ w2 x2 +w3 x 3 ++ w N x N

Therefore we have 10x0.8+( -20)x 0.2+4 x-1+-2X-0.9=1.8

8
Ashish Kushwaha, Ph.D Research Scholar (EE), Deptt. Of EE, SOE, Shiv Nadar
University

(a) In case of a linear neuron we will have the output to be the same as the sum of products
as done above i.e. 1.8
(b) In case of a McCulloch Pitts model, since the output of a neuron is greater than 0 i.e. 1.8
therefore it is a positive value means greater than T so the final output will be 1

Answer 1.10.
(a)In this case the logistic function given is that of unipolar sigmoid. The value of v is 1.8
therefore by using
(v )=

1
1+ev

We have
(v )=

1
=0.605
1+e(1.8 )

(b)For a McCulloch Pitts model

(v )=

Here

1
1+e1.8
( v ) >0

Here the output is greater than threshold value so the output will be one.
Answer 1.11 (a) and (b)
For the McCullochPitts model of a single neuron we can think about the calculation as
proceeding in two parts, input processing then the calculation of the output.

9
Ashish Kushwaha, Ph.D Research Scholar (EE), Deptt. Of EE, SOE, Shiv Nadar
University

The inputs to the neuron body are shown as

x1

x2

xN

. When the input

(from the previous neurons) reaches this neuron, then their values are multiplied by the synaptic
strength. These strengths are dependent on how much learning has occurred. These strengths are
w1
w 1 x1
called weights and are represented by
etc. So a single input is calculated as
and
the total weighted input to the neuron is
w 1 x 1+ w2 x2 +w3 x 3 ++ w N x N
This total input is then passed on to the output-calculation part of the neuron. Here, if the
total input is greater than some threshold , then the neuron will fire and produce an output.
Note, that the output of the neuron is effectively binary; fire or no-fire i.e 0 or 1.
It is possible to relax the binary nature of the output, to obtain real numbers (such as
decimals). This is done using the sigmoid function. The parameter k adjusts the slope of the
transition between fire and no-fire states. The mathematical description of the sigmoid function
is
f ( a )=

1
ka
1+ e

It is clear that choosing a large value of k approximates the binary McCulloch-Pitts

neuron more closely.

Sigmoid Functions. Left has k = 10, right has k = 5. Large k means steep slope.

10
Ashish Kushwaha, Ph.D Research Scholar (EE), Deptt. Of EE, SOE, Shiv Nadar
University