Forward Mode:- Lets suppose we have a function f , and we want to find out

first k derivatives by using Forward method. then we will start with an object u∇ ,
which is nothing just a ordered colleection of function and its derivativs.
then

0 00
(0.1) u∇ =< u, u , u , . . . , uk > .
Now we have some arthmetic rule to compute the A.D.

0 0 00 00
(0.2) u∇ + v∇ =< u + v, u + v , u + v , . . . , uk + v k >

0 0 00 0 0 00
(0.3) u∇ v∇ =< uv, u v + v u, u v + 2u v + uv , · · · >
0 0 00
(0.4) sin(u∇ ) =< sin(u), u cos(u), −sin(u)(u )2 + cos(u)u , · · · > .
So how we are getting these operations? simply our first element is the simple
arthmetic and after that we are just following the differentiation rules, each time
diffrentiate it thats it.
If we need to find out differentiation at some perticular point then let a we will
start with.

(0.5) x∇ =< a, 1, 0, 0 >

Example. f (x) = x.Sin(x) − 1. Find out first and second derivative at x = π/4.
First we need to decompose the whole function in simple operations.
let
v∇1 = x
v∇2 = Sin(v∇1 )
v∇3 = v∇1 v∇2
v∇4 = v∇3 − 1.
Note:- This decompostion will be usefull if Reverse mode also.
Now from equ. 5, we first get
v∇1 =< π/4, 1, 0 >=< 0.7854, 1, 0 >
then
0 00
v∇2 = Sin(< 0.7854, 1, 0 >) lets suppose ,< 0.7854, 1, 0 >=< u, u , u > then
use equation 4 and put the values in that. We should get.
v∇2 =< Sin(.7854), 1 × Cos(0.7854), Cos(0.7854) × (0) − Sin(0.7854) × (1)2 >
=< .7071, .7071, −.7071 > .
Now according to our decompostion now we will find v∇3
v∇3 =< .7854, 1, 0 >< .7071, .7071, −.7071 >Lets take these like
0 00 0 00
< .7854, 1, 0 >=< u, u , u > and < .7071, .7071, −.7071 >=< v, v , v > and
use equation 3.
Then v∇3 =< .5554, 1.263, 0.8589 >
Now to compute v∇4 we will take constant 1 =< 1, 0, 0 >
Then
v∇4 = v∇3 − < 1, 0, 0 >=< −.4446, 1.263, .08589 >
So our Results are.
f (x) = −.4446,
0
f (x) = 1.263,
1
 2

00
f (x) = .08589,
Reverse Mode:- In reverse mode we use the same decompostion but in every
step we calculate the differentiation of each intermediate variable.(Confuse!!!...Lets
see)
In our decompostion first equation was.
v∇1 = x (right!!)
Now differentiate it. We will get

0
(0.6) v∇1 = 1.
Second is v∇2 = Sin(v∇1 ) then we will get after differentiation
0 0
v∇2 = v∇1 Cos(v∇1 ) ⇔

0 0
(0.7) v∇1 Cos(v∇1 ) − v∇2 = 0
Now
v∇3 = v∇1 v∇2
0 0 0
v∇3 = v∇1 v∇2 + v∇1 v∇2

0 0 0
(0.8) v∇1 v∇2 + v∇1 v∇2 − v∇3 = 0
Now
v∇4 = v∇3 − 1.

0 0
(0.9) v∇3 − v∇4 = 0
From equation 6,7 ,8 and 9, we can make a system of equation and write that in
matrix form and can solve that.
like
  0   
1 0 0 0 v∇1 1
 Cos(v∇1 ) −1 0 0   0   0 
   v∇2
0
= 
 v∇2 v∇1 −1 0   v∇3   0 
0
0 0 1 −1 v∇4 0
By putting the values inside we can get the solution.
 0   
v∇1 1.0000
0
 v   .7071 
 ∇2
 v 0  =  1.2625 
  
∇3
0
v∇4 1.2625
0 0
So f (x = π/4) = v∇4 = 1.2625.And by magic its also equal to Forward Method.
Enjoy!!