Derivation of the Normal Distribution: Lecture VI

Charles B. Moss
September 2, 2010
Charles B. Moss () Derivation of the Normal Distribution: Lecture VI September 2, 2010 1 / 16
Outline
1 Derivation of the Normal Distribution Function
2 Expected Values
Charles B. Moss () Derivation of the Normal Distribution: Lecture VI September 2, 2010 2 / 16
Derivation of the Normal Distribution Function
Derivation of the Normal Distribution Function
The order of proof of the normal distribution function is to start with
the standard normal:
f (x) =
1

2
e

x
2
2
(1)
First, we need to demonstrate that the distribution function does
integrate to one over the entire sample space, which is to . This
is typically accomplished by proving the constant.
Let us start by assuming that
I =
_

y
2
2
dy (2)
Squaring this expression yields
I
2
=
_

y
2
2
dy
_

x
2
2
dx
_

y
2
+x
2
(3)
Charles B. Moss () Derivation of the Normal Distribution: Lecture VI September 2, 2010 3 / 16
Derivation of the Normal Distribution Function
The trick to this integration is changing the variables into a polar
form.
Polar Integration: The notion of polar integration is basically one of a
change in variables. Specically, some integrals may be ill-posed in the
traditional Cartesian plane, but easily solved in a polar space.
By polar space, any point y = f (x) can be written in a trigonometric
form
r =
_
x
2
+ y
2
= tan
1
(x/y)
y = r cos ()
x = r sin ()
(4)
As an example take f (x) = 15 + x 1/2x
2
. Some of the results for
this function are presented in Table 1
Charles B. Moss () Derivation of the Normal Distribution: Lecture VI September 2, 2010 4 / 16
Derivation of the Normal Distribution Function
Table: Results for Polar Transform
x f (x) r
-5.00 -2.5000 5.5902 0.4636
-4.00 3.0000 5.0000 -0.6435
-3.00 7.5000 8.0777 -1.1903
-2.00 11.0000 11.1803 -1.3909
-1.00 13.5000 13.5370 -1.4969
-0.50 14.3750 14.3837 -1.5360
-0.25 14.7188 14.7209 -1.5538
0.00 15.0000 15.0000
0.25 15.2188 15.2209 1.5544
0.50 15.3750 15.3831 1.5383
1.00 15.5000 15.5322 1.5064
2.00 15.0000 15.1327 1.4382
3.00 13.5000 13.8293 1.3521
4.00 11.0000 11.7047 1.2220
5.00 7.5000 9.0139 0.9828
Charles B. Moss () Derivation of the Normal Distribution: Lecture VI September 2, 2010 5 / 16
Derivation of the Normal Distribution Function
Adding a little more dention and plotting in Polar Space
Figure: Polar Map
Charles B. Moss () Derivation of the Normal Distribution: Lecture VI September 2, 2010 6 / 16
Derivation of the Normal Distribution Function
The workhorse in this proof is Greenes theorem. We know from
univariate calculus that
_
b
a
df (t)
dt
dt = f (b) f (a) (5)
In multivariate space, this becomes
_ _
S
(N
x
M
y
) dxdy =
_

Mdx +
_

Ndy (6)
To see this note that
_ _
S
(N
x
M
y
) dxdy =
_ _
S
N
x
dxdy
_ _
S
M
y
dxdy
=
_

Ndy +
_

Mdx
(7)
Charles B. Moss () Derivation of the Normal Distribution: Lecture VI September 2, 2010 7 / 16
Derivation of the Normal Distribution Function
e. Picture the general form
f (x) = exp
_

x
2
2
_
(8)
Charles B. Moss () Derivation of the Normal Distribution: Lecture VI September 2, 2010 8 / 16
Derivation of the Normal Distribution Function
Figure: Normal Distribution Function
Charles B. Moss () Derivation of the Normal Distribution: Lecture VI September 2, 2010 9 / 16
Derivation of the Normal Distribution Function
The Polar Form of this expression can be depicted as
Figure: Polar Form of the Normal Disturtion
Charles B. Moss () Derivation of the Normal Distribution: Lecture VI September 2, 2010 10 / 16
Derivation of the Normal Distribution Function
Thus, letting
M (x, y) =
_
y
(x)
f (x, y) dt, N (x, y) = 0
_ _
S
fdxdy =
_ _
S
(N
x
M
y
) dxdy =
4

i =1
Mdx
_ _
S
fdxdy =
_ _
s
f (r ) drd =
_
b
c
_
_
b
a
f (r cos () , r sin ()) rdr
_
d
(9)
Charles B. Moss () Derivation of the Normal Distribution: Lecture VI September 2, 2010 11 / 16
Derivation of the Normal Distribution Function
By substituting y = cos () and x = r sin () we derive
y
2
+ x
2
= r
2
cos
2
() + r
2
sin
2
()
= r
2
_
cos
2
() + sin
2
()

= r
2
(10)
Dening the derivatives of the change in variables
dydx = rdrd (11)
Folding these two results together we get
I
2
=
_
2
0
_

0
re
r
2
2
drd =
_
2
0
d = 2 (12)
Charles B. Moss () Derivation of the Normal Distribution: Lecture VI September 2, 2010 12 / 16
Derivation of the Normal Distribution Function
Taking the square root of each side yields
I =

2 (13)
Thus, we know that
_

2
e

y
2
2
dy = 1 (14)
Charles B. Moss () Derivation of the Normal Distribution: Lecture VI September 2, 2010 13 / 16
Derivation of the Normal Distribution Function
The expression above is the expression for the standard normal. A
more general form of the normal distribution function can be derived
by dening a transformation function. Dening
y = a + bx
x =
y a
b
(15)
By the change in variable technique, we have
f (x) =
1

2
e

(y a)
2
2b
2

1
b

(16)
Charles B. Moss () Derivation of the Normal Distribution: Lecture VI September 2, 2010 14 / 16
Expected Values
Expected Values
Denition 2.2.1. The expected value or mean of a random variable
g (X) denoted E[g (x)] is
E[g (X)] =
_ _

g (x) f (x) dx if x iscontinuous

xX
g (x) f
x
if x isdiscrete
(17)
The most general form used in this denition allows for taking the
expectation of the function g (X). Strictly speaking, the mean of the
distribution is found where g (X) = X , or
E[x] =
_

xf (x) dx (18)
For example, we take the exponential distribution
E[x] =
_

0
1

xe

dx
=
_
xe

0
+
_

0
e
x

dx
_

(19)
Charles B. Moss () Derivation of the Normal Distribution: Lecture VI September 2, 2010 15 / 16
Expected Values
Theorem 2.2.1. Let X be a random variable and let a, b, and c be
constants. Then for any functions g
1
(X) and g
2
(X) whose
expectations exist:
E[ag
1
(X) + bg
2
(X)] = aE[g
1
(X)] + bE[g
2
(X)] + c.
If g
1
(X) 0 for all X, then E[g
1
(X)] 0.
If g
1
(X) g
2
(X) for all X, then E[g
1
(X)] E[g
2
(X)].
If a g
1
(X) b for all X, then a E[g
1
(X)] b.
Charles B. Moss () Derivation of the Normal Distribution: Lecture VI September 2, 2010 16 / 16