Introduction to Computational Finance and

Financial Econometrics
Probability Theory Review: Part 1
Eric Zivot
June 26, 2012
1 Univariate Random Variables
Defnition: A random variable (rv) A is a variable that can take on a given
set of values, called the sample space S
A
, where the likelihood of the values
in S
A
is determined by the variables probability distribution function (pdf).
Examples
A = price of microsoft stock next month. S
A
= {R : 0 < A A}
A = simple return on a one month investment. S
A
= {R : 1
A < A}
A = 1 if stock price goes up; A = 0 if stock price goes down. S
A
=
{0, 1}
1.1 Discrete Random Variables
Denition: A discrete rv A is one that can take on a nite number of a
dierent values a
1
, , a
a
Denition: The pdf of a discrete rv A, j(a), is a function such that j(a) =
Pr(A = a). The pdf must satisfy
1. j(a) > 0 for all a S
A
; j(a) = 0 for all a S
A
2.
P
aS
A
j(a) = 1
3. j(a) 6 1 for all a S
A
State of Economy S
A
= Sample Space j(a) = Pr(A = a)
Depression -0.30 0.05
Recession 0.0 0.20
Normal 0.10 0.50
Mild Boom 0.20 0.20
Major Boom 0.50 0.05
Table 1: Discrete Distribution for Annual Return
Example: Probability Distribution for Annual Return on Microsoft
Example: Bernouli Distribution
Consider two mutually exclusive events generically called success and fail-
ure.
Let A = 1 if success occurs and let A = 0 if failure occurs.
Let Pr(A = 1) = , where 0 < < 1, denote the probability of success.
Then Pr(A = 0) = 1 is the probability of failure. A mathematical model
describing this distribution is
j(a) = Pr(A = a) =
a
(1 )
1a
, a = 0, 1.
When a = 0, j(0) =
0
(1 )
10
= 1 and when a = 1, j(1) =

1
(1 )
11
= .
1.2 Continuous Random Variables
Denition: A continuous rv A is one that can take on any real value
Denition: The pdf of a continuous rv A is a nonnegative function )(a)
such that for any interval on the real line
Pr(A ) =
Z

)(a)oa
Pr(A ) = "Area under probability curve over the interval ".
The pdf )(a) must satisfy
1. )(a) > 0;
R

)(a)oa = 1
Example: Uniform distribution over [o, b]
Let A v l [o, b], where "v" means "is distributed as". Then
)(a) =
(
1
bo
0
for o a b
otherwise
Properties:
)(a) 0, provided b o, and
Z

)(a)oa =
Z
b
o
1
b o
oa =
1
b o
Z
b
o
oa
=
1
b o
[a]
b
o
=
b o
b o
= 1
1.3 The Cumulative Distribution Function (CDF)
Denition The CDF, 1, of a rv A is 1(a) = Pr(A a) and
If a
1
< a
2
, then 1(a
1
) 1(a
2
)
1() = 0 and 1() = 1
Pr(A a) = 1 1(a)
Pr(a
1
< A a
2
) = 1(a
2
) 1(a
1
)

o
oa
1(a) = )(a) if A is a continuous rv.
Example: Uniform distribution over [0, 1]
A v l [0, 1]
)(a) =
(
1
10
= 1
0
for 0 a 1
otherwise
Then
1(a) = Pr(A a) =
Z
a
0
o:
= [:]
a
0
= a
and, for example,
Pr(0 A 0.5) = 1(0.5) 1(0)
= 0.5 0 = 0.5
Note
o
oa
1(a) = 1 = )(a)
Remark:
For a continuous rv
Pr(A a) = Pr(A < a)
Pr(A = a) = 0
1.4 Quantiles of a Distribution
A is a rv with continuous CDF 1
A
(a) = Pr(A a)
Denition: The c 100% quantile of 1
A
for c [0, 1] is the value q
c
such
that
1
A
(q
c
) = Pr(A q
c
) = c
The area under the probability curve to the left of q
c
is c. If the inverse CDF
1
1
A
exists then
q
c
= 1
1
A
(c)
Note: 1
1
A
is sometimes called the quantile function.
Example:
1% quantile = q
.01
5% quantile = q
.05
50% quantile = q
.5
= median
Example: Quantile function of uniform distn on [0,1]
1
A
(a) = a q
c
= c
q
.01
= 0.01
q
.5
= 0.5
1.5 The Standard Normal Distribution
Let A be a rv such that A v .(0, 1). Then
)(a) = (a) =
1

2
exp

1
2
a
2

, a
(a) = Pr(A a) =
Z
a

(:)o:
Shape Characteristics
Centered at zero
Symmetric about zero (same shape to left and right of zero)
Pr(1 a 1) = (1) (1) = 0.67
Pr(2 a 2) = (2) (2) = 0.95
Pr(3 a 3) = (3) (3) = 0.99
Finding Areas under the Normal Curve

2
c

1
2
a
2
oa = 1, via change of variables formula in calculus
Pr(o < A < b) =
R
b
o
1

2
c

1
2
a
2
oa = (b) (o), cannot be computed
analytically!
Special numerical algorithms are used to calculate (:)
Excel functions
1. NORMSDIST computes Pr(A :) = (:) or j(:) = (:)
2. NORMSINV computes the quantile :
c
=
1
(c)
R functions
1. pnorm computes Pr(A :) = (:)
2. qnorm computes the quantile :
c
=
1
(c)
3. dnorm computes the density (:)
Some Tricks for Computing Area under Normal Curve
.(0, 1) is symmeric about 0; total area =1
Pr(A :) = 1 Pr(A :)
Pr(A :) = Pr(A :)
Pr(A 0) = Pr(A 0) = 0.5
Example In Excel use
Pr(1 A 2) = Pr(A 2) Pr(A 1)
= NORMSDIST(2) NORMSDIST(-1)
= 0.97725 0.15866 = 0.81860
In R use
pnorm(2) pnorm(-1) = 0.81860
The 1%, 2.5%, 5% quantiles are
Excel::
.01
=
1
(0.01) = NORMSINV(0.01) = 2.33
R : qnorm(0.01) = 2.33
Excel::
.025
=
1
(0.025) = NORMSINV(0.025) = 1.96
R : qnorm(0.025) = 1.96
Excel::
.05
=
1
(.05) = NORMSINV(.05) = 1.645
R : qnorm(0.05) = 1.645
1.6 Shape Characteristics of pdfs
Expected Value or Mean - Center of Mass
Variance and Standard Deviation - Spread about mean
Skewness - Symmetry about mean
Kurtosis - Tail thickness
Expected Value - Discrete rv
1[A] = j
A
=
X
aS
A
a j(a)
=
X
aS
A
a Pr(A = a)
1[A] = probability weighted average of possible values of A
Expected Value - Continuous rv
1[A] = j
A
=
Z

a )(a)oa
Note: In continuous case,
P
aS
A
becames
R

Expected value of discrete random variable

Using the discrete distribution for the return on Microsoft stock in Table 1, the
expected return is
1[A] = (0.3) (0.05) + (0.0) (0.20) + (0.1) (0.5)
+ (0.2) (0.2) + (0.5) (0.05)
= 0.10.
Example: A v l[1, 2]
1[A] =
Z
2
1
aoa =
"
a
2
2
#
2
1
=
1
2
[4 1] =
3
2
Example: A v .(0, 1)
j
A
= 1[A] =
Z

a
1

2
c

1
2
a
2
oa = 0
Expectation of a Function of A
Denition: Let j(A) be some function of the rv A. Then
1[j(A)] =
X
aS
A
j(a) j(a) Discrete case
1[j(A)] =
Z

j(a) )(a)oa Continuous case

Variance and Standard Deviation
j(A) = (A 1[A])
2
= (A j
A
)
2
Var(A) = o
2
A
= 1[(A j
A
)
2
] = 1[A
2
] j
2
A
SD(A) = o
A
=
q
Var(A)
Note: Var(A) is in squared units of A, and SD(A) is in the same units as A.
Therefore, SD(A) is easier to interpret.
Computation of Var(A) and SD(A)
o
2
A
= 1[(A j
A
)
2
]
=
X
aS
A
(a j
A
)
2
j(a) if A is a discrete rv
=
Z

(a j
A
)
2
)(a)oa if A is a continuous rv
o
A
=
q
o
2
A
Remark: For bell-shaped data, o
A
measures the size of the typical deviation
from the mean value j
A
.
Example: Variance and standard deviation for a discrete random variable
Using the discrete distribution for the return on Microsoft stock in Table 1 and
the result that j
A
= 0.1, we have
Var(A) = (0.3 0.1)
2
(0.05) + (0.0 0.1)
2
(0.20)
+ (0.1 0.1)
2
(0.5) + (0.2 0.1)
2
(0.2)
+ (0.5 0.1)
2
(0.05)
= 0.020
SD(A) = o
A
=

0.020 = 0.141.
Given that the distribution is fairly bell-shaped we can say that typical values
deviate from the mean value of 0.10 by about 0.141
j o = 0.10 0.141 = [0.041, 0.241]
Example: A v .(0, 1).
j
A
=
Z

a
1

2
c

1
2
a
2
oa = 0
o
2
A
=
Z

(a 0)
2

2
c

1
2
a
2
oa = 1
o
A
=

1 = 1
size of typical deviation from j
A
= 0 is o
A
= 1
The General Normal Distribution
A v .(j
A
, o
2
A
)
)(a) =
1
q
2o
2
A
exp

1
2

a j
A
o
A
!
2

oa, a
1[A] = j
A
= mean value
Var(A) = o
2
A
= variance
SD(A) = o
A
= standard deviation
Shape Characteristics
Centered at j
A
Symmetric about j
A
Pr(j
A
o
A
A j
A
+o
A
) = 0.67
Pr(j
A
2 o
A
A j
A
+ 2 o
A
) = 0.95
Pr(j
A
3 o
A
A j
A
+ 3 o
A
) = 0.99
Quantiles of the general normal distribution:
q
c
= j
A
+o
A

1
(c) = j
A
+o
A
:
c
Remarks:
A v .(0, 1) : Standard Normal =j
A
= 0 and o
2
A
= 1
The pdf of the general Normal is completely determined by values of j
A
and o
2
A
Finding Areas under General Normal Curve
Excel Functions
NORMDIST(a, j
A
, o
A
,cumulative). If cumulative = true: Pr(A
a) is computed; If cumulative = false, )(a) =
1
q
2o
2
A
c

1
2
(
aj
A
o
A
)
2
is
computed
NORMINV(c, j
a
, o
a
) computes q
c
= j
A
+o
A
:
c
R Runctions
simulate data: rnorm(n, mean, sd)
compute CDF: pnorm(q, mean, sd)
compute quantiles: qnorm(p, mean, sd)
compute density: dnorm(x, mean, sd)
Standard Deviation as a Measure of Risk
1

= monthly return on asset A

1
1
= monthly return on assetB
1

v .(j

, o
2

), 1
1
v .(j
1
, o
2
1
)
where
j

= 1[1

] = expected monthly return on asset A

o

= SD(1

)
= std. deviation of monthly return on asset A
Typically, if
j

j
1
then
o

o
1
Example: Why the normal distribution may not be appropriate for simple
returns
1
t
=
1
t
1
t1
1
t1
= simple return
Assume 1
t
.(0.05, (0.50)
2
)
Note: 1
t
0 = 1
t
1. However, based on the assumed normal
distribution
Pr(1
t
< 1) = NORMDIST(-1,0.05,0.50,TRUE) = 0.018
This implies that there is a 1.8% chance that the asset price will be negative.
This is why the normal distribution may not be appropriate for simple returns.
Example: The normal distribution is more appropriate for cc returns
v
t
= ln(1 +1
t
) = cc return
1
t
= c
v
t
1 = simple return
Assume v
t
.(0.05, (0.50)
2
)
Unlike 1
t
, v
t
can take on values less than 1. For example,
v
t
= 2 = 1
t
= c
2
1 = 0.865
Pr(v
t
< 2) = Pr(1
t
< 0.865)
= NORMDIST(-2,0.05,0.50,TRUE) = 0.00002
The Log-Normal Distribution
A .(j
A
, o
2
A
), < A <
Y = exp(A) lognormal(j
A
, o
2
A
), 0 < Y <
1[Y ] = j
Y
= exp(j
A
+o
2
A
2)
Var(Y ) = o
2
Y
= exp(2j
A
+o
2
A
)(exp(o
2
A
) 1)
Example: log-normal distribution for simple returns
v
t
.(0.05, (0.50)
2
)
1 +1
t
lognormal(0.05, (0.50)
2
)
j
1+1
= exp(0.05 + (0.5)
2
2) = 1.191
o
2
1+1
= exp(2(0.05) + (0.5)
2
)(exp(0.5
2
) 1) = 0.563
R Runctions
simulate data: rlnorm(n, mean, sd)
compute CDF: plnorm(q, mean, sd)
compute quantiles: qlnorm(p, mean, sd)
compute density: dlnorm(y, mean, sd)
Skewness - Measure of symmetry
j(A) = ((A j
A
)o
A
)
3
Skew(A) = 1

A j
A
o
A
!
3

=
X
aS
A

a j
A
o
A
!
3
j(a) if A is discrete
=
Z

a j
A
o
A
!
3
)(a)oa if A is continuous
Intuition
If A has a symmetric distribution about j
A
then Skew(A) = 0
Skew(A) 0 = pdf has long right tail, and median < mean
Skew(A) < 0 = pdf has long left tail, and median mean
Example: Using the discrete distribution for the return on Microsoft stock in
Table 1, the results that j
A
= 0.1 and o
A
= 0.141, we have
skew(A) = [(0.3 0.1)
3
(0.05) + (0.0 0.1)
3
(0.20)
+ (0.1 0.1)
3
(0.5) + (0.2 0.1)
3
(0.2)
+ (0.5 0.1)
3
(0.05)](0.141)
3
= 0.0
Example: A v .(j
A
, o
2
A
). Then
Skew(A) =
Z

a j
A
o
A
!
3
1
q
2o
2
A
exp

1
2
(
aj
A
o
A
)
2
!
oa = 0
Example: Y v lognormal(j
A
, o
2
A
). Then
Skew(Y ) =

exp(o
2
A
) + 2

q
exp(o
2
A
) 1 0
Kurtosis - Measure of tail thickness
j(A) = ((A j
A
)o
A
)
4
Kurt(A) = 1

A j
A
o
A
!
4

=
X
aS
A

a j
A
o
A
!
4
j(a) if A is discrete
=
Z

a j
A
o
A
!
4
)(a)oa if A is continuous
Intuition
Values of a far from j
A
get blown up resulting in large values of kurtosis
Two extreme cases: fat tails (large kurtosis); thin tails (small kurtosis)
Example: Kurtosis for a discrete random variable
Using the discrete distribution for the return on Microsoft stock in Table 1, the
results that j
A
= 0.1 and o
A
= 0.141, we have
Kurt(A) = [(0.3 0.1)
4
(0.05) + (0.0 0.1)
4
(0.20)
+ (0.1 0.1)
4
(0.5) + (0.2 0.1)
4
(0.2)
+ (0.5 0.1)
4
(0.05)](0.141)
4
= 6.5
Example: A v .(j
A
, o
2
A
)
Kurt(A) =
Z

a j
A
o
A
!
4
1
q
2o
2
A
c

1
2
(
aj
A
o
A
)
2
oa = 3
Denition: Excess kurtosis = Kurt(A) 3 = kurtosis value in excess of
kurtosis of normal distribution.
Excess kurtosis (A) 0 A has fatter tails than normal distribution
Excess kurtosis (A) < 0 A has thinner tails than normal distribution
The Students-t Distribution
A distribution similar to the standard normal distribution but with fatter tails,
and hence larger kurtosis, is the Students t distribution. If A has a Students
t distribution with degrees of freedom parameter , denoted A t

, then its
pdf has the form
)(a) =

+1
2

1 +
a
2

+1
2

, < a < , 0.
where (:) =
R

0
t
:1
c
t
ot denotes the gamma function.
It can be shown that
1[A] = 0, 1
var(A) =

2
, 2,
skew(A) = 0, 3,
kurt(A) =
6
4
+ 3, 4.
The parameter controls the scale and tail thickness of distribution. If is
close to four, then the kurtosis is large and the tails are thick. If < 4, then
kurt(A) = . As the Students t pdf approaches that of a standard
normal random variable and kurt(A) = 3.
R Runctions
simulate data: rt(n, df)
compute CDF: pt(q, df)
compute quantiles: qt(p, df)
compute density: dt(x, df)
Here df is the degrees of freedom parameter .
1.7 Linear Functions of a Random Variable
Let A be a discrete or continuous rv with j
A
= 1[A], and o
2
A
= Var(A).
Dene a new rv Y to be a linear function of A :
Y = j(A) = o A +b
o and b are known constants
Then
j
Y
= 1[Y ] = 1[o A +b]
= o 1[A] +b = o j
A
+b
o
2
Y
= Var(Y ) = Var(o A +b)
= o
2
Var(A)
= o
2
o
2
A
o
Y
= o o
A
Linear Function of a Normal rv
Let A v .(j
A
, o
2
A
) and dene Y = o A +b. Then
Y .(j
Y
, o
2
Y
)
with
j
Y
= o j
A
+b
o
2
Y
= o
2
o
2
A
Remarks
Proof of result relies on change-of-variables formula for determining pdf of
a function of a rv
Result may or may not hold for random variables whose distributions are
not normal
Example - Standardizing a Normal rv
Let A .(j
A
, o
2
A
). The standardized rv Z is created using
Z =
A j
A
o
A
=
1
o
A
A
j
A
o
A
= o A +b
o =
1
o
A
, b =
j
A
o
A
Properties of Z
1[Z] =
1
o
A
1[A]
j
A
o
A
=
1
o
A
j
A

j
A
o
A
= 0
Var(Z) =

1
o
A
!
2
Var(A)
=

1
o
A
!
2
o
2
A
= 1
Z .(0, 1)
1.8 Value at Risk: Introduction
Consider a $10, 000 investment in Microsoft for 1 month. Assume
1 = simple monthly return on Microsoft
1 .(0.05, (0.10)
2
), j
1
= 0.05, o
1
= 0.10
Goal: Calculate how much we can lose with a specied probability c
Questions:
1. What is the probability distribution of end of month wealth, W
1
= $10, 000
(1 +1)?
2. What is Pr(W
1
< $9, 000)?
3. What value of 1 produces W
1
= $9, 000?
4. What is the monthly value-at-risk (VaR) on the $10,000 investment with
5% probability? That is, how much can we lose if 1 q
.05
?
Answers:
1. W
1
= $10, 000 (1 + 1) is a linear function of 1, and 1 is a normally
distributed rv. Therefore, W
1
is normally distributed with
1[W
1
] = $10, 000 (1 +1[1])
= $10, 000 (1 + 0.05) = $10, 500,
Var(W
1
) = ($10, 000)
2
Var(1)
= ($10, 000)
2
(0.1)
2
= 1, 000, 000
W
1
.($10, 500, ($1, 000)
2
)
2. Using W
1
.($10, 500, ($1, 000)
2
)
Pr (W
1
< $9, 000)
= NORMDIST(9000,10500,1000) = 0.067
3. To nd 1 that produces W
1
= $9, 000 solve
1 =
$9, 000 $10, 000
$10, 000
= 0.10.
Notice that 0.10 is the 6.7% quantile of the distribution of 1 :
q
.067
= Pr(1 < 0.10) = 0.067
4. Use 1 .(0.05, (0.10)
2
) and solve for the the 5% quantile:
Pr(1 < q
1
.05
) = 0.05
q
1
.05
= NORMINV(0.05, 0.05,0.10) = 0.114.
If 1 = 11.4% the loss in investment value is at least
$10, 000 (0.114) = $1, 144
= 5% VaR
In general, the c 100% Value-at-Risk (VaR
c
) for an initial investment of
$W
0
is computed as
VaR
c
= $W
0
q
c
q
c
= c 100% quantile of simple return distn
Remark:
Because VaR represents a loss, it is often reported as a positive number. For
example, $1, 144 represents a loss of $1, 144. So the VaR is reported as
$1, 144.
VaR for Continuously Compounded Returns
v = ln(1 +1), cc monthly return
1 = c
v
1, simple monthly return
Assume
v .(j
v
, o
2
v
)
W
0
= initial investment
100 c% VaR Computation
Compute c quantile of Normal Distribution for v:
q
v
c
= j
v
+o
v
:
c
Convert c quantile for v into c quantile for 1:
q
1
c
= c
q
v
c
1
Compute 100 c% VaR using q
1
c
:
VaR
c
= $W
0
q
1
c
Example: Compute 5% VaR assuming
v
t
.(0.05, (0.10)
2
), W
0
= $10, 000
The 5% cc return quantile is
q
v
.05
= j
v
+o
v
:
.05
= 0.05 + (0.10)(1.645) = 0.114
The 5% simple return quantile is
q
1
.05
= c
q
v
.05
1 = c
.114
1 = 0.108
The 5% VaR based on a $10,000 initial investment is
VaR
.05
= $10, 000 (0.108) = $1, 077