Introduction to Econometrics

Arnaud Maurel1
1 Department

of Economics
Duke University

Spring 2013

Econ 208D Handout 2 (Duke)

Probability Review

Spring 2013

1 / 28

Probability
Lets begin by introducing the notions of probability, randomness, and
random variables.
The use of probability to measure uncertainty and variability began
hundreds of years ago with the study of gambling.
Loosely speaking, a probability is the chance that an outcome or a set
of several outcomes (event) will happen.
The probability of an outcome (or event) is the proportion of the time
it occurs in the long run - this is called the frequentist interpretation
of probability. E.g.: coin tossed many times (say 1,000) will come up
heads and tails close to equally often.
Notations: for an event A, the number Pr(A) indicates the probability
that A will occur. We also denote by the set of all possible
outcomes (sample space).

Econ 208D Handout 2 (Duke)

Probability Review

Spring 2013

2 / 28

Probability(Contd)

In order to satisfy the mathematical definition of probability, Pr(A)

must satisfy certain conditions.
For every event A in the sample space , Pr(A) 0.
The probability that some event in occurs is 1, i.e. Pr() = 1.
If A and B are mutually exclusive, then Pr(A or B ) = Pr(A) + Pr(B ).

From these conditions, one can derive a number of properties like:

0 Pr(A) 1
and
Pr(not A) = 1 Pr(A)

Econ 208D Handout 2 (Duke)

Probability Review

Spring 2013

3 / 28

Random variables and distributions

A random variable is a real-valued function, that associates each

outcome in with a real number. E.g. (coin toss): function X which
returns 1 if head, 0 if tail.
We can calculate the probability that the variable takes on certain
values. Pr (X = 1) = Pr (X = 0) = 12 .
The distribution of the random variable describes the probability of
each possible outcome. These probabilities sum to 1.

Econ 208D Handout 2 (Duke)

Probability Review

Spring 2013

4 / 28

Random variables and distributions (Contd)

There are two major classes of random variables and distributions:

discrete and continuous.
A discrete random variable takes on only a discrete set of values.
E.g.: coin toss example; Number of phone calls you will receive today.

A continuous random variable takes on a continuum of values.

E.g.: amount of time you will spend on the phone.

Econ 208D Handout 2 (Duke)

Probability Review

Spring 2013

5 / 28

Discrete random variables

We characterize a discrete random variable X with a probability

function (pf).
A pf returns the probability for each possible discrete outcome.
The pf of a discrete random variable is defined as the function f such
that for every real number x,
f (x ) = Pr(X = x )
where X represents a random variable and x represents a realization
of that random variable.
The following slide contains a specific example. Note that the same
information is displayed in the table and graph.

Econ 208D Handout 2 (Duke)

Probability Review

Spring 2013

6 / 28

Probability function

Econ 208D Handout 2 (Duke)

Probability Review

Spring 2013

7 / 28

Cumulative distribution function (CDF)

Another way of characterizing the distribution of a random variable is

with a cumulative distribution function.
The cdf. gives the probability that a random variable is less than or
equal to a specific value (found by adding up the probabilities of lower
outcomes)
F (x ) = Pr(X x )

Econ 208D Handout 2 (Duke)

Probability Review

Spring 2013

8 / 28

Cumulative distribution function (CDF) (Contd)

The cumulative distribution is sometimes referred to as the
distribution function or cumulative risk profile.
x
Pr(X x )

Econ 208D Handout 2 (Duke)

0
0.8

1
0.9

2
0.96

Probability Review

3
0.99

4
1.0

Spring 2013

9 / 28

Continuous random variables

A random variable Y that can take on any real value within some
range is a continuous random variable.
E.g.: Time, temperature, height...

For continuous random variables, the probability of a particular value

(y = 2.3453345534332...) occurring is equal to zero
Pr(Y = y ) = 0
Interpretation: the outcome Y = 2.3453345534332... will almost
surely not happen. Note that this does not mean impossible!
In the continuous case, we typically speak of the probability that a
random variable is in an interval
Pr(a Y b )
Econ 208D Handout 2 (Duke)

Probability Review

Spring 2013

10 / 28

Probability density function (pdf)

The probabilities associated with a continuous random variable Y are
determined by the pdf. of Y , denoted by f (.). The pdf. summarizes
the distribution of Y , and has the following properties:
1
2

f (y ) 0, for all y .
The probability that the uncertain quantity Y will fall in the interval
(a, b ) is equal to the area under f (y ) between a and b :
P (a < Y < b ) =

Zb

f (y )dy .

a
3

The total area under the entire curve of f (y ) is equal to 1

Z

f (y )dy = 1

Econ 208D Handout 2 (Duke)

Probability Review

Spring 2013

11 / 28

Probability Density Function (pdf): graphical interpretation

Econ 208D Handout 2 (Duke)

Probability Review

Spring 2013

12 / 28

Cdf. of continuous random variables

The definition of a cdf. for a continuous random variable is the same

as that of a discrete random variable:
F (y ) = Pr(Y y )
With a continuous random variable, the cdf is a continuous function
over the entire real line, so we can write down a simple formula
F (y ) = Pr(Y y ) =

Zy

f (t )dt

Econ 208D Handout 2 (Duke)

Probability Review

Spring 2013

13 / 28

Cdf. of continuous random variables (Contd)

Furthermore, it follows that the pdf. can be calculated as

F 0 (y ) =

dF (y )
= f (y )
dy

We can easily see that:

Pr(Y > y ) = 1 F (y )
and
Pr(y1 < Y y2 ) = F (y2 ) F (y1 )

Econ 208D Handout 2 (Duke)

Probability Review

Spring 2013

14 / 28

Main points from last time

Random Variable is a variable that can take on various numeric values

with particular probabilities.

Random Variables can be either discrete or continuous.

pf (pdf) describes the probability with which a discrete (continuous)

random variable can take on each value.

cdf describes the probability that a RV (either discrete or continuous)

is less than a particular value.

Key properties of Probability, pdf and cdf.

Econ 208D Handout 2 (Duke)

Probability Review

Spring 2013

15 / 28

The cdf. provides an easy way to calculate the probability for any
interval.

Pr(15 < CT 20) = Pr(CT 20) Pr(CT 15) =

F (20) F (15) = 0.78 0.20 = .58
Pr(CT > 20) = 1 F (20) = 1 0.78 = .22
Econ 208D Handout 2 (Duke)

Probability Review

Spring 2013

16 / 28

Measures of central tendency for distributions

Mode
The mode is a value that occurs with the greatest probability (or pdf.
for a continuous random variable).
Median
The median is a value such that the probability of the random
variable being less than or equal to that value is at least 0.5 and the
probability of the random variable being greater than or equal to that
value is at least 0.5.
Example: What is the median age of students in this class?
Case 1
Age
19 20 21 22
Prop 0.4 0.2 0.2 0.2
Case 2
Age
Prop
Econ 208D Handout 2 (Duke)

19
0.4

20
0.1

Probability Review

21
0.3

22
0.2
Spring 2013

16 / 28

Measures of central tendency for distributions - Mean

(expected value)
The mean or expected value of a random variable X is the weighted
average of all its possible outcomes, where the weights are the
probabilities or pdfs.
It is unique.
For a discrete random variable X ,
n

E (X ) = x1 Pr(x1 ) + ... + xn Pr(xn ) =

xi Pr(X

= xi )

i =1

Example: expected value of throwing a dice:

E (X ) = 1 16 + 2 16 + 3 16 + 4 16 + 5 16 + 6 61 6 =
3.5
Econ 208D Handout 2 (Duke)

Probability Review

1
6

(1 + 2 + 3 + 4 + 5 + 6) =

Spring 2013

17 / 28

Measures of central tendency for distributions - Mean

(expected value) (Contd)

For a continuous random variable X with probability density function

f (x ), the mean of X is defined as
E (X ) =

xf (x )dx

The mean is the most widely used measure but a drawback of the
mean (relative to the median) is its sensitivity to outliers.
Relationship between mode, median and mean? Depends on the
symmetry of the distribution. If the probability function (or pdf.) is
symmetric, then mode=median=mean.

Econ 208D Handout 2 (Duke)

Probability Review

Spring 2013

18 / 28

Income Distribution

Econ 208D Handout 2 (Duke)

Probability Review

Spring 2013

19 / 28

Wealth Distribution

Econ 208D Handout 2 (Duke)

Probability Review

Spring 2013

20 / 28

Example: continuous distribution

f (x ) = 2x for 0 < x < 1 (and 0 otherwise).

R
R1
R1
1

Then E (X ) =
xf (x ) dx = x (2x ) dx = 2x 2 dx = 32 x 3 0 =

2
3

Question: what is F (x )? 2 cases: x 1:

Rx
Rx
 x
F (x ) =
f (t ) dt = 2tdt = t 2 0 = x 2 Otherwise, if x > 1:

F (x ) = 1.
What about the median?
F ( 12 ) = 12

Econ 208D Handout 2 (Duke)

Probability Review

Spring 2013

21 / 28

Expected value of a function of a random variable

We know that if X is a random variable with pdf f (x ) we can
calculate E (X ) as either
n

E (X ) =

xi Pr (xi ) or E (X ) =

i =1

xf (x )dx

But what if we want to calculate E (X 2 ) or E (ln(X ))?

In general, what is the expected value of a function g () of X ?
It can also be shown that
n

E (g (X )) =

g (xi )Pr (xi ) or E (g (X )) =

i =1

g (x )f (x )dx

For example, for a continuous distribution

E (X 2 ) =
Econ 208D Handout 2 (Duke)

x 2 f (x )dx

Probability Review

Spring 2013

22 / 28

Measures of dispersion for distributions - variance

The variance of a random variable measures the spread or dispersion
of the variable around its mean X
i
h
Var (X ) = E (X X )2
In the discrete case, this is simply the weighted average of the
squared deviations of X from its mean
n

Var (X ) =

[xi E (X )]2 Pr(xi )

i =1

For a continuous random variable X with probability density function

f (x ), the variance of X is
Var (X ) =
Econ 208D Handout 2 (Duke)

[x E (X )]2 f (x )dx.

Probability Review

Spring 2013

23 / 28

Variance & moments

Another (equivalent) formula that can be used in either case is:

Var (X ) = E (X 2 ) (E (X ))2
where the second moment E (X 2 ) is given by
E (X 2 ) = (x1 )2 Pr(x1 ) + (x2 )2 Pr(x2 ) + ... + (xn )2 Pr(xn )
or
2

E (X ) =

x 2 f (x )dx

Other (higher order) moments are defined similarly.

Econ 208D Handout 2 (Duke)

Probability Review

Spring 2013

24 / 28

Standard deviation

The variance of X is denoted Var (X ) or X2 .

The standard deviation of X , denoted X , is the square root of
Var (X ).
A large standard deviation (and variance) means that the probability
distribution is quite spread out: a large difference between the
outcome and the expected value is anticipated.

Econ 208D Handout 2 (Duke)

Probability Review

Spring 2013

25 / 28

Example
x
1
2
3
4
5
6

Pr(x )

x Pr(x )

(x )2 Pr(x )

1
6
1
6
1
6
1
6
1
6
1
6

1
6
1
3
1
2
2
3
5
6

1
6
2
3
3
2
8
3
25
6

E (X ) = 3.5

E (X 2 ) = 15 61

E (X ) = ni=1 xi Pr(xi ) = 3.5

E (X 2 ) = ni=1 (xi )2 Pr(xi ) = 15 61
Var (X ) = E (X 2 ) (E (X ))2 = 15 16 3.52 = 2.92

X = 2.92 = 1.71
Econ 208D Handout 2 (Duke)

Probability Review

Spring 2013

26 / 28

Example

f (x ) = 2x for 0 < x < 1 (and 0 otherwise)

We already know E (X ) = 23 . What is Var (X )?:
R
R1

2
Var (X ) =
x 23 2xdx =
[x E (X )]2 f (x )dx =

R1

2x 3

8 2
3x

8
9x

dx =

1 4
2x

8 3
9x

+ 49 x 2

1
0

1
2

98 +

4
9

1
18

or (using the other formula)

Var (X ) = E (X 2 ) (E (X ))2 =

R1
0

R1
0

2x 3 dx

4
9

1

Econ 208D Handout 2 (Duke)

2x


4 1
0

4
9

1
2

x 2 (2x ) dx ( 23 )2

4
9

1
18

Probability Review

Spring 2013

27 / 28

Expected value and variance of a linear function

If Y = a + bX , then E (Y ) = a + bE (X )
Example: Suppose E (X ) = 5, Z = 3X + 15
E (Z ) = 3 E (X ) + 15 = 3 5 + 15 = 0
If Y = a + bX , then Var (Y ) = b 2 Var (X )
Example: Var (X ) = 5, Z = 3X + 15
Var (Z ) = 9 Var (X ) = 9 5 = 45
These (very useful!) properties can be easily proven using the definitions
of expectation and variance.

Econ 208D Handout 2 (Duke)

Probability Review

Spring 2013

28 / 28