Chapter 2

Probability
2.1 Basic Ideas
Definition: An experiment is a process that results in an outcome that cannot be
predicted in advance with certainty.

Examples:
rolling a die
tossing a coin
weighing the contents of a box of cereal.


Definition: The set of all possible outcomes of an experiment is called the sample
space for the experiment.

Examples:
For rolling a fair die, the sample space is {1, 2, 3, 4, 5, 6}.
For a coin toss, the sample space is {heads, tails}.
For weighing a cereal box, the sample space is (0, ), a more reasonable sample space is
(12, 20) for a 16 oz. box.

More Terminology
Definition: A subset of a sample space is called an
event.

A given event is said to have occurred if the outcome
of the experiment is one of the outcomes in the event.
For example, if a die comes up 2, the events {2, 4, 6}
and {1, 2, 3} have both occurred, along with every
other event that contains the outcome 2.

Combining Events
The union of two events A and B, denoted A B, is the
set of outcomes that belong either to A, to B, or to
both. In words, A B means A or B. So the event
A or B occurs whenever either A or B (or both)
occurs.

Example: Let A = {1, 2, 3} and B = {2, 3, 4}.
Then A B = {1, 2, 3, 4}
Intersections
The intersection of two events A and B, denoted
by A B, is the set of outcomes that belong to
A and to B. In words, A B means A and B.
Thus the event A and B occurs whenever
both A and B occur.

Example: Let A = {1, 2, 3} and B = {2, 3, 4}.
Then A B = {2, 3}
Complements
The complement of an event A, denoted A
c
, is
the set of outcomes that do not belong to A. In
words, A
c
means not A. Thus the event not
A occurs whenever A does not occur.

Example: Consider rolling a fair sided die. Let
A be the event: rolling a six = {6}.
Then A
c
= not rolling a six = {1, 2, 3, 4, 5}.

Venn diagrams illustrating various events: (a) union; (b) intersection; (c) complement
Mutually Exclusive Events
Definition: The events A and B are said to be mutually
exclusive if they have no outcomes in common.

More generally, a collection of events
is said to be mutually exclusive if no
two of them have any outcomes in common.

Sometimes mutually exclusive events are referred to as disjoint
events.


1 2
, ,...,
n
A A A
Example
When you flip a coin, you
cannot have the coin
come up heads and tails.

The following Venn
diagram illustrates
mutually exclusive
events:




Probabilities
Definition: Each event in the sample space has a
probability of occurring. Intuitively, the probability
is a quantitative measure of how likely the event is to
occur.

Given any experiment and any event A:
The expression P(A) denotes the probability that the
event A occurs.
P(A) is the proportion of times that the event A would
occur in the long run, if the experiment were to be
repeated over and over again.

Axioms of Probability
1. Let S be a sample space. Then P(S) = 1.

2. For any event A, .

3. If A and B are mutually exclusive events, then
. More generally, if
are mutually exclusive events, then




0 ( ) 1 P A s s
( ) ( ) ( ) P A B P A P B = +
1 2
, ,..... A A
1 2 1 2
( ....) ( ) ( ) ... P A A P A P A = + +
A Few Useful Things
For any event A,
P( ) = 1 P(A).

Let denote the empty set. Then
P( ) = 0.

If A is an event, and A = { }, then
P(A) = P(E
1
) + P(E
2
) +.+ P(E
n
).

Addition Rule (for when A and B are not mutually
exclusive):


C
A
C
C
1 2
, ,...,
n
E E E
( ) ( ) ( ) ( ) P A B P A P B P A B = +
Exemple
A geology student is asked to identify three
mineral samples (a garnet, a galena, and
pyroxene) from a tray full of various minerals.
The probability that she identifies the garnet is
0.10, the probability that she identifies the
galena is 0.25, and the probability that she
identifies the pyroxene is 0.45.
What is the probability that she identifies all
three minerals?
What is the probability that she identifies none of
those three minerals?
P(identify all 3 minerals) =
P(garnet) + P(galena) + P(pyroxene)
= 0.10+0.25+0.45
= 0.80
P(missing all 3 minerals) = 1 0.80 = 0.20
The Multiplication Rule
If A and B are two independent events, then
P(A B) = P(A)P(B).

This result can be extended to any number of events.
Example: Environmentalist Classifications
Environmentalist engineers classify U.S. consumers into five groups based on
consumers feelings about environmentalism:
1. Basic browns claim they dont have the knowledge to understand
environmental problems.
2. True-blue greens use biodegradable products.
3. Greenback greens support requiring new cars to run on alternative fuel.
4. Sprouts recycle newspapers regularly.
5. Grousers believe industry, not individuals, should solve environmental
problems.
The proportion of consumers in each group is shown in the below table (from
The Orange County (Calif.) Reporter, Aug. 7, 1990

Basic Browns (BB) .28
True-blue greens (TG) .11
Greenback greens (GG) .11
Sprouts (S) .26
Grousers (G) .24


a) List the simple events for the experiment
BB, TG, GG, S, and G
b) Assign reasonable probabilities to the simple events.
Reasonable probabilities would be probabilities that correspond to the proportions.
Thus the probabilities are:

Simple event Probability
Basic Browns (BB) .28
True-blue greens (TG) .11
Greenback greens (GG) .11
Sprouts (S) .26
Grousers (G) .24

c) Find the probability that the consumer is either a basic brown or a grouser.

P(BB or G) = .28 + .24 = .52

d) Find the probability that the consumer supports environmentalism in some fashion
(i.e., the consumer is a true-blue green, a greenback green, or a sprout.

P(TG, GG, or S) = .11 + .11 + .26 = .48
Probability of an oil gusher
An oil-drilling venture involves the drilling of
six wildcat oil wells in different parts of the
country. Suppose that each drilling will
produce either a dry well or oil gusher.
Assuming that the simple events for this
experiment are equally likely, find the
probability that at least one oil well will be
discovered.
Solution
A simple event for this experiment consists of the outcomes
(dry well or oil gusher) for the 6 drilled wells. Using the
multiplication rule, there are a total of 2x2x2x2x2x2=64
simple events. If we let D represent a dry well and G
represent an oil gusher, a simple event will be of the
form DDGDGG, where each position corresponds to one
of the six wells. If each simple event is equally likely,
then each simple event has a probability of 1/64.
Define event A as the event that at least 1 oil gusher will be
discovered. Of the 64 simple events, only 1 (DDDDDD)
is not in A. Thus,

P(A) = 1 P(A
c
) = 1 1/64 = 63/64 = .984

2.2 Counting Methods (optional)
The Fundamental Counting Principle:
Assume that k operations are to be performed. If there are n
1
ways to
perform the first operation, and if for each of these ways there are n
2
ways
to perform the second calculation, and so on. Then the total number of
ways to perform the sequence of k operations is n
1
n
2
n
k
.

A permutation is an ordering of a collection of objects. The number of
permutations of n objects is n!.

The number of permutations of k objects chosen from a group of n objects
is n!/(n k)!

The number of combinations of k objects chosen from a group of n objects
is n!/[(n k)!k!]

The number of ways to divide a group of n objects into groups of k
1
, , k
n

objects where k
1
+ + k
n
= n, is n!/(k
1
!...k
n
!)
2. 3 Conditional Probability and
Independence
Definition: A probability that is based on part of the
sample space is called a conditional probability.
Let A and B be events with P(B) = 0. The conditional
probability of A given B is



( )
( | )
( )
P A B
P A B
P B

=
More Definitions
Definition: Two events A and B are independent
if the probability of each event remains the
same whether or not the other occurs.

If P(A) = 0 and P(B) = 0, then A and B are
independent if P(B|A) = P(B) or, equivalently,
P(A|B) = P(A).
If either P(A) = 0 or P(B) = 0, then A and B are
independent.
The Multiplication Rule
If A and B are two events and P(B) = 0, then
P(A B) = P(B)P(A|B).

If A and B are two events and P(A) = 0, then
P(A B) = P(A)P(B|A).

If P(A) = 0, and P(B) = 0, then both of the above hold.

If A and B are two independent events, then
P(A B) = P(A)P(B).

This result can be extended to any number of events.
Law of Total Probability
If A
1
,, A
n
are mutually exclusive and exhaustive
events, and B is any event, then
P(B) =


Equivalently, if P(A
i
) = 0 for each A
i
,
P(B) = P(B|A
1
)P(A
1
)++ P(B|A
n
)P(A
n
).



1
( ) ... ( )
n
P A B P A B + +
Example
Customers who purchase a certain make of car can
order an engine in any of three sizes. Of all the cars
sold, 45% have the smallest engine, 35% have a
medium-sized engine, and 20% have the largest. Of
cars with smallest engines, 10% fail an emissions test
within two years of purchase, while 12% of those
with the medium size and 15% of those with the
largest engine fail. What is the probability that a
randomly chosen car will fail an emissions test within
two years?
Solution
Let B denote the event that a car fails an emissions test
within two years. Let A
1
denote the event that a car
has a small engine, A
2
the event that a car has a
medium size engine, and A
3
the event that a car has a
large engine. Then P(A
1
) = 0.45, P(A
2
) = 0.35, and
P(A
3
) = 0.20. Also, P(B|A
1
) = 0.10, P(B|A
2
) = 0.12,
and P(B|A
3
) = 0.15. By the law of total probability,

P(B) = P(B|A
1
) P(A
1
) + P(B|A
2
)P(A
2
) + P(B|A
3
) P(A
3
)
= 0.10(0.45) + 0.12(0.35) + 0.15(0.20) = 0.117.
Risk in oil exploration
Managers of oil exploration portfolios make decisions on
which prospects to pursue based, in part, on the level of
risk associated with each venture. Kinchen (1986)
examined the problem of risk analysis in oil exploration
using the outcomes and associated probabilities for a
single prospect, shown in the table.

Outcome barrels Probability
0 (dry hole) .60
50,000 .10
100,000 .15
500,000 .10
1,000,000 .05
(a)What is the probability that a single oil
well prospect will result in no more than
100,000 barrels of oil?

P(0) + P(50,000) + P(100,000) = .60 +
.10 + .15 = .85

(b) What is the probability that a single oil
well prospect will strike oil?

1 P(0) = 1 - .60 = .40

Bayes Rule
Bayes Rule: Let A
1
,, A
n
be mutually exclusive and
exhaustive events, with P(A
i
) = 0 for each A
i
. Let B be
any event with P(B) = 0. Then

.

1
( | ) ( )
( | )
( | ) ( )
( | ) ( )
( | )
( | ) ( ) ( | ) ( )
k k
k
n
i i
i
C C
P B A P A
P A B
P B A P A
P B A P A
P A B
P B A P A P B A P A
=
=
=
+

Example
The proportion of people in a given community
who have a certain disease is 0.005. A test is
available to diagnose the disease. If a person has the
disease, the probability that the test will produce a
positive signal is 0.99. If a person does not have the
disease, the probability that the test will produce a
positive signal is 0.01. If a person tests positive, what
is the probability that the person actually has the
disease?
Solution
Let D represent the event that a person actually
has the disease, and let + represent the event
that the test gives a positive signal. We wish
to find P(D|+). We know P(D) = 0.005,
P(+|D) = 0.99, and P(+|D
C
) = 0.01.
Using Bayes rule:

. 332 . 0
) 995 . 0 ( 01 . 0 ) 005 . 0 ( 99 . 0
) 005 . 0 ( 99 . 0

) ( ) | ( ) ( ) | (
) ( ) | (
) | (
=
+
=
+ + +
+
= +
C C
D P D P D P D P
D P D P
D P
In central Oklahoma many rural property
owners have a well drilled in the hope of
accessing underground water, but there is
a risk of not striking sufficient water (i.e. a
maximum flow rate of less than 100
gallons per hour is considered insufficient)
and there is also a risk that the water is
not suitable for human consumption (i.e. is
not potable. It would be very helpful to
know the probability of the combination of
events of striking sufficient water that is
also potable: P(sufficient, potable).
Obtaining P(sufficient) is easy, because drilling
companies keep data for the numbers of sufficient and
insufficient well they have drilled. Unfortunately, they do
not have records of whether the water is potable,
because that is established later by a laboratory analysis
paid for by the property owner. Furthermore, laboratory
analyses of samples from new wells are usually only
done on those that yield sufficient water there would be
little point of assessing the water quality for an
insufficient well. Therefore, data from laboratory
analyses for potabillity only gives the conditional
probability P(potability|sufficient). Nevertheless, from the
two known probabilities, the chance of striking sufficient
and potable water can be calculated.
HOW?
P(sufficient, potable) = P(sufficient) X
P(potable|sufficient)
From drilling company records the likelihood of
striking sufficient water in central Oklahoma (P
sufficient) is 0.95 (so it is not surprising that one
company charges 5% more than its competitors
but guarantees to refund the drilling fee for any
well that does not strike sufficient water).
Laboratory records for water sample analyses
shoe that only 0.3 of sufficient wells yield potable
water (P potable|sufficient)
Type equation here.Therefore, the probability of
the two events sufficient and potable water is
only 0.285 (0.95 x 0.3), which means that the
chance of this occurring is slightly higher than .
If you were a central Oklahoma property owner
with a choice of two equally expensive
alternatives of
(a) installing additional rainwater tanks, or
(b) having a well drilled, what would you decide
on the basis of this probability?
A, B = B A B = (A) (B|A)
A B =
(A)(B|A)
(B)

Random Variables
Definition: A random variable assigns a
numerical value to each outcome in a sample
space.

Definition: A random variable is discrete if its
possible values form a discrete set.

Example
The number of flaws in a 1-inch length of copper wire
manufactured by a certain process varies from wire to wire.
Overall, 48% of the wires produced have no flaws, 39%
have one flaw, 12% have two flaws, and 1% have three
flaws. Let X be the number of flaws in a randomly selected
piece of wire. Then

P(X = 0) = 0.48, P(X = 1) = 0.39, P(X = 2) = 0.12, and
P(X = 3) = 0.01.

The list of possible values 0, 1, 2, and 3, along with the
probabilities of each, provide a complete description of the
population from which X was drawn.
Probability Mass Function
The description of the possible values of X and
the probabilities of each has a name: the
probability mass function.

Definition: The probability mass function
(pmf) of a discrete random variable X is the
function p(x) = P(X = x). The probability mass
function is sometimes called the probability
distribution.
Cumulative Distribution Function
The probability mass function specifies the
probability that a random variable is equal to a given
value.
A function called the cumulative distribution
function (cdf) specifies the probability that a random
variable is less than or equal to a given value.
The cumulative distribution function of the random
variable X is the function F(x) = P(X x).
Example
Recall the example of the number of flaws in a
randomly chosen piece of wire. The following is the
pdf: P(X = 0) = 0.48, P(X = 1) = 0.39, P(X = 2) = 0.12,
and P(X = 3) = 0.01.
For any value x, we compute F(x) by summing the
probabilities of all the possible values of x that are
less than or equal to x.
F(0) = P(X 0) = 0.48
F(1) = P(X 1) = 0.48 + 0.39 = 0.87
F(2) = P(X 2) = 0.48 + 0.39 + 0.12 = 0.99
F(3) = P(X 3) = 0.48 + 0.39 + 0.12 + 0.01 = 1
More on a Discrete Random Variable
Let X be a discrete random variable. Then
The probability mass function of X is the
function p(x) = P(X = x).
The cumulative distribution function of X is
the function F(x) = P(X x).

,where the sum is over all
the possible values of X.

( ) ( ) ( )
t x t x
F x p t P X t
s s
= = =

( ) ( ) 1
x x
p x P X x = = =

Mean and Variance for Discrete
Random Variables
The population mean (or expected value) of X is
given by
,
where the sum is over all possible values of X.
The population variance of X is given by



The standard deviation is the square root of the
variance.
( )
X
x
xP X x = =

2 2
2 2
( ) ( )
( ) .
X X
x
X
x
x P X x
x P X x
o

= =
= =

Exemple
A certain analytical procedure (MIP) in a petrophysics
lab is brought down for recalibration whenever the
precision of the measurements performed fall below
specs. Let X represent the number of times the
procedure is recalibrated during a month, and assume
that X has the following probability mass function:




Find the mean of X.

x 0 1 2 3 4
P(x) 0.35 0.25 0.20 0.15 0.05

X
=0(0.35) + 1(0.25) + 2(0.20) + 3(0.15) +
4(0.05) = 1.30
The Probability Histogram
When the possible values of a discrete random
variable are evenly spaced, the probability mass
function can be represented by a histogram, with
rectangles centered at the possible values of the
random variable.
The area of the rectangle centered at a value x is equal
to P(X = x).
Such a histogram is called a probability histogram,
because the areas represent probabilities.


Example
The following is a
probability histogram
for the example with
number of flaws in a
randomly chosen piece
of wire.


Continuous Random Variables
A random variable is continuous if its probabilities
are given by areas under a curve.
The curve is called a probability density function
(pdf) for the random variable. Sometimes the pdf is
called the probability distribution.
The function f(x) is the probability density function of
X.
Let X be a continuous random variable with
probability density function f(x). Then

( ) 1. f x dx

=
}
Computing Probabilities
Let X be a continuous random variable with
probability density function f(x). Let a and b
be any two numbers, with a < b. Then

In addition,

( ) ( ) ( ) ( )
b
a
P a X b P a X b P a X b f x dx s s = s < = < s =
}
( ) ( ) ( )
( ) ( ) ( )
( ) 1
a
a
P X a P X a f x dx
P X a P X a f x dx
f x dx

s = < =
> = > =
=
}
}
}
More on Continuous Random
Variables
Let X be a continuous random variable with
probability density function f(x). The cumulative
distribution function of X is the function

The mean of X is given by

The variance of X is given by
( ) ( ) ( ) .
x
F x P X x f t dt

= s =
}
( ) .
X
xf x dx

=
}
2 2
2 2
( ) ( )
( ) .
X X
X
x f x dx
x f x dx
o

=
=
}
}
Exemple
The level of impurity (in %) in a gold ore is a random
variable with probability density function



A. What is the probability that the impurity level is greater
than 3%?
B. What is the probability that the impurity level is
between 2% and 3%?
C. Find the mean impurity level.
D. Find the variance of the impurity level.
E. Find the cumulative distribution function of the
impurity level.

2
3
(4 ) 0 4
64
0 otherwise
x x x

< <

`

)
Median and Percentiles
Let X be a continuous random variable with probability mass
function f(x) and cumulative distribution function F(x).

The median of X is the point x
m
that solves the equation


If p is any number between 0 and 100, the pth percentile is the
point x
p
that solves the equation


The median is the 50
th
percentile.
( ) ( ) ( ) 0.5.
m
x
m m
F x P X x f x dx

= s = =
}
( ) ( ) ( ) /100.
p
x
p p
F x P X x f x dx p

= s = =
}
Linear Functions of Random Variables
If X is a random variable, and a and b are
constants, then

,

,

.




aX b X
a b
+
= +
2 2 2
aX b X
a o o
+
=
aX b X
a o o
+
=
More Linear Functions
If X and Y are random variables, and a and b are
constants, then


More generally, if X
1
, , X
n
are random
variables and c
1
, , c
n
are constants, then the
mean of the linear combination c
1
X
1
, , c
n
X
n

is given by
.
aX bY aX bY X Y
a b
+
= + = +
1 1 2 2 1 2
... 1 2
... .
n n n
c X c X c X X X n X
c c c
+ + +
= + + +
Exemple
The molarity of a solute in solution is defined by the
number of moles of solute per liter of solution (1 mole =
6.02 x 10
23
molecules). If the molarity of a stock solution
of concentrated sulfuric acid (H
2
SO
4
) is X, and if one part
of the solution is mixed with N parts of water, the
molarity Y of the dilute solution is given by Y = X/(N+1).
Assume that the stock solution is manufactured by a
process that produces a molarity with a mean 18 and a
standard deviation 0.1. If 100 mL of stock solution is
added to 300 mL of water, find the mean and standard
deviation of the molarity of the dilute solution.
The molarity of the dilute solution is Y = 0.25X [Y=
X/(3+1)]. The mean and standard deviation of X are mx
= 18 and o
x
= 0.1, respectively. Therefore

Y
=
0.25X
=0.25
X
=0.25(18.0)
=4.5

Also,
o
Y
=o
0.25X
=0.25o
X
=0.25(0.1)
=0.025
Two Independent Random Variables
If X and Y are independent random variables,
and S and T are sets of numbers, then


More generally, if X
1
, , X
n
are independent random variables,
and S
1
, , S
n
are sets, then
( and ) ( ) ( ). P X S Y T P X S P Y T e e = e e
1 1 2 2 1 1 2 2
( , ,..., ) ( ) ( )... ( ).
n n n n
P X S X S X S P X S P X S P X S e e e = e e e
Variance Properties
If X
1
, , X
n
are independent random variables,
then the variance of the sum X
1
+ + X
n
is
given by


If X
1
, , X
n
are independent random variables
and c
1
, , c
n
are constants, then the variance
of the linear combination c
1
X
1
+ + c
n
X
n
is
given by


1 2 1 2
2 2 2 2
...
.... .
n n
X X X X X X
o o o o
+ + +
= + + +
1 1 2 2 1 2
2 2 2 2 2 2 2
... 1 2
.... .
n n n
c X c X c X X X n X
c c c o o o o
+ + +
= + + +
More Variance Properties
If X and Y are independent random variables
with variances , then the variance of
the sum X + Y is


The variance of the difference X Y is

2 2
and
X Y
o o
2 2 2
.
X Y X Y
o o o
+
= +
2 2 2
.
X Y X Y
o o o

= +
Independence and Simple Random
Samples
Definition: If X
1
, , X
n
is a simple
random sample, then X
1
, , X
n
may be
treated as independent random variables,
all from the same population.

Properties of
If X
1
, , X
n
is a simple random sample from a
population with mean and variance o
2
, then
the sample mean is a random variable with



The standard deviation of is

X
X
X
=
X
2
2
.
X
n
o
o =
.
X
n
o
o =
Exemple
The article Abyssal Peridotites > 3800 Ma from Southern
West Greenland: Field Relationships, Petrography,
Geochronology, Whole-Rock and Mineral Chemistry of
Dunite and Harzburgite Inclusions in the Itsaq Gneiss
Complex (C. Friend, V. Bennet, and A. Nutman, Contrib.
Mineral. Petrol., 2002:71-92) describes the chemical
compositions of certain minerals in the early Archean
mantle. For a certain type of olivine assembly, the silicon
dioxide (SiO
2
) content (in weight percent) in a randomly
chosen rock has a mean 40.25 and a standard deviation
0.36.
A. Find the mean and standard deviation of the sample mean SiO2
content in a random sample of 10 rocks.
B. How many rocks must be samples so that the standard deviation
of the sample mean SiO
2
content is 0.05?
Jointly Distributed Random Variables
(sec. 2.6-optional)
If X and Y are jointly discrete random variables:
The joint probability mass function of X and Y is the function

The marginal probability mass functions of X and Y can be
obtained from the joint probability mass function as follows:

where the sums are taken over all the possible values of Y and
of X, respectively.
The joint probability mass function has the property that

where the sum is taken over all the possible values of X and Y.
( , ) ( and ) p x y P X x Y y = = =
( ) ( ) ( , ) ( ) ( ) ( , )
X Y
y x
p x P X x p x y p y P Y y p x y = = = = = =

( , ) 1
x y
p x y =

Jointly Continuous Random Variables

If X and Y are jointly continuous random
variables, with joint probability density
function f(x,y), and a < b, c < d, then


The joint probability density function has the
property that
( and ) ( , ) .
b d
a c
P a X b c Y d f x y dydx s s s s =
} }
( , ) 1. f x y dydx


=
} }
Marginals of X and Y
If X and Y are jointly continuous with joint
probability density function f(x,y), then the
marginal probability density functions of X and
Y are given, respectively, by

( ) ( , )
X
f x f x y dy

=
}
( ) ( , ) .
Y
f y f x y dx

=
}
More Than Two Random Variables
If the random variables X
1
, , X
n
are jointly
discrete, the joint probability mass function is

If the random variables X
1
, , X
n
are jointly
continuous, they have a joint probability
density function f(x
1
, x
2
,, x
n
), where


for any constants a
1
b
1
, , a
n
b
n
.
1 1 1
( ,..., ) ( ,..., ).
n n n
p x x P X x X x = = =
1
1
1 1 1 1 1
( ,...., ) ( ,..., ) ... .
n
n
b b
n n n n n
a a
P a X b a X b f x x dx dx s s s s =
} }
Means of Functions of Random
Variables
Let X be a random variable, and let h(X) be a
function of X. Then:
If X is a discrete with probability mass function p(x), then
mean of h(X) is given by

where the sum is taken over all the possible values
of X.

If X is continuous with probability density function f(x), the
mean of h(x) is given by


( )
( ) ( ).
h x
x
h x p x =

( )
( ) ( ) .
h x
h x f x dx

=
}
Functions of Joint Random Variables
If X and Y are jointly distributed random variables, and
h(X,Y) is a function of X and Y, then

If X and Y are jointly discrete with joint probability mass function
p(x,y),


where the sum is taken over all possible values of X and Y.

If X and Y are jointly continuous with joint probability mass
function f(x,y),

( , )
( , ) ( , ).
h X Y
x y
h x y p x y =

( , )
( , ) ( , ) .
h X Y
h x y f x y dxdy


=
} }
Conditional Distributions
Let X and Y be jointly discrete random variables, with joint
probability density function p(x,y), let p
X
(x) denote the
marginal probability mass function of X and let x be any
number for which p
X
(x) > 0.

The conditional probability mass function of Y given X = x is



Note that for any particular values of x and y, the value of
p
Y|X
(y|x) is just the conditional probability P(Y=y|X=x).


|
( , )
( | ) .
( )
Y X
p x y
p y x
p x
=
Continuous Conditional Distributions
Let X and Y be jointly continuous random
variables, with joint probability density
function f(x,y). Let f
X
(x) denote the marginal
density function of X and let x be any number
for which f
X
(x) > 0.

The conditional distribution function of Y given
X = x is

|
( , )
( | ) .
( )
Y X
f x y
f y x
f x
=
Conditional Expectation
Expectation is another term for mean.
A conditional expectation is an expectation,
or mean, calculated using the conditional
probability mass function or conditional
probability density function.
The conditional expectation of Y given X = x is
denoted by E(Y|X = x) or
Y|X
.

Independence
Random variables X
1
, , X
n
are independent,
provided that:

If X
1
, , X
n
are jointly discrete, the joint
probability mass function is equal to the product of
the marginals:

If X
1
, , X
n
are jointly continuous, the joint
probability density function is equal to the product
of the marginals:

1
1 1
( ,..., ) ( )... ( ).
n
n X X n
p x x p x p x =
1 1
( ,..., ) ( )... ( ).
n n
f x x f x f x =
Independence (cont.)
If X and Y are independent random variables,
then:

If X and Y are jointly discrete, and x is a value for
which p
X
(x) > 0, then
p
Y|X
(y|x)= p
Y
(y).

If X and Y are jointly continuous, and x is a value
for which f
X
(x) > 0, then
f
Y|X
(y|x)= f
Y
(y).
Covariance
Let X and Y be random variables with means

X
and
Y
.

The covariance of X and Y is


An alternative formula is

( )( )
Cov( , ) .
X Y
X Y
X Y

=
Cov( , ) .
XY X Y
X Y =
Correlation
Let X and Y be jointly distributed random variables
with standard deviations o
X
and o
Y
.

The correlation between X and Y is denoted
X,Y
and
is given by


For any two random variables X and Y;
-1
X,Y
1.
,
Cov( , )
.
X Y
X Y
X Y

o o
=
Covariance, Correlation, and Independence
If Cov(X,Y) =
X,Y
= 0, then X and Y are said to
be uncorrelated.
If X and Y are independent, then X and Y are
uncorrelated.
It is mathematically possible for X and Y to be
uncorrelated without being independent. This
rarely occurs in practice.
Covariance of Random Variables
If X
1
, , X
n
are random variables and c
1
, , c
n

are constants, then
1 1 1
... 1
...
n n n
c X c X X n X
c c
+ +
= + +
1 1 1
1
2 2 2 2 2
... 1
1 1
... 2 Cov( , ).
n n n
n n
c X c X X n X i j i j
i j i
c c c c X X o o o

+ +
= = +
= + + +

Covariance of Independent Random
Variables
If X
1
, , X
n
are independent random variables
and c
1
, , c
n
are constants, then


In particular,

1 1 1
2 2 2 2 2
... 1
... .
n n n
c X c X X n X
c c o o o
+ +
= + +
1 1
2 2 2
...
... .
n n
X X X X
o o o
+ +
= + +
Summary
Probability and rules
Counting techniques
Conditional probability
Independence
Random variables: discrete and continuous
Probability mass functions

Summary Continued
Probability density functions
Cumulative distribution functions
Means and variances for random variables
Linear functions of random variables
Mean and variance of a sample mean
Jointly distributed random variables