Important Terms

Probability
Random Experiment – a process leading to

an uncertain outcome.

Basic Outcome – a possible outcome of a
random experiment: s ∈ S.
Chapter 6

Sample Space – the collection of all possible
outcomes of a random experiment: S.

Event – any subset of basic outcomes from
the sample space: A ⊆ S.

Important Terms Union of Events

Discrete sample space: S is finite or
numerable.
Outcomes of rolling a die:
S = {1, 2, 3, 4, 5, 6}

Continuous sample space: S is non-
numerable.
If A and B are two events in a sample
space S, then the union, A U B, is the set
of all outcomes in S that belong to either A
or B.
S The entire shaded area
represents
A B
AUB

Waiting time for a bus: S = [0, ∞)

3 4
 Intersection of Events Mutually Exclusive Events
If A and B are two events in a sample A and B are Mutually Exclusive Events if
space S, then the intersection, A ∩ B, is they have no basic outcomes in common,
the set of all outcomes in S that belong to i.e., the set A ∩ B is empty.
both A and B.
S
S
A∩B=∅
A B
A ∩B
A∩ B

5 6

Partition of the sample space Complement of an event

Events E1, E2, 8 Ek are Collectively Exhaustive
events if
E1 U E2 U . . . U Ek = S
i.e., the events completely cover the sample
space

Events E1, E2, 8 Ek are a Partition of the sample
space S if they are collectively exhaustive and
mutually exclusive events:
E1 U E2 U . . . U Ek = S
and Ei ∩ Ek = ∅, for every i ≠ k 7 8
The Complement of an event A is the set
of all basic outcomes in the sample space
that do not belong to A.
The complement is denoted by: Ᾱ
S
A
Ᾱ
A and Ᾱ are a partition of the sample space.
Difference of events Example
Let the Discrete Sample Space be the collection
The Difference of two events, A \ B, is the
of all possible outcomes of rolling one dice:
set of all basic outcomes that belong to A
but do not belong to B.
Ω S = [1, 2, 3, 4, 5, 6]
A B
A\ B = A∩ B Let A be the event: “Number rolled is even”
Let B be the event: “Number rolled is at least 4”
Obtain: Ᾱ, A U B, A U Ᾱ, A ∩ B, Ᾱ ∩ B, A \ B.

9 10

Properties
Some basic properties: Probability Definitions

A∪ B = A∩ B

A∩ B = A∪ B

A ∪ (B ∩ C ) = ( A ∪ B ) ∩ ( A ∪ C )

A ∩ (B ∪ C ) = ( A ∩ B ) ∪ ( A ∩ C )
11
 Probability Assessing Probability

Let A be an event: A ⊆ S 1 Certain There are three approaches to assessing the

Probability – the chance that an
uncertain event A will occur
(always between 0 and 1)
0 ≤ P(A) ≤ 1 For any event A
probability of an uncertain event:

First approach: Classical Probability (a priori
probability)
.5

Impossible Event – an event that has
Second approach: Empirical Probability

no chance of occurring (P(A) = 0)

Certain Event – an event that is sure
Third approach: Subjective Probability

to occur (P(A) = 1) 0 Impossible
13 14

First approach. Classical Probability Second approach. Empirical Probability

Laplace (1812)
Relative frequency probability.

Assumes all outcomes in the
The probability of an event is the limit of the
sample space are equally likely to proportion of times that an event A occurs in a
occur large number of trials, n. n

The number of outcomes in the P(A) = lim A
n →∞ n
sample space should be finite

Counting the number of outcomes
It is assumed that the
that satisfy the event A random experiment can
NA number of outcomes that satisfy the event A be repeated many
P(A) = = times under the same
N total number of outcomes in the sample space S conditions.

Example: A = {at least one 5 when rolling two dice} P (A) = 11 = 0.3055 15 16
36
Third approach. Subjective Probability

Savage (1953)

There are random experiment that can
Axiomatic Theory of Probability
not be repeated or whose basic
outcomes are not equally likely to
occur.

The probability of an event A is an
individual opinion or belief about the
probability of occurrence.

Subjective probabilities are based on a
combination of an individual’s past
experience, personal opinion, and
analysis of a particular situation 17

Probability Measures Probability Postulates

Kolmogorov (1933) Every probability measure fulfils the following

We assume same probability three postulates:
postulates that every 1. If A is any event in the sample space S,
probability measure should then 0 ≤ P(A) ≤ 1.
fulfil.

Once these postulates are
assumed, a set of properties 2. If A and B are two mutually exclusive
are proved, by using events, A ∩ B = Ø, then
mathematical techniques.

This probability approach is the
P(A∪B) = P(A) + P(B)
basis of the modern Statistical
Inference.
19
3. P(S) = 1 20
Probability Rules A Probability Table
Probabilities and joint probabilities for two events A

The Complement rule: P(Ᾱ) = 1 – P(A), and B are summarized in this table:
i.e. P(A) + P(Ᾱ) = 1

P(Ø) = 0 B B

The Addition rule. The probability of the
union of two events is: A P(A∩B) P(A ∩ B ) P(A)

P(A∪B) = P(A) + P(B) – P(A∩B) Ā P(Ā∩B) P(A ∩ B ) P(Ā)

If the event A is contained in the event B,
A ⊆ B, then: P(A\B) = P(A) – P(B) P(B) P( B ) P(S) = 1

21 22

Addition Rule Example Addition Rule Example

Consider a standard deck of 52 cards, with four suits: P(Red U Ace) = P(Red) + P(Ace) - P(Red ∩ Ace)
♥♣♦♠ = 26/52 + 4/52 - 2/52 = 28/52
Let event A = {the selected card is an Ace} Don’t count
the two red
Let event B = {the selected card is from a red suit} Color aces twice!
Type Red Black Total
Ace 2 2 4
Non-Ace 24 24 48
Total 26 26 52
23 24
 Conditional Probability

Conditional Probability A conditional probability is the probability

of one event, given that another event has
occurred:
The conditional
P(A ∩ B)
P(A/B) = probability of A given
that B has occurred,
P(B)
P(B)>0

P(A ∩ B) The conditional

P(B/A) = probability of B
P(A) given that A has
occurred, P(A)>0
26

Example Example

Of the cars on a used car lot, 70% have air conditioning Given AC, we only consider the top row (70% of
(AC) and 40% have a CD player (CD). 20% of the cars
have both. the cars). Of these, 20% have a CD player. 20%

What is the probability that a car has a CD player, given
of 70% is 28.57%.
that it has AC ? CD No CD Total

CD No CD Total AC .2 .5 .7
AC .2 .5 .7 No AC .2 .1 .3
No AC .2 .1 .3 Total .4 .6 1.0
Total .4 .6 1.0
P(CD ∩ AC) .2
P(CD | AC) = = = .2857
We want to find P(CD/AC) 27
P(AC) .7 28
 Multiplication Rule Example
P(Red ∩ Ace) = P(Red/Ace)P(Ace)

Multiplication rule for two events A and B:  2  4  2
=    =
P(A∩B) = P(A)—P(B/A)  4  52  52
number of cards that are red and ace 2
= =

Also P(A∩B) = P(B)—P(A/B) total number of cards 52

Color

Multiplication rule for three events A, B Type Red Black Total
and C: Ace 2 2 4
P(A∩B∩C) = P(A)—P(B/A)—P(C/A∩B)
Non-Ace 24 24 48
29 Total 26 26 52 30

Statistical Independence of events Example

CD No CD Total

Two events A and B are statistically independent AC .2 .5 .7
if:
No AC .2 .1 .3
P(A∩B) = P(A)—P(B)

Two events are independent when the Total .4 .6 1.0
probability of one event is not affected by the P(AC ∩ CD) = 0.2
other event.

Two events A and B are statistically independent P(AC) = 0.7
P(AC)P(CD) = (0.7)(0.4) = 0.28
if P(CD) = 0.4
P(B/A) = P(B) or P(A/B) = P(A) P(AC ∩ CD) = 0.2 ≠ P(AC)P(CD) = 0.28
The two events AC and CD are not statistically independent
31 32
 “a priori” and “a posteriori” probabilities Example
CD No CD Total

Without information about the event, the
AC .2 .5 .7
probability is called a priori:
P(A) ≡ “a priori” probability No AC .2 .1 .3

When it is known that the event B has Total .4 .6 1.0
occurred, the conditional probability of A,
given B, is called a posteriori: A posteriori A priori
P(A/B) ≡ “a posteriori” probability
P(AC/CD) = 0.5 P(AC) = 0.7

The a priori and a posteriori probabilities are
linked by:
P (B / A ) P(CD/AC) = 0.2857 P(CD) = 0.4
P ( A / B ) = P ( A) ⋅
P (B ) 33 34

Tree diagram .2 Tree diagram .2

.7 P(AC ∩ CD) = .2 .4 P(CD ∩ AC) = .2

P(AC ∩ CD) = .5 P(CD ∩ AC) = .2

.5 .2
Conditional .7 Conditional .4
All Probabilities All Probabilities

cars cars
.2 .5
.3 P(AC ∩ CD) = .2 .6 P(CD ∩ AC) = .5

.1 P(AC ∩ CD) = .1 .1 P(CD ∩ AC) = .1

.3 35
.6 36
Total Probability Theorem Total Probability Theorem

Let B1, B2, 8, Bk be a partition of the
S B1 B2 A
sample space S, that is, B3
B1 U B2 U ... U Bk = S
Bi ∩ Bk = ∅, for every i ≠ k B5
B4 B6

Let A be an arbitrary event. Then
k k
P ( A) = ∑ P ( A ∩ B ) = ∑ P ( A / B ) ⋅ P (B )
i i i B7 B8 B9
i =1 i =1
37 38

Example Odds
P(Ace)
2 2 4

The odds in favor of a particular event
= P(Ace ∩ Red) + P(Ace ∩ Black) = + = are given by the ratio of the probability
52 52 52
of the event divided by the probability of
Color its complement.
Type Red Black Total

The odds in favor of A are
Ace 2 2 4
24 24 48 P(A) P(A)
Non-Ace odds = =
Total 26 26 52
1- P(A) P(A)
39 40
Example Bayes’ Theorem
Calculate the probability of winning if the odds of
winning are 3 to 1:
Let B1, B2, 8, Bk be a partition of the
sample space S, that is,
3 P(A)
odds = = B1 U B2 U ... U Bk = S
1 1- P(A)
Multiply both sides by 1 – P(A) and solve for P(A): Bi ∩ Bk = ∅, for every i ≠ k
3 x (1 – P(A)) = P(A)
Let A be an arbitrary event. Then
P (A / B j ) P(A / B j )⋅ P (B j )
3 – 3P(A) = P(A)
3 = 4P(A) P (B j / A) = P (B j )⋅ =
P ( A) k
P(A) = 0.75
∑ P ( A / B ) ⋅ P (B )
i =1
i i
41 A posteriori A priori 42

Example

In a box production chain, machines A, B and C

manufacture, respectively, 50%, 35% and 20% of
the total weekly output. Historical records tell us
that 5%, 15% and 25% of the boxes made by
machines A, B and C, respectively, are defective.
Obtain the probability that, if a box was defective,
it was made by machine A.