1

PROBABILITY
2
Definition
Formal study of chance
Has its origins in gambling games (some
mathematicians fascination on the odds of
winning)
3
Random Experiment
A process of making observations that
Leads to well defined outcomes
Capable of repetition under basically the same
conditions
One cannot predict with certainty the outcome
every time the experiment/process is performed
(referred to as a trial)

4
Random Experiment
Examples
1. Tossing a coin
2. Selecting 3 persons from a group of 100.
3. Observing the number of absences incurred by
students in a school
4. Recording the amount of time a particular drug takes
into effect.
5
Sample Space
The set of all outcomes of a random
experiment.
Denoted by S
Example:

Random Experiment toss two coins
S={HH, HT, TH, TT}

6
Events
Any subset of the sample
space (including the empty set
and sample space itself)
Types:
Simple/Elementary contains
only one element of S
Compound contains more than
one element of S
Null contains no element in S
S
E
7
Events (Examples)
Random Expt. Toss 2 coins
S={HH, HT, TH, TT}
Let E - event of getting exactly one head
E= {HT, TH}

OBJECTIVE: To assign a value, P(E), referred to as
the probability of event E, which is a measure of
how likely (or unlikely) the event is going to happen
if the experiment where to be performed.
8
Properties of P(E)
1. For any event E in S, 0 P(E) 1.

The closer P(E) is to 1, the more likely the event
is going to happen and the closer it is to 0 the
less likely it is going to happen.



9
Properties of P(E)
2. P(u) = 0 and P(S)=1

In here u (null event) is called the impossible event (it
cannot happen) while S (sample space) is called the sure
event.

In some situations, P(E)=0 does not necessarily mean that
E= u. P(E) is just so small that it is not any much different
from 0. In this case, we call E as an improbable event. A
similar thing can be said when P(E)=1. In here, we call E
as a probable event.



10
Properties of P(E)
3. If E
1
,E
2
,,E
n
are non-overlapping events in
S, then
P(E
1
E
2
E
n
)=P(E
1
)+P(E
2
)++P(E
n
)


11
Approaches in Assigning
Probabilities to Events
A Priori (Theoretical/Classical/Axiomatic)
A Posteriori (Relative Frequency)
Subjective
12
A Priori Approach
Requires the construction of a model with
underlying assumptions
MODEL sample space (a representation of
the random experiment)
ASSUMPTION how each element
(outcome) of the sample space will occur if
the random experiment where to be
performed. Example: Equally likely
assumption.
13
A Priori Approach
Under the assumption of equally-likely outcomes


number of outcomes in E
( )
number of outcomes in S
P E =
14
Example
Random Expt: Toss 3 coins
S = {HHH, HHT, HTH, THH, HTT, THT, TTH, TTT}
E event of getting exactly 2 heads
E= {HHT, HTH, THH}

Under the equally-likely assumption: P(E)=3/8.



15
A Posteriori Approach
If the random experiment is repeated over
and over again (may be from previous
experiences), the P(E) is computed by taking
the relative frequency of the event E
happening/observed.
16
A Posteriori Approach
Suppose three coins were tossed 100 times and the
outcomes were recorded as follows:

Outcome HHH HHT HTH THH HTT THT TTH TTT
Frequency 11 8 12 14 12 19 17 7
In here, P(E)= (8+12+14)/100= 34/100 = 0.34
17
Subjective Approach
Depends on ones personal assessment
18
Law of Large Numbers
The a-posteriori probability approaches the a-
priori probability as the number of trials
increases.
A-Priori Probability
P(E)
Number of trials/repetition
A-Posteriori Probability
19
Counting Methods
Recall that under the A-Priori approach in
assigning probabilities to events under the
equally likely sample space assumption:




Requires counting outcomes in E and S
( )
( )
( )
n E
P E
n S
=
20
Fundamental Principle of
Counting
If a process can be performed by
simultaneously performing two operations
together in which the first operation can be
done in n
1
ways, and the second in n
2
ways,
then together, the entire process can be
performed in n
1
x n
2
ways.
21
Fundamental Principle of
Counting
Example: How many outcomes will result
when we toss a pair of dice.
Two operations outcomes in die 1 (6) and
outcomes in die 2 (6). Together, we have 6 x
6 = 36 possible outcomes.
22
Fundamental Principle of
Counting
Outcomes can be represented by tree
diagram:
Die 1 Die 2 Outcome
1 (1,1)
1 2 (1,2)
3 (1,3)
4 (1,4)
5 (1,5)
6 (1,6)
.
.
.

36 possible
outcomes.
23
Fundamental Principle of
Counting (Extension)
If a process can be performed by
simultaneously performing k operations
together in which the first operation can be
done in n
1
ways, and the second in n
2

ways,, and operation k in n
k
ways then
together, the entire process can be
performed in n
1
x n
2
x x n
k
ways.
24
Permutations
A permutation is an ordered arrangement of
objects all at a time or r at a time.
The total number of ways n object can be
arranged (rearranged) if order is important
is n! = n(n-1)(n-2)(2)(1).
The total number of ways n objects can be
arranged r at a time with respect to order is
!/ !
n r
P n r =
25
Combinations
A combination refers to the number of r
distinct objects from n.
The total number of combinations of r
distinct objects from n is
!
!( )!
n r
n
n
C
r r n r
| |
= =
|

\ .
26
Union of Events
An event may be formed by
combining two or more events
(in the same S) by an
admissible operation such as
unions or intersections.
Let A and B be events in S.
Note that E=AB _ S and
hence E is another event in S
formed by taking the union of
A and B
S
A B
E
27
Union of Events
E is interpreted as the event in
which we are interested in
event A happening or event B
happening or both A and B
happening at the same time.
In this case,
P(AB)=P(A)+P(B)-P(AB)
If A and B are mutually
exclusive, then
P(AB)=P(A)+P(B)

S
A B
E
28
Union of Events
Example: Random Expt
toss a die and a coin.
S={H1, H2, H3, H4, H5, H6,
T1, T2, T3, T4, T5, T6}
A event of getting a head on
the coin. A={H1, H2, H3, H4,
H5, H6}
B event of getting even
number of dots on the die.
B={H2, H4, H6, T2, T4, T6}
S
H2
H4
H6


T2
T4
T6
H1
H3
H5
T1
T2
T3
B A
E={H1,H2,H3,H4,H5,H6,T2,T4,T6}
A B={H2,H4,H6}
29
Union of Events
Assuming Equally Likely
Outcomes,
P(A)=6/12=0.5
P(B)=6/12=0.5
P(AB)=3/12=0.25

Thus,
P(AB)=0.5+0.5-0.25=0.75
S
H2
H4
H6


T2
T4
T6
H1
H3
H5
T1
T2
T3
B A
E={H1,H2,H3,H4,H5,H6,T2,T4,T6}
A B={H2,H4,H6}
30
Intersection of Events
Let A and B be events in S.
Note that E=AB _ S and
hence E is another event in S
formed by taking the
intersection of A and B.
In here, E is interpreted as the
event when A and B occurred
at the same time.
S
A B
E
31
Intersection of Events
Assuming Equally Likely
Outcomes,
E= P(AB)=3/12=0.25


S
H2
H4
H6


T2
T4
T6
H1
H3
H5
T1
T2
T3
B A
E={H1,H2,H3,H4,H5,H6,T2,T4,T6}
A B={H2,H4,H6}
32
Mutually Exclusive Events
Let A and B be events in S. A and B
are said to be mutually exclusive if
there are no outcomes in S that are
associated with both A and B.
A and B cannot happen at the same
time
A and B are disjoint. AB=C
P(AB)=P(C)=0
A
B
S
33
Complementary Events
Let A be an event in S. The
complement of A, denoted by or
(A
c
or A) consists of all outcomes in
S but is not associated with A.
Note that A =S and A =C.
That is A and are collectively
exhaustive and mutually exclusive.
P(A)=1-P() ; P() = 1 P(A).
A

34
Conditional Probability
In some situations, we may be interested in determining
the probability of one event happening given that another
event (conditioning event) has already (earlier) occurred.
Example: Consider the random expt of tossing a coin
and a die. Suppose we toss the coin first and a head
occurred (A has occurred). Given this, we want to find
the probability that even number of heads will occur
when the die will be tossed (B given A or B|A).

35
Conditional Probability
In this case,



In here, A is referred to as the conditioning event

Similarly




B conditioning event

0 =

= ) A ( P ,
) A ( P
) B A ( P
) A | B ( P
( )
( | ) , ( ) 0
( )
P A B
P A B P B
P B

= =
36
Conditional Probability
Example: Toss a coin and a die

5 . 0 6 / 3
) 12 / 6 (
) 12 / 3 (
) (
) (
) | (
5 . 0 6 / 3
) 12 / 6 (
) 12 / 3 (
) (
) (
) | (
= = =

=
= = =

=
A P
B A P
A B P
Similarly
B P
B A P
B A P
37
Independent Events
Two events A and B in S are said to be independent if
the occurrence of one DOES NOT affect the occurrence
of another.
That is, P(A|B)=P(A) and P(B|A)=P(B)
We say that A and B are INDEPENDENT. Otherwise,
they are DEPENDENT.
38
Independent and Mutually
Exclusive Events
If two events, A and B in S, are MUTUALLY
EXCLUSIVE, can they be independent?
ANSWER: No. (Why?)
If two events, A and B in S, are INDEPENDENT, can
they be mutually exclusive?
ANSWER: No. (Why?)
39
Multiplication Rule
) A ( P ) A | B ( P ) B ( P ) B | A ( P ) B A ( P or =
( ) ( ) ( ) P A B P A P B =
If A and B are independent, then




As a consequence
) ( ) ( ) ( ) ( ) ( B P A P B P A P B A P + =
40
Example
One hundred forty students were classified according to the
number of simultaneous romantic relationship and academic
performance.
Scholastic
Performance
Number of Romantic Relationships
None 1-3 > 3
Poor 15 20 30
Satisfactory 15 20 15
Good 15 5 5