MATH-361

Probability and Statistics

Lecture 5

Conditional Probability, Bayes Theorem,

Independence of Events

A/P Kamran Aziz Bhatti | Assistant Professor | Dept. of Electrical Engg. | NUST College of Electrical & Mechanical Engineering | Pakistan
 Steps to find Probability Law

• From the random experiment, measurements and observations, find

the set of all possible outcomes (The Sample Space)

• Determine Initial Probability Assignment of certain events (It should

satisfy all axioms of probability)

• For discrete S, probabilities of elementary events are required

• For continuous S, probabilities of intervals/regions are required

• Probability of all other events can be extracted accordingly by using

probability axioms and/or corollaries

2
Conditional Probability
 Conditional Probability

• Probability of event A given that event B has occurred is

termed as Conditional Probability P[A|B]
𝑃[𝐴 𝐵]
𝑃[𝐴|𝐵] =
𝑃[𝐵]
• SS is reduced to event B
• Normalization of probability of events that occur jointly with
B, i.e. 𝑃[𝐴 𝐵]
• Satisfies axioms of probability
• Conditional Probability is useful for finding the probability in
sequential experiments
4
Conditional Probability

5
Conditional Probability

6
 Conditional Probability

• Theorem on Total Probability: Probability of an event A can

be defined in terms of the intersection of A with the mutually
exclusive events that form the complete SS
𝑃 𝐴 = 𝑃 𝐴 𝐵1 𝑃 𝐵1 + 𝑃 𝐴 𝐵2 𝑃 𝐵2 + ⋯ + 𝑃 𝐴 𝐵𝑛 𝑃 𝐵𝑛
• Useful for sequential random experiment (tree diagram)

7
Bayes’ Rule
 Bayes’ Rule

• Let 𝐵1 , 𝐵2 , … , 𝐵𝑛 be mutually exclusive events that form the

complete SS, Given an event A is occurred then

𝑃[𝐴 𝐵𝑗 ] 𝑃 𝐴 𝐵𝑗 𝑃[𝐵𝑗 ]
𝑃[𝐵𝑗 |𝐴] = = 𝑛
𝑃[𝐴] 𝑘=1 𝑃 𝐴 𝐵𝑘 𝑃[𝐵𝑘 ]

is given by Bayes’ rule

• Useful for cases where ‘a-priori’ probabilities of events (that
form the SS) are known and we have to find the ‘a-posteriori’
probabilities of these event(s) after the occurrence of event A

9
Bayes’ Rule

10
Independence of Events
 Independence of Events

• Events A and B are said to be independent, if the knowledge

of occurrence of B doesn’t alter the probability of event A
• In this case, P[A] = P[A|B]
• In other words, the proportion of outcomes in S that lead to
the occurrence of A is equal to the proportion of outcomes in
B that lead to A

12
 Independence of Events

• Mutually exclusive vs independent?

• A General Rule: If two events have non-zero probabilities and
are mutually exclusive, they cannot be independent

13
 Independence of Events

• Three events are mutually independent if and only if:

• They are pair-wise independent
• Probability of the intersection of three events is equal to the
product of the probabilities of the individual events

• If the events are pair-wise independent, it is not necessary

that they are mutually independent
• Square region example – revisited

14
Independence of Events

15
 Independent Experiments

• General rule for independence of events

• This rule becomes complicated for more events
• Independent experiments
• Independent coin toss experiment

16