Int

Quantitative Techniques-I
Amitava Mukherjee;

Course Outline
Sl.
No.

Topic

Appox. No.
of Sessions

Faculty

Theory of Probability

Myself

Discrete Distributions

Myself

Continuous Distributions

Prof. S. K. De.

Elementary Notions of
Stochastic Processes

Prof. S. K. De.

Decision Theory

Prof. S. K. De.

Pedagogy For First 10 Sessions

Completely Lecture Based
Slides will be used in various sessions
Lecture slides will be shared time to time
All Slides up to Lecture-5 will be shared before the Quiz-I
and up-to Lecture 10 before Mid Term
Slides will not be allowed to use during Examinations
You are encouraged to take running notes in class which will be
useful for preparation for Quiz/ Mid-Term and End Term.
Reading Material will be shared online and students are requested
to come to class by reading the material for the corresponding
session for better understanding of the class.
3

Evaluation System
Component

Syllabus

Weight

Quiz I
Tentatively after 5
sessions
[Announced but
closed Book/Notes]

Topics covered till

Session 5 from
beginning

20%

Mid Term
Tentatively after
[Announced but
10 sessions
closed Book/Notes]

Topics covered till

Session 10 from
beginning

20%

Quiz-II

To be informed By To be informed By
Prof. S. K. De.
Prof. S. K. De.

20%

End Term

After all 20
sessions

40%

Time Line

Entire Course

Class Participation and Evaluation

Total Marks for CP is 0 but could go down to -10

Class Test will be taken if Total Attendance is less than 45 in a day in a

section

Marks may or may not be used during final grading

Please Ask Questions in Class if necessary and Kindly Listen to Others

No Two Persons Should Speak at the Same time and Penalty of 1

mark/class may be imposed if class become Fish Market

Please refrain from asking a question that two of your friends have
already asked Rather meet personally and ask for more
clarifications

Tentative Grading Pattern Subject to

Acceptance by Deans Office
Relative Grading

Grading
System
Rating

Fail

Grade

D+

C+

B+

A+

Quality
Points

** [10- [20- [34- [50- [75- [9020) 34) 50) 75) 90) 100]

Equivalent Academic
Percentiles Indiscipline

Poor

Average

Good

Excellent

* To be decided based on score: No D for score over and above 25

** To be decided based on score: No D+ for score over and above 40
6

Appointment
Always Welcome
Fix an Appointment by sending a Direct Email to me
No Voice Calling is entertained, in any form
Specially, Ill reserve Wednesday late evening to meet
students

No correspondence via Class Representative or any other channel

Bargaining for Marks/Grades will not be entertained and may be
awarded -2 to -5.
Human Error in Marking, such as, in omission/counting, if any,
will be corrected
7

Theory of Probability
Session-1

Why Should We Teach and Learn

Theory of Probability
in A Premier B- School?

Uncertainty and Inductive Reasoning

Unavoidably an element of uncertainty is always present regarding
the truth or correspondence with facts of the conclusion reached
through inductive reasoning.
Pure Mathematics or Applied Mechanics you have studied so far
are broadly based on prolongation of deductive reasoning.
In deduction, given the premises, the conclusion necessarily follows
from them.
If a piece of deductive reasoning is free from fallacy, its conclusion
is formally valid.
10

Uncertainty and Inductive Reasoning

If the premises are materially valid so is the conclusion, but
deduction as such is not concerned with material validity.
In induction the premises only lend some support to the
conclusion relate to the contingent (i.e. situated in space and
time) world.
Apart from formal validity, the question of the material
validity or validity as the basis of practical action of the
conclusion (over and above the same of the premises)
naturally arises.
11

Degree of Uncertainty

In the case of every exercise at induction, the question of

assessing the degree of uncertainty, or in other words, the
extent of support given to the conclusion by the evidence, is
relevant.
Until the operations generating the observations (including
additional observations if extensibility is assumed) are
performed, the evidence (extended evidence) that would be
realized remains uncertain and this uncertainty can be
assessed in terms of probability.
12

Degree of Uncertainty

In the case of every exercise at induction, the question of

assessing the degree of uncertainty, or in other words, the
extent of support given to the conclusion by the evidence, is
relevant.
Until the operations generating the observations (including
additional observations if extensibility is assumed) are
performed, the evidence (extended evidence) that would be
realized remains uncertain and this uncertainty can be
assessed in terms of probability.
13

Uncertainty Objective Approach

In controlled experiments where a set of units are subjected
to different treatments before observation, allocation of the
units to the treatments is similarly randomized.
Example:
1. Inspection of diameters of cork stoppers in a productionline
2. Controlled clinical Trials

14

Uncertainty Objective Approach

Uncertainty in the evidence may arise due to one or more of
the causes, natural variation, errors of measurement,
sampling variation (incorporating any randomness
deliberately introduced), unforeseen contingencies etc.

15

Uncertainty Subjective Approach

In the subjective approach there is no question of repetition
of observations.
Here uncertainty only means absence of knowledge about
the evidence and extended evidence, before the generating
operations are performed.
Because of this, the scope for induction is somewhat wider
in the subjective than in the objective approach.
Example:
1. Number of working hours that might be wasted due to
contact labour strike in next six months.
2. Exchange rate at the next morning
16

Definition of Probability
Various Aspects

Meaning of probability
As regards probability which expresses the uncertainty about the
observables, it is given radically different interpretations in the
objective and subjective approaches.
In the former, roughly speaking, we assume in effect that the
unpredictable variation of the evidence is such that the relative
frequency with which it would belong to any meaningful set in the
evidential domain would tend to stabilize around some idealized
value, if the number of repetitions were made indefinitely large.
The meaningful sets, technically called measurable sets, are those
which are of practical interest and are theoretically easy to handle

18

Meaning of probability
The basis of this assumption, which we call frequential regularity
is our experience with diverse types of particular repetitive
experiments.
This is commonly called statistical regularity [A misnomer]
For any set of interest the probability that the uncertain evidence
will belong to it is identified with the corresponding idealized
long-term relative frequency.
Probabilities, so defined, of all meaningful sets in the evidential
domain determine a probability distribution over the domain and
this gives an objective representation of the evidential uncertainty.
19

Meaning of probability
In the subjective approach probability exists only in one's mind and
may vary from person to person.
For a particular person the probability of any set of interest
represents the person's degree of belief in the materialization of the
event that the evidence (extended evidence) generated through the
operations when they are performed would belong to that set.
In practice this degree of belief can be quantified introspectively,
e.g. by ascertaining the maximum price one is prepared to pay
outright for a unit gain contingent on the actual realization of the
event.
20

Meaning of probability
Ideally one should attach numerical degrees of belief to
different sets of interest in a consistent or coherent manner.
Coherent probabilities for different meaningful sets in the
domain define a probability distribution over it.
Since uncertainty here means absence of knowledge, such a
probability distribution may cover evidence extended
backward or collaterally to involve unobserved characters
belonging to the past or the present.
21

Gambling and Games of Chances

A Fascinating History of Development of
Theory of Probability

Cardanoan unrecognized pioneer

Gerolamo Cardano
24 September 1501 21 September
1576
A renowned physician, mathematician,
astrologer, and an inveterate gambler

23

Cardanoan unrecognized pioneer

He wrote a book entitled Liber de Ludo Aleae (The Book on Games
of Chance) around 1564
The book remained unpublished possibly because of various
misfortunes and tragedies that befell the author towards the end of
his life and saw the light of day only in 1663.
Cardano suffered a number of other tragedies as well. Cardano's son
Giambatista poisoned his wife.
Cardano was jailed briefly for heresy (in part for casting the
horoscope of Jesus).
Cardano supposedly predicted the date of his own death, a
prediction that he perhaps ensured by suicide.
24

Basic Ideas and Rules of Probability Theory:

Conceptualized by Cardano
1. The chance of an event in a random trial represents its longrun relative frequency.
2. If a die is honest its different faces have equal chance of
appearing.
In fact Cardano makes the statement, I am as able to throw 1,
3, or 5 as 2, 4, or 6 which suggests that he had something like
propensity in mind. From this he identifies the set of equally
likely cases (the sets of all 36 or 216 permutations) when two or
three honest dice are thrown.
He uses the term circuit for such a set.
25

Basic Ideas and Rules of Probability Theory:

Conceptualized by Cardano
3. When the circuit for a trial is well-identified, the chance of an
event is represented by the portion of the whole circuit favourable
to it.
Cardano gives the rule that to obtain the odds we have to consider
in how many ways the favourable result can occur and compare
that number to the remainder of the circuit

4. Cardano correctly uses the rule for addition of probabilities in terms

of disjoint events.
In throwing two dice of 36 equally likely cases, 11 are favourable to
the event at least one ace, 9 additional cases become favourable if
we take the larger event at least one ace or deuce, 7 further cases
come if we consider the still larger at least one ace, deuce, or trey
and so on.
Similar computations are made for three dice.
26

Basic Ideas and Rules of Probability Theory:

Conceptualized by Cardano
5. Cardano also correctly formulates the product rule for computing
the chance of the simultaneous occurrence of events defined for
independent trials Details will be discussed later.
6. In the case of throwing two dice the odds on getting at least one ace,
deuce, or trey are 3:1.
Cardano states that if the player who wants an ace, deuce, or trey
wagers three ducats [a standard unit of currency at that time] and
the other player one, then the former would win three times and
would gain three ducats and the other once and would win three
ducats; therefore in the circuit of four throws [impliedly in the long
run] they would always be equal.
27

Galileo Galilei Sought to resolve a

puzzle about a dice game
Galileo Galilei
15 February 1564 8 January 1642

One of the pioneers in introducing

experimental methods in science

28

Basic Ideas and Rules of Probability Theory:

Conceptualized by Galileo
In throwing three dice, the numbers of unordered partitions producing the
total scores 9 ({1, 2, 6}, {1, 3, 5}, {1, 4, 4}, {2, 2, 5}, {2, 3, 4}, {3, 3, 3})
and 10 ({1,3, 6}, {1, 4, 5}, {2, 2, 6}, {2, 3, 5}, {2, 4, 4}, {3, 3, 4}) are
both equal to 6. Yet, why is it that long observation has made dice-players
consider 10 to be more advantageous than 9?
Galileo pointed out that there is a very simple explanation, namely that
some numbers are more easily and more frequently made than others,
which depends on their being able to be made up with more variety of
numbers.
A variety of numbers making up a score here represents an ordered
partition. There being 27 such ordered partitions for the score 10 and 25
for the score 9 and all ordered partitions or permutations being equally
likely, the chance of getting a 10 is higher.
29

Probability is officially bornPascal

and Fermat
Blaise Pascal
19 June 1623 19 August 1662
A French (Parisian) mathematician,
physicist, inventor, writer and Christian
philosopher.
Pascal solved some problems on Games of
chances including the failed attempts of
Cardano through correspondence with his
friend, Pierre de Fermat (16011665) stationed at Toulouse
30

Probability is officially bornPascal

and Fermat
Pierre de Fermat
17 August 1601 (or 1607) 12
January 1665
Although a jurist by profession, Fermat had
become famous for his contributions to
mathematics and the other branches of
knowledge

31

First Published Book on Probability

Christiaan Huygens
14 April 1629 8 July 1695
A was a prominent Dutch mathematician
and scientist. He is known particularly as
an astronomer, physicist, probabilist and
horologist.
Wrote the book entitled De Ratiociniis in
Ludo Aleae (Computations in Games of
Chance), published in 1657 - The first
published book on probability
32

Applications: Probability in Finance :

Consider only two players, they alternate moves, each is
immediately informed of the others moves, and one or the
other wins.
In such a game, one player has a winning strategy, and so we
do not need the subtle solution concepts now at the center of
game theory in economics and the other social sciences.
Reference: Probability and Finance: Its Only a Game!, by
Glenn Shafer and Vladimir Vovk. 2001 by JohnWiley & Sons,
Inc.

33

Probability in Finance :
Consider a straightforward but rigorous framework for elaboration, with no
extraneous mathematical or philosophical baggage, of two ideas that are
fundamental to both probability and finance:
The Principle of Pricing by Dynamic Hedging : [Can be discerned in the
letters of Blaise Pascal to Pierre de Fermat in 1654] When simple gambles
can be combined over time to produce more complex gambles, prices for
the simple gambles determine prices for the more complex gambles.
The Hypothesis of the Impossibility of a Gambling System: Sometimes
we hypothesize that no system for selecting gambles from those offered to
us can both (1) be certain to avoid bankruptcy and (2) have a reasonable
chance of making us rich.

34

Probability in Marketing :

A company might like to estimate the probability for the

increase in volume of sales by Rupees ten million given a
particular marketing campaign.

Probability models are used to measure consumer lifetime

value.

35

Theory of Probability
Session-2

Calculus of Probability :
Connections with Set Theory
Set Theory

Probability Theory

Notations

Element

Outcome / Elementary Event

Set

(Compound) Event Collection of

Elementary Events

Universal Set

Sample Space or Sure Event

 or

Null set

Impossible Event

Complement of a set A

Complementary event of A

is a subset of
( is a superset of )

Occurrence of event implies

occurrence of event




Union of sets and

Occurrences of events or

37

Calculus of Probability :
Connections with Set Theory
Set Theory

Probability Theory

Intersection of sets and Joint occurrences of events

and

38

Notations

and are disjoint set

and are mutually

exclusive events

and are exhaustive set

and are exhaustive

events

Power Set - The set of

all subsets of S, including
the empty set and S itself

(Countable) Sigma- Field

Classical Definition of Probability :

As in Thorie analytique des probabilits
by Pierre-Simon Laplace
The probability of an event is the ratio of the number of cases
favorable to it, to the number of all cases possible when
nothing leads us to expect that any one of these cases should
occur more than any other, which renders them, for us,
equally possible
The Probability of an event A is defined a-priori without
actual experimentation as



provided all these outcomes are equally likely.
39

Simple Examples
Consider a box with n white and m red balls. In this case,
there are two elementary outcomes: white ball or red ball.
Probability of selecting a white ball is

We can use classical definition to determine the probability

that a given number is divisible by a prime p. If p is a prime
number, then every pth number (starting with p) is divisible
by p. Thus among p consecutive integers there is one
favorable outcome, and hence

 

40

Frequentist Definition of Probability :

The frequentist view may have been foreshadowed by

Aristotle, in Rhetoric, when he wrote:

the probable is that which for the most part happens

41

Frequentist Definition of Probability :

In the frequentist interpretation, probabilities are discussed
only when dealing with well-defined random experiments (or
random samples).
The set of all possible outcomes of a random experiment is
called the sample space of the experiment.
An event is defined as a particular subset of the sample space
to be considered.

42

Frequentist Definition of Probability :

For any given event, only one of two possibilities may hold: it
occurs or it does not.
The relative frequency of occurrence of an event, observed in
a number of repetitions of the experiment, is a measure of the
probability of that event.
This is the core conception of probability in the frequentist
interpretation.

43

Frequentist Definition of Probability :

Thus, if t is the total number of trials and is the number
of trials where the event occurred, the probability of
the event occurring will be approximated by the relative
frequency as follows:

  

Clearly, as the number of trials is increased, one might expect

the relative frequency to become a better approximation of a
"true frequency".

44

Frequentist Definition of Probability :

A claim of the frequentist approach is that in the "long run,"
as the number of trials approaches infinity, the relative
frequency will converge exactly to the true probability:


  


45

Travelers Choices
There are three major possible options available to a travel
agency for its customer who wants to travel to New Delhi
from Jamshedpur
Direct train to New Delhi
By Road/Rail to Ranchi and Flight from Ranchi
By Rail/road to Kolkata and Flight from Kolkata
Agency has records of previous bookings in the same route
over last few years that will help them to assess probable
choice of customers
46

Other Applications
Proportion of loan application rejected by a a major bank from
SMEs micro, small and medium-sized enterprises
Proportion of defective items produced by a manufacturing unit
In estimating life time of a product, proportion of electric bulb
survived after 1000 hours in operations.
In fact, estimating probability of survival of a
electric/electronic device after certain hours, we can actually
have a lifetime distribution that we study later.
Not only for consumer durables, we may think of Consumer
lifetime as well.
47

Combinatorics :
Arrangement of r balls in n cells

Four possible cases according to

Whether balls are distinguishable of not

Whether Exclusion principle followed (cells cannot more

than one ball) or not

48

Combinatorics :
Arrangement of r balls in n cells
Exclusion principle
followed

Exclusion principle not

followed

 (Maxwell-Boltzman

 Statistics)

Balls are

indistinguishable

(Bose-Einstein Statistics)
(FermiDirac
Special Case:
statistics)
No cell remain empty:

Balls are
distinguishable

49



Application
A random sample of size with replacement is taken from a
population of elements. What is the probability that in the
sample, no element appears twice, that is, the sample could
have been realized also by sampling without replacement?
We see that there are possible sample in all of which
satisfies the stipulated condition. Assuming that all
arrangements have equal probability, we conclude that
probability of no repetition in our sample is 

50

Industrial Implications
If in a coal mines, 12 accidents occur in each year, then
practically all year will contain months with two or more
accidents. The probability that all months will have one
accident each is only 0.0000537.
On the average, only one year out of 18614 years, will
show a uniform distribution of one accident per month
This example reveal an unexpected characteristic of pure
randomness
This type of argument is often used for fraud detection
51

Extensions
The number of ways to deposit distinct objects into cells
with objects in cell no. ( are non-negative integers
summing to )is


(ordering of bins is important but within each bin the
ordering is not important).

52

More Example
A throw of twelve dice can result in different outcomes
which we consider equally likely. The event that each face
appears twice can occur in as many ways as twelve dice can
be arranged in six groups of two each. The probability of
that event is therefore

53

Application In Industrial Quality Control

Items are sampled from a collection of items and inspected for
defects. Assume that there are n defective items in the lot of
items. What is the probability of sampling defective items
out of items?
Problems of these types lead to genesis of hypergeometric
distribution.
In practice, the total population size as well as and are
known but the number of defected items in the population is
unknown.
o The latter may be estimated by maximizing the likelihood of the sample,
and may be given confidence interval using standard statistical estimation.
54

Estimating Population Size of Fish in a

Lake [capture-recapture ]
Consider the following experiment in an attempt to estimate
the number of fish in a lake. First, fishes are captures,
marked, and released. At a later time, fishes are caught with
of them bearing the mark of the original capture. Assuming
the size of the population of fishes is , the probability of
getting marked fishes in the second capture is

In this case, are known but is unknown. We can estimate or

construct confidence intervals using the likelihood (probability of
observed data as a function of the unknown parameter ).
For example, if       - we have approximately 93%
confidence interval that belongs to (8500; 12000).
55

Criticism of
Classical Definition of Probability :
Mathematicians find the definition to be circular.
The probability for a "fair" coin is... A "fair" coin is defined by a
probability of...
The definition is very limited. It says nothing about cases where no
physical symmetry exists.
Insurance premiums, for example, can only be rationally priced by
measured rates of loss.
It is not trivial to justify the principle of indifference except in the
simplest and most idealized of cases. Coins are not truly symmetric.
Can we assign equal probabilities to each side? Can we assign equal
probabilities to any real world experience?
56

V - algebra
A non-empty collection of subsets of Sample space is
called a sigma algebra (or Borel field for events over real
line), denoted by  B), if it satisfied the following two
properties:
a. If , then
( is closed under complementation).
b. If , , , then

( is closed under countable unions).
To show that is closed under finite unions
To show ( the empty set is an element of ) and
( the sample space is an element of ).
57

V - algebra
Easy to realize that the superset generated by countable
sample space : is always a V - field.
Let :  :. Can we
consider as a V - field?
Probability is a measure (set-function) defined on :
: is known as probabilizable space

58

More on V - algebra
Example-1. (Sigma algebra-I) If S is finite or countable, we
define for a given sample space S, B = {all subsets of S,
including S itself}. If S has n elements, there are sets in
B. For example, if S = {1, 2, 3}, then B is the following
collection of 2 3 = 8 sets: {1}, {1, 2}, {1, 2, 3}, {2}, {1, 3},
, {3}, {2, 3}.
Example-2. (Sigma algebra-II) Let S = (, ), the real line.
Then B is chosen to contain all sets of the form [a, b], (a,
b], (a, b), [a, b) for all real numbers a and b. Also, from the
properties of B, it follows that B contains all sets that can
be formed by taking (possibly countably infinite) unions
and interactions of sets of the above varieties.
59

Theory of Probability
Session-3

Axiomatic Definition of Probability

[By Andrey Kolmogorov]
Probability of an event A, denoted by P(A) is a set function (also
called a measure ) with sample space and V-field (also called event
space) satisfying the following axioms:
Axiom of nonnegativity: The probability of an event is a non
negative real number:

Axiom of Unity: the probability that at least one of the elementary
events in the entire sample space will occur is 1. More specifically,
there are no elementary events outside the sample space.

Axiom of Countable Additivity: For any countable sequence of
disjoint (synonymous with mutually exclusive) events ,

 

61

Andrey Kolmogorov
Andrey Nikolaevich Kolmogorov
25 April 1903 20 October 1987
A 20th-century Russian mathematician
who made significant contributions to the
mathematics of probability theory,
topology, intuitionistic logic, turbulence,
classical
mechanics,
algorithmic
information theory and computational
complexity.

62

Important Results Follows From

The Probability axioms
Result-1: For the impossible event we have necessarily  .
Result-2: Probability function P is finitely additive; that is, if
(for ) and if these events are disjoint, then

Result-2.A. If be exhaustive and mutually

exclusive events in , then

Result-2.B. Rule for Complementary Probability of any event 

For any event ( )

Important Results - Continued

Result-3. The probability function P is monotone; that is, if and
are events in , such that then .
The numeric bound: It immediately follows from the
monotonicity property that for  ,  

Result-4. The probability function P is subtractive; that is, if

and are events in , such that then

Important Results - Continued

Result-5. Rule for Union of Probability for any events  

not necessarily mutually exclusive: If and are any two events
in , then

Imagine the rule for three or more events

Example
A company recruits 10 students for summer internship from twoyear full-time BM programme of XLRI for four functional areas,
namely, Analytics, Finance, Marketing and Operations. The
students are nearly equally efficient in terms of their expertise in
each of these four functional areas. On the first day of their
reporting, The HR manager in a hurry, allotted them almost at
random to the four functional areas, without taking much care
about the requirement of various areas. What is the probability
that Analytics area will receive exactly 4 of the students?

Hint Answer
A company recruits 10 students for summer internship from twoyear full-time BM programme of XLRI for four functional areas,
namely, Analytics, Finance, Marketing and Operations. The
students are nearly equally efficient in terms of their expertise in
each of these four functional areas. On the first day of their
reporting, The HR manager in a hurry, allotted them almost at
random to the four functional areas, without taking much care
about the requirement of various areas. What is the probability
that Analytics area will receive exactly 4 of the students?
Check:

Example
A large shopping complex has 15 entry gates. Usually, one security
personnel is deployed to each of these gates. Security personnel can
usually chat with their colleagues deployed in the adjacent (both right
and left) gates. The personnel in 1st and 15th gates will be able talk
with only one of their colleagues. It was observed from past CCTV
footages that two personnel, say, and whenever deployed in
adjacent gates gossips more and do not take the job seriously!
Management has ordered the chief-security officer that and
should be so deployed that there should be 10 other personnel in
between them. On one day, the chief-security officer was absent and
another person who had no idea about the order, allotted 15 personnel
in 15 gates at random. What is the probability that the requirement
will be met even in that case?

Hint Answer

Required Probability =

When Exact Probability is Untraceable

Result-6. Booles Inequality If be any events in ,

then

Result-7. Bonferronis Inequality If be any events in

, then

Example
Over the years, the culture of binge drinking spread in premier
the B-schools across the globe despite honest effort of various
managements to curb irresponsible drinking behaviour of
students. After a booze night in a hostel, it was reported that
80% of the student of one Hostel consumed Beer, 70% enjoyed
Whisky and 60% relished Vodka. What is the proportion of
stalwarts who tried all three in that night?

Union of More than Two

Events

Poincares Theorem
For any sequence of events  , (not
necessarily mutually exclusive )

For proof, use induction method starting from Result-5.

Example of Booze Night - continued

It was further reported that because of the popular perception

that Whisky after Beer, there is no fear; Beer after Whisky, it
is risky; 60% of the students tried Whisky after Beer but no
one tried Beer after Whisky. Moreover 50% tried both Beer
and Vodka and 40% actually tried Baseball Pleasure (a
cocktail of Vodka and Whisky). About 35% tried all three in
someway or other. What is the probability that a student did
not drink at all in the Booze night?

Conditional Probability

Perception of Conditioning in Random

Experiment: Relative Frequency Context
In N independent trials, suppose denote the
number of times events A, B and AB occur respectively.
According to the frequency interpretation of probability, for
large N




Among the NA occurrences of A, only NAB of them are also

may
found among the NB occurrences of B. Thus the ratio

be looked upon as a measure of the event A given that B has

already occurred. Now,

76

Definition of Conditional Probability

Consider the probability space The conditional
probability of an event given that another event also
belong to same has occurred, denoted as is defined
as by

provided .
If
The above definition satisfies all probability axioms discussed
earlier [Please Check by yourself]
77

Justification of the Definition of Conditional

Probability in the Light of Three Axioms
(i) by definition and by axiom of nonnegativity.
Therefore,

(ii) Note that :   Therefore,


:

(iii) Suppose are mutually disjoint for all . Then for any

Therefore

Hence satisfies all probability axioms and thus, defines a legitimate

probability measure.
78

Theory of Probability
Session-4

Properties of Conditional Probability

(i) If
If    Hence,

Since the occurrence of B implies automatic occurrence of the

event A.

Example: Probability that a G20 member is selected given that it is a

Brics member
A=A All 20 members ={Argentina, Australia, Brazil, Canada, China,
France, Germany, India, Indonesia, Italy, Japan, South Korea, Mexico,
Russia, Saudi Arabia, South Africa, Turkey, UK, USA, EU}
B= All Brics member = {Brazil, Russia, India, China, South Africa}
80

Properties of Conditional Probability

(ii) If
If
  
Hence,

(iii) The Law of Compound Probability.

When expressed in the product form we get,
_
81

Theorem of Compound Probability

When we have 3 events A, B and C, we have
_
_ _
By an easy induction we obtain for n events we obtain for n
events

82

Probability of Complementary Event

with Conditioning Event
Suppose that, given , either or both belong to
can take place.
So we have

We complement the event that is conditioned, not the

conditioning events in computing probabilities.

83

Law of independence

How to interpret the equation: 


It shows that As occurrence has had no impact on B. We say
then that B is independent of A.
We now ask the following: If B is independent of A, then is A
also independent of B?
The answer is yes, as the equation _  also
implies the equation _
Thus the relationship of independence is symmetric. So from
now on we shall say A and B are independent events,
whenever anyone is independent of the other.
85

Testing Independence of Two Events

This will mean
_
_ 
and

To show that A and B are independent events, we may verify
any of the above three equations.

86

Testing Independence of Two Events

Example : Three coins are tossed

 first coin is heads;  second coin is heads

Then and

So,

This verifies that A and B are independent events.

87

Testing Independence of Two Events

If and both belongs to and are mutually independent,
then
 and are independent
 and are independent
 and are independent

88

Difference between Mutually Exclusive

Events and Independent Events
Note carefully that, if A and B are mutually exclusive, then
. From the definition of conditional probability,
we see that
_ _
From this, we would have, respectively
_ _  
Thus, unless either A or B is the null eventI, A and B are not
independent when they are mutually exclusive. Alternatively, if
A and B are mutually exclusive, the occurrence of B must depend
on A, since if A occurs, then B can never do so. If either A or B
is equal toI, then A andI, or B andI, are independent.
89

Difference between Pairwise Independence

and Complete Independence of Events
If A, B and C are three
independent if

events in they will be pairwise

A, B and C will be completely independent if along with above

three equations following also holds

How many equations need to be satisfied, if events
have to be completely independent?
90

Some Important Theorems

Theorem of Total Probability

There are three machines that are producing cork stoppers in a
manufacturing unit. One machine is old and produce about 5%
defective. Other two machines respectively produce 2% and 3%
defective items. Probability of having a defective item at the
time of inspection is actually connected with
I.

The machine in which that cork-stopper is produced

II. Proportional contribution of three machine in total

production of cork stoppers under inspection
We can use the conditional probability to express the probability
of a complicated event in terms of simpler related events. The
theorem of total probability helps us in achieving this.
92

Theorem of Total Probability

If a sequence of events     forms a
finite or countably infinite partition of a sample space (in other
words, set of events are exhaustive as well as mutually
exclusive) and provided for each , then for any
event  

     

If  for some , we should take away those events

and work with the rests.
93

Theorem of Total Probability

The summation can be interpreted as a weighted average, and
consequently the marginal probability, , is sometimes
called average probability.

Special Case: If so that   

    are both defined, then,
         

94

and

Example: Theorem of Total Probability

Suppose that two factories supply light bulbs to the market.
Factory X's bulbs work for over 5000 hours in 99% of cases,
whereas factory Y's bulbs work for over 5000 hours in 95% of
cases. It is known that factory X supplies 60% of the total
bulbs available. What is the chance that a purchased bulb will
work for longer than 5000 hours?

95

Bayes' Theorem

96

Laplace form of Bayes Theorem

L      be a sequence of exhaustive and
mutually exclusive events in s for each
, then for any event  
  

    
provided, of course, .

97

Rev. Thomas Bayes

Thomas Bayes (c. 1701 7 April 1761)
An English statistician, philosopher and Presbyterian minister
Bayes never published what would eventually become his
most famous accomplishment
His notes were edited and published after his death
by Richard Price.

98

Application of Bayes Theorem

The entire output of a factory is produced on three machines.
The three machines account for 20%, 30%, and 50% of the
output, respectively. The fraction of defective items
produced is this: for the first machine, 5%; for the second
machine, 3%; for the third machine, 1%. If an item is chosen
at random from the total output and is found to be defective,
what is the probability that it was produced by the third
machine?

99

Bayesian (or epistemological) interpretation

of the Theorem
Measure of a degree of belief: Bayes' theorem links the degree of
belief in a proposition before and after accounting for evidence.
Example: Suppose it is believed with 50% certainty that a coin is
twice as likely to land heads than tails. If the coin is flipped a number
of times and the outcomes observed, that degree of belief may rise,
fall or remain the same depending on the results.
For proposition A and evidence B,
P(A), the prior, is the initial degree of belief in A.
P(A|B), the posterior, is the degree of belief having accounted for B.
the quotient P(B|A)/P(B) represents the support B provides for A.
100