1

Introduction to probability
Statistics is concerned with making inferences about a population on the basis of information
contained in an observed sample (a part of the population). Probability theory is the tool that
enables us to infer about population characteristics using the corresponding sample
characteristics.
The word probability may be used in two different contexts. First, it may be used in regard to
some proposition. Take, for instance, the statement: It is very probable that India will adhere to
the democracy system of government till the end of this century or It is very improbable that
the centurys brain drain will stop in the future. Probability here means the degree of belief in
the proposition of the person making the statement. This may be called subjective probability.
Alternatively, the word may be used in regard to the outcomes of an experiment that can,
conceivably be repeated an infinite number of times under essentially similar conditions.
The probability of an event here refers to the proportion of cases in which the event occurs in
such repetition of the experiment. This type of probability may be called objective, being a part
of the real world, and it is with sense of the word that we shall be concerned in the present
discussion.
Probability plays an important role in decision-making. To illustrate, suppose you have an
opportunity to invest in an oil exploration company. Past records shows that for ten out of ten
previous oil drilling (a simple of the companys experiences), all ten resulted in dry well. What
do you conclude? Do you think the chances are better then 50-50 that the company will hit a
production well? Should you invest in this companys? We think your answer to this question
will be emphatic no. If companys exploratory prowess in sufficient to hit a production well 50%
of the time, a record of ten dry wells out of ten drilling is an event that is just too improbable. Do
you agree?
Probability may be loosely interpreted as the measure of the degree of ones belief in the
occurrence of a future event. We see people making informal probability statement such as
follows: it is likely that the Bombay stock exchange (BSE) Sensex will touch the 18000 marks
by the end of the year . There is fifty -fifty chances that India will win the India Australia test
series. The odds do not favor a quick settlement of the labor strike . It is expected that at
least Rs.500,000/-will be raised at a charity football match .Probability theory deals with
methods for quantifying chances of various possible outcomes in a particular situation and allow
one to express informal statement such as given above in a precise manner.
Sources:

2
1.
2.
3.

Fundamentals of Statistics- Vol I by A. M. Goon, M, K Gupta, B. Dasgupta

Statistics for Management: Levin and Rubin
McClave & Benson

A probabilistic model for an "experiment"

2.1. Some Definitions:

Experiment: An act that can be repeated under similar conditions

Random Experiment: An experiment whose all possible outcomes are known in advance
but which one will occur exactly in a particular repetition is not known in advance.
Examples:
1.
2.
3.
4.
5.
6.

Toss a coin
Choice of a perfume to gift
Selection of Pizza toppings for the next party
DJ hours to be booked for next batch-party
Sales of Ford in India
Lifetime of a laptop battery

A random experiment where the variation in the outcomes could be controlled (but cannot be
made zero) is called a controlled experiment. For example, budget may control in case of
examples 2, 3 and 4 above. However, most of the real cases are beyond control of the
surveyor/experimenter. For example, economic growth of a country, stock market index
movement, impact of an advertisement are such examples. These types of random experiments
are called uncontrolled experiments and in particular Econometric experiments in relation
with economics and business.

Sample space and sample point:

Given a particular random experiment, the collection of all possible outcomes is called the
sample space, which is usually denoted by or S. An element (or a member) of the sample
space is called as the sample point.
Event:
Any subset (or sub-collection) of a sample space . is called an event. Usually, events are
denoted by A, B, C, etc. or A1, A2, A3 etc.

For example, suppose a coin is tossed twice.

A: .there is at least one head.
Event A can be decomposed into following simpler events:
A1 : Head in bo.th the tosses (H1 H2=HH)
A2: Head in first toss and tail in second toss (H1 T2:=HT)
A3: Tail in first toss and head in second toss (T1 H 2=TH)
a) Simple or elementary events: An event consisting of a single sample point is called a simple
or elementary event. Such an event cannot be further decomposed in the context of the
experiment.
In example above there are four simple events HH, HT, TH, TT.
b) Compound event: An event composed of two or more sample points is called a compound
event.
c) Impossible event (): An event, whose occurrence is impossible, in the context of a random
experiment is called an impossible event.

Example: A coin is tossed twice. Thus. = {HH, HT, TH, TT}, B = {HHH} =

Some operations with events

(Use Venn diagrams to verify)
,
1) Union :

AU B is the event which occurs if at least one of A and B occurs.

Suppose A2... An are n events in. in the context of some experiment. Then
A 1 A 2 An = i=1 n Ai

, Ai is the event which occurs if at least one of A1, A2... An

occurs.

2) Intersection:
A B occurs if A and B occur together.
i=1 n A i

is the event which occurs if A1, A.2,..., An occur simultaneously.

3) Differences:
A-B is the event which occurs, if A occurs but not B. Similarly, B-A implies B occurs but not A.
Note that A-B B-A
Symmetric differences:
A B = (A- B) U (B- A) =>Exact1y one of A or B occurs. Here only either (A-B) or (B-A) can
occur.
4) Complement of an event:
Denoted as =Ac = A; it represents non-occurrence of event A.

Rules on these operations

A) Associativity: If there are more than two events and the union and intersection operations are
considered, then the order of these doesn't matter.
(A U B) U C = A U (B U C)
(A B) C = A (B C)
b) Commutativity: (A U B) U C = A U (B U C)

C) Distributivity: (A U B) C = (A C) Uection
(B gets distributed)
Similarly (A B) U C = (AU C) (BU C)
d) Idempotency: AUA=A AA=A
e) De Morgan's laws:
(AUB)c=Ac Bc
(AB)c=Ac U Bc

Some important results

1.Ac=-A
2.A-B=ABc
3.AU =
4.A=A
5.AU=A
6.A=
7. c =

8.c =
9.AUAc =
10.AAc =

Some special types of events:

a) Mutually exclusive (disjoint events)
A and B are mutually exclusive if and only if they cannot occur together, i.e., if and
Only if A B =
A1, A.2, ..., An. are mutually exclusive if arid only if no two of them can occur
Simultaneously, i.e., if and only if Ai Aj= , i<j & i,j=l,2,..n
b) Exhaustive events: A1, A.2, ..An are exhaustive for if A1 U A2 U ..U An= .
c) Partition of a sample space: A1, A.2, ..., An form a partition if they are mutually exclusive as
well as exhaustive for
[For example, AUAc = . => A and AC form a partition]

Probability as the long-run relative frequency

Suppose an experiment is (hypothetically) repeated n times. Consider some event 'A' which may
occur in the context of the experiment and A, A

Pn (A) =

fn(A)
= relative frequency of occurrence of Event A
n

fn(A) : the frequency of occurrence of event A

As n, consider the limit of Pn(A). If the limit exists, it is called probability of occurrence of
event A, i.e., P(A) by definition is limnPn (A) .

Observe that
a) For any event A, 0 P(A) 1
b) If A and B are mutually exclusive, P(A U B) = P(A) + P(B). Generally, if A1 A2.... are mutually
exclusive, then
i=1

P ( Ai ) = P( Ai)
i=1

c) For a sure event , P()=1.

Remark: The axiomatic approach developed by Kolmogorov is based on observations a), b) and
c) and these are treated as axioms. Using a), b) and c) as axioms one can obtain the following
results d)-i):
d) Let B A, i.e., B is contained in A then P(B)P(A)
e) If B A as well as A B, then A B
f) If A B, P(A) = P(B) this is because, A B => P(A) P(B) & B A => P(B)
P(A) => P(A)= P(B)
g) Converse of (f) is not true I.e., if P (A) = P (B) does not imply that A B. For example,
consider tossing a fair coin twice, the event A is at least 1 head and event B is at least 1 tail.
h) If B A

A, then P (A-B) = P (A)-P (B). This property is called subtractivity of

'probability Function.
Proof: If B A, A= (A-B) U B where A-B and B are mutually exclusive.
=> P (A) = P [(A-B) U B] = P (A-B) + P (B) , by axiom 2 => P (A-B) = P(A)-P(B)
1) General result: For any two events A and B, P (A-B) =P (A)-P (A B)

Classical Definition of Probability

Suppose is discrete and finite having N ( ) elementary events and all N elementary events
are equally likely. Then, probability of an event A is defined as P (A) = N(A)/N where N(A) is
the number of elementary events favoring the occurrence of event A.

But the sample points may be infinite in number and they may not be equally likely. In such a
general case one can employ what is called the "sample point method" to compute the probability
of A.

Sample points of a sample space are naturally assigned (nonnegative) probabilities such that
the assigned probabilities add up to 1. Then the probability of an event A is calculated summing
the probabilities assigned to the sample points in A.
Examples:
1. The odds are two to one that when A and B play tennis, A wins. Suppose A and B play two
matches. What is the probability that A wins at least one match?
Solution: here = {AA, AB, BA, BB}, where, for example, AB implies A wins the first match
and b wins the second. Then assigned probabilities to the sample points AA, AB, BA, BB are:
(2/3)(2/3), (2/3)(1/3), (1/3)(2/3), (1/3)(1/3). Therefore, by the sample point method probability
that A wins at least one match = (2/3)(2/3) +(2/3)(1/3) + (1/3)(2/3)
1. A fair coin is tossed repeatedly until "heads" appear. Find the probability of requiring at most 4
tosses to get heads.
Solution: Here .Q = {H, TH, TTH, TTTH,}. Natural probabilities assigned to the sample points
H, TH, TTH, TTTH, are 1/2, 1/4, 1/8, 1/16, Therefore, by the sample point method, probability
of requiring at most 4 tosses to get "heads" = 1/2 + 1/4 + 1/8 + 1/16= 15/16.
3. Monty Hall Problem (New York Times, July 21, 1991): Monty Hall, a thoroughly honest
game show host, has placed a car behind one of the three doors. There is a goat behind each of
the other two doors. "First, you point to a door" he says, "then I will open one of the other doors
to reveal a goat. After I have shown you the goat, you make your final choice, either to stick with
the door you first selected, or switch over to the remaining door and you win what is behind that
door". Should you switch to improve your chances of winning the car?

Solution: We try to solve it by applying the sample point method. Without loss of generality,
suppose the car is placed behind door #1 and goats behind doors #2, #3. The contestant chooses
first either door#1 or '#2 or #3 with probability 1/3. In case the first door (with car behind)
is chosen then the host shows either door #2 or #3 with probability 1/6. Consider-two scenarios:
Scenario A when the contestant does not switch and Scenario B when he/she switches. For each
scenario, we list in. a table the sample points and the probabilities naturally assigned to them.
Scenario A. Contestant does not switch
Sample point
#
1

Door #1

Door #2

Door #3

(Car)

(Goat)

(Goat)

Contestant's

Results

Prob.
Assigned to
sample point

win

1/6

win

1/6

Host shows

loss

1/3

Contestant'
s

loss

1/3

Results

Prob.
Assigned to
sample point

Host shows

first choice
2

Contestant's

Host shows

first choice
3

Contestant's
first choice

Host shows

first choice
Scenario B. Contestant does not switch
Sample point
#
1

Door #1

Door #2

Door #3

(Car)

(Goat)

(Goat)

Contestant's

Switch

Host shows

Loss

1/6

Host shows

Switch

Loss

1/6

first choice
2

Contestant's

10

first choice
3

Switch

Contestant's

Host shows

win

1/3

Contestant'
s

win

1/3

first choice
4

Switch

Host shows

first choice

The total number of elements in an event A or may not be small or easy to list. In such a case
we need a counting system. If the sample points are finite and equally likely (See classical
definition of probability) then combinatorial mathematics may be useful.
Number of different samples of n elements from a population of N distinct objects
Case 1: Drawing with replacement => Nn
Case 2: Drawing without replacement, order matters => NPn (number of permutation)
N

Pn =

N!
( N n ) !

=(N-1).(N-n+1), where k! = k(k-1) (k-2) ..21

Case 3: Drawing without replacement, order matters => NCn (number of combination)

N!

Cn = ( N n ) ! n !

Examples:

Suppose you plan to invest equal amount of money in each of five common stocks. If you
have chosen 20 stock from which to make the selection, how many different sample of five
stocks can be selected from the twenty?

11

Solution: Answer in 20C5 because here we have a case of drawing without replacement where
order does not matter.

(From MWS) A company orders supplies from M distributors and wishes to place n distinct
orders. Assume that the company places orders in a manner that allows every distributor an
equal chance of obtaining anyone order and there is no restriction on the number of orders
that can be placed with any distributor. Let A be the event that a particular distributor,
say X, gets exactly k orders (k n). Find the probability of A.

Solution: Mn sample points in and all are equally likely. Event A has nCk (M-l)n-k sample points.
Therefore, by the sample point method, P (A) = nCk (M-l)n-k/ Mn.

We choose 10 items at a time at random from a lot consisting of 20 defective items and
80 non-defective items. What is the probability of finding exactly 4 defective and 6 nondefective items in the sample drawn?

Solution: 2OC4 8OC6/ l00c10.Here we have a case of drawing without replacement where order does
not matter.

An inspector visits 10 different machines during the day. In order to prevent operators

From knowing when he will inspect he varies the order of his visits. In how many ways
May this be done?
Solution: 10!

A personnel director for a corporation has hired ten new managers. If 3 distinctly different
positions are open at a Mumbai branch; in how many ways can the positions be filled?

Solution: 10P3
Case 4: Number of ways n distinct objects are divided into k distinct groups containing
nl,n2, ...,nk objects is n!/(nl! n2!...nk!)
Example: A fleet of 10 taxis to be dispatched to three airports in such a way that 3 go to airport
A, 5 go to airport B and 2 go to airport C. Taxis are allocated at random (a) II1J'!.9I.nany
Ways can this be done? (b) Assume that exactly one taxi .:i is
Probability that the taxi requiring repair is dispatched to Airport C?

12

Solution: a) 10!/(3!5!2!)
(b) [9!/(3!5!1!)] / [10!/(3!5!2!)]
2.2.2. Event Composition Method
Earlier the Sample Point Method of computing probabilities has been discussed. Often
it is not easy to determine the probabilities of the sample points of a sample space.
Instead one may have knowledge of probabilities of certain events. An event E may be expressed
as a composition of such events using union, intersection and complementation operations, and
then one can compute probability of E using some special formulae. This method of computing
probabilities is called the Event Composition Method.
Theorem: For any event A, P(Ac) = 1- p(A).
Theorem: For any two events A and B, P (A U B) = P(A) + P(B)- P(A B).
Proof (A U B) = (A - B)U(AB )U (B - A)
=> P (AU B) =P (A- B) + P(A B) + P (B - A)
=P (A)-P (A B) + P(A B) + P(B) P(BA)
=>P (AU B = P(A) + P(B) P(B A)
Corollary: If A and B are mutually exclusive, Then P (AU B) = P(A) + P(B)
Theorem: Suppose AI, A2, ..An are any n events, then

13

A consumer group is concerned about the possibility that there is systematic under weighing
in the meat department of a local supermarket. A representative is sent to purchase 1kg packages of ground meat. If there are currently 25 such packages on display and 8 of them
are actually underweight, what is the probability that at least 2 of the 5 packages purchased
are underweight?

Solution: 1-P (at most 1 underweight) = 1- (l7C4 8C1 /25C5)- (17C5 8C0 / 25C5)

The 3 most popular options on a certain type of car are automatic transmission (A), power
steering (B), and a radio (C). Suppose that 70% of all purchasers request A, 80%B, 75% C,
85% A or B, 90% A or C, 95% B or C and 98% A or B or C. Compute the probability that the
next purchaser will select (a) none of the three options, (b) only radio of the three options, (c)
exactly one of the three options.

Solution: (a) .02, (b) .13

A box has 20 balls, 12 red and 8 black. Three balls are drawn at random without replacement
from the box. Find probability
o That there are 2 red and 1 black ball drawn.
o That the first 2 drawn give red balls and the third drawn gives a black ball.

14

Result: For any two events A and B,

P(A)+P(B) 1 P (A B) P (A) or P(B) P(A UB) P(A) + P(B)