LECTURE NOTES NO.

Chapter 3
Discrete Probability Distributions
Random Variables
Example: Tossing a coin: we could get Heads or
Tails.
Let's give them the values Heads=0 and Tails=1 and
we have a Random Variable "X":
 

In short:
x = 0,1
 

1
 We have an experiment 
 We give values to each event
 A random variable is a set of possible
values from a random experiment, is a
quantity whose value change
Not Like an Algebra
In Algebra a variable, like x, is an unknown
value:
Example: x + 2 = 6
In this case we can find that x=4
But a Random Variable is different ...

 A Random Variable has a whole set of

values  and it could take on any of those
values, randomly.
Let’s say you flip a coin six times. How
many tails could you come up with?

2
There are a finite number of possible values.
Values such as "1.5" or "2.5923" don’t make
sense for this type of problem.
Examples:    
- the number of children in a family.
- the number of defective light bulbs in a box
- the number of arrivals per hour.
Random variables denoted by capital letter,
like X , Y, Z ,...
Discrete Probability Distributions
Example: Toss a coin 2 times
S={HH,HT,TH,TT}
Not interested in the outcomes but on
some numerical quantities
3
Let X: the number of heads
X(HH) = 2
X(HT) = 1
X(TH) = 1
X(TT) = 0
The possible values of x = 0, 1, 2
A random variable is a function that
maps events to numbers.
X:S (sample space) → R(set of real numbers)

X:S →R
- A probability distribution is a table
showing the probabilities of different
values of the random variable.
It is called probability mass function
(p.m.f)
4
 - The usual notation that is used is
P(X = x) for every value x that X can
take.
P(X = x) is a probability mass
function if
1. P(X=x) ≥ 0, for all values of x
∑ P ( X = x )=1
2. x

Example: Suppose you flip a coin two

times. There are four possible outcomes:
HH, HT, TH, and TT. Let the random
variable X: no. number of Heads. x = 0, 1,
or 2.

5
The probability distribution is

No. of heads Outcomes P(X=x)

x
0 TT 1/4
1 HT, TH 2/4=1/2
2 HH 1/4

Example: Throw a die once

Random Variable X = "The score shown on

the top face".
X could be 1, 2, 3, 4, 5 or 6
In this case they are all equally likely, so the
probability of any one is 1/6
6
 P(X = 1) = 1/6
 P(X = 2) = 1/6
 P(X = 3) = 1/6
 P(X = 4) = 1/6
 P(X = 5) = 1/6
 P(X = 6) = 1/6

1
P(X=x)=¿ ,
6 { }
if x=1,2,3,4,5,6 ¿ ¿ {}

Uniform distribution.
Example: Two dice are tossed.

The Random Variable is X = "The sum of the

scores on the two dice". Let's make a table of all
possible values:
Second die

7
 1 2 3 4 5 6

1 1,1 1,2 1,3 1,4 1,5 1,6

2 2,1 2,2 2,3 2,4 2,5 2,6

first die
3 3,1 3,2 3,3 3,4 3,5 3,6

4 4,1 4,2 4,3 4,4 4,5 4,6

5 5,1 5,2 5,3 5,4 5,5 5,6

6 6,1 6,2 6,3 6,4 6,5 6,6

There are 6 × 6 = 36 of them,

X: S → R
X((1,1)) = 2
X((1,2)) = 3
.
.
.
X((6,6)) = 12
The possible values of x = 2, 3, 4,…,12

8
Let's count how often each value occurs, and
work out the probabilities:

Outcome (first
die, second die) Sum Probability
(1,1) 2 1/36
(1,2), (2,1) 3 2/36
(1,3), (2,2), (3,1) 4 3/36
(1,4), (2,3), (3,2),
(4,1) 5 4/36
(1,5), (2,4), (3,4),
(4,2), (5,1) 6 5/36
... ... ...
(6,6) 12 1/36

The probability distribution of X is:

9
 1 1 1
x 2 3 4 5 6 7 8 9
0 1 2
P(X= 1 2 3 4 5 6 5 4 3 2 1
x) 36 36 36 36 36 36 36 36 36 36 36

The probability density function of X is

displayed in the following graph.

10
Example: Toss three coins.

11
 X = "number
    
of Heads"

HHH   3

HHT   2

HTH   2

HTT   1

THH   2

THT   1

TTH   1

TTT   0

X = "The number of Heads"

Possible values of x = 0, 1, 2, 3
Note: 1 case of Three Heads, but 3 cases of Two
Heads, 3 cases of One Head, and 1 case of Zero
Heads. So:

 P(X = 3) = 1/8
 P(X = 2) = 3/8
 P(X = 1) = 3/8
 P(X = 0) = 1/8

12
The probability distribution of X
x 0 1 2 3
P(X=x) 1/ 3/ 3/ 1/
8 8 8 8

Example: Toss a coin until a head appears

Let X :number of tosses until a head appears

S={H,TH,TTH,TTTH,…}

X: S → R
X(H) = 1
X(TH) = 2

13
X(TTH)=3
.
.
The possible values of x = 1,2,3,…
a. Find the p.m.f of X?
1
P( X= 1)=
2
1 1
P( X=1)= ×
2 2
.
1 1 1 1
P( X=n)= × ×.. .× = n , n=1,2 , ..
2 2 2 2

Expectation
Expected Value (Mean)
Definition: Let X be a discrete random
variable whose p.m.f is P(X = x. The

14
 expected value of X is the number,
denoted by E(X), defined by

E( X )=∑ xP ( X=x )
x
Provided that the sum exists.
The expected value of a random variable X,
E(X) is more commonly referred its mean µ
(or µx)

Example: Suppose you flip a coin two times.

There are four possible outcomes: HH, HT,
TH, and TT. Now, let the random variable X
represent the number of Heads. The
probability distribution appears below.
x P(X=x)
0 1/4
1 1/2
15
 2 1/4
E ( X )=∑ xP ( X = x )
x

1 1 1
E ( X )=0× +1× + 2× =1
4 2 4
Interpretation: Repeating the experiment of
tossing a coin 2 times large number of times, for
each experiment you get a value of X, the
average of these values in the long run is 1.

Expected value is the long run average or mean

Example: consider a die-rolling game that
costs $10 per play. A 6-sided die is rolled
once, and your cash winnings depend on the
number rolled. Rolling a 6 wins you $30;

16
rolling a 5 wins you $20; rolling any other
number results in no payout.
Die-Rolling Game ($10)
Number Prize Overall Probability
Rolled $ Outcome $
x P(X=x) x×P(X=x)
1 0 -10 Loss 1/6 -10(1/6)
2 0 -10 Loss 1/6 -10(1/6)
3 0 -10 Loss 1/6 -10(1/6)
4 0 -10 Loss 1/6 -10(1/6)
5 20 +10 Gain 1/6 +10(1/6)
6 30 +20 Gain 1/6 +20(1/6)
Expected value
Do not play !!! = -1.67$

Definition:

17
Let X be a random variable. Let g(X) be
a function of X. Then the expected value
of g(X) is defined by

E( g( X ))=∑ g ( x )P( X=x )

x
Provided that the sum/integral exists.
Example:

2 2
E( X )=∑ X P( X =x )
x

Result: E(a+bX) = a + b E(X)

Example: Let X be a random variable

such that E(X)=1, then
E(2X+3)=2E(X)+3=2(1)+3=5
18
Example: Let X be a random variable
such that E(X)=1/2, and E(X2)=1/3,then
E(X2+X-1/3)=E(X2)+E(X)-1/3
=1/3+1/2-1/3=1/2

Variance:
The variance of a discrete random
variable denoted by σ2 , is defined by
2 2 2
σ =E( X−μ ) =∑ ( x−μ ) P( X=x )
X

Result:

2 2 2
σ =E( X )−( μ )
Where
19
σ: standard deviation of X (or σX)
Example: Suppose you flip a coin two
times. X: number of Heads. The
probability distribution is
x P(X=x)
0 1/4
1 1/2
2 1/4
E( X )=1
Find σ2 ?
 Using the definition
2 2 2
σ =E( X−μ ) =∑ ( x−μ ) P( X=x )
X

Note μ=1

20
 2 2
σ= ∑ ( x−1) P( X =x)
x=0,1,2
2 2 2 2
σ =( 0−1) P( X=0)+(1−1) P( X=1)+(2−1) P( X=2)
2 1 1 1 1
σ =1( )+0( )+1( )=
4 2 4 2
 Using the result
2 2 2
σ =E( X )−( μ )
2 2
E ( X )=∑ x × P ( X = x )
X

2 1 22 1 2 1 3
E( X )=0 ( )+1 ( )+2 ( )=
4 2 4 2
2 3 2 1
σ = −( 1 ) =
2 2
Interpretation of variance

21
Example: Consider the following three
distributions
y -3 0 3
P(Y=y 0.2 0.6 0.2
)
E(Y) = 0
Var(Y) = 3.6
z -3 0 3
P(Z=z 0.4 0.2 0.4
)
E(Z) = 0
Var(Z) = 7.2
Note: Variance increases if probability of
far values increases
w -10 0 10
22
P(W=w 0.4 0.2 0.4
)
E(W) = 0
Var(W) = 80
Note: More spread out, we expect
variance to be large
2
Result: Var(a+bX) = b Var(X)
Where a and b are real constants

23
 Some Discrete Distributions
1. Discrete Uniform Distribution
2. Bernoulli Distribution
3. Binomial Distribution
4. Hypergeometric Distribution
5. Geometric Distribution
6. Poisson Distribution

24
1.Discrete Uniform Distribution
Consider n events, each of which have the
same probability 1/n (all outcomes are
equally likely). The random variable X
follows a discrete uniform distribution and
its p.m.f is:

1
{ }
P(X=x)=¿ , if x=x 1 ,x 2 ,..., x n ¿ ¿ {}
n
Suppose X is uniform over on {1,2,…,n}

1
{
P(X=i)=¿ , if i=1,2,...,n ¿ ¿ {}
n }
Examples
 A coin also has a uniform distribution
because the probability of getting either
heads or tails in a coin toss is the same,
25
  distribution is the rolling of a die. Each of
the six numbers is equally likely to come up
with a 1/6 chance.
Example: Throw a die once
Random Variable X = "The score shown on
the top face".
X could be 1, 2, 3, 4, 5 or 6
In this case they are all equally likely, so the
probability of any one is 1/6
 P(X = 1) = 1/6
 P(X = 2) = 1/6
 P(X = 3) = 1/6
 P(X = 4) = 1/6
 P(X = 5) = 1/6
 P(X = 6) = 1/6

1
P(X=x)=¿ ,
6 { }
if x=1,2,3,4,5,6 ¿ ¿ {}

Uniform distribution.

26
Result: Expected Value
n+1
E( X )=
2

Proof:
E( X )=∑ xP ( X=x )
x
n
E( X )=∑ iP ( X =i )
i =1
n
1
E ( X )=∑ i
i =1 n ( )
n
1 1 n(n+ 1) n+1
E( X )= ∑ i =
n i=1 n 2 ( =
2)
2.Bernoulli Distribution
27
 A Bernoulli trial is any trial with only two
possible outcomes, denoted by success and
failure. Let p be the success probability,
and 1-p be the failure probability. X is a
Bernoulli (Binary) random variable takes
two values
Success, x = 1
Failure, x = 0
So the Bernoulli distribution is

x 0 1
Result: P(X=x 1- p E(X)=p,
) p Var(X)=p(1-
p)
Proof:
E(X)= 0(1-p)+1(p) = p
Var(X)=E(X2)-(E(X))2
Var(X)=[02(1-p)+12(p)] – (p)2=p(1-p)

3.Binomial Distribution
28
A binomial experiment is a probability
experiment that satisfies the following four
requirements:
1. Each trial can have only two outcomes. These
outcomes can be considered as either success or
failure.
2. There must be a fixed number of trials.
3. The outcomes of each trial must be
independent of each other.
4. The probability of a success, denoted by p,
must remain the same for each trial.
The number of successes X in n trials of a
binomial experiment is called a binomial
random variable.
Examples:
Children with a bacterial infection might
respond to antibiotic therapy or not.

29
Light bulbs selected from a factory might be
defective or non-defective.

The Binomial Distribution Model

P( X=k successes )=¿ ( n ¿ ) ¿ ¿ ¿

¿
where

( n ¿) ¿¿ ¿
¿
The latter expression is known as the binomial
coefficient, stated as "n choose k," or the
number of possible ways to choose k
"successes" from n observations. For example,
the number of ways to achieve 2 heads in a set
of four tosses is "4 choose 2", or 4!/2!2! =
(4*3)/(2*1) = 6. The possibilities are {HHTT,
HTHT, HTTH, TTHH, THHT, THTH}, where
"H" represents a head and "T" represents a tail.
The binomial coefficient multiplies the
30
probability of one of these possibilities (which
is (1/2)²(1/2)² = 1/16 for a fair coin) by the
number of ways the outcome may be achieved,
for a total probability of 6/16.
Binomial theorem
n
( x+ y )n =∑ ( n ¿ ) ¿ ¿¿ ¿
k =1 ¿

For Binomial distribution check

n
∑ P( X =k )=1 ?
k =0

n
∑ ( n ¿) ¿ ¿ ¿ ¿
k =0 ¿
using Binomial theorem, for this reason it is
called Binomial Distribution
Result: The expected value (or the mean μ) is
E(X)= np
Proof

31
 E( X )=∑ xp ( X=x )
x

n
E( X )=∑ ip( X =i)
i=0

n
E( X )=∑ ialignl ( n ¿ ) ¿ ¿ ¿ ¿
i=1 ¿

Note:

ialignl ( n ¿ ) ¿ ¿ ¿
¿
n
E( X )=∑ nalignl ( n−1 ¿ ) ¿ ¿ ¿ ¿
i=1 ¿
n
E( X )=∑ nalignl ( n−1 ¿ ) ¿ ¿ ¿ ¿
i =1 ¿
n
E ( X )=np ∑ ( n−1 ¿ ) ¿ ¿ ¿ ¿
i=1 ¿

Let m=n-1, j=i-1

32
 m
E ( X )=np ∑ ( m ¿ ) ¿ ¿ ¿ ¿
j=0 ¿

Result: Var(X) =σ2= np(1-p)

The binomial distribution model is an

important probability model that is used
when there are two possible outcomes (i.e.,
it is dichotomous).
Example 1: A coin is tossed three times.
Find the probability of getting exactly two
heads.
Solution:
X: number of heads
X: B(n, p), X: B(3, 1/2)

P( X=2)=¿ ( 3 ¿ ) ¿ ¿¿
¿

33
This problem can be solved by looking that the
sample space. There are three ways to get two
heads.
HHH, HHT, HTH, THH, TTH, THT, HTT, TTT
The answer is 3 / 8
Example 2: If a student randomly guesses at
five multiple-choice questions, find the
probability that the student gets exactly three
correct. Each question has five possible
choices.
Solution
In this case n = 5, X = 3, and p = 1/5, since
there is one chance in five of guessing a
correct answer. Then,

P( X=3)=¿ ( 5 ¿ ) ¿ ¿¿
¿

34
The expected or mean number of correct
answers = 5 (1/5) = 1
Example 3: If a student randomly guesses at
five multiple-choice questions, find the
probability that the student gets exactly three
correct. What is the expected number of
correct answers?
Solution; In this case n = 5, X = 3, and p =
1/5, since there is one chance in five of
guessing a correct answer. Then,

P( X=3)=¿ ( 5 ¿ ) ¿ ¿¿
¿
The expected or mean number of correct =
np=5 (1/5) = 1
Example 4: Suppose that the chance of
hitting a target in a single shot is 0.40. If
three shots are fired independently at the
target, what is the probability that
35
 a. Exactly one shot will hit the target?
X: no. of shots that hit the target
X: follow Binomial distribution with
n=3 and p=0.4
Need P(X=1)?

P( X=1)=¿ ( 3 ¿ ) ¿ ¿¿
¿
b. At least one shot will hit the target?
Need P(X ≥ 1)?
P( X≥1)=1-P ( X <1)=1-P( X =0)

=1−¿ ( 3 ¿ ) ¿ ¿¿
¿
Relation between Bernoulli and
Binomial distributions
X: B(n, p)

36
X: The number of successes in n independent
Bernoulli trials
X1 Bernoulli random variable (x1=1, success,
x1=0 failure)
X2 Bernoulli random variable (x2=1, success,
x2=0 failure)
.
.
.
Xn Bernoulli random variable (xn=1, success,
xn=0 failure)
X1+X2+…+Xn= total number of success in n trials

=X :B(n, p)

So, the Binomial random variable is a sum

of n independent Bernoulli random variables
each with success probability p.

37
4.Hypergeometric Distribution
Closely related to Binomial distribution
Example: Sampling Problem
A box contains N balls of which M are
white and the rest are red (N-M)
Draw n balls randomly
What’s probability that k are white?
X: number of white balls in the sample
(a) Sampling with replacement
X is binomial where p = M/N
(b) Sampling without replacement

( M ¿
P(X=k)= ¿¿¿¿ ) ¿
¿
38
Where
( N ¿ )¿ ¿ ¿
¿
( M ¿) ¿ ¿ ¿
¿
( N − M ¿) ¿ ¿ ¿
¿
Hypergeometric distribution with
parameters: N, M, and n

Result:
Expected value or mean is
M
E( X )=n
N
Variance is

39
 N −n M M
Var ( X )= n
N−1 N
1−(N )
Example: A box containing 10 electric
bulbs, among the three are 4 defectives. If
three bulbs are drawn without replacement
from the box, what is the probability of
getting
a. Exactly one defective bulb?
Solution

( 4¿
P(X=1)= ¿¿¿¿) ¿
¿
b. Not more than 2 defective bulbs?

40
 Solution
Need P(X ≤ 2)?
P(X ≤ 2)=P(X = 0)+P(X = 1)+P(X = 2)

= ( 4 ¿ ) ¿ ¿¿ ¿ ¿
¿
=0.1667+0.5+0.3=0.9667

5.Geometric Distribution
We have a sequence of independent Bernoulli

trials with success probability equals p,

Let X: number of trials to get the first success

41
S={S,FS,FFS,FFFS,…}

X(S)=1, X(FS)=2, X(FFS)=3, …

Then the possible values of X are 1,2,3,…

The p.m.f of X is given by

n−1
P( X=n)=(1− p) p, n=1,2,3,...
This is called the geometric distribution
Check
n
∑ P( X =k )=1 ?
k =0
∞
∑ ( 1− p )n−1 p
n=1

Let m=n-1
∞ ∞
1
∑ (1− p)m p= p ∑ (1− p)m= p 1−(1− p)
=1
m=0 m=0

42
Result:
Expected value or mean is
1
E( X )=
p
Variance is
1− p
Var ( X )= 2
p
Example 1: Roll a die until a “6” shows up
X: number of rolls needed, the p.m.f of X is
n−1
5 1
P( X=n)=
6 () 6
, n=1,2,3 ,. ..

Example 2: If the probability of hitting a

target in a single shot is 0.7. If shots are to
be fired independently, what is the
43
probability that the first hit to the target will
be in the third shot ?
Need P(X=3)?
P(X=3)=(0.3)2(0.7)=0.063

6.Poisson Distribution
The Poisson Distribution is a discrete
distribution which takes on the values X = 0, 1,
2, 3, ... . It is often used as a model for the
number of events occurring within a given time
interval, such as
 number of arrivals at some service station.
 You observe that the number of telephone
calls that arrive each day on your mobile
phone over a period of a year, and note that
the average is 3
 Records indicate that messages arrive to a
computer server at the rate of 6 per hour.

44
 Let X be the number of messages arriving
in any one hour.
 the occurrence of earthquakes could be
considered to be a random event. If there are 5
major earthquakes each year, then the number of
earthquakes in any given year will have a
Poisson distribution with parameter 5. 
The Poisson random variable satisfies the
following conditions:
1.The probability of one occurrence in a certain
period of time is directly proportional to the
length of that interval.
2.The probability of two or more occurrences is
negligible if the length of the interval is small
3.The occurrences are independent.

Poisson random variable also applies to disjoint

regions of space
The Poisson probability mass function is

45
 −λ x
e λ
P( X= x )= , x=0,1,2 ,. .. , n
x!
λ indicates the average number of events in the given
time interval.

For Poisson distribution check

n
∑ P( X =k )=1 ?
k =0

∞
e− λ λ i −λ ∞ λi −λ λ
∑ i ! =e ∑ i! =e e =1
i=0 i=0
∞ i
x
using ∑ =e− x
i=0 i!

Result: Mean of Poisson Distribution

E(X) =μ= λ
Proof
E( X )=∑ xp ( X=x )
x

46
 E( X )=∑ ip( X =i)
i

∞ −λ i
e λ
E( X )=∑ i
i =0 i!
∞ −λ i
e λ
E( X )=∑ i
i =1 i( i−1) !

Let j=i-1
∞ −λ j+1∞ −λ j
e λ e λ
E( X )= ∑ =λ ∑ =λ
j=0 j! j=0 j !

Result: Variance
Var(X) = λ

47
The following is the plot of the Poisson probability
density function for four values of λ.

48
Example 1: The number of traffic accidents
that occurs on a particular stretch of road
during a month follows a Poisson
distribution with a mean of 9.4. Find the
probability that less than two accidents will
occur on this stretch of road during a
randomly selected month.
Solution
Need P(X<2) ?
P(x < 2) = P(x = 0) + P(x = 1) = 0.000860
−9. 4 0 −9 . 4 1
e 9.4 e 9.4
= + =0 . 00086
0! 1!
Example 2: Consider, in an office 2
customers arrived today.

49
a. Calculate the possibilities for exactly 3
customers to be arrived on one day?
−2 3
e 2
P( X=3)= =0 .18
3!
b. Calculate the possibilities for more than
two customers to be arrived on one day?
.

P( X >2 )=1− p( X≤2 )

=1−[ P ( X=0 )+P( X =1)+ P( X =2)
=1-(0.1353+0.2707+0.2707)=0.3233

c. . Calculate the possibilities for no

customers to be arrived on one day?
−2 0
e 2
P( X=0)= =0 .1353
0!
d. Calculate the possibilities for exactly 15
customers to be arrived on one week.
Arrival rate 2 per day
Then arrival rate 2(7)=14 per week
  
 

50
 −14 15
e 14
P( X=15)= =0 . 099
15 !
Example 3: Suppose we are interested in the
number of defects in a section of highway
shortly after it is resurfaced. It seems reasonable
to assume that the probability of a defect is the
same for any two intervals of equal length and
that the occurrence of a defect in one interval is
independent of a defect of any other interval.
Assume that after resurfacing defects can be
expected to occur at the mean of two per mile.
Find the probability of no defects in a particular
3 mile stretch of the highway?
Solution:
2 defects per mile
(2 defects per mile)(3 miles)= 6 represents the
expected number of defects over 3 mile stretch
51
 −6 0
e 6
P(no defects=X =0 )= =0 .0025
0!
it’s unlikely that there will be no defects in 3
mile section
Probability that at least one defect=1-(no
defects)

P( X≥1)=1−P ( X=0)=1−0 . 0025=0 . 9975

52