Discrete Probability

Lecture 14

Discrete Probability
The study of probability uses special jargon
A sample space in a nonempty set
An experiment is a process that produces a point in a
sample space
An event is a subset of sample space
The event space is the power set of the sample space
Experiment: Toss a coin
Sample space: U={H,T}
Event Space: P(U)={, {H}, {T}, {H,T}}
2

Discrete Probability
Remark: Often some of the more interesting events have
special name
Experiment: Toss Two Coin
Sample space: U={HH, HT, TH, TT}
Event Space: Subsets of U
Named events:
Match ={HH, TT}
At least one head = {HT, TH, HH}
3

Uniform Probability Measure

Def: The uniform probability measure on a finite sample
space S assigns to each event E the probability

|E|
p( E ) =
|S|
Sample space: U={H,T}
Event
, {H}, {T}, {H,T}
Probability
0

Uniform Probability Measure

Def: The uniform probability measure on a finite sample
space S assigns to each event E the probability

|E|
p( E ) =
|S|
Experiment: Toss Two Coin
Sample space: U={HH, HT, TH, TT}
Event Name
Event
Probability

At least one head

{HT, TH, HH}

one of each
{HT, TH}
2/4
5

Uniform Probability Measure

Def: The uniform probability measure on a finite sample
space S assigns to each event E the probability

|E|
p( E ) =
|S|
Experiment: Roll a die
Sample space: U={1,2,3,4,5,6}
Event Name
odd
even
Even
{1,3,5} {2,4,6}
Probability
3/6
3/6

2 mod 3
{2,5}
2/6

Uniform Probability Measure

Def: The uniform probability measure on a finite sample
space S assigns to each event E the probability

|E|
p( E ) =
|S|
Experiment: Roll two die
Sample space: U={11,.,16,,61,66}
Event Name
double
sum is 9
Even
{11, 22,.,66} {36,45,54,63}
Probability
6/36
4/36

Uniform Probability Measure

Experiment: deal one card from deck

Sample space: standard 52 card deck
Event Name
heart
seven
Even
13/52
4/52
Probability

Probability of Complementary Event

Let E be an event in finite sample space S, under the
uniform probability distribution. Then

p( E ) = 1 p( E )
Pf:
1. |S| = |E| +|E|

Probability of a Union Event

Let E1 and E2 be an events in a finite sample space S,
under the uniform probability distribution. Then

p( E1 E2 ) = p ( E1 ) + p( E2 ) p( E1 E2 )
Pf:
1. An integer is chosen from the internal [1,2,.,100]. Find
the probability that it is divisible either by 6 or by 15.
P(6\n v 15\n) = p(6\n) +p(15\n)-p(30\n)
= 16/100 + 6/100 -3/100
= 19/100
10

Non-uniform Probability Measure

Here are two non-uniform probability measure for some
familiar sample spaces.
The standard loaded coin has probability p(H)=0.8 and
p(T)=0.2

The standard loaded die has sample space U={1,2,3,4,5,6}

Then assign the singleton {j} the probability j/21

11

Bernoulli distribution
A Bernoulli distribution is a probability measure on a
sample space with exactly two points
Flip standard loaded coin
Sample space = {H,T}
Event Space = { {H}, {T}, {H,T}}
pr({H}) = 4/5

pr ({T}) = 1/5

12

Bernoulli distribution
A Bernoulli distribution is a probability measure on a
sample space with exactly two points
The standard loaded die induces a Bernoulli distribution
with
pr(even) = 4/7

pr ({odd}) = 3/7

13

Conditional probability
Let p be a probability distribution on a sample space U
and let Y be an event. Then the conditional probability of
event E given that event Y has occurred is

pr ( E Y )
pr ( E | Y ) =
pr (Y )
p ( HH | TT ) =
p ( HH )
P(TT )
1/ 4
=
3/ 4
1
=
3

p ( HH TT )
P(TT )

14

14

Conditional Independence
Let pr be a probability distribution on a sample space U.
Event E is probabilistically independent of event Y if and
only if

pr ( E | Y ) = pr ( E )
pr ( E | Y ) = pr ( E ) = pr ( E Y ) / pr (Y )
Pr(Y ) = pr ( E Y ) / pr ( E )
= pr (Y E ) / pr ( E )
= pr (Y | E )

15

Conditional Independence
Roll two fair dice. Let Y be the event that the first die is a
one and the event that the sum is odd. Then
pr(E) = 1.1/2.1/2=1/2
pr ( E Y )
pr ( Y )
p ( 12 ,14 ,16 )
1 / 6
3 / 36
1 / 6
3
6
1
2

pr ( E | Y ) =
=
=
=
=

16

Random Variable
A random variable is a real valued function on the domain
of a probability sapce.

Flip a coin three times. Let the X(t) be the number of

heads that occurs. Then
X(TTT) = 0
X(TTH)=X(THT)=X(HTT) =1
X(THH)=X(HHT)=X(HTH)=2
X(HHH)=3
Since a random variable is a function, it is not a variable
and it is not random.

17

Bayes Theorem
Start with the joint probability distribution:
toothache

toothache
catch

catch

catch

catch

cavity

0.108

0.012

0.072

0.008

cavity

0.016

0.064

0.144

0.576

18

Inference by enumeration
Start with the joint probability distribution:
toothache

toothache
catch

catch

catch

catch

cavity

0.108

0.012

0.072

0.008

cavity

0.016

0.064

0.144

0.576

P(toothache) = 0.108 + 0.012 + 0.016 + 0.064 = 0.2

19

Inference by enumeration
Start with the joint probability distribution:
toothache

toothache
catch

catch

catch

catch

cavity

0.108

0.012

0.072

0.008

cavity

0.016

0.064

0.144

0.576

Can also compute conditional probabilities:

P(cavity | toothache)
= P(cavity toothache)
P(toothache)
=
0.016+0.064
0.108 + 0.012 + 0.016 + 0.064
= 0.4
20

Normalization
toothache

toothache
catch

catch

catch

catch

cavity

0.108

0.012

0.072

0.008

cavity

0.016

0.064

0.144

0.576

Denominator can be viewed as a normalization constant

P(Cavity | toothache) = , P(Cavity,toothache)
= , [P(Cavity,toothache,catch) + P(Cavity,toothache, catch)]
= , [<0.108,0.016> + <0.012,0.064>]
= , <0.12,0.08> = <0.6,0.4>
21

Thank You

22