BASIC CONCEPTS OF

Submitted by:
Pasaoa, Almarie Graceilla C.
Viernes, Aurea Noreen
Orinday, Patrick
Clacio, Joyce Elaine

5.1 Objective and

Subjective Probability

References

http://www.mathsisfun.com/data/probabil
ity.html
http://www.slideshare.net/gsriharsha/fun
damentals-probability-080720091804620

5.2 SAMPLE SPACE

AND EVENTS

DEFINITION: SAMPLE SPACE &

EVENT
SAMPLE

SPACE
A set of possible outcomes of a random
experiment
The symbol S will be used to denote the
sample space

DEFINITION: SAMPLE SPACE &

EVENTS
EVENT

It is a subset of the sample space

-May consists of one or more simple events
- EVENTS are denoted using capital letters such as E

EXAMPLE :

A coin is tossed twice and the outcome of

each is recorded. Then,

S= {(H, H), (H, T) , (T, H) (T,T)}

The event that the second toss was a Head is
the subset
E= {(H ,H), (T, H)}

EXAMPLE:

Tossing a die

The sample space is S= {1,2,3,4,5,6} .

E= {2,4,6} is an event, which can be
described in words as the number is even

EXAMPLE:
Tossing a coin twice.
The sample is S= {HH, HT, TH, TT}.
E= {HH, HT} is an event, which can be
described in words as the first toss results
in a Heads

EXAMPLE:

A spinner has 4 equal sectors colored

yellow, blue, green, and red. What is the
probability of landing on each color after
spinning this spinner?

SAMPLE SPACE: {yellow, blue, green, red}

REFERENCES:
http://cims.nyu.edu/~kiryl/Probability/Chap
ter%202.pdf
http://www.math.uiuc.edu/~kkirkpat/Sample
Space.pdf
http://www.mathgoodies.com/lessons/vol6/s
ample_spaces.html

5.3 AXIOMS
OF
PROBABILITY

REFERENCES
http://www.math.utah.edu/~lzhang/teaching/3070summer2008/
DailyUpdates/jun10/sec2_2.pdf
http://www.slideshare.net/mathscontent/probability-and-itsaxioms-3209538

5.4 Some Important Theorems on

Probability

5.4 Some Important Theorems on Probability

5.4 Some Important Theorems on Probability

5.4 Some Important Theorems on Probability

5.4 Some Important Theorems on Probability

Examples
Experiment 1:

A single 6-sided die

is rolled. What is the
probability of rolling a
2 or a 5?

Probabilities:
1
P(2)

1
P(5)

Experiment 1:
P(2 or 5)

P(2)

A single 6-sided die is rolled. What is

the probability of rolling a 2 or a 5?
+

1
=

2
=

P(5)

1
+

Examples

Experiment 2:

On New Year's Eve, the probability of a

person having a car accident is 0.09. The
probability of a person driving while
Experiment 6:
On New Year's
Eve, the is 0.32 and probability of a
intoxicated
probability of person
a person
having a car accident while
having a car accident
is is 0.15. What is the probability of
intoxicated
0.09. The probability
a
a personofdriving
while intoxicated or having a
person drivingcar
while
accident?
intoxicated is 0.32 and
probability of a person

P(intoxicated
having a car accident
+
P(accident)
-
and
P(intoxicated or = P(intoxicated)
while intoxicated is 0.15.

accident)
accident)
What is the probability of
a person driving while
intoxicated or having a
car accident?

Probabilities:=
0.32
+
0.09
-
0.15

=
0.26

Examples

Example: Rolling a "5" or "6"

Event A: {5, 6}
Number of ways it can happen: 2
Total number of outcomes: 6

The Complement of Event A is {1, 2, 3, 4}

Number of ways it can happen: 4
Total number of outcomes: 6
Let us add them:
Yep, that makes 1
It makes sense, right? Event A plus all outcomes that are not Event A make up all possible outcomes.

2
P(A')
= =
P(A)

4/6

=
=

2/3

P(A) + P(A') =

1/3

2/3

3/3

= 1

Examples

Example: Scoring Goals

If the probability of:
scoring no goals (Event "A") is20%
scoring exactly 1 goal (Event "B") is15%
Then:
The probability of scoring no goalsand1 goal
is0(Impossible)
The probability of scoring no goalsor1 goal is 20% +
15% =35%

Which is written:
P(AB) = 0
P(AB) = 20% + 15% = 35%

Examples

Example: A Deck of Cards

In a Deck of 52 Cards:
the probability of a King is 1/13, soP(King)=1/13
the probability of an Ace is also 1/13, soP(Ace)=1/13

When we combine those two Events:

The probability of a card being a Kingandan Ace
is0(Impossible)
The probability of a card being a Kingoran Ace is
(1/13) + (1/13) =2/13
Which is written like this:
P(King and Ace) = 0
P(King or Ace) = (1/13) + (1/13) = 2/13

Examples
Example: 16 people study French,
21 study Spanish and there are 30
altogether. Work out the
probabilities!
This is definitely a case
ofnotMutually Exclusive (you can
study French AND Spanish).
Let's saybis how many study both
languages:
-people studying French Only must
be16-b
-people studying Spanish Only must
be21-b
And we get:

And we can put in the correct

numbers:

So we know all this now:

P(French) = 16/30
P(Spanish) = 21/30
P(French Only) = 9/30
P(Spanish Only) = 14/30
P(French or Spanish) = 30/30 = 1

And we know there are30people,

so:
(16b) + b + (21b) = 30
37 b = 30

P(French and Spanish) = 7/30

Lastly, let's check with our formula:
P(A or B) = P(A) + P(B) P(A
and B)
Put the values in:
30/30 = 16/30 + 21/30 7/30

Activities
1. A number is chosen at random from the set of two-digit numbers
from 10 to 99 inclusive.
What is the probability the number contains at least one digit 2?
2. There are 30 children in a class and they all have at least one cat or
dog.
14 children have a cat, 19 children have a dog.
What is the probability that a child chosen at random from the class
has both a cat and a dog?
3. A fair die is thrown. What is the probability that the score is not a
factor of 6?
4. A spinner is made from a piece of card in the shape of a regular
pentagon with a toothpick pushed through the center. When the
spinner is spun and it lands on an edge, each of the numbers from 1
to 5 is equally likely.
If the spinner is spun twice, what is the probability the two scores are
different?
5. A day of the week is chosen at random. What is the probability of
choosing a Monday or Tuesday?
6. In a pet store, there are 6 puppies, 9 kittens, 4 gerbils and 7
parakeets. If a pet is chosen at random, what is the probability of

Interesting Trivia

Each time you inhale, you have a

greater than 99.9% chance of
inhaling at least 1 molecule
exhaled by Caesar's last dying
breath.

References
http://www.mathgoodies.com/lessons/vol6/addition_rules.html
https://www.mathsisfun.com/data/probability-complement.html
http://www.mathsisfun.com/data/probability-events-mutuallyexclusive.html

5.5 Conditional
Probability
and Independent
Events

When dealing with two events (usually called A

and B), sometimes the events are so related to
each other, that the probability of one depends
on whether the other event has occurred.
When we talk about probabilities based on the
fact that something else has already happened
we call this conditional probability.
What changes when dealing with conditional
probability is that we know for certain that
something else has already happened. This
means that in our definition of probability that
says

where the total number of ways is based on

the fact that we know something else has
already occurred.
There are two ways to approach conditional
probability and depends on the type of problem
that you are given. In a situation where you are
given percentages and probabilities (usually but
not always in a table format) we make use of
the conditional probability rule. In situations
where you are trying to compute probabilities on
your own (instead of them being given to you)
most of the time it is easier to not to use the
formula.

Conditional Probability Rule:

Consider events A and B.

The line between A and B is read given.

So translated, this reads, "the probability of
A given that B has happened." The event
on the right side of the line is the event
that has already happened.

What The Rule Means:

This rule is applied when you have two events
and you already know the outcome of one of the
events. In doing the computations, you will need
to be able to find the probability of A and B, that
is, P(AB). Problems of this type make use of the
multiplication rule. If you need help with the
multiplication rule or understanding what type of
problems make use of the rule.

References:

http://www.regentsprep.or
g/regents/math/algebra/AP
R3/Lconditional
http://www.cut-theknot.org/Curriculum/Proba
bility/ConditionalProbabilit
y
https://www.mathsisfun.co
m/data/probability-eventsconditional
http://www.wyzant.com/re
sources/lessons/math/stati
stics_and_probability/prob
ability/further_concepts_in
_probability

EXAMPL
E #1:

1. There is a 2/5 chance of pulling out a

Blue marble, and a 3/5 chance for Red:

We can even go one step further

and see what happens when we
select a second marble

If a blue marble was selected first there is now a 1/4 chance

of getting a blue marble and a 3/4 chance of getting a red
marble.
If a red marble was selected first there is now a 2/4 chance
of getting a blue marble and a 2/4 chance of getting a red
marble.
"What are the chances of drawing 2 blue marbles?"
Answer: it is a 2/5 chance followed by
a 1/4 chance:

The chances of drawing 2 blue

marbles is 1/10.

P(A) means "Probability Of Event A"

In our marbles example Event A is "get a Blue Marble
first" with a probability of 2/5:
P(A) = 2/5
And Event B is "get a Blue Marble second" ... but for
that we have 2 choices:
If we got a Blue Marble first the chance is now 1/4
If we got a Red Marble first the chance is now 2/4
So we have to say which one we want, and use the
symbol "|" to mean "given":
P(B|A) means "Event B given Event A"
In other words, event A has already happened, now what is
the chance of event B?
P(B|A) is also called the "Conditional Probability" of B given A.
And in our case:
P(B|A) = 1/4

So the probability of getting 2 blue

marbles is:

And we write it as:

Probability of event A and event B equals

the probability of event A times the
probability of event B given event A"

EXAMPLE
#2

In a school of 1200 students,

250 are seniors, 150 students
take math, and 40 students are
seniors and are also taking
math. What is the probability
that a randomly chosen student
who is a senior, is taking
math?

These questions can be

confusing. It sounds, at first
read, that they are asking for the
probability of choosing a student
who is a senior and who is taking
math.

Answer:
It helps to re-word the question into:
Find the probability that the student is
taking math, given that the student is a
senior.
B = the student is taking
math
n(A) = the student is a senior =
250.
n(A and B) = the student is a senior
and is taking math = 40.

SOLUTION:

EXAMPLE
#3

In a card game, suppose a player needs to draw

two cards of the same suit in order to win. Of
the 52 cards, there are 13 cards in each suit.
Suppose first the player draws a heart. Now the
player wishes to draw a second heart. Since one
heart has already been chosen, there are now
12 hearts remaining in a deck of 51 cards. So
the conditional probability P(Draw second
heart|First card a heart) = 12/51.

Suppose an individual applying to a college

determines that he has an 80% chance of being
accepted, and he knows that dormitory housing
will only be provided for 60% of all of the
accepted students. The chance of the student
being accepted and receiving dormitory housing
is defined by
P(Accepted and Dormitory Housing) =
P(Dormitory Housing|Accepted)P(Accepted) =
(0.60)*(0.80) = 0.48.

LET'S
PRACTICE!

1. A survey of 500 adults asked about

college expenses. The survey asked
questions about whether or not the
person had a child in college and about
the cost of attending college. Results
are shown in the table below.

Suppose one person is chosen at

random. Given that the person has a
child in college, what is the probability
that he or she ranks the cost of
attending college as cost too much?

2. Suppose you draw two

cards from a standard deck
without replacement. Given
that the first card is an ace,
what is the probability that
the second card is a queen?

ANSWERS
:

1. P(cost too much child in college) =

P(cost too muchchild in college) can be found from the
table as 0.30
P(child in college) can be found by adding 0.30 + 0.13
+ 0.01 = 0.44
Substituting these values into the equation for
conditional probability we get

2. P(ace queen) =
P(ace|queen) is found using the multiplication
rule. Since the problem states that there is no
replacement we can find
P(ace|queen) as (4/52)(4/51) = 4/663
We know that
P(queen) = 4/52
Substituting values into the conditional
probability formula we get

5.6 Bayes
Theorem

 relates current probability to prior probability

Used to find probabilities of various events.

A big advantage of a Bayesian approach

Allows a principled approach to the exploitation of
all available data
with an emphasis on continually updating ones
models as data accumulate
as seen in the consideration of what is learned
from a positive mammogram

Bayesian Reasoning
ASSUMPTIONS
1% of women aged forty who participate in a routine
screening have breast cancer
80% of women with breast cancer will get positive
tests
9.6% of women without breast cancer will also get
positive tests
EVIDENCE
A woman in this age group had a positive test in a
routine screening
PROBLEM
Whats the probability that she has breast cancer?

Bayesian Reasoning
ASSUMPTIONS
10 out of 1000 women aged forty who participate in a
routine screening have breast cancer
800 out of 1000 of women with breast cancer will get
positive tests
95 out of 1000 women without breast cancer will also
get positive tests
PROBLEM
If 1000 women in this age group undergo a routine
screening, about what fraction of women with positive
tests will actually have breast cancer?

Bayesian Reasoning
ASSUMPTIONS
100 out of 10,000 women aged forty who participate
in a routine screening have breast cancer
80 of every 100 women with breast cancer will get
positive tests
950 out of 9,900 women without breast cancer will
also get positive tests
PROBLEM
If 10,000 women in this age group undergo a routine
screening, about what fraction of women with positive
tests will actually have breast cancer?

Bayesian Reasoning
Before the screening:
100 women with breast cancer
9,900 women without breast cancer
After the screening:
A = 80 women with breast cancer and positive test
B = 20 women with breast cancer and negative test
C = 950 women without breast cancer and positive test
D = 8,950 women without breast cancer and negative test
Proportion of cancer patients with positive results, within
the group of ALL patients with positive results:
A/(A+C) = 80/(80+950) = 80/1030 = 0.078 = 7.8%

Compact Formulation
C = cancer present, T = positive test
p(A|B) = probability of A, given B, ~ = not
PRIOR PROBABILITY
p(C) = 1%
PRIORS

CONDITIONAL PROBABILITIES
p(T|C) = 80%
p(T|~C) = 9.6%

POSTERIOR PROBABILITY (or REVISED PROBABILITY)

p(C|T) = ?

Bayesian Reasoning
Before the screening:
100 women with breast cancer
9,900 women without breast cancer
After the screening:
A = 80 women with breast cancer and positive test
B = 20 women with breast cancer and negative test
C = 950 women without breast cancer and positive test
D = 8,950 women without breast cancer and negative test
Proportion of cancer patients with positive results, within
the group of ALL patients with positive results:
A/(A+C) = 80/(80+950) = 80/1030 = 0.078 = 7.8%

Bayesian Reasoning

Prior Probabilities:

100/10,000 = 1/100 = 1% = p(C)

9,900/10,000 = 99/100 = 99% = p(~C)

Conditional Probabilities:
A = 80/10,000 = (80/100)*(1/100) = p(T|C)*p(C) = 0.008
B = 20/10,000 = (20/100)*(1/100) = p(~T|C)*p(C) = 0.002
C = 950/10,000 = (9.6/100)*(99/100) = p(T|~C)*p(~C) = 0.095
D = 8,950/10,000 = (90.4/100)*(99/100) = p(~T|~C) *p(~C) = 0.895

Rate of cancer patients with positive results, within the

group of ALL patients with positive results:

A/(A+C) = 0.008/(0.008+0.095) = 0.008/0.103 = 0.078 = 7.8%

-----> Bayes theorem

A

p(T|C)*p(C)
p(C|T) = ______________________
P(T|C)*p(C) + p(T|~C)*p(~C)

A + C

Comments
Common mistake: to ignore the prior probability
The conditional probability slides the revised
probability in its direction but doesnt replace the
prior probability
A NATURAL FREQUENCIES presentation is one in
which the information about the prior probability is
embedded in the conditional probabilities (the
proportion of people using Bayesian reasoning rises to
around half).
Test sensitivity issue (or: if two conditional
probabilities are equal, the revised probability equals
the prior probability)
Where do the priors come from?

-----> Bayes theorem

p(X|A)*p(A)
p(A|X) = ______________________
P(X|A)*p(A) + p(X|~A)*p(~A)

Given some phenomenon A that we want to investigate, and an

observation X that is evidence about A, we can update the
original probability of A, given the new evidence X.

ACTIVITIES
Suppose a drug test is 99%sensitiveand 99%
specific. That is, the test will produce 99% true
positive results for drug users and 99% true
negative results for non-drug users. Suppose that
0.5% of people are users of the drug. If a
randomly selected individual tests positive, what
is theprobabilityhe or she is a user?

P(User +)= P(+ User) P(User)

P(+ User) P(User) + P(+

No=user)
P(Non-user

Answer
=

0.99 x 0.005
0.99 x 0.005 + 0.01 x 0.995

References

Statistics: A Bayesian Perspective D. Berry, 1996, Duxbury Press.

o excellent introductory textbook, if you really want to understand what

its all about.

http://ftp.isds.duke.edu/WorkingPapers/97-21.ps

o Using a Bayesian Approach in Medical Device Development, also by

Berry

http://www.pnl.gov/bayesian/Berry/
o a powerpoint presentation by Berry

http://yudkowsky.net/bayes/bayes.html

o Extremely clear presentation of the mammography example; highly

polemical and fun too!

http://www.stat.ucla.edu/history/essay.pdf
o Bayes original essay

Jaynes, E. T., 1956, `Probability Theory in Science and

Engineering,' (No. 4 in `Colloquium Lectures in Pure and Applied
Science,' Socony-Mobil Oil Co. USA.
http://bayes.wustl.edu/etj/articles/mobil.pdf
o A physicists take on Bayesian approaches. Proposes an interesting
metric of probability using decibels (yes, the unit used for sound
levels!).

http://www.sportsci.org/resource/stats/

o a skeptical account of Bayesian approaches. The rest of the site is very

informative and sensible about basic statistical issues.

5.7 CONTINGENCY
TABLES

OVERVIEW

Hypotheses of equal proportions

Hypotheses of independence
Exact distributions and Fishers test
The Chi squared approximation
Median test
Measures of dependence
The Chi squared goodness-of-fit test
Cochrans test

CONTINGENCY TABLE EXAMPLES

Countries - religion by government
States dominant political party by
geographic region
Mutual funds - style by family
Companies - industry by location of
headquarters

MORE EXAMPLES Countries - government by GDP categories

States - divorce laws by divorce rate categories
Mutual funds - family by Morning Star rankings
Companies - industry by price earnings ratio
category

STATISTICAL INFERENCE HYPOTHESIS OF

EQUAL PROPORTIONS
H0: all probabilities (estimated by proportions,
relative frequencies) in the same column are
equal,
H1:at least two of the probabilities in the same
column are not equal
Here, for an r x c contingency table, r populations
are sampled with fixed row totals, n1, n2, nr.

HYPOTHESIS OF INDEPENDENCE
H0: no association
i.e. row and column variable are independent,

H1: an association,
i.e. row and column variable are not independent

Here, one populations is sampled with

sample size N. Row totals are random
variables.

EXACT DISTRIBUTION FOR 2 X 2 TABLES:

HYPOTHESIS OF EQUAL PROPORTIONS; N1 =
N2 = 2

FISHERS EXACT TEST

For 2 x 2 tables assuming fixed row and column
totals r, N-r, c, N-c:
Test statistic = x, the frequency of cell11
Probability = hyper-geometric probability of x
successes in a sample of size r from a population
of size N with c successes

LARGE SAMPLE APPROXIMATION FOR

EITHER TEST
Chi squared
= Observed - Expected]2 /Expected
Observed frequency for cell ij comes from crosstabulation of data
Expected frequency for cell ij
= Probability Cell ij * N

Degrees of freedom (r-1)*(c-1)

COMPUTING CELL PROBABILITIES

Assumes independence or equal probabilities (the
null hypothesis)
Probability Cell ij = Probability Row i
* Probability Column j
= (R i/N) * (C j/N)
Expected frequency ij = (R/N)*(C/N)*N
= R*C/N.

DISTRIBUTION OF THE SUM

Chi Square with (r-1)*(c-1) degrees of freedom
Assumes
Observed - Expected]2 /Expected
is standard normal squared

 Implies
Observed - Expected] /Square root[Expected]
is standard normal

Implies
and Observed is a Poisson RV

Poisson is approximately normal if > 5, traditional guideline

Conovers relaxed guideline page 201

MEASURES OF STRENGTH:
CATEGORICAL VARIABLES

Phi 2x2
Cramer's V for rxc
Pearson's Contingency Coefficient
Tschuprow's T

MEASURES OF
STRENGTH: ORDINAL
VARIABLES

Lambda A .. Rows dependent

Lambda B .. Columns dependent
Symmetric Lambda
Kendall's tau-B
Kendall's tau-C
Gamma

STEPS OF STATISTICAL ANALYSIS

Significance - Strength
1- Test for significance of the observed association
2 - If significant, measure the strength of the
association

CONSIDER THE CORRELATION

COEFFICIENT
a measure of association (linear relationship
between two quantitative variables)
significant but not strong
significant and strong
not significant but strong
not significant and not strong

R AND PROB (P-VALUE)

r
r
r
r

=
=
=
=

.20
.90
.90
.20

p-value
p-value
p-value
p-value

<
<
>
>

.05
.05
.05
.05

CONCEPTS
Predictive associations must be both
significant and strong
In a particular application, an association may
be important even if it is not predictive (I.e.
strong)

MORE CONCEPTS
Highly significant , weak associations result from
large samples
Insignificant strong associations result from
small samples - they may prove to be either
predictive or weak with larger samples

EXAMPLES
Heart attack Outcomes by Anticoagulant
Treatment
Admission Decisions by Gender

SUMMARY
Is there an association?
Investigate with Chi square p-value

If so, how strong is it?

Select the appropriate measure of strength of
association

Where does it occur?

Examine cell contributions