Formulated by Reverend Thomas Bayes in 1761. The theorem deals with conditional probability.

Bayes theorem is used to find P[A|B], when the available information is not readily compatible with that required to apply the definition of conditional probability. Sometimes it can be thought of as a formula for evaluating a certain kind of inverse probability. If we know the probability P[A|Bi] for all I, the theorem enables us to compute conditional probabilities in the other direction, P[Bj|A].

Bayes Theorem
Let A1,A2,An be a collection of mutually exclusive

events whose union is S. let B be an event such that P[B]0. Then for any of the events Aj, where j=1,2,3..n

P[Aj|B] = P[B|Aj]P[Aj] P[B|Ai]PAi]

I know the formula looks overwhelming, but lets look at a simple example to help ease our present thoughts.

Example: Urn I contains two white chips and one red chip. Urn II contains one white chip and two red chips. One chip is drawn from urn I and transferred to urn II. Then one chip is drawn from urn II. Suppose a red chip is selected from urn II. What is the probability the chip transferred was white? First lets see a picture of what is transpiring on the next slide.

Urn I
transferred

Urn II
drawn

2 3

What is the questioned to be answered?

Given that a red chip was selected from urn II, what is the probability that a white chip was transferred.
P [White chip transferred | Red chip was drawn] This looks like a simple conditional probability problem, but it is not. You will see this in the next upcoming slide when we write down all the probabilities that we know from the description. Bayes theorem will have to be used to solve this problem.

What do we know?
Let A1 be a red chip is transferred from urn I. Then the

P[A1] = 1/3. Let A2 be a white chip is transferred from urn I. Then the P[A2] = 2/3. Let B be a red chip is drawn from urn II. Then the P[B|A1]=3/4 and P[B|A2]=2/4. However, the questioned asked is P[A2|B]. We do not have this conditional probability. Therefore , we must use Bayes Thm.

P[Aj|B] = P[B|Aj]P[Aj] P[B|Ai]PAi]

P[A2|B] =

P[B|A2]P[A2] P[B|A2]P[A2]+P[B|A1]P[A1]

Looking at Bayes theorem written out in the context of this problem, it should be easily seen that we do know the probabilities to solve this question. P[A2|B] = (2/4)(2/3) = 4/7 = 0.5714 (2/4)(2/3)+(3/4)(1/3) Therefore, there is a 57.14% chance that a white chip was transferred.

One thing to remember about Bayes Theorem is that you will have to sum all possibilities in the denominator. For our example, there were only two possibilities. Either a white chip was transferred from urn I or a red chip was transferred from urn I. The numerator will always be the opposite conditional probability that was asked for. They asked for P[A2|B] and the term in the numerator was the opposite conditional probability, P[B|A2].

Lets take a look at the sample space for this problem.

The sample space will be written out in ordered pairs, such as (chip transferred, chip drawn). S={ (W1,W1) , (W1,W3) , (W2,W2), (W2,W3) , (R1,R1) , (R1,R2) , (R1,R3) , (R1,W3) , (W1,R2) , (W1,R3) , (W2,R2) , (W2,R3) }

If you look at the sample space, you will see that there are only seven events that involve a red chip being drawn (the second one in the ordered pair being red). If you look a little closer, you will notice that out of those seven events only four have a white chip being transferred (white chip being first in the ordered pair).