AI & AT

Module III
Chapter 2
Statistical Reasoning
Bayes’ Rule Review

P( A | B)  P( B)
P( B | A) 
P( A)
P(B) is the prior probability of B
P(B | A) is the posterior probability of
B
P (H|E) is used to represent the
probability that some hypothesis,
H, is true, given evidence E.
Let us suppose we have a set of
hypotheses H1…Hn.
For each H
i

P( E | H i )  P( H i )
P( H i | E ) 
P( E )
P(Hi|E) = the probability that
hypothesis Hi is true given
evidence E
P(E|Hi) = the probability that we
will observe evidence E given the
hypothesis Hi is true
P(Hi) = the a priori probability
that hypothesis Hi is true in the
absence of any specific evidence
k = the number of possible
Bayes’ Theorem

P( E | H i )  P( H i )
P( H i | E ) 

k
n 1
P ( E | Hn ).P ( Hn )
Certainty Factors and
Rule-Based Systems
MYCIN
Early expert system to identify bacteria
causing severe infections and to
recommend antibiotics, with the dosage
adjusted for patient’s body weight
Ph. D work of Edward Shortlifee, Stanford
University , 1970
The MYCIN developers realized that a
Bayesian approach was intractable, as
too much data and/or
suppositions/estimates are required
MYCIN represents most of its
diagnostic knowledge as a set of
rules
Each rule has associated with it a
certainty factor, which is a
measure of the extent to which
the evidence that is described by
the antecedent of the rule
supports the conclusion that is
given in the rule’s consequent
Certainty Factors (CFs) are similar to
conditional probabilities, but somewhat
different
Rather than representing the degree of
probability of an outcome, they represent a
measure of belief in the outcome
Where probabilities range from

0 (false) to 1 (true), CFs range from:

◦ -1 believed not to be the case
◦ 1 believed to be the case
◦ The absolute size of the CF measures the degree of
belief
◦ The sign indicates belief vs disbelief
 A Typical MYCIN rule
If:(1) the stain of the organism is gram-positive, and
(2) the morphology of the organism is coccus, and
(3) the growth conformation of the organism is clumps,
then there is suggestive evidence (0.7) that the
identity of the organism is staphylococcus

The rule is represented internally using LISP

PREMISE: ($AND (SAME CNTXT GRAM GRAMPOS)
(SAME CNTXT MORPH COCCUS)
(SAME CNTXT CONFORM CLUMPS))
ACTION: (CONCLUDE CNTXT IDENT STAPHYLOCOCCUS TALLY 0.7)
Bayesian Network
Data Structure which represents
the dependence between
variables
Gives concise specification of the
joint probability distribution
A
Bayesian Network is a graph in
which the following holds:
1. A set of random variables makes up
the nodes in the network.
2. A set of directed links or arrows
connects pairs of nodes.
3. Each node has a conditional
probability table that quantifies the
effects the parents have on the node.
4. Directed, acyclic graph (DAG), i.e. no
directed cycles.
Example
C = that you will go to college
S = that you will study
P = that you will party
E = that you will be successful in
your exams
F = that you will have fun
 C

S P

E F
P(C) S P P(E)
0.2 true true 0.6
C P(S) true false 0.9
false true 0.1
true 0.8
false 0.2 false false 0.2

C P(P) P P(F)
true 0.6 true 0.9
false 0.5 false 0.7
Dempster-Shafer Theory
Dempster-Shafer theory is an approach to
combining evidence
Dempster (1967) developed means for
combining degrees of belief derived from
independent items of evidence
His student, Glenn Shafer (1976), developed
method for obtaining degrees of belief for
one question from subjective probabilities
for a related question
People working in Expert Systems in the
1980s saw their approach as ideally suitable
for such systems
This approach considers sets of
propositions and assigns to each
of them an interval [Belief,
Plausibility] in which the degree
of belief must lie
Belief (usually denoted Bel)
measures the strength of the
evidence in favor of a set of
propositions
It ranges from 0 (indicating no
Plausibility is defined to be
Pl(s) = 1 – Bel(¬s)
It also ranges from 0 to 1
It measures the extent to which
evidence in favor of ¬s leaves
room for belief in s
Fuzzy Logic
Fuzzy logic attempts to reflect
the human way of thinking
Fuzzy Logic was initiated in 1965
by
Lotfi A. Zadeh , Professor for
Computer Science at the
University of California in
Berkeley
Fuzzy Logic (FL) is a multivalued
logic, that allows intermediate
values to be defined between
conventional evaluations like
true/false, yes/no, high/low, etc.
Notions like rather tall or very
fast can be formulated
mathematically and processed by
computers, in order to apply a
more human−like way of thinking
in the programming of computers
Traditional set theory defines set
membership as a Boolean
predicate
Fuzzy set theory allows us to
represent set membership as a
possibility distribution
Characteristic Function of a
Crisp Set
Characteristic Function of a
Fuzzy Set
Fuzzy Logic applications:
Controlling and steering of
systems and complex industrial
processes
Household and entertainment
electronics