Fuzzy Logic and Fuzzy Set Theory

with examples from Image Processing

By: Rafi Steinberg
4/2/2008 1
Some Fuzzy Background
Lofti Zadeh has coined the term Fuzzy Set in 1965 and
opened a new field of research and applications
A Fuzzy Set is a class with different degrees of membership.
Almost all real world classes are fuzzy!
Examples of fuzzy sets include: {Tall people}, {Nice day},
{Round object}

If a persons height is 1.88 meters is he considered tall?
What if we also know that he is an NBA player?

2
Some Related Fields
Fuzzy
Logic &
Fuzzy Set
Theory
Evidence
Theory
Pattern
Recognition
& Image
Processing
Control
Theory
Knowledge
Engineering
3
Overview
L. Zadeh
D. Dubois
H. Prade
J.C. Bezdek
R.R. Yager
M. Sugeno
E.H. Mamdani
G.J. Klir
J.J. Buckley

4
Membership
Functions
Linguistic
Hedges
Aggregation
Operations
Image
Processing
Fuzzy
Morphology
Fuzzy
Measures

Fuzzy
Integrals
Fuzzy
Expert
Systems
Speech
Spectrogram
Reading
A Crisp Definition of Fuzzy Logic
Does not exist, however
- Fuzzifies bivalent Aristotelian (Crisp) logic
Is The sky are blue True or False?
Modus Ponens
IF <Antecedent == True> THEN <Do Consequent>
IF (X is a prime number) THEN (Send TCP packet)
Generalized Modus Ponens
IF a region is green and highly textured
AND the region is somewhat below a sky region
THEN the region contains trees with high confidence
5
Fuzzy Inference (Expert) Systems
Input_1
Fuzzy
IF-THEN
Rules

Output
Input_2
Input_3
6
Fuzzy Vs. Probability
Walking in the desert, close to being
dehydrated, you find two bottles of water:
The first contains deadly poison with a
probability of 0.1
The second has a 0.9 membership value in the
Fuzzy Set Safe drinks
Which one will you choose to drink from???

7
Membership Functions (MFs)
What is a MF?
Linguistic Variable
A Normal MF attains 1 and 0 for some input

How do we construct MFs?
Heuristic
Rank ordering
Mathematical Models
Adaptive (Neural Networks, Genetic Algorithms )


( ) ( )
{ }
1 2 1 2
, 1, 0
A A
x x x x - eO = =
8
Membership Function Examples
Trapezoidal
Triangular
( )
( )
1
, ,
1
smf
a x c
f x a c
e

=
+
Sigmoid
( )
( )
2
2
2
; ,
x c
gmf
f x c e
o
o

=
Gaussian
( ) ; , , , max min , 1, , 0
x a d x
f x a b c d
b a d c
| |
| |
=
| |

\ .
\ .
( ) ; , , max min , , 0
x a c x
f x a b c
b a c b
| |
| |
=
| |

\ .
\ .
9
Alpha Cuts
( )
{ }
A
A x X x
o
o = e >
( )
{ }
A
A x X x
o
o
+
= e >
Strong Alpha Cut
Alpha Cut
0 o =
0.2 o = 0.5 o = 0.8 o = 1 o =
10
Linguistic Hedges
Operate on the Membership Function (Linguistic Variable)
1. Expansive (Less, Very Little)
2. Restrictive (Very, Extremely)
3. Reinforcing/Weakening (Really, Relatively)
( ) Less x =
( )
4
Very Little x =
( ) ( )
2
Very x =
( ) ( )
4
Extremely x =
( ) ( )
A A
x x c
11
Aggregation Operations
( )
o o o o
o
1
2 1
2 1
, , ,
|
|
.
|

\
|
+ + +
=
n
a a a
a a a h
n
n

( )
0 0, ,1
i
and a i i n o o e9 = > e s s
, min
1 ,
0 ,
1 ,
, max
h
h Harmonic Mean
h Geometric Mean
h Al gebraic Mean
h
o
o
o
o
o
o
o
o
o
o

= =
=
= =

Generalized Mean:
12
Aggregation Operations (2)
Fixed Norms (Drastic, Product, Min)
Parametric Norms (Yager)
T-norms:
( )
, 1
, , 1
0 ,
D
b if a
T a b a if b
otherwise
=

= =

Drastic Product
( ) ( )
, min ,
Z
T a b a b =
( )
, T a b a b
-
=
Zadehian
( )
,
BS
S a b a b a b = +
( )
, 0
, , 0
1 ,
D
b if a
S a b a if b
otherwise
=

= =

S-Norm Duals:
( ) ( )
, max ,
Z
S a b a b =
Bounded Sum Drastic Zadehian
13
Aggregation Operations (3)
Drastic
T-Norm
Product
Zadehian
min
Generalized Mean
Zadehian
max

Bounded
Sum
Drastic
S-Norm
Algebraic (Mean)
Geometric
Harmonic
b (=0.8) a (=0.3)
( ) ( )
( )
( )
1
, min 1, 0,
w w
w
u a b a b for w = + e
Yager S-Norm
Yager S-Norm for varying w
14
Crisp Vs. Fuzzy
Fuzzy Sets
Membership values on [0,1]
Law of Excluded Middle and Non-
Contradiction do not necessarily
hold:




Fuzzy Membership Function
Flexibility in choosing the
Intersection (T-Norm), Union (S-
Norm) and Negation operations

Crisp Sets
True/False {0,1}
Law of Excluded Middle and Non-
Contradiction hold:




Crisp Membership Function
Intersection (AND) , Union (OR),
and Negation (NOT) are fixed

A A
A A
= C
= O
A A
A A
. > C
v s O
15
Image Processing
Binary
Gray Level
Color (RGB,HSV etc.)

Can we give a crisp definition to light blue?
16
Fuzziness Vs. Vagueness
Vagueness=Insufficient Specificity
I will be back
sometime
Fuzzy Vague
I will be back in
a few minutes
Fuzzy
Fuzziness=Unsharp Boundaries
17
Fuzziness
As the complexity of a system increases, our ability
to make precise and yet significant statements
about its behavior diminishes L. Zadeh
A possible definition of fuzziness of an image:

( )
2
min ,
ij ij
i j
Fuzz
M N
=

18
Example: Finding an Image Threshold
Membership Value
Gray Level
( )
( )
1
, ,
1
smf
a x c
f x a c
e

=
+
19
Mathematical Morphology
Operates on predefined geometrical objects in an image
Structured Element (SE) represents the shape of interest
Initially developed for binary images; extended to grayscale using
aggregation operations from Fuzzy Logic





Some Examples: Dilation, Erosion, Open, Close, Hit&Miss, Skeleton
20
0 1 0
0 1 0
0 1 0
SE
| |
|
=
|
|
\ .
68 12 4 32 60
16 12 4 32 60
16 40 28 56 12
16 40 52 8 12
40 72 76 8 12
imerode
| |
|
|
| =
|
|
|
\ .
68 12 4 32 60
92 80 28 56 64
16 40 52 80 88
40 100 76 84 12
44 72 100 8 36
im
| |
|
|
| =
|
|
|
\ .
Fuzzy Mathematical Morphology
Does it fit How well does a SE fit
( )
| |
| | | | { }
( )
| |
| | | | { }
,
,
, max , ,
, min , ,
G
j k B
G
j k B
D A B a m j n k b j k
E A B a m j n k b j k
e
e
= +
=
For B=0
( ) ( )
( ) ( )
, max
, min
G
B
G
B
D A B A
E A B A
=
=
( )
( )
( )
( )
max min
min max
G
B B
G
B B
O A
C A
=
=
21
Some Basic Concepts
{ }
, ,
# 2
L
L a b c
elements in the Power Set
=
=
Universe of Discourse:
Power Set of X= P(X)= {Null , {a} , {b} , {c} ,{a , b},{b , c}, {a , c}, {a,b,c}}
Singletons of the Power Set of X: { {a} , {b} , {c} }
An Event=An Element of the Power Set
Basic Probability Assignment (BPA)

Focal Element
m(A)=0.2
m(C)=0.5
m(B)=0.3
m(A)=0.2
Consonant Body of
Evidence
22
( )
( )
( )
1
0
i
i
A P X
m A
m |
e
=
=

Fuzzy Measures









Additive, Sub Additive, Super Additive Measures
Examples: {Probability}, {Belief, Plausibility}, {Necessity, Possibility}
( ) ( )
0 , 1 g g X | = =
( ) ( ) ( )
, , if A B A B P X then g A g B _ e s
(2) Monotonicity:
(1) Boundary Condition:
( ) | |
: 0,1 set function g P x
(3) Uniform Convergence
{ }
1
i
i
A

=

increasing sequence of measurable sets we have uniform convergence:

( ) ( ) ( )
( )
lim lim
i i
i i
A A

=
23
Example: Fuzzy Measure
Membership
in the Set
Our Line
Symbolic Representation of
the Measure
Observed
Image
1
0.7
0.5
0.7
0.2
0.3
0.2
0
( )
1 3
, x x
( )
1 2
, x x
( )
1 2 3
, , x x x
( )
2 3
, x x
( )
3
x
( )
2
x
( )
1
x
( )
|
{ }
1 2 3
, , Line x x x =
24
The Choquet Integral

Is defined over a Fuzzy Measure
Consider a gray level input

f(x
3
) f(x
2
) f(x
1
)
1 0.4 0.8
25
Example: Choquet Integral Calculation
Corresponding
Measure
Set
Representation
0.2 1 0 0
0.2 1 0 0
0.5 1 0 1
0.5 1 0 1
0.5 1 0 1
0.5 1 0 1
1 1 1 1
1 1 1 1
1 1 1 1
1 1 1 1
0.64 1 0.4 0.8
10
f
9
f
7
f
8
f
5
f
6
f
3
f
4
f
1
f
2
f
/10

1
x
2
x
3
x
26
Sugeno Measures
( )
( )
( )
( )
1
, 1,
1
A
A
A

= e
+
( ) ( ) ( ) ( ) ( )
( )
1,
A B A B A B
for and A B


|
v = + +
e =
Sugeno Inverse for ={-0.99, -0.9, -0.5, 0, 1, 10}
Sugeno Inverse:
Sugeno Measures Additional Axiom:
( ) ( )
1 1
i
i
x + = +
[
Compute from the normalization rule:
Optimistic/Pessimistic Aggregation of Evidence
27
Finding the Sugeno Measure
We need to solve the third order equation:

Solutions: {0, -15, 5/3}
Since =0 is the trivial additive solution and since
=-15 is out of range (>-1) we choose =5/3 and
obtain:

( )( )
2
1 1 0.3 1 0.2 + = + +
( )
1 3
, 0.47 x x =
( ) ( )
2 3 1 2
, , 0.6 x x x x = =
28
Example: Sugeno Integral Calculation
( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( )
3 1 3 1 2 3
, , , , , h g h a g x h b g x x h c g x x x

= v . . . =
}
( ) ( ) ( ) ( )
( ) ( ) ( )
0.9 0.2 , 0.5 0.47 , 0 1
max min 0.9, 0.2 , min 0.5, 0.47 0.47
= v . . . =
= =
-> We cannot aggregate with the Sugeno Union since the segmenting
alpha cut values are not part of our initial frame of discernment

-> Zadehian Max-Min are good default operators

29
h(q) is the alpha cut that entirely includes the measure of q.
Example: Finding Edges
1
2
min 1, min ,
1
ij
mn ij
i j
ij
W

| |
| |

= |
|
|
|
+
\ .
\ .

( ) ( )
( ) ( )
( )
( )
max min
min
1
max max max
ij ij ij
ij
spatial spatial
spatial
ij ij
ij ij ij
spatial global spatial
g g g
g g

| |
| |
|
|
|
\ .
= =
|

|
\ .
30
O.K. So Now What?
We have a fuzzy result, however in many cases we
need to make a crisp decision (On/Off)
Methods of defuzzifying are:
Centroid (Center of Mass)
Maximum
Other methods
( )
( )
A
A
x x dx
Centroid
x dx

}
}
31
Fuzzy Inference (Expert) Systems
Service
Time
Fuzzy
IF-THEN
Rules
Tip Level
Food
Quality
Ambiance
Fuzzify:
Apply MF on
input
Generalized Modus Ponens
with specified aggregation
operations
Defuzzify:
Method of Centroid,
Maximum, ...
32
Automatic Speech Recognition (ASR) via
Automatic Reading of Speech Spectrograms
Phoneme Classes:
Vowels
Semi-vowels/Diphthongs
Nasals
Plosives
Fricatives
Silence
Examples of Fuzzy Variables:
Distance between formants (Large/Small)
Formant location (High/Mid/Low)
Formant length (Long/Average/Short)
Zero crossings (Many/Few)
Formant movement (Descending/Ascending/Fixed)
VOT= Voice Onset Time (Long/Short)
Phoneme duration (Long/Average/Short)
Pitch frequency (High/Low/Undetermined)
Blob (F1/F2/F3/F4/None)
Dont ask me to carry"
33
Applying the Segmentation Algorithm
34
Suggested Fuzzy Inference System
Feature Vector
from Spectrogram
Identify Phoneme
Class using Fuzzy
IF-THEN Rules
Vowels Find Vowel
Fricatives
Nasals
Output Fuzzy MF
for each
Phoneme

35
Assign a Fuzzy Value for
each Phoneme, Output
Highest N Values to a
Linguistic model
Summary
36
Fuzzy Logic can be useful in solving Human related tasks
Evidence Theory gives tools to handle knowledge
Membership functions and Aggregation methods can be selected
according to the problem at hand

Some things we didnt talk about:
Fuzzy C-Means (FCM) clustering algorithm
Dempster-Schafer theory of combining evidence
Fuzzy Relation Equations (FRE)
Compositions
Fuzzy Entropy




References

[1] G. J. Klir ,U. S. Clair, B. YuanFuzzy Set Theory: Foundations and Applications , Prentice Hall PTR 1997, ISBN:
978-0133410587
[2] H.R. Tizhoosh;Fast fuzzy edge detection Fuzzy Information Processing Society, Annual Meeting of the North
American, pp. 239 242, 27-29 June 2002.
[3] A.K. Hocaoglu; P.D. Gader; An interpretation of discrete Choquet integrals in morphological image processing
Fuzzy Systems , Fuzzy Systems, FUZZ '03. Vol. 2, 25-28, pp. 1291 1295, May 2003.
[4] E.R. Daugherty, An introduction to Morphological Image Processing, SPlE Optical Engineering Press,
Bellingham, Wash., 1992.
[5] A. Dumitras, G. Moschytz, Understanding Fuzzy Logic An interview with Lofti Zadeh, IEEE Signal Processing
Magazine, May 2007
[6] J.M. Yang; J.H. Kim, A multisensor decision fusion strategy using fuzzy measure theory , Intelligent Control,
Proceedings of the 1995 IEEE International Symposium on, pp. 157 162, Aug. 1995
[7] R. Steinberg, D. OShaugnessy ,Segmentation of a Speech Spectrogram using Mathematical Morphology ,To
be presented at ICASSP 2008.
[8] J.C. Bezdek, J. Keller, R. Krisnapuram, N.R. Pal, Fuzzy Models and Algorithms for Pattern Recognition and Image
Processing Springer 2005, ISBN: 0-387-245 15-4
[9] W. Siler, J.J. Buckley,Fuzzy Expert Systems and Fuzzy Reasoning, John Wiley & Sons, 2005, Online ISBN:
9780471698500
[10] http://pami.uwaterloo.ca/tizhoosh/fip.htm
[11] "Heavy-tailed distribution." Wikipedia, The Free Encyclopedia. 22 Jan 2008, 17:43 UTC. Wikimedia Foundation,
Inc. 3 Feb 2008 http://en.wikipedia.org/w/index.php?title=Heavy-tailed_distribution&oldid=186151469
[12] T.J. Ross, Fuzzy Logic with Engineering Applications, McGraw-Hill 1997. ISBN: 0070539170


37
Heavy-Tail Distributions
38
Examples: Alpha Stable (Cauchy, Pareto), Weibull, Student-T, Log-Normal



Problem different samples with very low probability occur very frequently
Solution: Smoothing the probability density function; Good or Bad??
Another Solution: Use Possibility (Membership function) and Necessity as
envelopes

Example: Amazon sells far more books that are very unpopular than
popular books

Another example: Automatic translation most words in English have a
very low frequency of occurrence. However, we often find such rare words
in a sentence.
| | ( )
lim Pr , 0
x
x
e X x

> = >