Pattern Recognition(An Overview)

Data
Object Data
Relational Data
Pattern Recognition Tasks
Clustering: Finding groups in data (Fuzzy C-Mean)
Classification: Partitioning the feature space (Rule-Based
Classifier, Fuzzy K-NN Algorithm)
Feature Analysis: Feature selection, Feature ranking,
Dimensionality Reduction
Classification and Clustering
Class 1
Class 2
?
Classification
Clustering
Problem:
How to partition
How many clusters
Problem:
Which class to assign
Pattern Recognition(An Overview)
According to fuzzy Algebra every element of the universe
can belong to any fuzzy set with a degree of membership that
varies from 0 to 1 taking real values.
If an element of the universe belongs to a Fuzzy Set with a
degree of 1 then it belongs to its complement with a degree
of 1 1
Fuzzy clustering allows each feature vector to belong to
more than one cluster with different membership degrees
(between 0 and 1) and fuzzy boundaries between clusters.
Fuzzy Clustering
The Fuzzy C-means algorithm
The Fuzzy C-means algorithm
The Fuzzy C-means algorithm
The Fuzzy C-means algorithm
The Fuzzy C-means algorithm
The Fuzzy C-means algorithm
The Fuzzy C-means algorithm
_ a
1 2
, , ,
n
= X x x x
n is the number of data point in X
p
k
T x
p
is the number of features in each vector
A c-partition of X, which is matrix U c n v
Set of vectors
_ a
1 2
, , ,
p
c
= T V v v v
i
v
is called cluster center
Input: Unlabeled data set
Main Output
Common Additional Output
The Fuzzy C-means algorithm
Input, Output
This algorithm is based upon iterative optimization of the
objective function, with update of membership and cluster
centers.
This is based upon initial membership matrix for each item
in a cluster.
Center of clusters are calculated based upon the membership
function.
Once the centers are determined the membership matrix is
updated
When the difference between two sequential membership
matrix is less than the initial termination criterion the
algorithm is stopped. Otherwise step 2 and 3 are repeated.
The Fuzzy C-means algorithm
2
( , )
1 1
min ( , )
c n
m
m ik ik
i k
J u D
= =
|
=
, `
| )

U V
U V
Objective Function
2
2
ik k i
D =
A
x v
Distance
1 mu
Degree of
Fuzzification
1
1 ,
c
ik
i
u k
=
= V

Constraint
The Fuzzy C-means algorithm
Objective Function
Using Lagrange Multiplier technique, one can derive the following
update equations for the partition matrix and the prototype vectors
i
u
u
j i
D
D
u
n
k
m
ik
n
k
k
m
ik
i
c
k
m
ik
ij
ij
V

'
+

'

=
V

'
+

'

=
=

1
1
1
1
1
2
x
v
,
1)
2)
Fuzzy Classification
K- nearest neighbor algorithm: Voting on crisp labels
1
0
0

'

+
'

0
1
0

'

+
'

0
0
1

'

+
'

Class 1 Class 2 Class 3

z
K-nn Classification
The crisp K-nn rule can be generalized to generate fuzzy labels.
Take the average of the class labels of each neighbor:
D( )
.
.
.
z =

'

+
'

'

+
'

'

+
'

'

+
'

2
1
0
0
3
0
1
0
1
0
0
1
6
033
050
017
This method can be used in case the vectors have fuzzy or
possibilistic labels also.
K-nn Classification
Suppose the six neighbors of z have fuzzy labels as:
x x x x x x
1 2 3 4 5 6
09
0 0
01
09
01
0 0
03
0 6
01
0 03
095
0 02
0 2
08
0 0
03
0 0
0 7
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

'

+
'

'

+
'

'

+
'

'

+
'

'

+
'

'

+
'

D( )
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
z =

'

+
'

'

+
'

'

+
'

'

+
'

'

+
'

'

+
'

'

+
'

09
0 0
01
09
01
0 0
03
0 6
01
0 03
095
0 02
0 2
08
0 0
03
0 0
0 7
6
0 44
0 41
015
Rule1:
If x is CLOSE to a
1
and y is
CLOSE to b
1
then (x,y) is in class
is 1
Rule 2:
If x is CLOSE to a
2
and y is
CLOSE to b
2
then (x,y) is in class
is 2
How to get such rules!!
Fuzzy Rule Based Classifiers
An expert may provide us with classification rules.
We may extract rules from training data.
Clustering in the input space may be a possible way to
extract initial rules.
A
x
B
x
B
y
A
y
If x is CLOSE TO A
x
& y
is CLOSE TO A
y
Then
Class is
If x is CLOSE TO B
x
& y
is CLOSE TO B
y
Then
Class is
THANK YOU