K-Means & Fuzzy C-Means

Clustering

Clustering

Subset

Clustering

Partitioning Methods
Hierarchical Methods
Density-based Methods
Grid-based Methods
Model-Based Methods

Clustering-

Clustering-

Clustering-

Cell,

Clustering-

Clustering-
N
K K < N

Clustering-
K

Clustering-

K-MEANS

K-MEDOIDS

Clara

CLARANS

Fuzzy C-Means

Vehicle Example
Vehicle
V1
V2
V3
V4
V5
V6
V7
V8
V9

Top speed
(km/h)
220
230
260
140
155
130
100
105
110

Colour
red
black
red
gray
blue
white
black
red
gray

Air
resistance
0.30
0.32
0.29
0.35
0.33
0.40
0.50
0.60
0.55

Weight
Kg
1300
1400
1500
800
950
600
3000
2500
3500

Vehicle Clusterings
3500

3000

Lorries

Weight kg

2500

Sports cars

2000

1500

Medium market cars

1000

500
100

150

200
Top speed km/h

250

300

Terminology
Object or data point
3500

feature space
3000

Lorries

Weight kg

2500

cluster
Sports cars

2000

label
1500

Medium market cars

feature
1000

500
100

150

200
Top speed km/h

feature

250

300

K-MeansC-Means
NK
K

i 1

i 1 j 1

J J i w ji X j Ci

Ji i
K
Xj j
Ci i
wji Xj Ci

K-means
K

w
i 1

ji

1, j 1,..., N ;

w
i 1 j 1

ji

1, if X j Ci X j Cm , m j
w ji
otherwise
0,

Xj

1 0 1 0 0 0 1 0 0
W 0 1 0 1 0 1 0 0 0 Ci
0 0 0 0 1 0 0 1 1

K-means
1. kCii=1k

2. Xj
i wji = 1 0
1, if X j Ci X j Cm , m j
w ji
otherwise
0,

w
i 1

ji

1, j 1, ..., n,

w
i 1 j 1

ji

K-means
3. 1 J J

4
J

J
i 1

w
i 1

j 1

ji

Ci

4. 42
n

Ci

w
j 1
n

ji

Xj

w
j 1

4
ji

K-means
1.
2.
3.

Ktmax>0
Cj(0)0 < j K
for t=1,......,tmax
Afor j=1,......,N
(t )
( t 1)
d
ij
X j Ci
; i 1,...K
i

1, arg min iK1{d (jit ) }

iiwji
0, otherwise
N

Cit

(t )
w
ji X j
j 1

(t )
w
ji

; i 1,...K

j 1

1. C

E (t ) J (t ) J (t 1)
E(t)= C (t ) C (t 1)

K-Means
K

K-Means

K-Means

Cluster

K-Means
K-MeansHard
Clustering

200

210

250

20

20

20

20

20

20

Cluster 1: mean=35
Cluster 2: mean=230


Cluster 1

Cluster 1 or 2 ?

Cluster 2

Fuzzy C-Means
Dunn Ruspini
,, 1973 Jim Bezdek
Dunn m
Fuzzy C-Means KMeans W

http://www.cs.uwf.edu/~jbezdek/

Fuzzy C-Means
J 5
N m
J J i w ji X j Ci
i 1
i 1 j 1
K

Xj
Ci
N
K
m

K-Means
Xj
0.0025 0.0091 0.0129 0.0001 0.0107 0.9393 0.9638 0.9574 0.9906 0.9807
W

0.9975 0.9909 0.9871 0.9999 0.9893 0.0607 0.0362 0.0426 0.0094 0.0193

Ci

m
BezdekHathaway
mn/(n-2),n

PalBezdekm
[1.5, 2.5]
CheungChen
1.251.75

m
m,

Fuzzy C-means
1. k 0~1
1 6
K

w
i 1

ji

1, w ji 0,1 , j 1,......N , 0 w ji N 6
j 1

2. 7
N

Ci

m
w
ji X j
j 1
N

m
w
ji
j 1

Fuzzy C-Means
3. 5
4

E (t ) J (t ) J (t 1)
4. W8 2
w ji

1
X j Ci

X j Cs
s 1

2
m1

Tiles data: o = whole tiles, * = cracked tiles, x = centres

2
1

log(intensity) 557 Hz

0
-1
-2
-3
-4
-5
-6
-7
-8
-8

-6

-4

-2

log(intensity) 475 Hz

1.

Place two cluster centres

2.

Assign a fuzzy membership to each data point depending on distance

Tiles data: o = whole tiles, * = cracked tiles, x = centres

2
1

log(intensity) 557 Hz

0
-1
-2
-3
-4
-5
-6
-7
-8
-8

-6

-4

-2

log(intensity) 475 Hz

1.
2.

Compute the new centre of each class

Move the crosses (x)

Iteration 2

Tiles data: o = whole tiles, * = cracked tiles, x = centres

2
1

log(intensity) 557 Hz

0
-1
-2
-3
-4
-5
-6
-7
-8
-8

-6

-4

-2

log(intensity) 475 Hz

Iteration 5

Tiles data: o = whole tiles, * = cracked tiles, x = centres

2
1

log(intensity) 557 Hz

0
-1
-2
-3
-4
-5
-6
-7
-8
-8

-6

-4

-2

log(intensity) 475 Hz

Iteration 10

Tiles data: o = whole tiles, * = cracked tiles, x = centres

2
1

log(intensity) 557 Hz

0
-1
-2
-3
-4
-5
-6
-7
-8
-8

-6

-4

-2

log(intensity) 475 Hz

Iteration 13 (then stop, because no visible change)

Each data point belongs to the two clusters to a degree

The membership matrix W :

Xj
0.0025 0.0091 0.0129 0.0001 0.0107 0.9393 0.9638 0.9574 0.9906 0.9807
W

0.9975 0.9909 0.9871 0.9999 0.9893 0.0607 0.0362 0.0426 0.0094 0.0193

Ci

200

210

250

20

20

20

20

20

20

Image

0.2

0.2

0.2

0.9

0.9

0.9

0.9

0.9

0.9

Membership of cluster 1

0.8

0.8

0.8

c1
c2

0.2

0.2

0.2

0.9

0.9

0.9

0.9

0.9

0.9

0.8

0.8

0.8

0.1

0.1

0.1

0.1

0.1

0.1

Membership matrix
0.1

0.1

0.1

0.1

0.1

0.1

Membership of cluster 2

Fuzzy C-Means
Ktmax>0
Cj(0)0 < j K
for t=1,......,tmax
1
; i 1,..., K
Afor j=1,......,N , w(jit )
(
t
)
K X C

j
i

(t )

s 1
Bfor i=1,......,K,.
X j Cs
1.
2.
3.

2
m1

Ci(t )

m (t )
w
ji X j
j 1
N

m (t )
w
ji
j 1

C E (t ) J J

(t )
(t )

( t 1)

E (t ) C C (t 1)

Fuzzy C-Means

K-MeansFuzzy CMeans

Soft Clustering

Fuzzy C-Means

m


m=2
ep=0.3

37 ep=0.01