Hi ngh ton quc v iu khin v T ng ho - VCCA-2011

Ci tin gii thut mng noron SOM p dng phn cm mu nh Improving som neural network algorithm for color image clustering problem
L Anh T1, Nguyn Quang Hoan2
1

Trng i hc Cng ngh thng tin v truyn thng, i hc Thi Nguyn Email: anhtucntt@gmail.com 2 Hc vin Cng ngh Bu chnh Vin thng Email: quanghoanptit@yahoo.com.vn
BMU Best Matching Unit

Tm tt
Trong qu trnh th nghim mng noron SOM gii bi ton phn cm mu nh, chng ti nhn thy vic p dng thut ton tch t to nhm l khng ph hp v vic p dng gii thut loang khng em li kt qu nh mong mun. T , bi bo ny xut gii thut SOM ci tin cho php thc hin vic phn cm noron trn lp Kohonen ngay trong qu trnh hun luyn. Chng ti ci t thut ton ci tin v th nghim phn cm mu trn nh s. Qua quan st kt qu trc quan cho thy gii thut t c nhng kt qu kh n tng.

1. t vn
Mng noron SOM c Teuvo Kohonen pht trin vo nhng nm 80 [1,2,3,4]. y l mng truyn thng s dng thut hc cnh tranh, khng gim st [1,2]. SOM l mt cng c thch hp trong khai ph d liu [5]. N c ng dng trong nhiu bi ton [6] nh nhn dng ting ni Kohonen, 1989 , robotics Ritter et al., 1989 , my o Oja, 1992 , ti u t hp Fort, 1988 , phn lp nh (Kohonen, 1984 Mng noron SOM c thit k ch yu gim s chiu d liu v trc quan thng tin. Tuy nhin, c th kt hp SOM vi cc k thut xc nh cm khc phn cm d liu [11]. Hin ti, trong cc bi ton phn cm d liu nu s dng mng noron SOM cn thc hin theo hai giai on (H.1 : giai on th nht l thc hin gii thut SOM to bn phn b c trng ca tp d liu, giai on th hai trc quan bn v hnh thnh ng bin gia cc cm trn bn (ma trn) trc quan. Giai on mt c ngha quan trng bi n to ra tp nh x c trng trn lp ra Kohonen t tp d liu vo, trong cc d liu c s tng ng nhau s c c trng bi cc noron gn nhau trn lp Kohonen. Xt v bn cht, mi noron trn lp ra Kohonen l i din cho mt hoc mt s mu d liu, do vy vic phn cm d liu chnh l phn cm cc noron (xc nh mi noron thuc v cm no). Tuy nhin, gii thut SOM ch ng vai tr gom cm d liu m khng thc s quyt nh chnh xc d liu g thuc v cm no. V th, hiu qu phn cm ph thuc vo k thut trc quan bn v ph thuc vo thut ton p dng xc nh ranh gii cm trn ma trn trc quan thuc giai on hai. K thut trc quan c s dng ph bin l ma trn khong cch, trong mi phn t ca ma trn l khong cch ca mt n v noron trong SOM n cc lng ging (noron) lin k ca n. V d k thut trc quan dng ma trn hp nht khong cch UMatrix [7] hoc dng gi tr khong cch trung bnh

Abstract
During SOM Neural network testing to solve color image clustering problem, we find out that the application of agglomerative algorithm to create groups is less inappropriate and the application of the spreading algorithm does not produce desired results. Therefore, the paper proposed an improved SOM algorithm allowing implementation of the neural clustering in the Kohonen layer during the training. We implement improved SOM algorithm and experiment color clustering on digital images. Based on visual observation results, the algorithm has achieved quite impressive results.

K hiu
K hiu

n v

ngha hm ni suy theo thi gian hc Hm xc nh bn knh ln cn ti thi im t Bn knh khi to Vector trng s ti thi im t Hm ni suy tc hc ti thi im t Gi tr khi to ca tc hc

t
0
w t

t
0
Ch vit tt
SOM

SelfOrganizing Map

VCCA-2011

743

Hi ngh ton quc v iu khin v T ng ho - VCCA-2011

ti cc lng ging [8]. pht hin bin cm thng p dng thut ton tch t (Agglomerative Algorithm) hoc thut ton loang (Spreading Algorithm) theo ti thiu cc b t im nhn ca mi cm [9]. Tuy nhin, khi p dng thut ton tch t phi bit trc s cm cn hnh thnh, iu ny khng ph hp trong mt s bi ton thc t, v d nh phn cm mu trn nh s (v s lng cm ph thuc vo ni dung ca nh . Trng hp s khc bit ca cc phn t trn ma trn trc quan b (bin thin lin tc, nh), thut ton loang khng mang li kt qu nh mong i Tp hun luyn Mng noron SOM (H.5c). Ngoi ra, khi lng tnh ton ca cc thut ton ny tng i ln, nh hng ng k ti hiu qu ca ton b qu trnh. Trong bi bo ny, chng ti xut mt gii thut SOM ci tin (H.2), cho php hnh thnh cc cm noron ngay trong qu trnh hun luyn m khng cn thc hin thm giai on 2 nh m hnh H.1 . Nh vy, gii tht SOM ci tin ch cn mt giai on s gim ng k thi gian tnh ton do khng phi thc hin cc thut ton giai on hai nh nh m hnh H.1. Trc quan mng (to ma trn trc quan v pht hin bin cm trn ma trn trc quan)
Giai on 2

Gii thut SOM chun

Giai on 1 H. 1

Hnh thnh cm trn ma trn trc quan

Gii thut SOM chun p dng gii bi ton phn cm d liu

Tp hun luyn

Mng noron SOM

Gii thut SOM ci tin

H. 2

Hnh thnh cm trn lp Kohonen

Gii thut SOM ci tin p dng gii bi ton phn cm d liu

2. Mng noron SOM

2.1 Cu trc

- Chn ngu nhin mt u vo v t tp d liu, tnh khong cch dist gia v v vector trng s w tt c cc noron. Noron b c dist nh nht c chn lm noron chin thng (BMU). C th s dng cc hm khong cch nh Euclidian thng dng , Manhattan, hay Vector Dot Product.

dist || v
- Xc inh

w b || min || v mi ||
i

(1) ca BMU:

bn

knh

ln

cn

s t

s0 exp

t l hm ni suy bn knh l

H. 3

Cu trc mng SOM

gim dn theo s ln lp , vi 0 l bn knh ti K thi im t0; hng s thi gian l , vi log s0 K l t ng s ln lp. - Cp nht li trng s cho cc noron ln cn ca BMU theo hng gn hn vi vector u vo v

Mng noron SOM gm 2 lp: lp tn hiu vo v lp ra noron Kohonen. Lp Kohonen thng c t chc di dng mt ma trn 2 chiu cc noron. Mi n v i (noron) trng lp Kohonen c gn mt vector trng s wi= [wi1, wi2, ,win], vi n l kch thc vector u vo; wij c hiu l trng s ca noron i ng vi u vo j). 2.2 Thut ton hc Qu trnh hun luyn mng c lp nhiu ln, ti ln lp th t thc hin:

wi t

wi t

a t hbi t v

wi t

(2)

Trong : + a t

a0 exp

t l hm ni suy tc hc, l

vi 0 l gi tr khi to ca tc hc. + hbi(t) hm ni suy theo thi gian hc, th hin s tc ng ca khong cch i vi qu trnh hc,

VCCA-2011

744

Hi ngh ton quc v iu khin v T ng ho - VCCA-2011

c tnh theo
2

cng

thc

hbi t

exp

|| rb ri || 2s 2 t

trong rb v ri l v

tr ca noron b v noron i trong ma trn Kohonen. Hm li ca SOM xt trong trng hp tp d liu ri

N M

rc E
i 1 j 1

hbj || vi

w j ||2 trong , N l s mu
H. 4 Minh ha U-Matrix c m ha theo mc xm

u vo, M s noron trong lp Kohonen Thut ton hc SOM c tm lc trong 5 bc c bn nh sau: Bc 1: Xc nh cu hnh mng, v cc tham s. Bc 2: Khi to ngu nhin trng s ca cc noron lp Kohonen Lp li Bc 3: c mu hc v, a vo lp vo. Bc 4: Tm noron chin thng (BMU) trong lp Kohonen (gn mu hc v nht) Bc 5: Duyt mi noron trong bn knh ln cn ca BMU: - Cp nht li trng s theo cng thc (2) Cho n khi tha mn iu kin dng. Cc im cn lu trong thut ton hc: - Khi to cc tham s: SOM b nh hng rt nhiu bi qu trnh la chn cc tham s ca mng. Cc tham s ny bao gm: kch thc ca bn (Width/Height), s ln lp (K), bn knh khi to (0), gi tr khi to cho tc hc (0). Thc t khng c mt hng dn c th cho vic la chn cc tham s ny ng vi cc bi ton p dng SOM, do vy, vic th sai (trial and error) l cn thit nhm xc nh tp cc gi tr thch hp ng vi tp d liu u vo. - iu kin dng: C th da trn s ln lp, s mu hc, hay cn bng ca mng cc trng s thay i di mt ngng nht nh . 2.3 Trc quan mng s dng ma trn khong cch Phng php ny s dng khong cch gia cc vector trng s ca mi noron nh l mt tiu chun xc nh ranh gii gia cc cm. Gii php thng dng l xy dng ma trn U-Matrix [7], sau m ha ma trn ny theo mc xm (H.4). Cc cm trong ma trn l cc vng mu sng khong cch gia vector trng s ca cc noron nh , cc vng mu ti khong cch ln l min phn cch gia cc cm. xc nh mi n v trong ma trn thuc v cm no, ngi ta thng s dng thut ton tch t [9,10]. Cc bc ca thut ton nh sau: Bc 1: Quy cho mi n v trong ma trn mt cm ring Bc 2: Tnh ton khong cch gia tt c cc cm Bc 3: Ghp hai cm gn nht Bc 4: Nu s cm tn ti bng s cm do ngi dng nh ngha trc th dng, nu khng lp li t bc 2

Cng c th p dng thut ton loang pht hin cc cm mt cch t nhin. Trong trng hp gi tr trng s ca cc noron bin thin nh th gii thut ny hu nh tht bi do kh xc nh c chnh xc ranh gii ca cm . 2.4 Hn ch ca m hnh phn cm s dng gii thut SOM chun khi p dng phn cm mu trn nh Kh khn th nht l vic pht hin s lng v bin cm theo ni dung ca nh: nu p dng thut ton tch t cn bit trc s cm cn hnh thnh; cn p dng thut ton loang thng tht bi do kh xc nh c ranh gii chnh xc gia cc cm khi cc im mu trn nh c s bin thin nh H.5c). Kh khn th hai l tc hi t ca gii thut chm do khi lng tnh ton ln. Xt mt mng SOM vi kch thc 20x30=600 noron, phn gii ca bc nh u vo c tnh bng n v megapixel tc l c ti hng triu im nh. Nh vy ring trong qu trnh hun luyn, vic tm BMU phi duyt qua khong 600 triu ln cc noron. Ngoi ra, cc thut ton hnh thnh cm cng i hi s ln lp tng i ln.

a) Tp u vo b) Ma trn c) Kt qu hnh ngu nhin Kohonen thnh cm H. 5 Phn cm tht bi khi s dng thut ton loang

khc phc hai hn ch cn mt gii php hnh thnh cm ph hp v tng tc gii thut. l gii thut SOM ci tin ca bi bo ny.

3. Gii thut SOM ci tin

3.1 Cc nguyn tc thit lp cm Gii thut SOM ci tin gi l PSOM xut pht t tng xc nh ch s cm ca mi noron ngay trong qu trnh hun luyn, vi gi thit ban u mi noron cha c ch s thuc bt k cm no. Vi mi noron chin thng BMU ti mt bc trong qu trnh hun luyn ta xt 4 nguyn tc thit lp cm nh sau:

VCCA-2011

745

Hi ngh ton quc v iu khin v T ng ho - VCCA-2011

Nguyn tc hnh thnh cm: nu BMU cha cha thuc v mt cm no th hnh thnh mt cm mi, gm BMU v nhng noron khc trong bn knh tc ng ca BMU nu nhng noron ny cng cha thuc mt cm no. Nu nhng noron trong bn knh tc ng ca BMU thuc mt cm no th xy ra tranh chp noron gia cm mi v cm c. Nguyn tc phn ly cm: nu BMU thuc mt cm no th xt iu kin phn ly: nu c trng ca BMU khc so vi c trng ca cm cha n mt gi tr k >=T ngng phn ly th bn thn BMU s hnh thnh mt cm mi theo nguyn tc hnh thnh cm; ngc li BMU s cng c cm theo nguyn tc m rng cm. Nguyn tc m rng cm: nu BMU thuc mt cm no nhng khng tha iu kin phn ly th n s cng c cho chnh cm cha n bng cch kt np thm nhng noron trong bn knh tc ng ca n vo cm theo nguyn tc tng t nh hnh thnh cm. Nguyn tc tranh chp noron: mt noron M c coi l b tranh chp nu n thuc mt cm G1 v li nm trong bn knh tc ng ca mt BMU. Gi k1 l gi tr khc bit gia c trng ca M v c trng ca G1; k2 l gi tr khc bit gia c trng ca M v BMU. Nu k1<k2 th M vn thuc v cm G1. Ngc li, M thuc v cm ca BMU. 3.2 Gii thut PSOM Gii thut PSOM thc cht l gii thut SOM c b sung thm bc su. C th nh sau: Bc 1: Xc nh cu hnh mng, v cc tham s. Bc 2: Khi to ngu nhin trng s ca cc noron lp Kohonen Lp li Bc 3: c mu hc v, a vo lp u vo. Bc 4: Tm BMU trong lp Kohonen. Bc 5: - Xt 2 nguyn tc hnh thnh cm mi hoc phn ly cm - Duyt mi noron trong bn knh ln cn ca BMU: +Cp nht li trng s theo cng thc (2) +Cng c cm theo 2 nguyn tc m rng cm v tranh chp noron Cho n khi tha mn iu kin dng. Nhn xt: Bc 5 sa i c b sung thm vic xt cc nguyn tc thit lp cm. - Nu BMU cha thuc cm no th hnh thnh cm mi, ngc li nu thuc mt cm no th xt iu kin phn ly. Nu tha iu kin phn ly th tch BMU ra khi nhm c hnh thnh nhm mi. - Duyt mi noron Ni trong bn knh ln cn ca BMU, thc hin: + Cp nht li trng s theo cng thc 2) + Nu Ni cha thuc cm no th kt np Ni vo cm ca BMU, ngc li thuc mt cm GNi no th xy ra tranh chp noron Ni gia GBMU cm cha BMU v GNi cm cha Ni). Ni dung Bc 5 c th hin di dng ngn ng gi lp trnh nh sau: /*Nu BMU cha thuc mt cm no*/ If BMU.GroupName=null Then GroupCount = GroupCount+1; /*Hnh thnh cm mi*/ BMU.GroupName= GroupCount; Else /*Nu BMU thuc mt cm no */ K=Dist(WBMU,WNhm cha BMU); /*Hnh thnh cm mi do phn ly*/ If K>=T Then/*T l ngng phn ly*/ GroupCount = GroupCount+1; BMU.GroupName= GroupCount; End If End If /*Duyt qua mi noron Ni trong tp NR (l tp noron nm trong bn knh ln cn ca BMU)*/ For Each Ni in NR /*Cp nht trng s Ni theo cng thc (2)*/ UpdateWeight(Ni); /*Nu Ni cha thuc cm no*/ If Ni.GroupName=null Then Ni.GroupName=BMU.GroupName Else If Ni.GroupName BMU.GroupName Then /*Cm G(BMU) (cha BMU) cnh tranh vi cm G(Ni) (cha Ni) ginh noron Ni*/ If Compete(G(BMU),G(Ni)) = true Then Ni.GroupName=BMU.GroupName; End if; End if; Next; trong : W l vector trng s c trng ca noron GroupName ch ra tn cm m noron thuc v, khi noron cha thuc v mt cm no thuc tnh ny nhn gi tr null. Bin GroupCount cho bit t ng s cm vo thi im hin ti. Hm Dist(WBMU,WCm cha BMU) tr v gi tr khc bit khong cch gia vector c trng BMU v vector c trng ca cm cha BMU. C nhiu cch xc nh vector c trng ca cm. Trong bi bo ny chng ti s dng trung bnh cng ca cc vector c trng ca mi noron thnh phn trong cm. Hm Compete(G(BMU),GNi)=true nu kt qu cnh tranh noron Ni thuc v cm G(BMU), ngc li tr v gi tr fase. Function Compete(G(BMU),G(Ni)) C=false; If Dist(WG(BMU),WNi)<Dist(WG(Ni),WNi) Then

VCCA-2011

746

Hi ngh ton quc v iu khin v T ng ho - VCCA-2011

C=true; return C; End Function

4. Thc nghim
4.1 Thit k cu trc mng phn cm mu nh Mi im nh c coi l mt vector u vo vi 3 thnh phn Red, Green, Blue. Nh vy, mng gm c 3 u vo tng ng vi 3 gi tr mu ca im nh. Kch thc bn (width, height) l (20,40). 4.2 Hun luyn mng s dng PSOM Hnh H.6 th hin trc quan kt qu hun luyn v phn cm cc noron trc tip trn ma trn Kohonen bng gii thut PSOM. Kt qu H.6c vi cc cm c hnh thnh r rng, hn hn so vi kt qu H.5c s dng thut ton loang pht hin bin cm .
b)Ma trn c)Kt qu hnh a)Tp u vo Kohonen thnh cm ngu nhin H. 6 PSOM phn cm trn lp Kohonen vi tp hun luyn ngu nhin

4.3 Th nghim kh nng phn cm mu trn nh s S dng kt qu hun luyn ca PSOM phn cm mu trn cc nh. C th, mu 1 l nh vo vi cc chi tit phn bit mu sc kh r; mu 2 l nh chp t nhin. C hai trng hp u cho kt qu phn cm kh quan H.7 . Vi cc nh chp t nhin khc, kt qu th nghim l tng t. Bng trc quan cho thy cc cm c nhn dng chnh xc. Gii thut PSOM c hiu qu.
Hnh thnh cm trn ma trn Kohonen Phn cm mu trn nh

Mu

nh vo

Hnh nh ma trn Kohonen

H. 7

Kh nng phn cm mu nh s dng PSOM

Ti liu tham kho

[1] Juha Vesanto, Data Mining techniques based on the Self-Organizing Map, Thesis for the degree of Master in Engineering, Helsinki University of Technology, 1997. [2] Juha Vesanto, Using SOM in Data Mining, Licentiates thesis, Helsinki University of Technology, 2000. [3] Merja Oja, Samuel Kaski, and Teuvo Kohonen, Bibliography of Self-Organizing Map (SOM) Papers, 1998-2001 Addendum, Neural Computing Surveys (3), pp.1-156, 2003. [4] Teuvo Kohonen, Self-Organizing Springer, 3rd Edition, 2001. Maps,

[5] J. Han and M. Kamber, Data Mining - Concepts and Techniques, Morgan Kaufmann, Chapter 8, 2001. [6] Jari Kangas and Teuvo Kohonen, Developments and Application of the Self-Organizing Map and Related Algorithms, Mathematics and Computers in Simulation, 1996. [7] Alfred Ultsch and H. Peter Siemon, Kohonen's self-organizing feature maps for exploratory data analysis, Proceedings of the International Neural Network Conference (INNC'90), pages 305-308. Kluwer, 1990. [8] M. A. Kraaijveldd, J. Mao, and A. K. Jain, A nonlinear projection method based on kohonens

VCCA-2011

747

Hi ngh ton quc v iu khin v T ng ho - VCCA-2011

topology preserving maps, IEEE Transactions on Neural Networks, 6(3):548-559, 1995. [9] Anil K. Jain, Richard C. Dubes, Algorithms for clustering data, Prentice Hall, 1988. [10] Fuliang Weng, Andreas Stolcke, and Ananth Sankar, Hub4 Language Modeling Using Domain Interpolation and Data Clustering, Speech Technology And Research Laboratory, SRI International Menlo Park, California, 1997. [11]http://www.ifs.tuwien.ac.at/dm/somtoolbox/visual isations.html Nguyn Quang Hoan nhn bng K s T ng iu khin-K ngh My tnh ti Trng i hc ng st Mockva Lin x c nm 1973, hc v Tin s ti Vin Cng ngh Thng tin, Vin Khoa hc, Cng ngh Vit Nam (Vin Hn lm Khoa Hc: theo tn ngoi giao) nm 1997, hc hm PGS nm 2002. PGS Nguyn Quang Hoan lm nghin cu vin ti Vin Cng ngh Thng tin Vin (Vin Hn lm: theo tn ngoi giao) Khoa hc, Cng ngh Vit Nam 24 nm 1974-1998); ging vin 12 nm 1998-2010), trng khoa Cng ngh Thng tin v gi mt vi v tr khc ti Hc vin Cng ngh Bu chnh Vin thng. ng quan tm v iu khin, tin hc s dng cc phng php tin tin, c bit N ron v cc h lai vi N ron. PGS ang l Ging vin, Ch tch Hi ng Khoa hc o to ngnh CNTT ti Trng i hc S phm K thut Hng yn; ngi t vn (gi mt vi v tr) v nghin cu khoa hc, o to v cc d n u t lin quan mt s n v khc. L Anh T sinh nm 1980, nhn bng thc s ngnh Cng ngh thng tin ca i hc Thi Nguyn. Hin ang lm nghin cu sinh ti Vin Cng ngh thng tin Vin Khoa hc Vit Nam. Ging vin ti Trng i hc Cng ngh thng tin v truyn thng i hc Thi Nguyn. Hng nghin cu chnh v mng khng dy di ng v mng noron nhn to.

VCCA-2011

748