Asia Pacific Journal of Information Systems

Vol. 22, No. 4, December 2012

1)

Relationships between Collective Intelligence Quality,

Its Determinants, and Usefulness: A Comparative Study
between Wiki Service and Q&A Service in Perspective
of Korean Users*
Jaehun Joo**, Ismatilla R. Normatov***

Innovation can come from inside or outside organizations. Recently, organizations have begun turning to
external knowledge more often, through various forms of collective intelligence (CI) as collaborative platform
to solve complex problems. Several factors facilitate this CI utilization phenomenon. First, with the rapid development of Internet and social media, numerous web applications have become available to millions of the
Internet users over the past few decades. Web 2.0 and social media have become innovative web applications that provide an environment for human social interaction and collaboration. Second, the diffusion of
simple and easy-to-use technologies that enable users to interact and design web applications without programming skills have led to vast, previously unknown amounts of user-generated content. Finally, the Internet
has enabled communities to connect and collaborate, creating a virtual world of CI. In this study, web
enabled CI is defined as a composed ability of individuals who are acting as a single cognitive unit to
achieve common goals, think reasonably, solve problems, make decisions, carry out complex tasks, and
develop creative ideas collectively through participation and collaboration on the web.
Although CI plays a critical role in organizational innovation and collaboration, the dubious quality of CI
is still problem that is difficult to solve. In general, the quality level of content collected from the crowd
is lower than that from professionals. Thus, it is important to identify determinants of CI quality and to analyze
the relationship between CI quality and its usefulness. However, there is a lack of empirical study on the
quality factors of web-enabled CI. There exist a variety of web enabled CI sites such as Threadless, iStockphoto
or InnoCentive, Wikipedia, and Youtube. One of the most successful forms of web-enabled CI is the Wikipedia
online encyclopedia, accessible all over the world. Another one example is Naver KnowledgeiN, a typical
and popular CI site offering question and answer (Q&A) services. It is necessary to study whether or not
*

This work was supported by the National Research Foundation of Korea Grant funded by the Korean Government
(NRF-2012S1A5A2A01014345).
** Corresponding Author, Professor, College of Business and Tourism, Dongguk University-Gyeongju
*** Master of Business Adminstration, Cooperative Department of Electronic Commerce, Dongguk University-Gyeongju

Relationships between Collective Intelligence Quality, Its Determinants, and Usefulness

different types of CI have a different effect on CI quality and its usefulness. Thus, the purpose of this paper
is to answer to following research questions:

What determinants are important to CI quality?

What is the relationship between CI quality factors and the usefulness of web-enabled CI?
Does CI type have a moderating effect on the relationship between CI quality, its determinants, and
CI usefulness?
Online survey using Google Docs with email and Kakao Talk was conducted for collecting data from
Wikipedia and Naver KnowledgeiN users. A totoal of 490 valid responses were collected, where users of
Wikipedia were 220 while users of Naver KnowledgeiN were 270. Expertise of contributors, community size,
and diversity of contributors were identified as core determinants of perceived CI quality. Perceived CI quality
has significantly influenced perceived CI usefulness from a users perspective. For improving CI quality, it
is believed that organizations should ensure proper crowd size, facilitate CI contributors diversity and attract
as many expert contributors as possible. Hypotheses that CI type plays a role of moderator were partially
supported. First, the relationship between expertise of contributors and perceived CI quality was different
according to CI type. The expertise of contributors played a more important role in CI quality in the case
of Q&A services such as Knowledge iN compared to wiki services such as Wikipedia. This implies that Q&A
service requires more expertise and experiences in particular areas rather than the case of Wiki service to
improve service quality. Second, the relationship between community size and perceived CI quality was different
according to CI type. The community size has a greater effect on CI quality in case of Wiki service than
that of Q&A service. The number of contributors in Wikipeda is important because Wiki is an encyclopedia
service which is edited and revised repeatedly from many contributors while the answer given in Naver
Knowledge iN can not be corrected by others. Finally, CI quality has a greater effect on its usefulness in
case of Wiki service rather than Q&A service. In this paper, we suggested implications for practitioners and
theorists. Organizations offering services based on collective intelligence try to improve expertise of contributeros,
to increase the number of contributors, and to facilitate participation of various contributors.
Keywords : IS Management, Collective Intelligence, Collective Intelligence Quality, Collaborative Platform,
Wiki Service, Q&A Service, Wikipedia

, , :
Q&A
,

76 Asia Pacific Journal of Information Systems

. 2.0

[Leimeister,

Vol. 22, No. 4

Relationships between Collective Intelligence Quality, Its Determinants, and Usefulness

2010]. ,
SNS(social
network service), ,

[Alag, 2008;
Doan et al., 2011; Dutton, 2008].

iN[Black, 2008;
Nam et al., 2009].
. Chesbrough[2003]

. ,

.

.

,

[Hu et al., 2007; Javanmardi
et al., 2009; Lykourentzou et al., 2010]. Furtado
et al.[2010] 2.0

.

iN .


[Kim, 2010].
,


[Kim, 2010].


. , (Yahoo! Answers),
iN Q&A

Vol. 22, No. 4

[Kim,
2010]. ,


.



. ,

. ,

.

,
,
.

.


.

iN

. ,
iN Q&A
.



.

.
2.1
1910
Asia Pacific Journal of Information Systems 77

Relationships between Collective Intelligence Quality, Its Determinants, and Usefulness

[Wheeler, 1910].


[Bonadeau et al., 2001].
, ,
,
[Levy, 1997].


[Weschsler,
1971]. Russell[1995]

.

.

,

[Doan et al., 2011].


. Malone et al.
[2010]


.
.


, ,
.
UCC(User Created Content),
, (), (voting), ,
, ,
Q&A
. Doan et al.[2011]
, (explicit) (implicit)
78 Asia Pacific Journal of Information Systems

.
.
: , ,
: , ,
(Answers)
: , ,

Building artifacts: (
, , ), (),
(Digg.com)

[Doan et al., 2011].

. .


.
,
, .


.

.

.
( )
Q&A
.

2.2
< 1>
Vol. 22, No. 4

Relationships between Collective Intelligence Quality, Its Determinants, and Usefulness

< 1>

,
,

[]

Alag[2009]

Black[2008]

Boder[2006]

,
.

Colombino et al.
[2007]

,
.

Gholami and
Safavi[2010]

2.0
.

Lane[2010]

(, )
.

Leimester[2010]

,
.

Rubio et al.[2007]

2.0 (, ) .

Urzay[2008]

PandaLabs Panda Security

Solution .

Brabham[2008]

iStockphoto
.

Brabham[2010]

Threadless ,
.

Kuznetsov[2006]

Leimeister et al.
[2009]

, , (group identification)
.

Malone et al.[2010]

(genes) ,
love, glory, money .

Nam et al.[2009]

iN .

Prasarnpanich and
Wagner[2009]

Chesney[2006]

Denning et al.[2005] 6 .
Giles[2005]
Hu et al.[2007]

Vol. 22, No. 4

Javanmardi et al.
[2009]

(Britannica)
.
.
( )
.

Lykourentzou et al.
() .
[2010]
Stvilia et al.
[2005]

,
.

Lee and Ahn[2010]

[Davis et al., 1989] Sussman and Siegal[2003]

.

Normatov and Joo

[2011]

, ,
, .

Asia Pacific Journal of Information Systems 79

Relationships between Collective Intelligence Quality, Its Determinants, and Usefulness

3 .
, , .
Alag[2009]
. , ,
, , , , , ,
(voting), (user) ,
. Gholami and Safavi[2010]
2.0

. Urzay[2008]
Panda Security Solution
.

?
. Kuznetsov[2006]

. Prasarnpanich and Wagner
[2009]
. Malone
et al.[2010] , , , ,

, , ,
.

[Hu et al., 2007; Javanmardi et al., 2009;
Lykourentzou et al., 2010]. Lykourenzou et al.
[2010]
() ,

,

. Stvilia et al.[2005]


. < 1>

.
, [2010]
80 Asia Pacific Journal of Information Systems

. Normatov
[2011] 3
.
(,
, , )
.
, , ,
.
(crowd trait)
, ,
. Normatov
[2011]

.

.
3.1
Normatov and Joo[2011] Malone et al.[2010]


. , ,

,
,
.

. ,

.

. , ,

.
,
.

Vol. 22, No. 4

Relationships between Collective Intelligence Quality, Its Determinants, and Usefulness

,
.


.

.
,
, .

.

. iN

.

. iN


. ,
,

.


.

.
.


[ , 2010].

.
< 1>

.

Vol. 22, No. 4

[Alag, 2009].
, ,
.
,
,
.

H1

H2

H6

H3

H7-9

H5

H10

H4

< 1>

3.2


.


[Bonadeau, 2009]. Danielson
[2005] Q&A

.


.
iN . iN

Q&A .
iN

Asia Pacific Journal of Information Systems 81

Relationships between Collective Intelligence Quality, Its Determinants, and Usefulness

.

.

.
.
H1:
(+) .
Surowiecki[2004] ,

.

.
,

. Yetton and Bottger[1982]

.
Javanmardi et al.[2009]
,
. Lih
[2004]

.

.
,
,
.


. Callahan and Herring[2011]

.
27 : 1.

82 Asia Pacific Journal of Information Systems

.

.
.
H2: (+)
.
H3:
(+) .

[Bray et al., 2008].


[Leimeister, 2010].

.
Surowiecki[2004]




.


[Johansson,
2006]. .


[Lane, 2010]. Page[2007]


.

.

Vol. 22, No. 4

Relationships between Collective Intelligence Quality, Its Determinants, and Usefulness

[Surowiecki, 2004; Bonabeau, 2009].
,

.
.

.


. .
H4:
(+) .

[Jin et al., 2009;
Saeed and Abdinnour-Helm, 2008]. Sussman
and Siegal[2003] ELM(Elaboration Likelihood
Model)
.
[2009]
.

.
.
.
H5:
(+) .

[Javanmardi et al., 2009; Noubel,
2004; Rubio et al., 2007]. Dutton[2008]

(sharing) 1.0, (contributing)
2.0, (co-creating) 3.0
. ,
InnoCentive
1.0 .
Digg YouTube

2.0
. iN
2.0 .
3.0 .




.

iN Q&A
.


.
.



.
.

Alag[2009] .
,
.

H6:
.

H7:

.

Vol. 22, No. 4

Asia Pacific Journal of Information Systems 83

Relationships between Collective Intelligence Quality, Its Determinants, and Usefulness

< 2>

iN

Brabham[2010];
Convertino et al.[2010];
Surowiecki[2004]

iN

Javanmardi et al.[2009];
Lane[2010]

iN
(, , )

Juho[2009];
Leimeister[2010];
Page[2007]; Surowiecki
[2004]

,
,

Lee et al.[2002];
Nicolaou and
McKnight[2006]

Davis [1989];
Venkatesh and Davis
[2000]

H8:
.
H9:

.
H10:

.

3.3

.
.
< 2> ,
,

.


.
84 Asia Pacific Journal of Information Systems

.
4.1
.

.


.
iN.
Dutton[2008]
.
iN
Q&A . < 3>
.
.
(
iN )
.

Vol. 22, No. 4

Relationships between Collective Intelligence Quality, Its Determinants, and Usefulness

< 3>

(EC)

iN( )
(EC1).
(EC2).
(EC3).
( ) (CS1).

(CS)

(CS2).
(CS3).
(DC1).

(DC)

(DC2).
(DC3).
(DC4).

(PQ)

(PQ1).
(PQ2).
(PQ3).
(PU1).
(PU2).

(PU)

(PU3).
(PU4).
(PU5).
(PU6).

. ,

.
.

. 490 ,
iN 270,
220.

4.2
220(44.9%),
iN 270(55.1%) 490
Vol. 22, No. 4

.
, (296, 60.4%)
(194, 39.6%) 20
335(68.4%)
.
1~2
136(27.8%) ,
118(24.1%)

. ,
(368, 75.1%)
.
Asia Pacific Journal of Information Systems 85

Relationships between Collective Intelligence Quality, Its Determinants, and Usefulness

< 4>

4.3

iN

10
20
30
40
50

3~5
1~2
1~2

3~5
1~2
1~2

(%)
220
44.9
270
55.1
296
60.4
194
39.6
7
1.4
335
68.4
89
18.2
47
9.6
12
2.4
11
2.2
15
3.1
25
5.1
71
14.5
368
75.1
118
24.1
59
12.0
103
21.0
71
14.5
136
27.8
3
0.6

SPSS
AMOS . SPSS
1
5 .
< 5> .
0.7
[Hair et al.,
2010; Nunnally and Berstein, 1994]
.
(common method bias)
Harman
(Harman, 1967; Podsakoff
et al., 2003). < 5>
1 5,
37%.
.
< 6> AMOS
. < 6>

< 5>
/

(EC)

(DC)

(CS)

(PQ)

(PU)

EC1
EC2
EC3
DC1
DC2
DC3
DC4
CS1
CS2
CS3
PQ1
PQ2
PQ3
PU1
PU2
PU3
PU4
PU5
PU6

.124
.242
.172
.398
.019
.198
.221
.219
.240
.140
.289
.234
.186
.700
.768
.686
.719
.646
.609

.110
.086
.109
.632
.760
.737
.727
.081
.204
.241
.062
.101
.261
.233
.202
.098
.174
.096
.078

.829
.816
.781
.127
.056
.230
.153
.059
.146
.068
.146
.237
.105
.197
.064
.111
.055
.174
.231

86 Asia Pacific Journal of Information Systems

.121
.150
.151
.070
.146
.177
.131
.096
.167
.137
.796
.794
.656
.062
.120
.352
.321
.151
.109

.096
.099
.059
.168
.127
.170
.133
.765
.713
.782
.171
.047
.236
.191
.178
.142
.102
.139
.130

Cronbachs
(%)
Alpha

1.420

7.472
(53.451)

.814

1.700

8.945
(45979)

.781

1.098

5.778
(65.498)

.750

1.191

6.268
(59.719)

.791

7.036

37.034(37.03
4)

.853

Vol. 22, No. 4

Relationships between Collective Intelligence Quality, Its Determinants, and Usefulness

2

, GFI, NFI, RMSEA

.
< 7>

< 6> ( )

2/d.f.

GFI

AGFI

NFI

CFI

RMR

RMSEA

2.576

0.926

0.901

0.909

0.942

0.042

0.057

1. 3

0.9

0.9

0.9

0.9

0.05

0.08

< 7>
/

(EC)

(DC)

(CS)

(PQ)

(PU)

t-

EC1

0.766

0.069

14.645

EC2

0.846

0.074

15.197

EC3

0.706

DC1

0.663

0.058

13.809

DC2

0.556

0.061

11.517

DC3

0.800

DC4

0.728

0.060

15.106

CS1

0.633

CS2

0.761

0.098

12.140

CS3

0.730

0.093

11.926

PQ1

0.811

PQ2

0.783

0.060

16.295

PQ3

0.657

0.057

13.903

PU1

0.732

0.059

15.979

PU2

0.759

PU3

0.755

0.066

16.509

PU4

0.760

0.063

16.620

PU5

0.619

0.061

13.335

CR

AVE

0.827

0.600

0.779

0.480

0.775

0.504

0.806

0.567

0.864

0.497

*CR(Composite Reliability), AVE(Average Variance Extracted).

< 8>

EC

DC

CS

PQ

(EC)

0.775

(DC)

0.451

0.693

(CS)

0.386

0.626

0.710

(PQ)

0.518

0.489

0.536

0.753

(PU)

0.518

0.630

0.621

0.683

Vol. 22, No. 4

PU

0.705

Asia Pacific Journal of Information Systems 87

Relationships between Collective Intelligence Quality, Its Determinants, and Usefulness

(CR) (AVE)
. CR 0.7
(Hair et al., 2010).
< 8> AVE
.

(Gefen et al., 2000).

4.4

AMOS .
< 9> . < 9>
AGFI RMR
, GFI, CFI, RMSEA

.
< 10> < 2>
. ,
,
(+) (

0.1% 1%),

( 0.1%). , H1- H5
.

H1
0.346***

H2
0.453***

H5
0.737***

H3
0.894***

H4
0.200**

Note: * p < 0.05, ** p < 0.01. *** p < 0.001.

< 2>

4.5 iN
< 3> iN
.
.

< 9>

2/d.f.

GFI

AGFI

NFI

CFI

RMR

RMSEA

3.127

0.909

0.881

0.886

0.919

0.065

0.066

1. 3

0.9

0.9

0.9

0.9

0.05

0.08

< 10>

H1

ECPQ

0.346

***

H2

CSPQ

0.453

***

H3

CSDC

0.894

***

H4

DCPQ

0.200

**

H5

PQPU

0.737

***

Note: * p < 0.05. ** p < 0.01. *** p < 0.001.

88 Asia Pacific Journal of Information Systems

Vol. 22, No. 4

Relationships between Collective Intelligence Quality, Its Determinants, and Usefulness

(n = 220)

iN(n = 270)

0.211**

0.620***

0.433***

0.966***

0.278*

0.540***

0.822***

0.950***

0.272**

0.169*

Note: * p < 0.05, ** p < 0.01, *** p < 0.001.

< 3> iN
< 11>

t-

()

H6

CS DC

0.701

No

H7

EC PQ

1.883

Yes (0.05)*

H8

CS PQ

1.721

Yes (0.05)*

H9

DC PQ

0.809

No

H10

PQ PU

3.754

Yes(0.0005)***

t- Chin(2000) .

, Spooled = SQRT[{
, Spooled: pooled estimator for the variance
t
:
:
:

(multi-group analysis) .
< 11>
.
< 11>
EC PQ CS PQ
5%, PQ PU 0.1%
.
Vol. 22, No. 4

,
,
.
H7, H8, H10 .

.
, ,
Asia Pacific Journal of Information Systems 89

Relationships between Collective Intelligence Quality, Its Determinants, and Usefulness

?

(H6)
(H9)
. Q&A


. ( )

.

(H7)
(H8)
(H10)
.


(0.211) iN(0.433)
. iN Q&A

. iN
Q&A
.
iN

.

. Danielson[2005]
Q&A

. Q&A

. Q&A

[Kim, 2010].


90 Asia Pacific Journal of Information Systems

.
-
iN

.
iN(0.278) (0.620)

.


. Anthony et al.[2005]
.
Lih[2004]

.
H10,


. ,
(0.966) iN (0.540)
. Wilkinson and
Huberman[2007]
.


.

. iN

,
[Kim, 2010].
iN

[Kim,
2010].
,

. iN
Vol. 22, No. 4

Relationships between Collective Intelligence Quality, Its Determinants, and Usefulness

.
iN
.


[Stvilia et al.,
2005]. iN

.

.


.
,

.

. ,

,
,

, . ,



.
, iN Q&A


.
2002 iN 2009
, ,
.

.
Vol. 22, No. 4

.
,

.


. ,
,
.

,
,

.



.

.
,
,

[Brabham, 2008; Brabham, 2010; Gruber, 2008;
Narasimhan et al., 2007].
.

,

. ,
, ,
.


,
Asia Pacific Journal of Information Systems 91

Relationships between Collective Intelligence Quality, Its Determinants, and Usefulness

,
.
,
,
.


.
.

iN Q&A

. iN

. Q&A

.
Q&A

.

,

.

<References>
[1] Alag, S., Collective Intelligence in Action,
Manning Publications Co., USA, 2009.
[2] Anthony, D., Smith, S.W., and Williamson,
T., "Explaining quality in internet collective
goods: Zealots and good samaritans the case
of wikipedia," 2005, Retrieved from http://
descargas.ismarketing.com/QUALITY_COL
LECTIVE_GOODS_IX.pdf.
[3] Black, E.W., "Wikipedia and academic peer
review? Wikipedia as a recognized medium
for scholarly publication?," Online Information
Review, Vol. 32, No. 1, 2008, pp. 73-88.
[4] Boder, A., "Collective intelligence: A keystone in knowledge management," Journal
of Knowledge Management, Vol. 10, No. 1,
2006, pp. 81-93.
[5] Bonabeau, E. and Meyer, C., "Swarm intelligence: A whole new way to think about
business," Harvard Business Revie, Vol. 79,
2001, pp. 107-114.
[6] Bonabeau, E., "Decisions 2.0: The power of
collective intelligence," Sloan Management
Review, Vol. 50, No. 2, 2009, pp. 45-52.
92 Asia Pacific Journal of Information Systems

[7] Brabham, D.C., "Moving the crowd at threadless," Information, Communication and Society,
Vol. 13, No. 8, 2010, pp. 1122-1145.
[8] Brabham, D.C., "Crowdsourcing as a model
for problem solving: An introduction and cases," Convergence: The International Journal of
Research into New Media Technologies, Vol. 14,
No.1, 2008, pp. 75-90.
9] Bray, D., Laubavher, R., and Malone, T.,
"Collective intelligence: Promoting diversity,
crowd performance algorithms, and better decision outcomes," MIT Sloan School of Management,
2008, Retrieved from http://ssrn.com/abst
ract=1215863.
[10] Callahan, E.S. and Herring, S.C., "Cultural
bias in Wikipedia content on famous persons," Journal of the American Society for Information Science, Vol. 62, No. 10, 2011, pp.
1899-1915.
[11] Chesbrough, H.W., Open innovation: The
new imperative for creating and profiting from
technology, MA: Harvard Business School
Press, Boston, 2003.
Vol. 22, No. 4

Relationships between Collective Intelligence Quality, Its Determinants, and Usefulness

[12] Chesney, T., "An empirical examination of

Wikipedias Credibility," First Monday, Vol.
11, No. 11, 2006.
[13] Chin, W.W., "Frequently asked questionsPartial least squares and PLS-Graph," 2000,
Retrieved from http://disc-nt.cba.uh.edu/
chin/plsfaq.htm.
[14] Colombino, T., Grasso, A., and Martin, D.,
"Collective intelligence and the creation,
use and management of citizen-led journalism," 2007, Retrieved from http://www.
parc.com/content/events/attachments/co
lombino-CitizenJournalism.pdf.
[15] Convertino, G., Grasso, A., DiMicco, J.,
Michelis, D.G., and Chi, H.E., "Collective intelligence in organizations: Toward a research
agenda," Proceedings of the 2010 ACM Conference
on Computer Supported Cooperative Work, 2010,
pp. 613-614.
[16] Danielson, D.R., "Web credibility," Encyclopedia of Human-computer Interaction, In C. Ghaoui
(Ed.), Hershey, PA: Idea Group, 2005, pp.
713-721.
[17] Davis, F.D., "Perceived usefulness, perceived
ease of use and user acceptance of nformation
technology," MIS Quarterly, Vol. 13, No. 3,
1989, pp. 319-340.
[18] Denning, P., Horning, J., Parnas, D., and
Weinstein, L., "Wikipedia Risks," Communications of ACM, Vol. 48, No. 12, 2005, pp.
152.
[19] Doan, A., Ramakrishnan, R., and Halevy, A.,
"Crowdsourcing systems on the World-Wide
Web," Communications of the ACM, Vol. 54,
No. 4, 2011, pp. 86-96.
[20] Dutton, W.H., "The Wisdon of Collaborative
Network Organizations: Capturing the Value
of Networked Individuals," Prometheus, Vol.
Vol. 22, No. 4

[21]

[22]

[23]

[24]

[25]

[26]

[27]

[28]

26, No. 3, 2008, pp. 211-230.

Furtado, V., Ayres, L., Oliveira, M., Vasconcelos,
E., Caminha, C., DOrleans, J., and Belchior,
M., "Collective intelligence in law enforcement-The WikiCrimes system," Information
Sciences, Vol. 180, 2010, pp. 4-17.
Gefen, D., Straub, D.W., and Boudreau, M.-C.,
"Structural equation modeling and regression: Guidelines for research practice,"
Communications of the AIS, Vol. 4, No. 7, 2000,
pp. 1-79.
Gholami, B. and Safavi, R., "Harnessing collective intelligence: Wiki and social network
from end-user perspective," The Proceedings
of International Conference on e-Educatio, e-Business, e-Management and e-Learning, 2010, pp.
242-246.
Giles, J., "Internet encyclopedias go head to
head nature," Nature, Vol. 438, 2005, pp.
900-901.
Gruber, T.R., "Collective knowledge systems:
Where the social web meets the semantic
web," Web Semantics, Vol. 6, No. 1, 2008, pp.
4-13.
Hair, J.F., Black, W.C., Babin, B.J., and Anderson,
R.E., Multivariate Data Analysis, 7th ed., Pearso
n Education, 2010.
http://www.doria.fi/bitstream/handle/100
24/46992/nbnfi-fe200908031990.pdf.
https://scholarsbank.uoregon.edu/xmlui/
bitstream/handle/1794/10210/Lane-2010.p
df.
Hu, M., Lim, E.P., Sun, A., Lauw, H.W., and
Vuong, B.Q., "Measuring article quality in
Wikipedia: Models and evaluation," Proceedin
gs of CIKM07, Lisboa, Portugal, 2007, pp. 243-2
52.
Hur, S., Ryoo, S., and Jeon, S., "Determinants

Asia Pacific Journal of Information Systems 93

Relationships between Collective Intelligence Quality, Its Determinants, and Usefulness

[29]

[30]

[31]

[32]

[33]

[34]

[35]

[36]

of online review adoption: Focusing on online

review quality and consensus," Journal of Infor
mation Technology Applications and Managemen
t, Vol. 16, No. 4, 2009, pp. 41-58.
Javanmardi, S., Ganjisaffar, Y., Lopes, C., and
Baldi, P., "User contribution and trust in
Wikipedia," Proceedings of Collaborative Computing: Networking, Applications and Worksharing Conference, Washington DC. USA. 2009,
pp. 1-6.
Jin, X.L., Cheung, C.M.K., Lee, M.K.O., and
Chen, H.P., "How to keep members using
the information in a computer-supported social network," Computers in Human Behavior,
Vol. 25, No. 5, 2009, pp. 1172-1181.
Johansson, F., The Medici effect, what elephants
and epidemics can teach us about innovation,
Boston, MA, Harvard Business School Press,
2006.
Juho, S., "Applying collective intelligence to
idea evaluation at the front end of innovation," Lappeenranta University of Technology,
Masters thesis, 2009, Retrieved from.
Kim, S., "Answerers strategies to provide
credible information in question answering
community," Journal of the Korean Society for
Information Management, Vol. 27, No. 2, 2010,
pp. 21-35.
Kim, S., "Collective intelligence or communities? Charactrisitcis of the Korean internet
culture from a historical perspective," Journal
of Cybercommunication Academic Society, Vol.
27, No. 4, 2010, pp. 45-92.
Kuznetsov, S., "Motivations of contributors
to Wikipedia," ACM SIGCAS Computers and
Society, Vol. 36, No. 2, 2006, pp. 1-7.
Lane, S., "Collective intelligence for competitive advantage: Crowdsourcing and open

94 Asia Pacific Journal of Information Systems

[37]

[38]

[39]

[40]

[41]
[42]

[43]

[44]

innovation," Capston Report, University of

Oregon, 2010. Retrieved from
Lee, H. and Ahn, H., "A study on the user
acceptance model of mass collective intelligence," Journal of Information Technology
Applications and Management, Vol. 17, No. 4,
2010, pp. 1-17.
Lee, Y., Strong, D., Kahn, B., and Wang, R.,
"AIMQ: A methodology for information quality assessment," Information and Management,
Vol. 40, 2002, pp. 133-146.
Leimeister, J.M., "Collective intelligence,"
Business and Information Systems Engineering,
Vol. 2, No. 4, 2010, pp. 245-248.
Leimeister, J.M., Bretschneider, U., and Krcmar,
H., "Leveraging crowdsourcing: Activationsupporting components for IT-based ideas competition," Journal of Management Information
Systems, Vol. 26, No. 1, 2009, pp. 197-224.
Levy, P., Collective Intelligence-Mankinds Emerging World Cyberspace, Basic Books, 1997.
Lih, A., "Wikipedia as participatory journalism: Reliable sources? Metrics for evaluating
collaborative media as a news resource,"
Proceedings of 5th International Symposium on
Online Journalism, 2004, Retrieved from htt
p://jmsc.hku.hk/faculty/alih/publications/
utaustin-2004-wikipedia-rc2.pdf.
Lykourentzou, I., Papadaki, K., Vergados, D.
J., Polemi, D., and Loumos, V., "CorpWiki:
A self-regulating Wiki to promote corporate
collective intelligence through expert peer
matching," Information Sciences, Vol. 180, No.
1, 2010, pp. 18-38.
Malone, T.W., Laubacher, R., and Dellarocas,
C., "The collective intelligence genome," Sloan
Management Review, Vol. 51, No. 3, 2010, pp.
21-32.
Vol. 22, No. 4

Relationships between Collective Intelligence Quality, Its Determinants, and Usefulness

[45] Nam, K., Ackerman, M.S., and Adamic, L.A.,

"Question in, knowledge IN? A study of
Navers question answering community,"
Proceedings of the 27th International Conference
on Human Factors in Computing Systems, New
York: ACM, 2009, pp. 779-788.
[46] Narasimhan, E., Banerjee, U., and Nanda, H.,
"Applying collective intelligence to knowlnd
edge management," The Proceeding of the 2
International Conference on Informatics, 2007,
pp. 7-13.
[47] Nicolaou, A.I. and Mcknight, D.H., "Perceived
information quality in data exchanges: Effects
on risk, trust, and intention to use," Information System Research, Vol. 17, No. 3, 2006,
pp. 332-351.
[48] Normatov, I.R. and Joo, J., "A study on quality
factors of web enabled collective intelligence
as a donor for business success," The Journal
of Information Systems Research, Vol. 20, No.
3, 2011, pp. 209-235.
[49] Noubel, J.F., "Collective intelligence, the invisible
revolution," The Transitioner.org, 2004, Retrieved from http://www.thetransitioner.org/
Collective_Intelligence_Invisible_Revolution_J
FNoubel.pdf.
[50] Nunnally, J.C. and Bernstein, I.H., Psychometric
rd
Theory, 3 ed., NY: McGraw-Hill, 1994.
[51] Page, S., The differece: How the power of diversity
creates better groups, firms, schools, and societie,
Princeton: Princeton University Press, 2007.
[52] Podsakoff, P.M., MacKenzie, S.B., and
Podsakoff, N.P., "Common Method Biases in
Behavioral Research: A Critical Review of the
Literature and Recommended Remedies,"
Journal of Applied Psychology, Vol. 88, No. 5,
2003, pp. 879-903.
[53] Prasarnpanich P. and Wagner C., "The role
Vol. 22, No. 4

[54]

[55]
[56]

[57]

[58]

[59]
[60]

of Wiki technology and altruism in collaborative knowledge creation," Computer Information

Systems, Vol. 49, No. 4, 2009, pp. 33-41.
Rubio, R., Martn, S., Morn, S., "Collaborative
web learning tools: Wikis and Blogs," Comput.
Appl. Eng. Educ, Vol. 18, No. 3, 2007, pp. 502-511.
Russell, P., The Gglobal Bbrain Awakens, Miles
River Press, 1995.
Saeed, K.A. and Abdinnour-Helm, S., "Examining the effects of information system characteristics and perceived usefulness on post
adoption usage of information systems,"
Information and Management, Vol. 45, No. 9,
2008, pp. 376-386.
Steinbock, D., Kaplan, C., Rodriguez, M., Diaz,
J., Der, N., and Garcia, S., "Collective intelligence quantified for computer-mediated
group problem solving," Retrieved from
http://files.uniteddiversity.com/Decision_
Making_and_Democracy/Collective_Intelli
gence_for_Group_Problem_Solving.pdf,
University of California Santa Cruz, 2000.
Stvilia, B., Twidale, M., Gasser, L., and Smith,
L., "Information quality discussions in Wikipedia," UIUCLIS, Technical Report ISRN 2005/
2+CSCW, 2005, Retrieved from http://mailer.
fsu.edu/~bstvilia/papers/stvilia_wikipedia
_infoWork_p.pdf.
Surowiecki, J., The wisdom of the crowds,
London: Random House, 2004.
Sussman, S.W. and Siegal, W.S., "Informational Influence in oraganizaions: An integrated approach to knowledge adoption,"
Information Systems Research, Vol. 14, No. 1,
2003, pp. 47-65.

[61] Urzay, I., "Collective intelligence approaches

to malware recognition," Network Security,
Vol. 2008, No. 5, 2008, pp. 14-16.
Asia Pacific Journal of Information Systems 95

Relationships between Collective Intelligence Quality, Its Determinants, and Usefulness

[62] Venkatesh, V. and Davis, F.D., "A theoretical

extension of the technology acceptance model:
Four longitudinal field studies," Management
Science, Vol. 46, No. 2, 2000, pp. 186-204.
[63] Weschsler, D., "Concept of collective intelligence," American Psychologist, Vol. 26,
No.10, 1971, pp. 904-907.
[64] Wheeler, W.M., Ants: Their structure, development and behavior, New York, Columbia University Press, 1910.

96 Asia Pacific Journal of Information Systems

[65] Wilkinson, D.M. and Huberman, B.A., "Assessing the value of cooperation in Wikipedia,"
First Monday, Vol. 12, No. 4, 2007, Retrieved
from http://journals.uic.edu/fm/article/view/
1763/1643.
[66] Yetton, P.W. and Bottger, P.C., "Individual
versus group problem solving: An empirical
test of a best-member strategy," Organizational
Behavior and Human Decision Processes, Vol.
29, No. 3, 1982, pp. 307-321.

Vol. 22, No. 4

Relationships between Collective Intelligence Quality, Its Determinants, and Usefulness

<> iN
: ( )
( )
: ( ) 10~19 ( ) 20~29 ( ) 30~39 ( ) 40~49 ( ) 50~59 ( ) 60
: ( ) ( ) ( ) ( ) ( ) ______________
iN ?
( ) ( )
( ) 3-5
( ) 1~2
( ) 1~2
( )
5. iN ?
( ) 1~2 ( ) 1~2
( ) ( ) 3~5
( )

1.
2.
3.
4.

iN (Question) (Answer) .
(V) .

iN
.

iN
.

iN
.

iN
.

iN
.

iN .

iN .

iN ( )
.

iN
.

10

iN
.

11

iN .

12

iN .

13

iN .

14

iN .

15

iN .

16

iN .

17

iN .

18

iN .

19

iN .

Vol. 22, No. 4

Asia Pacific Journal of Information Systems 97

Relationships between Collective Intelligence Quality, Its Determinants, and Usefulness

.
.
.
.
.
.
.
( )
.
9. .
10. .
11. .
12. .
13. .
14. .
15. .
16. .
17. .
18 .
19. .
1.
2.
3.
4.
5.
6.
7.
8.

98 Asia Pacific Journal of Information Systems

Vol. 22, No. 4

Relationships between Collective Intelligence Quality, Its Determinants, and Usefulness

About the Authors

Jaehun Joo
Jaehun Joo is a professor of Department of Information Management at
Dongguk University-Gyeongju Campus. He was a visiting professor of
Department of Management at University of Nebraska-Lincoln. He received
his Ph. D. from Pusan National University. His areas of research interest
are e-commerce, collective intelligence, Semantic Web/ontology, business
ecosystems, and knowledge management. He published many papers in
International Journal of Human-Computer Studies, Journal of Sustainable
Tourism, Decision Support Systems, Information Systems Management,
International Journal of Industrial Engineering, Expert Systems with
Applications, Journal of Computer Information Systems, etc.
Ismatilla R. Normatov
Ismatilla R. Normatov has received his Bachelor of Arts in Business
Administration degree from Westminster International University in
Tashkent, Uzbekistan and MBA degree in Cooperative Department of
Electronic Commerce at Dongguk University Gyeongju Campus. He posses
Uzbek (native), Russian and English languages. His interest area of research
is electronic commerce models and technologies as well as knowledge
management, particularly collective intelligence.

SubmittedJuly 20, 2012

AcceptedDecember 10, 2012

1st revision October 29, 2012

Vol. 22, No. 4

Asia Pacific Journal of Information Systems 99