D A V I D H .

C R O P L E Y
J A M E S C . K A U F M A N
Measuring Functional Creativity: Non-Expert
Raters and the Creative Solution Diagnosis
Scale
ABSTRACT
The Creative Solution Diagnosis Scale (CSDS) is a 30-item scale based on a core
of four criteria: Relevance & Effectiveness, Novelty, Elegance, and Genesis. The CSDS
offers potential for the consensual assessment of functional product creativity. This
article describes an empirical study in which non-expert judges rated a series of
mousetrap designs using the 30-item CSDS. A conrmatory factor analysis revealed
a simple structure that corresponded closely to the a priori theoretical model of
functional creativity, resulting in a revised 24-item CSDS. Non-expert judges were
able to use the scale with a high degree of reliability and internal consistency. The
revised CSDS paves the way for further research into the use of non-expert judges as
a possible replacement for more costly, harder-to-obtain experts when measuring
product creativity using the Consensual Assessment Technique (CAT).
Keywords: factor analysis, functional creativity, product, scale.
INTRODUCTION
There is widespread agreement in the literature that creativity requires the ability
to produce outcomes that are novel, high quality, and appropriate to the task (e.g.,
Sternberg, Kaufman, & Pretz, 2002). Cropley and Cropley (2010a) expressed this
concept as the generation of effective novelty. These outcomes may include products,
services, ideas, processes, or procedures (Woodman, Sawyer, & Grifn, 1993). Many
have argued that creativity drives the broader innovation process of modern
economies (Florida, 2002). It does this both by underpinning the individual and
organizational skills needed to adapt to the pace and nature of change in the
modern world (Carnevale, Gainer, & Meltzer, 1990), and by acting as a key
ingredient in the process of generating new business opportunities, whether in the
form of a product, process, system, or service.
Puccio and Cabra (2010), in discussing the role that both individual and
organizational creativity play in driving innovation, also draw attention to the fact
119 The Journal of Creative Behavior, Vol. 46, Iss. 2, pp. 119137 2012 by the Creative Education Foundation, Inc. DOI: 10.1002/jocb.9
that innovation comes about as the result of the interaction among people, the
processes they engage in, and the environment in which they work (p. 149).
However, it is the end result, the way that these variables interact to form a product,
which is critical (Puccio, Murdock, & Mance, 2007). Given the importance of
creative products to the success of the innovation system, it therefore is axiomatic
that we understand how to characterize outcomes that are novel, high quality, and
task appropriate (Sternberg et al., 2002). The very nature of product creativity, as a
possible resource organizations can use to compete, emphasizes the need for its
measurement (Horn & Salvendy, 2009). The creative product is not simply the
conclusion of a process of creativity; it is the embodiment of that creativity.
MEASURING PRODUCT CREATIVITY
There have been extensive studies on how to measure a creative product in its
broadest sense. OQuin and Besemer (1999) describe three common approaches
used to measure product creativity: Indirect measurement, global judgment, and
criterion-based measurement. These approaches have been developed both in a
domain-general and a domain-specic context. Some of the possible solutions
include the use of expert raters (Amabile, 1996), divergent thinking-based scoring of
creative products for originality or uency (Reiter-Palmon, Illies Young, Kobe,
Buboltz, & Nimps, 2009), or its historical impact (Simonton, 2009a).
Horn and Salvendy (2006) offer a detailed comparison of specic product
creativity measurement tools, including rating scales and subjective assessments. The
former include Besemer and OQuins (1987, 1999) Creative Product Semantic Scale
(CPSS) and Reis and Renzullis (1991) Student Product Assessment Form, while the
latter is based on Amabiles (1983) Consensual Assessment Technique (CAT). Horn
and Salvendy (2006) also report that the rating scales have been tested in a variety
of domains, including art work, cartoons, chairs, advertisements, scientic and
creative writing, audio-visual products, and social studies. The CAT has been applied
to stories, art, poetry, and other aesthetic products. Much of the research has been
geared toward evaluating either aesthetic or organizational products. The assessment
of aesthetic work (such as a painting or poem) has been extensively investigated for
nearly a century (Baer, Kaufman, & Gentile, 2004; Cattell, Glascock, & Washburn,
1918; Child & Iwao, 1968). Within industrial/organizational psychology or business,
assessing creative products may mean studying group creativity or the performance
of teams (e.g., Shalley, 2002). There are, however, surprisingly few studies aimed at
assessing the creativity of products in the sense of tangible, scientic, or
technological productsthat is, engineered artifacts or manufactured consumer
goods. Where studies do relate to products, in the sense just described, it is
primarily in connection with related concepts, such as usability (see, e.g., Han,
Yun, Kim, & Kwahk, 2000). Looking at one such domain (mathematics), Mann
(2009) argues that many of the current assessments are time consuming to score;
they also tend to be separate instruments designed to measure the specic domains.
As a result, most of the work on mathematical creativity assessment cannot be easily
applied to related domains (such as engineering).
120
Measuring Functional Creativity
Simonton (2009b) argues for a hierarchy within domains, with hard sciences at
one end of the extreme (highest), soft sciences in the middle, and arts and
humanities at the other end (lowest). In essence, what much of the eld of creativity
assessment has done is to focus on the middle and lower end of Simontons
spectrum, perhaps to the neglect of the hard sciences.
FUNCTIONAL CREATIVITY
Cropley (2005) as well as Cropley and Cropley (2005, 2008, 2010a) have sought to
address this gap in research by focusing on novel products that serve some useful
social purpose, labeling their special quality functional creativity. This is consistent
with, and builds on, existing denitions of creativity, ranging from Bodens (1994)
three-criterion model (novel, valuable, and surprising), through Ochse (1990), and
Besemer and OQuins (1987) model (novelty, resolution, elaboration, and synthesis),
to Sternberg and Lubarts (1999) summary of novel (i.e., original and unexpected)
and appropriate (i.e., useful, adaptive concerning task constraints). Cropley and
Cropley argued that novelty seems intuitively to take precedence over usefulness in
determining creativity. However, in the practical world of products, processes, sys-
tems, and services, the most important aspect of an artifact that excites admiration in
the beholder is effectiveness. To take a simple example, an automobile must transport
people quickly, economically, and comfortably over long distances. If it fails to satisfy
requirements like these, then it lacks effectiveness and thus cannot be regarded as
creative, no matter how novel it is. Einstein, however, argued that it is not difcult to
nd novel solutions to problems that achieve the desired effect: The difcult part is
nding solutions that are elegant (see Miller, 1992). Grudin (1990) reinforced this
idea when he referred to the grace of great things [our italics]. Such solutions not
infrequently cause a more or less instantaneous shock of recognition when they
occur, and provoke a why didnt I think of that? reaction. Indeed, an elegant
solution may look so simple and obviousafter the factthat people may underrate
its creativity or denigrate it as banal. In addition, products that are not only useful
in the situation for which they were generated but can also be applied in other
apparently unrelated situations embody genesis. A product may introduce a new way
of conceptualizing an area, for instance, by opening up new approaches to existing
problems, or by drawing attention to the existence of previously unnoticed problems.
Cropley and Cropley (2005) classied creative products using the four dimensions
listed above. They arranged them in a hierarchy ranging from the routine product
(characterized by effectiveness alone) at one pole, to the innovative product
(characterized by effectiveness, novelty, elegance, and genesis) at the other, with
original and elegant products between these poles. This classication system is
shown in Table 1, where a plus sign means that a property is necessary for this kind
of product, a minus sign that it is not. The schematic in Table 1 can also be used to
demonstrate the position of pseudo- and quasi-creativity: that is, novelty without
effectiveness. The table shows that as a product moves from routine to innovative, it
incorporates all the properties of products at lower levels, but adds something to
them.
121
Journal of Creative Behavior
Routine products should not be dismissed out of hand. They may be very useful:
In areas such as engineering, for example, a very large number of products perform
important functions that benet humankind and contribute to the advancement of
society (i.e., they are useful), even though they are devoid of novelty. However,
because they lack novelty their creativity is qualitatively different. Changes to these
products, rather than representing functional creativity, may instead take the form
of replicationthat is, minor adaptations to existing ideas (Sternberg et al., 2002)
that develop what already exists according to existing lines of thought. It is only
when products move beyond repetition and effectiveness, and begin to incorporate
novelty that they enter the realm of creativity.
The hierarchical organization of products shown in Table 1 introduces a further
important principle into the discussion: Creativity is not an all-or-nothing quality of
a productthere are both levels and kinds of creativity. Creativity is not something
that products either have or do not have. Different products can have creativity to
greater or lesser degrees, or they can display different kinds of it. Beghetto, Kaufman
and Baxter (2011) give the metaphor of the all-or-nothing viewpoint as envisioning
creativity as a simple light switch; in more recent years, most theorists have
embraced the view of creativity more as a dimmer switch.
INDICATORS OF CREATIVITY
The functional model of creativity (Cropley & Cropley, 2005) provides a
number of broad properties of products that can be used to describe the level and
kind of creativity they possess. The question that arises at this point is that of the
observable characteristics of products that reveal the presence of these properties.
We refer to such characteristics as indicators of creativity. Scales developed by
psychologists for rating the creativity of products served as a basis for enriching
the functional model of creativity. An early example is Taylors (1975) Creative
Product Inventory, which measured the dimensions: Generation, reformulation,
originality, relevancy, hedonics, complexity, and condensation. More recently,
Besemer and OQuins (1987) Creative Product Semantic Scale dened the
creativity of products in terms of three dimensions: Novelty (the product is
original, surprising, and germinal), Resolution (the product is valuable, logical,
useful, and understandable), and Elaboration and Synthesis (the product is organic,
elegant, complex, and well-crafted).
TABLE 1. Levels and Kinds of Creativity in Products.
Criterion
Kind of Product
Routine Original Elegant Innovative Aesthetic
Effectiveness + + + +
Novelty + + + +
Elegance + + ?
Genesis + ?
122
Measuring Functional Creativity
Criteria such as hedonics or elegance are reminiscent of Jackson and Messicks
(1965) distinction between internal criteria, such as logic, harmony among
the elements of the product, pleasingness, and external criteria (i.e., is it relevant,
is it useful?). The indicators suggested by Taylor and Besemer and OQuin
give greater weight to internal criteria, such as elegance, complexity, or logic.
They involve a mixture of pure aesthetic, formalist, and technical properties (see
Slater, 2006), whereas they vary in the level of openness or closedness they dis-
play.
By contrast, the propulsion model (Sternberg, 1999; Sternberg, Kaufman, &
Pretz, 2003; Sternberg et al., 2002) turns directly to external indicators. According to
them, a creative product achieves its external effect by propelling a eld. They
suggested a number of ways in which this can occur: These include conceptual
replication (the known is transferred to a new setting), redenition (the known is
seen in a new way), forward and advanced forward incrementation (the known is
extended in various ways), redirection (the known is extended in a new direction),
reconstruction (new life is breathed into an approach previously abandoned), and
reinitiation (thinking begins at a radically different point from the current one and
takes off in a new direction). Savransky (2000) also discussed the processes through
which existing knowledge leads to effective novelty in the external world: He argued
that inventive solutions to problems always involve changing what already exists,
and discerned a variety of ways in which this can occur, including improvement,
diagnostics, synthesis, and genesis.
Cropley and Cropley (2005) enriched their hierarchical, four-criterion model of
functional creativity with the indicators described above to dene a Creative
Solution Diagnosis Scale (CSDS). Table 2 shows the full CSDS.
RECOGNIZING CREATIVITY
To serve as the basis of an instrument for measuring the creativity of products,
the internal and external properties (indicators) outlined in the previous section
conjointly with Table 2 would have to be recognizable to observers. The most
straightforward way of checking whether people really can recognize creativity when
they see it is to ask them. This idea is at the heart of the method of consensual
assessment (for a summary, see Amabile, 1996). Amabile along with her colleagues
has developed and rened this approach, furthermore, the CAT is now well known
among creativity researchers (Kaufman, Plucker, & Baer, 2008b). The method
frequently involves asking judges, usually experts in the eld to which the product
belongs, to rate the creativity of a product.
The need for experts highlights a limitation of the Consensual Assessment
Technique. It can be expensive (to obtain multiple expert raters), cumbersome (all
products must be viewed separately by the experts), and time consuming (Kaufman,
Baer, Cole, & Sexton, 2008a).
The role that expertise plays in assessing creativity is by no means settled.
Besemer and OQuin (1999), for example, demonstrated that people without spe-
cial expertise were able to distinguish consistently among products (four chairs of
123
Journal of Creative Behavior
T
A
B
L
E
2
.
T
h
e
C
o
m
p
l
e
t
e
C
r
e
a
t
i
v
e
S
o
l
u
t
i
o
n
D
i
a
g
n
o
s
i
s
S
c
a
l
e
.
C
r
i
t
e
r
i
o
n
o
f
C
r
e
a
t
i
v
i
t
y
K
i
n
d
o
f
S
o
l
u
t
i
o
n
P
r
o
p
e
r
t
y
o
f
t
h
e
S
o
l
u
t
i
o
n
I
n
d
i
c
a
t
o
r
R
e
l
e
v
a
n
c
e
&
E
f
f
e
c
t
i
v
e
n
e
s
s
R
o
u
t
i
n
e
S
o
l
u
t
i
o
n
d
i
s
p
l
a
y
s
k
n
o
w
l
e
d
g
e
o
f
e
x
i
s
t
i
n
g
f
a
c
t
s
a
n
d
p
r
i
n
c
i
p
l
e
s
a
n
d
s
a
t
i
s

e
s
t
h
e
r
e
q
u
i
r
e
m
e
n
t
i
n
t
h
e
p
r
o
b
l
e
m
s
t
a
t
e
m
e
n
t
C
O
R
R
E
C
T
N
E
S
S
(
t
h
e
s
o
l
u
t
i
o
n
a
c
c
u
r
a
t
e
l
y
r
e

e
c
t
s
c
o
n
v
e
n
t
i
o
n
a
l
k
n
o
w
l
e
d
g
e
a
n
d
/
o
r
t
e
c
h
n
i
q
u
e
s
)
P
E
R
F
O
R
M
A
N
C
E
(
t
h
e
s
o
l
u
t
i
o
n
d
o
e
s
w
h
a
t
i
t
i
s
s
u
p
p
o
s
e
d
t
o
d
o
)
A
P
P
R
O
P
R
I
A
T
E
N
E
S
S
(
t
h
e
s
o
l
u
t
i
o
n

t
s
w
i
t
h
i
n
t
a
s
k
c
o
n
s
t
r
a
i
n
t
s
)
O
P
E
R
A
B
I
L
I
T
Y
(
t
h
e
s
o
l
u
t
i
o
n
i
s
e
a
s
y
t
o
u
s
e
)
S
A
F
E
T
Y
(
t
h
e
s
o
l
u
t
i
o
n
i
s
s
a
f
e
t
o
u
s
e
)
D
U
R
A
B
I
L
I
T
Y
(
t
h
e
s
o
l
u
t
i
o
n
i
s
r
e
a
s
o
n
a
b
l
y
s
t
r
o
n
g
)
N
o
v
e
l
t
y
O
r
i
g
i
n
a
l
P
r
o
b
l
e
m
a
t
i
z
a
t
i
o
n
(
s
o
l
u
t
i
o
n
d
r
a
w
s
a
t
t
e
n
t
i
o
n
t
o
p
r
o
b
l
e
m
s
i
n
w
h
a
t
a
l
r
e
a
d
y
e
x
i
s
t
s
)
D
I
A
G
N
O
S
I
S
(
t
h
e
s
o
l
u
t
i
o
n
d
r
a
w
s
a
t
t
e
n
t
i
o
n
t
o
s
h
o
r
t
c
o
m
i
n
g
s
i
n
o
t
h
e
r
e
x
i
s
t
i
n
g
s
o
l
u
t
i
o
n
s
)
P
R
E
S
C
R
I
P
T
I
O
N
(
t
h
e
s
o
l
u
t
i
o
n
s
h
o
w
s
h
o
w
e
x
i
s
t
i
n
g
s
o
l
u
t
i
o
n
s
c
o
u
l
d
b
e
i
m
p
r
o
v
e
d
)
P
R
O
G
N
O
S
I
S
(
t
h
e
s
o
l
u
t
i
o
n
h
e
l
p
s
t
h
e
b
e
h
o
l
d
e
r
t
o
a
n
t
i
c
i
p
a
t
e
l
i
k
e
l
y
e
f
f
e
c
t
s
o
f
c
h
a
n
g
e
s
)
S
o
l
u
t
i
o
n
a
d
d
s
t
o
e
x
i
s
t
i
n
g
k
n
o
w
l
e
d
g
e
R
E
P
L
I
C
A
T
I
O
N
(
t
h
e
s
o
l
u
t
i
o
n
u
s
e
s
e
x
i
s
t
i
n
g
k
n
o
w
l
e
d
g
e
t
o
g
e
n
e
r
a
t
e
n
o
v
e
l
t
y
)
C
O
M
B
I
N
A
T
I
O
N
(
t
h
e
s
o
l
u
t
i
o
n
m
a
k
e
s
u
s
e
o
f
n
e
w
m
i
x
t
u
r
e
(
s
)
o
f
e
x
i
s
t
i
n
g
e
l
e
m
e
n
t
s
)
I
N
C
R
E
M
E
N
T
A
T
I
O
N
(
t
h
e
s
o
l
u
t
i
o
n
e
x
t
e
n
d
s
t
h
e
k
n
o
w
n
i
n
a
n
e
x
i
s
t
i
n
g
d
i
r
e
c
t
i
o
n
)
S
o
l
u
t
i
o
n
d
e
v
e
l
o
p
s
n
e
w
k
n
o
w
l
e
d
g
e
R
E
D
I
R
E
C
T
I
O
N
(
t
h
e
s
o
l
u
t
i
o
n
s
h
o
w
s
h
o
w
t
o
e
x
t
e
n
d
t
h
e
k
n
o
w
n
i
n
a
n
e
w
d
i
r
e
c
t
i
o
n
)
R
E
C
O
N
S
T
R
U
C
T
I
O
N
(
t
h
e
s
o
l
u
t
i
o
n
s
h
o
w
s
t
h
a
t
a
n
a
p
p
r
o
a
c
h
p
r
e
v
i
o
u
s
l
y
a
b
a
n
d
o
n
e
d
i
s
s
t
i
l
l
u
s
e
f
u
l
)
R
E
I
N
I
T
I
A
T
I
O
N
(
t
h
e
s
o
l
u
t
i
o
n
i
n
d
i
c
a
t
e
s
a
r
a
d
i
c
a
l
l
y
n
e
w
a
p
p
r
o
a
c
h
)
R
E
D
E
F
I
N
I
T
I
O
N
(
t
h
e
s
o
l
u
t
i
o
n
h
e
l
p
s
t
h
e
b
e
h
o
l
d
e
r
s
e
e
n
e
w
a
n
d
d
i
f
f
e
r
e
n
t
w
a
y
s
o
f
u
s
i
n
g
t
h
e
s
o
l
u
t
i
o
n
)
G
E
N
E
R
A
T
I
O
N
(
t
h
e
s
o
l
u
t
i
o
n
o
f
f
e
r
s
a
f
u
n
d
a
m
e
n
t
a
l
l
y
n
e
w
p
e
r
s
p
e
c
t
i
v
e
o
n
p
o
s
s
i
b
l
e
s
o
l
u
t
i
o
n
s
)
124
Measuring Functional Creativity
T
A
B
L
E
2
.
(
C
o
n
t
i
n
u
e
d
)
.
C
r
i
t
e
r
i
o
n
o
f
C
r
e
a
t
i
v
i
t
y
K
i
n
d
o
f
S
o
l
u
t
i
o
n
P
r
o
p
e
r
t
y
o
f
t
h
e
S
o
l
u
t
i
o
n
I
n
d
i
c
a
t
o
r
E
l
e
g
a
n
c
e
E
l
e
g
a
n
t
S
o
l
u
t
i
o
n
s
t
r
i
k
e
s
o
b
s
e
r
v
e
r
s
a
s
b
e
a
u
t
i
f
u
l
(
e
x
t
e
r
n
a
l
e
l
e
g
a
n
c
e
)
R
E
C
O
G
N
I
T
I
O
N
(
t
h
e
b
e
h
o
l
d
e
r
s
e
e
s
a
t
o
n
c
e
t
h
a
t
t
h
e
s
o
l
u
t
i
o
n

m
a
k
e
s
s
e
n
s
e

)
C
O
N
V
I
N
C
I
N
G
N
E
S
S
(
t
h
e
b
e
h
o
l
d
e
r
s
e
e
s
t
h
e
s
o
l
u
t
i
o
n
a
s
s
k
i
l
l
f
u
l
l
y
e
x
e
c
u
t
e
d
,
w
e
l
l
-

n
i
s
h
e
d
)
P
L
E
A
S
I
N
G
N
E
S
S
(
t
h
e
b
e
h
o
l
d
e
r

n
d
s
t
h
e
s
o
l
u
t
i
o
n
n
e
a
t
,
w
e
l
l
-
d
o
n
e
)
S
o
l
u
t
i
o
n
i
s
w
e
l
l
w
o
r
k
e
d
o
u
t
a
n
d
h
a
n
g
s
t
o
g
e
t
h
e
r
(
i
n
t
e
r
n
a
l
e
l
e
g
a
n
c
e
)
C
O
M
P
L
E
T
E
N
E
S
S
(
t
h
e
s
o
l
u
t
i
o
n
i
s
w
e
l
l
w
o
r
k
e
d
o
u
t
a
n
d

r
o
u
n
d
e
d

)
G
R
A
C
E
F
U
L
N
E
S
S
(
t
h
e
s
o
l
u
t
i
o
n
w
e
l
l
-
p
r
o
p
o
r
t
i
o
n
e
d
,
n
i
c
e
l
y
f
o
r
m
e
d
)
H
A
R
M
O
N
I
O
U
S
N
E
S
S
(
t
h
e
e
l
e
m
e
n
t
s
o
f
t
h
e
s
o
l
u
t
i
o
n

t
t
o
g
e
t
h
e
r
i
n
a
c
o
n
s
i
s
t
e
n
t
w
a
y
)
S
U
S
T
A
I
N
A
B
I
L
I
T
Y
(
t
h
e
s
o
l
u
t
i
o
n
i
s
e
n
v
i
r
o
n
m
e
n
t
a
l
l
y
f
r
i
e
n
d
l
y
)
G
e
n
e
s
i
s
I
n
n
o
v
a
t
i
v
e
I
d
e
a
s
i
n
t
h
e
s
o
l
u
t
i
o
n
g
o
b
e
y
o
n
d
t
h
e
i
m
m
e
d
i
a
t
e
s
i
t
u
a
t
i
o
n
F
O
U
N
D
A
T
I
O
N
A
L
I
T
Y
(
t
h
e
s
o
l
u
t
i
o
n
s
u
g
g
e
s
t
s
a
n
o
v
e
l
b
a
s
i
s
f
o
r
f
u
r
t
h
e
r
w
o
r
k
)
T
R
A
N
S
F
E
R
A
B
I
L
I
T
Y
(
t
h
e
s
o
l
u
t
i
o
n
o
f
f
e
r
s
i
d
e
a
s
f
o
r
s
o
l
v
i
n
g
a
p
p
a
r
e
n
t
l
y
u
n
r
e
l
a
t
e
d
p
r
o
b
l
e
m
s
)
G
E
R
M
I
N
A
L
I
T
Y
(
t
h
e
s
o
l
u
t
i
o
n
s
u
g
g
e
s
t
s
n
e
w
w
a
y
s
o
f
l
o
o
k
i
n
g
a
t
e
x
i
s
t
i
n
g
p
r
o
b
l
e
m
s
)
S
E
M
I
N
A
L
I
T
Y
(
t
h
e
s
o
l
u
t
i
o
n
d
r
a
w
s
a
t
t
e
n
t
i
o
n
t
o
p
r
e
v
i
o
u
s
l
y
u
n
n
o
t
i
c
e
d
p
r
o
b
l
e
m
s
)
V
I
S
I
O
N
(
t
h
e
s
o
l
u
t
i
o
n
s
u
g
g
e
s
t
s
n
e
w
n
o
r
m
s
f
o
r
j
u
d
g
i
n
g
o
t
h
e
r
s
o
l
u
t
i
o
n
s
e
x
i
s
t
i
n
g
o
r
n
e
w
)
P
A
T
H
F
I
N
D
I
N
G
(
t
h
e
s
o
l
u
t
i
o
n
o
p
e
n
s
u
p
a
n
e
w
c
o
n
c
e
p
t
u
a
l
i
z
a
t
i
o
n
o
f
t
h
e
i
s
s
u
e
s
)
125
Journal of Creative Behavior
quite different design). Reliabilities of their ratings ranged from 0.69 to 0.86
(alpha coefcients), with the majority of coefcients being in excess of 0.80. By
contrast, Kaufman et al. (2008a,b) found that non-expert raters judgments of
creativity were inconsistent (showing low inter-rater reliability) and did not
match those of experts. Lee, Lee and Young (2005) found similar problems with
novice ratings. Plucker, Kaufman, Temple and Qian (2009) found, however, a
degree of uncertainty in the assessment of product creativity by different catego-
ries of raters, with their results suggesting a continuum of creative evaluation in
which the distinctions between categories such as novice, amateur, and
expert are blurry and often overlap This was supported by Hekkert and
Van Wieringen (1996) who also found overlaps between raters with different lev-
els of expertise. Baer (2010) discusses this issue in the context of the debate on
domain specicity and creativity and questions the validity of replacing expert
judges with novices, citing Chen, Himsel, Kasof, Greenberger and Dmitreiva
(2006) to highlight the need for data showing that expert raters in a domain
can be reliably replaced by novices
A key step in addressing the debate surrounding the role of expertise in creativity
assessment is therefore to examine whether or not it is possible for different people
to show a common and reliable understanding of novelty, complexity, elegance, and
the like. If people can recognize creative products when they see them and can
express their judgments of the level of the characteristics in a quantiable way,
without the need for special knowledge or expertise, then not only does this open
up the utility of creativity measurement but it may also contribute to the wider
debate regarding domain specicity and creativity.
The Creative Solution Diagnosis Scale was developed on the basis of a theoretical
framework for product creativity, namely the concept of functional creativity
(Cropley & Cropley, 2005), enriched by a set of indicators drawn from the literature
of product creativity. The scale has yet to be tested on a large group of raters to see
if the hypothesized structure is borne out by factor analysis. The goal of this study
is to look at a large number of novice raters to see the level of reliability and the
factor structure of the responses.
METHOD
PARTICIPANTS
The participants who used the CSDS consisted of 203 college students from a
public university in Southern California. Participants took part in the study online
for extra credit. The sample included 157 participants who identied as female
(77.3%), 21 who identied as male (10.3%), and a further 25 (12.3%) who chose to
give no answer for this demographic. The most common age group was 1824 years
old, (58.1%) followed by 2529 years old (13.3%). The demographic breakdown of
the sample was as follows: 70 Hispanic American; 67 European American/Caucasian;
15 African American; 12 Asian American; 9 of mixed or biracial ethnicity; and 30
who chose not to identify their ethnicity.
126
Measuring Functional Creativity
PROCEDURE
Participants were directed to a website where the measures were hosted online.
Participants were presented, sequentially, with an image of one of ve different
mousetraps of varying designs (stimuli 15). Images of the mousetraps were selected
from Google image search to represent a diverse range of possible mousetraps (see
Appendix A for descriptions). Participants were asked to rate each of the different
mousetraps using the 30 items of the (CSDS, Table 2). Each item was rated using a
5-point Likert-type scale (ranging from not at all through somewhat to very
much) to indicate the degree to which the CSDS item is applied to the given
mousetrap. In addition, each item was rated using a 5-point Likert-type scale
(ranging from not at all through somewhat to very much) to indicate how
creative, overall, each mousetrap was. Participants were asked to complete a basic
demographic questionnaire, debriefed, and given extra credit when applicable for
their participation.
RESULTS
INTER-RATER RELIABILITY
Consistency among the participants was evaluated with Cronbachs coefcient
alpha (Cronbach, 1951). Coefcient alpha is a standard measure of internal
consistency and has been used in creativity research as a measure of inter-rater
reliability, treating raters as items (see Kaufman et al., 2008a). For the purpose of
assessing the consistency of the raters, missing data were addressed by excluding
respondents who missed more than 10% of the total. This removed 27 of the 203
respondents (n = 176). The consistency of raters was then computed for each indi-
vidual mousetrap by rotating the dataset to treat the raters as items. This resulted in
a 30 9 176 matrix for each stimulus, with the following results for coefcient alpha
for inter-rater reliability:
Stimulus 1: 0.976; Stimulus 2: 0.956; Stimulus 3: 0.971; Stimulus 4: 0.941;
Stimulus 5: 0.853.
In addition, the consistency of raters (inter-rater reliability) for all ve stimuli
combined was also computed using the same approach. The 30 9 880 matrix
formed by treating raters as items across all ve stimuli resulted in a coefcient
alpha of 0.956. This places the consistency of the raters in the excellent range
(Nunnally & Bernstein, 1994). Because of the high inter-rater reliability of scores
across all mousetraps, ratings for the ve different stimuli were combined for the
purpose of a conrmatory factor analysis of the CSDS.
SCALE RELIABILITY
Scale reliability was calculated using Cronbachs alpha to assess the mean inter-
item correlations for the ve different stimuli. Scale reliability was computed for
each separate stimulus based on a matrix of 203 responses across 30 CSDS items.
After listwise deletions, the following scale reliabilities were obtained: Stimulus 1:
0.948 (160 valid cases); Stimulus 2: 0.947 (168 valid cases); Stimulus 3: 0.954 (152
valid cases); Stimulus 4: 0.965 (164 valid cases); Stimulus 5: 0.968 (149 valid cases).
127
Journal of Creative Behavior
The mean scale reliability was therefore 0.956. This places both the individual scale
reliabilities, and the mean value, in the excellent range (Nunnally & Bernstein,
1994).
FACTOR ANALYSIS
The Creative Solution Diagnosis Scale is based on a theoretical model of func-
tional creativity that links four dimensions (Relevance & Effectiveness; Novelty; Ele-
gance; Genesis) to a larger set of 30 variables of indicators. The proposed structure
was created to balance the competing needs of parsimony (in explaining and mea-
suring product creativity in a simple and economical manner) and utility (in provid-
ing an instrument that explains creativity in sufcient detail to render it useful as,
e.g., a diagnostic tool). Factor analysis is used most frequently in the development
of instruments and in testing theories about instruments or the theories on which
instruments are based (Tinsley & Tinsley, 1987).
Exploratory Factor Analysis (EFA) is used to discover structure among a relatively
large set of variables. EFA does not advance any a priori hypotheses about the results
or structure. EFA does not, for example, specify the number of factors before the
analysis is conducted.
Conrmatory Factor Analysis (CFA), by contrast, involves the specication and
estimation of one or more possible models of a factor structurethat is, a structure
that relates an underlying construct, in the present case Functional Creativity, to a
set of variables (individual measures, or items). In the case of the present study, a
given theoretical model (Table 3) proposes a set of latent variables (e.g., Relevance
& Effectiveness) to account for covariances among a set of observed variables (e.g.,
the 30 CSDS indicators) (Bagozzi, 1980; Bollen, 1989).
Conrmatory Factor Analysis requires a priori designation of plausible factor pat-
terns from previous theoretical or empirical work; these plausible alternative models
are then explicitly tested statistically against sample data. Some researchers (e.g.,
Tinsley & Tinsley, 1987) are more prescriptive in stating that conrmatory factor
analysis procedures require the investigator to specify the number of factors before
factoring. Conrmatory factor analysis has been used extensively in a wide range of
domains, including psychology, marketing, and counseling for validating instruments
by testing alternative models (e.g., Besemer & OQuin, 1999; Harvey, Billings &
Nilan, 1985; Kumar & Sashi, 1989; Marsh, 1985; Marsh & Hocevar, 1985; Thacker,
Fields, & Tetrick, 1989).
It is therefore necessary to empirically test the CSDS to determine:
1 The degree to which the proposed CSDS structure (Table 1) ts a CFA.
2 The reliability/internal consistency (degree to which each item statistically ts
other items) of the CSDS.
3 The convergent validity of the CSDSdoes the proposed instrument measure
creativity? Can we correlate measures on the CSDS to another established mea-
sure of creativity?
128
Measuring Functional Creativity
T
A
B
L
E
3
.
C
S
D
S
T
h
e
o
r
e
t
i
c
a
l
M
o
d
e
l
.
R
e
l
e
v
a
n
c
e
&
E
f
f
e
c
t
i
v
e
n
e
s
s
N
o
v
e
l
t
y
E
l
e
g
a
n
c
e
G
e
n
e
s
i
s
P
r
o
b
l
e
m
a
t
i
z
a
t
i
o
n
E
x
i
s
t
i
n
g
K
n
o
w
l
e
d
g
e
N
e
w
K
n
o
w
l
e
d
g
e
E
x
t
e
r
n
a
l
I
n
t
e
r
n
a
l
C
o
r
r
e
c
t
n
e
s
s
P
r
e
s
c
r
i
p
t
i
o
n
R
e
p
l
i
c
a
t
i
o
n
R
e
d
i
r
e
c
t
i
o
n
R
e
c
o
g
n
i
t
i
o
n
C
o
m
p
l
e
t
e
n
e
s
s
F
o
u
n
d
a
t
i
o
n
a
l
i
t
y
P
e
r
f
o
r
m
a
n
c
e
P
r
o
g
n
o
s
i
s
C
o
m
b
i
n
a
t
i
o
n
R
e
c
o
n
s
t
r
u
c
t
i
o
n
C
o
n
v
i
n
c
i
n
g
n
e
s
s
G
r
a
c
e
f
u
l
n
e
s
s
T
r
a
n
s
f
e
r
a
b
i
l
i
t
y
A
p
p
r
o
p
r
i
a
t
e
n
e
s
s
D
i
a
g
n
o
s
i
s
I
n
c
r
e
m
e
n
t
a
t
i
o
n
R
e
i
n
i
t
i
a
t
i
o
n
P
l
e
a
s
i
n
g
n
e
s
s
H
a
r
m
o
n
i
o
u
s
n
e
s
s
G
e
r
m
i
n
a
l
i
t
y
O
p
e
r
a
b
i
l
i
t
y
R
e
d
e

n
i
t
i
o
n
S
u
s
t
a
i
n
a
b
i
l
i
t
y
S
e
m
i
n
a
l
i
t
y
S
a
f
e
t
y
G
e
n
e
r
a
t
i
o
n
V
i
s
i
o
n
D
u
r
a
b
i
l
i
t
y
P
a
t
h

n
d
i
n
g
129
Journal of Creative Behavior
Four primary factors (Relevance & Effectiveness, Novelty, Elegance, and Genesis)
provide a theoretical framework for explaining the Functional Creativity construct
by identifying underlying components that permit more precision in formulating
and testing research hypotheses in the eld of product creativity. The analysis that
was conducted was conrmatory in the sense that the procedure was used to test
the t of the 30 indicators to the hypothesized four-factor CSDS structure. However,
it may also be viewed as exploratory, in the sense that in the analysis we did not x
the number of factors at four, but allowed this to emerge from the procedure.
Prior to commencing factor analysis, the data were tested for their suitability.
The Kaiser-Meyer-Olkin Measure of Sampling Adequacy was 0.943, and Bartletts
Test of Sphericity resulted in an approximate Chi-Square value of 15832.166,
df = 276, and a signicance of p < 0.001, indicating that the data were suitable for
factor analysis.
Principle Axis Factoring was selected as the extraction method for its ability to
examine shared variance and to uncover the structure of the underlying variables.
An Oblimin rotation was used for its assumption that correlations exist between the
items of the CSDS, in contrast to a Varimax rotation which assumes that items are
uncorrelated. Missing values were deleted listwise from the data. Although this con-
dition is more stringent than pairwise deletion (i.e., replacing missing values with
means), it yielded 793 valid responses for each of the 30 CSDS items.
The criterion that was applied for accepting an item as loading onto a factor was
that the loading should be > 0.4, whereas items loading at < 0.3 were regarded as
failing to load onto a given factor. These criteria are consistent with accepted values
(see, for example, Comrey & Lee, 1992).
The factor analysis was conducted with all 30 CSDS items. Items with cross-
loadings > 0.3 were excluded one at a time, and the factor analysis repeated with
the goal of achieving a simple structure in which each item loaded onto a single
factor at > 0.4. Table 4 shows the simple structure that was achieved after the exclu-
sion of six items that either loaded onto no factor or which cross-loaded at > 0.3
and were therefore considered to be redundant.
The simple structure in Table 4 resulted in the exclusion of six redundant or irrele-
vant items from the CSDS scale: Operability, Durability, Replication, Incrementation,
Reconstruction, and Recognition. This structure satises typical criteria for a simple
structure, including a minimum of three items per factor. The majority of items in this
structure have loading rated as either very good or excellent (Comrey & Lee, 1992:
> 0.7 excellent; > 0.63 very good; > 0.55 good; > 0.45 fair). Finally, the ve factors in
the simple structure shown in Table 4 account for 68.127% of the total variance.
LINEAR REGRESSION
A stepwise linear regression was conducted to see how the 24 items of the CSDS
(after exclusion of the redundant items indicated by the conrmatory factor analy-
sis) predicted the overall creativity rating for the mousetraps. As can be seen in
Table 5, eight of the items signicantly predicted the overall creativity score, F (8,
798) = 69.962, p < 0.0001. The model summary indicates that 41.2% (R
2
) of the
130
Measuring Functional Creativity
variance of overall creativity has been signicantly explained by these eight indepen-
dent variables.
FACTOR CORRELATION MATRIX
A factor correlation matrix for the CSDS data was also computed (Table 6). This
factor correlation matrix for the new structure indicates two important features of
the revised CSDS scale. First, all ve factors are positively correlated with a degree.
This ts the theoretical model, in the sense that all contribute to the construct Func-
tional Creativity. Second, it indicates stronger positive correlations between some
factors. For example, Genesis and Propulsion correlate strongly (0.705), as do
TABLE 4. Conrmatory Factor AnalysisCreative Solution Diagnosis Scale Pat-
tern Matrix.
Factor
1 2 3 4 5
Vision .841 .041 .031 .003 .004
Transferability .818 .008 .024 .009 .019
Seminality .807 .080 .111 .093 .049
Pathnding .777 .015 .009 .109 .015
Germinality .732 .033 .048 .173 .006
Foundationality .495 .115 .007 .162 .001
Performance .023 .911 .037 .027 .034
Appropriateness .029 .869 .019 .002 .026
Correctness .002 .752 .008 .037 .092
Prescription .005 .008 .886 .026 .005
Prognosis .002 .003 .808 .065 .025
Diagnosis .013 .006 .643 .032 .013
Redenition .012 .001 .040 .832 .077
Reinitiation .093 .066 .036 .799 .054
Generation .118 .001 .047 .749 .031
Redirection .061 .086 .112 .650 .048
Combination .004 .125 .178 .488 .091
Pleasingness .026 .064 .021 .002 .927
Completeness .013 .051 .032 .005 .921
Sustainability .111 .114 .059 .119 .754
Gracefulness .026 .079 .036 .072 .741
Convincingness .053 .149 .059 .006 .698
Harmoniousness .130 .174 .053 .071 .678
Safety .079 .153 .041 .141 .480
Extraction Method: Principal Axis Factoring. Rotation Method: Oblimin with Kaiser
Normalization. Rotation converged in 13 iterations.
131
Journal of Creative Behavior
Relevance, Effectiveness, and Elegance (0.638). This is supported by the underlying
relationships of the criteria that dene functional creativity. Genesis and Propulsion,
for example, both relate to new knowledge, ideas and solutionsPropulsion is con-
cerned more with novelty, as it relates to the problem in hand, whereas Genesis is
concerned more with future novelty and possibilities. Relevance, Effectiveness, and
Elegance are also conceptually related. Relevance and Effectiveness examines the
degree to which a product satises the dened need (what it does), whereas Elegance
examines how well it does so.
DISCUSSION
The simple structure shown in Table 4 has a close correspondence to the
theoretical model described in Table 3. The six items that load onto Factor 1 are
all items identied in the theoretical model as indicators under the criterion
Genesis. Factor 1 will therefore be named Genesis. Three items, all drawn
from the criterion Relevance and Effectiveness, load onto Factor 2, which will
therefore be named Relevance and Effectiveness. Factor 3 is represented by
TABLE 5. Linear Regression, 24-Item Creative Solution Diagnosis Scale.
Variable B Beta
Redirection 0.219** 0.193
Correctness 0.216** 0.199
Redenition 0.115* 0.102
Germinality 0.104* 0.092
Combination 0.121** 0.107
Diagnosis 0.092** 0.078
Harmoniousness 0.107** 0.096
Reinitiation 0.115** 0.111
R
2
= 0.412, * p < 0.05, ** p < 0.01.
TABLE 6. Creative Solution Diagnosis Scale Factor Correlation Matrix.
Factor Problematization Genesis Propulsion
Relevance &
Effectiveness
Elegance
Problematization 1.000 .556 .447 .483 .406
Genesis .556 1.000 .705 .350 .448
Propulsion .447 .705 1.000 .322 .424
Relevance &
Effectiveness
.483 .350 .322 1.000 .638
Elegance .406 .448 .424 .638 1.000
Extraction Method: Principal Axis Factoring. Rotation Method: Oblimin with Kaiser
Normalization.
132
Measuring Functional Creativity
three items that were identied by the secondary criterion Problematization in
the theoretical model, and will be designated Problematization. Factor 4 is
characterized by items that represent new and existing knowledge (Novelty) in
the theoretical model. The items which load onto Factor 4 can all be character-
ized by the Propulsion Model (Sternberg et al., 2002) and Factor 4 will therefore
be designated Propulsion. Factor 5 is characterized by items which represent
the criterion Elegance and other qualities of good products (for example, Safety)
that will be designated as Elegance.
Table 7 captures the revised version of the CSDS derived from the conrmatory
factor analysis. The analysis is conrmatory in that it supports the general structure
of the theoretical model (Table 3), however, it identies a renement with the addi-
tion of Problematization as a distinct, secondary criterion, and has eliminated six
redundant or irrelevant indicators from the original 30 (Table 2).
CONCLUSION
The conrmatory factor analysis undertaken has shown that six items in the 30-
item CSDS are redundant and can be eliminated. The remaining 24 items result in a
simple structure of ve factors that support the criteria and structure dened by
Cropley and Cropley (2005) as a model of functional creativity.
The analysis has also shown that non-expert judges can reliably assess the creativ-
ity of products using a more highly differentiated scale (the CSDS) with no formal
training. These results suggest that, given the right tool, non-experts judges are able
to recognize and quantify widely accepted characteristics of creative products,
including effectiveness and novelty. Whereas further questions, of course, remain to
be investigated, not least whether these non-expert raters produce scores that are
correlated with expert ratings, the current results do address a key criticism (Baer,
2010) regarding reliability in previous studies. If the expertnovice agreement level
is comparable to the low levels found using the Consensual Assessment Technique,
as some studies suggest, then novice CSDS ratings may nevertheless be used as an
alternate rating representing public perception. If experts and novices agree at a
TABLE 7. Revised Creative Solution Diagnosis Scale.
Functional Creativity
Relevance &
Effectiveness
Problematization Propulsion Elegance Genesis
Performance Prescription Redenition Pleasingness Vision
Appropriateness Prognosis Reinitiation Completeness Transferability
Correctness Diagnosis Generation Sustainability Seminality
Redirection Gracefulness Pathnding
Combination Convincingness Germinality
Harmoniousness Foundationality
Safety
133
Journal of Creative Behavior
stronger rate, then CSDS ratings may be able to supplement or even replace expert
ratings. A reliable instrument for non-expert judgments of creativity (the CSDS) also
permits more research to be undertaken in related questions. One key variable will
be whether the questions in the CSDS help shape novice judgments in a way that
serves as a type of informal training. Past research (Dollinger & Shafran, 2005) has
indicated such training can help novice ratings correspond more closely with expert
ratings. The CSDS may also shed light on the nature of expert assessments. Past
research (Runco, McCarthy, & Svenson, 1994) has suggested that expert ratings of
artworks may be more severe than ratings by peers, or by the artists themselves. The
availability of a reliable instrument like the CSDS may serve to smooth differences
between experts that have confounded previous studies (e.g., Hickey, 2001) by
balancing individual, expert conceptions of creativity with a uniform framework of
categories (effectiveness, novelty, elegance, etc.). Future research therefore needs to
compare non-expert CSDS ratings with unconstrained expert ratings, as well as with
expert rating made using the CSDS.
Linear regression analysis lends support to the revised CSDS, indicating that each
of the ve factors identied is linked to at least one CSDS item that is a signicant
predictor of overall creativity (Table 5). Furthermore, the factor Propulsion, with
four of ve individual items being strong predictors of overall creativity, is impor-
tant to the overall construct of functional creativity. Lastly, the linear regression
analysis also suggests that some of the factors (Relevance & Effectiveness; Problema-
tization; Elegance; Genesis) may be harder for novices to rate, or that novice raters
may nd it harder to perceive how these concepts are related to creativity.
On a broader level, we hope that such free and publicly available instruments as
the CSDS can help encourage more research on functional creativity. Aesthetic crea-
tivity is certainly meaningful, but the extent to which creativity measurement focuses
on art or writing is likely borne out of simplicity; it is easier to have people create
and then rate a drawing or a poem than design a mousetrap. If the rating aspect
can become more accessible, then more functional creativity research may become
more feasible.
APPENDIX
MOUSETRAP DESCRIPTIONS
Stimulus 1 = Bottle MousetrapThis design attracts a mouse to a food source in
a glass bottle, but uses a coil of wire, inserted in the open top of the bottle, to trap
the mouse as it attempts to enter the bottle to retrieve the food.
Stimulus 2 = House MousetrapThis proprietary design, shaped like a small
house, is made from clear plastic and uses a hinged ap to allow a mouse to enter,
trapping it inside. The design contains the mouse inside the device, allowing it to be
released at an appropriate time/place.
Stimulus 3 = Cage MousetrapThis design uses a sprung trap door to contain
the mouse in a wire cage. Upon entering the trap, the mouse will trigger the trap-
door, preventing it from exiting.
134
Measuring Functional Creativity
Stimulus 4 = Hi-tech MousetrapThis design uses a complex, computerized
control mechanism to trigger the trap. Upon entering the trap, the device is acti-
vated. A motorized arm releases the upper part of the trap (a plastic container),
which falls over a similarly shaped lower container, closing the entry point and con-
taining the mouse inside the trap.
Stimulus 5 = Plastic MousetrapThis proprietary design resembles a conven-
tional wooden, spring-loaded mousetrap. The device is made from plastic. When
activated, a plastic arm snaps down on the mouse.
REFERENCES
AMABILE, T. M. (1983). The social psychology of creativity. New York: Springer.
AMABILE, T. M. (1996). Creativity in context: Update to The Social Psychology of Creativity. Boulder, CO:
Westview Press.
BAER, J. (2010). Is creativity domain specic? In J. C. Kaufman & R. J. Sternberg (Eds.), Cambridge Hand-
book of Creativity (pp. 321341). New York: Cambridge University Press.
BAER, J., KAUFMAN, J. C., & GENTILE, C. A. (2004). Extension of the consensual assessment technique to
nonparallel creative products. Creativity Research Journal, 16, 113117.
BAGOZZI, R. P. (1980) Causal modeling in marketing. New York: Wiley & Sons.
BEGHETTO, R. A., KAUFMAN, J. C., & BAXTER, J. (2011). Answering the unexpected questions: Exploring
the relationship between students creative self-efcacy and teacher ratings of creativity. Psychology of Aes-
thetics, Creativity, and the Arts, 5(4), 342349.
BESEMER, S. P., & OQUIN, K. (1987). Creative product analysis: Testing a model by developing a judging
instrument. In S. G. Isaksen (Ed.), Frontiers of creativity research: Beyond the basics (pp. 367389). Buf-
falo, NY: Bearly.
BESEMER, S. P., & OQUIN, K. (1999). Conrming the three-factor creative product analysis matrix model
in an American sample. Creativity Research Journal, 12, 287296.
BODEN, M. A. (1994). What is creativity? In M. A. Boden (Ed.), Dimensions of creativity (pp. 75118). Cam-
bridge, MA: MIT Press.
BOLLEN, K. A. (1989) Structural equations with latent variables. New York, NY: Wiley & Sons.
CARNEVALE, A. P., GAINER, L. J., & MELTZER, A. (1990) Workplace basics: The essential skills employers
want. San Francisco: Jossey-Bass.
CATTELL, J., GLASCOCK, J., & WASHBURN, M. F. (1918). Experiments on a possible test of aesthetic
judgment of pictures. American Journal of Psychology, 29, 333336.
CHEN, C., HIMSEL, A., KASOF, J., GREENBERGER, E., & DMITREIVA, J. (2006). Boundless creativity: Evi-
dence for domain generality of individual differences in creativity. Journal of Creative Behavior, 40, 179
199.
CHILD, I. L., & IWAO, S. (1968). Personality and esthetic sensitivity: Extension of ndings to younger age
and to different culture. Journal of Personality and Social Psychology, 8, 308312.
COMREY, A. L., & LEE, H. B. (1992). A rst course in factor analysis (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.
CRONBACH, L. J. (1951). Coefcient alpha and the internal structure of tests. Psychometrica, 16, 297334.
CROPLEY, A. J. (2005) Creativity and problem-solving: Implications for classroom assessment. Leicester: British
Psychological Society.
CROPLEY, D. H., & CROPLEY, A. J. (2005). Engineering creativity: A systems concept of functional creativ-
ity. In J. C. Kaufman, & J. Baer (Eds.), Creativity across domains: Faces of the muse (pp. 169185). Hills-
dale, NJ: Lawrence Erlbaum.
CROPLEY, D. H., & CROPLEY, A. J. (2008). Elements of a universal aesthetic of creativity. Psychology of Aes-
thetics, Creativity and the Arts, 2, 155161.
CROPLEY, D. H., & CROPLEY, A. J. (2010a). Functional creativity: Products and the generation of effective
novelty. In J. C. Kaufman & R. J. Sternberg (Eds.), Cambridge Handbook of Creativity (pp. 301320).
New York: Cambridge University Press.
135
Journal of Creative Behavior
DOLLINGER, S. J., & SHAFRAN, M. (2005). Note on the consensual assessment technique in creativity
research. Perceptual and Motor Skills, 100, 592598.
FLORIDA, R. (2002) The rise of the creative class and how its transforming work, life, community and everyday
life. New York: Basic Books.
GRUDIN, R. (1990) The grace of great things: Creativity and innovation. New York: Ticknor and Fields.
HAN, S. H., YUN, M. H., KIM, K., & KWAHK, J. (2000). Evaluation of product usability: Development and
validation of usability dimensions and design elements based on empirical models. International Journal
of Industrial Ergonomics, 26, 477488.
HARVEY, R. J., BILLINGS, R., & NILAN, K. J. (1985). Conrmatory factor analysis of the job diagnostic sur-
vey: Good news and bad news. Journal of Applied Psychology, 70, 461468.
HEKKERT, P., & VAN WIERINGEN, P. C. W. (1996). Beauty in the eye of expert and nonexpert beholders:
A study in the appraisal of art. American Journal of Psychology, 109, 389407.
HICKEY, M. (2001). An application of Amabiles consensual assessment technique for rating the creativity of
childrens musical compositions. Journal of Research in Music Education, 49, 234244.
HORN, D., & SALVENDY, G. (2006). Consumer-based assessment of product creativity: A review and reap-
praisal. Human Factors and Ergonomics in Manufacturing & Service Industries, 16, 155175.
HORN, D., & SALVENDY, G. (2009). Measuring consumer perception of product creativity: Impact on satis-
faction and purchasability. Human Factors and Ergonomics in Manufacturing & Service Industries, 19,
223240.
JACKSON, P. W., & MESSICK, S. (1965). The person, the product, and the response: Conceptual problems
in the assessment of creativity. Journal of Personality, 33, 11221131.
KAUFMAN, J. C., BAER, J., COLE, J. C., & SEXTON, J. D. (2008a). A comparison of expert and nonexpert
raters using the consensual assessment technique. Creativity Research Journal, 20, 171178.
KAUFMAN, J. C., PLUCKER, J. A., & BAER, J. (2008b) Essentials of creativity assessment. New York: Wiley.
KUMAR, A., & SASHI, C. M. (1989). Conrmatory analysis of aggregate hierarchical market structures:
Inferences from brand-switching behavior. Journal of Marketing Research, 26(4), 444453.
LEE, S., LEE, J., & YOUNG, C.-Y. (2005). A variation of CAT for measuring creativity in business products.
Korean Journal of Thinking and Problem Solving, 15, 143153.
MANN, E. L. (2009). The search for mathematical creativity: Identifying creative potential in middle school
students. Creativity Research Journal, 21, 338348.
MARSH, H. W. (1985). The structure of masculinity/femininity: An application of conrmatory factor analysis
to higher-order factor structures and factorial invariance. Multivariate Behavioral Research, 20, 427449.
MARSH, H. W., & HOCEVAR, D. (1985). Application of conrmatory factor analysis to the study of self-
concept: First- and higher-order factor models and their invariance across groups. Psychological Bulletin,
97, 562582.
MILLER, A. I. (1992). Scientic creativity: A comparative study of Henri Poincare and Albert Einstein. Crea-
tivity Research Journal, 5, 385418.
NUNNALLY, J. C., & BERNSTEIN, I. H. (1994). Psychometric theory (3rd ed.). New York: McGraw-Hill.
OCHSE, R. (1990) Before the gates of excellence. New York: Cambridge University Press.
OQUIN, K. & BESEMER, S. P. (1999). Creative products. In M. A. Runco, & S. R. Pritzker (Eds.), Encyclo-
pedia of creativity (pp. 413422). Boston: Academic Press.
PLUCKER, J. A., KAUFMAN, J. C., TEMPLE, J. S., & QIAN, M. (2009). Do experts and novices evaluate
movies the same way? Psychology and Marketing, 26, 470478.
PUCCIO, G. J., & CABRA, J. F. (2010). Organizational creativity: A systems approach. In J. C. Kaufman, & R. J.
Sternberg (Eds.), Cambridge handbook of creativity (pp. 145173). New York: Cambridge University Press.
PUCCIO, G. J., MURDOCK, M. C., & MANCE, M. (2007) Creative leadership: Skills that drive change. Thou-
sand Oaks, CA: Sage.
REIS, S. M., & RENZULLI, J. S. (1991). The assessment of creative products in programs for gifted and tal-
ented students. Gifted Child Quarterly, 35, 128134.
REITER-PALMON, R., ILLIES YOUNG, M., KOBE, L., BUBOLTZ, C., & NIMPS, T. (2009). Creativity and
domain specicity: The effect of task type of multiple indices on creative problem solving. Psychology of
Aesthetics, Creativity, and the Arts, 3, 7380.
136
Measuring Functional Creativity
RUNCO, M. A., MCCARTHY, K. A., & SVENSON, E. (1994). Judgements of the creativity of artwork from
students and professional artists. Journal of Psychology, 128, 2331.
SAVRANSKY, S. D. (2000) Engineering of creativity. Boca Raton, FL: CRC Press.
SHALLEY, C. E. (2002). How valid and useful is the integrative model for understanding work groups crea-
tivity and innovation? Applied Psychology: An International Review, 51, 406410.
SIMONTON, D. K. (2009a) Genius 101. New York: Springer.
SIMONTON, D. K. (2009b). Varieties of (scientic) creativity: A hierarchical model of disposition, develop-
ment, and achievement. Perspectives on Psychological Science, 4, 441452.
SLATER, B. H. (2006). Aesthetics. Retrieved February 22, 2011, from http://www.iep.utm.edu/a/aestheti.
htm#H2.
STERNBERG, R. J. (1999). A propulsion model of types of creative contributions. Review of General Psychol-
ogy, 3, 83100.
STERNBERG, R. J., KAUFMAN, J. C., & PRETZ, J. E. (2002) The creativity conundrum: A propulsion model
of kinds of creative contributions. New York: Psychology Press.
STERNBERG, R. J., KAUFMAN, J. C., & PRETZ, J. E. (2003). A propulsion model of creative leadership.
Leadership Quarterly, 14, 455473.
STERNBERG, R. J., & LUBART, T. I. (1999). The concept of creativity: Prospects and paradigms. In R. J.
Sternberg (Ed.), Handbook of creativity (pp. 315). New York: Cambridge University Press.
TAYLOR, A. (1975). An emerging view of creative actions. In I. A. Taylor & J. W. Getzels (Eds.), Perspectives
in creativity (pp. 297325). Chicago: Aldine.
THACKER, J. W., FIELDS, M. W., & TETRICK, L. E. (1989). The factor structure of union commitment: An
application of conrmatory factor analysis. Journal of Applied Psychology, 74, 228232.
TINSLEY, H. E. A., & TINSLEY, D. J. (1987). Uses of factor analysis in counseling psychology research. Jour-
nal of Counseling Psychology, 34(4), 414424.
WOODMAN, R. W., SAWYER, J. E., & GRIFFIN, R. W. (1993). Toward a theory of organizational creativity.
Academy of Management Review, 18, 293321.
David H. Cropley, University of South Australia
James C. Kaufman, California State University
Correspondence concerning this article should be addressed to David H. Cropley, Defence and Systems
Institute, University of South Australia, Building W, Mawson Lakes Campus, Mawson Lakes, SA 5158,
Australia. Email: david.cropley@unisa.edu.au
ACKNOWLEDGMENTS
The authors wish to thank Arielle White for her assistance with data collection and Raul Salcedo for his help
in preparing the manuscript.
137
Journal of Creative Behavior