See

discussions, stats, and author profiles for this publication at: https://www.researchgate.net/publication/263917340

Lost in Translation: Thoughts Regarding the

Translation of Existing Psychological Measures
Into Other Languages

Article in European Journal of Psychological Assessment May 2013

DOI: 10.1027/1015-5759/a000167

CITATIONS READS

16 718

2 authors:

Matthias Ziegler Doreen Bensch

Humboldt-Universitt zu Berlin Humboldt-Universitt zu Berlin
126 PUBLICATIONS 1,577 CITATIONS 8 PUBLICATIONS 50 CITATIONS

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

European Journal of Psychological Assessment View project

S-Pan [dropout phenomena in higher education with special regard to primary (science) education]
View project

All content following this page was uploaded by Matthias Ziegler on 14 September 2015.

The user has requested enhancement of the downloaded file.

 European Journal of Psychological Assessment

2013 Hogrefe Publishing

Editor-in-Chief Matthias Ziegler, Department of Psychology, Humboldt University Berlin, Rudower Chaussee 18, 12489 Berlin,
Germany (Tel. +49 30 2093 9447, Fax +49 30 2093 9361, E-mail zieglema@hu-berlin.de)

Editorial Assistant Doreen Bensch, Department of Psychology, Humboldt University Berlin, Rudower Chaussee 18, 12489 Berlin,
Germany (Tel. +49 30 2093 9441, Fax +49 30 2093 9360, E-mail benschdx@cms.hu-berlin.de)

Associate Editors Itziar Alonso-Arbiol, Spain; Martin Bckstrm, Sweden; Laurence Claes, Belgium; Johnny Fontaine, Belgium;
Richard Griffith, USA; Carolyn MacCann, Australia; Symeon Vlachopoulos, Greece

Consulting Editors Paul De Boeck, Belgium Helfried Moosbrugger, Germany

Christine DiStefano, USA Janos Nagy, Hungary
Anastasia Efklides, Greece Willibald Ruch, Germany
Roco Fernndez-Ballesteros, Spain Manfred Schmitt, Germany
Brian F. French, USA Stphane Vautier, France
Christoph Kemper, Germany Fons J.R. van de Vijver, The Netherlands
Klaus Kubinger, Austria Cilia Witteman, The Netherlands
Kerry Lee, Singapore

Founders Roco Fernndez-Ballesteros and Fernando Silva

Supporting The journal is the official organ of the European Association of Psychological Assessment (EAPA). The EAPA was
Organizations founded to promote the practice and study of psychological assessment in Europe as well as to foster the exchange
of information on this discipline around the world. Members of the EAPA receive the journal in the scope of their
membership fees. Further, the Division for Psychological Assessment and Evaluation, Division 2, of the Interna-
tional Association of Applied Psychology (IAAP) is sponsoring the journal: Members of this association receive the
journal at a special rate (see below).

Publisher Hogrefe Publishing, Merkelstr. 3, D-37085 Gttingen, Germany, Tel. +49 551 999-500, Fax +49 551 999-50111,
E-mail publishing@hogrefe.com, Web http://www.hogrefe.com
North America: Hogrefe Publishing, 38 Chauncy Street, Suite 1002, Boston, MA 02111, USA, Tel. +1 866 823-
4726, Fax +1 617 354-6875, E-mail customerservice@hogrefe-publishing.com, Web http://www.hogrefe.com

Production Regina Pinks-Freybott, Hogrefe Publishing, Merkelstr. 3, D-37085 Gttingen, Germany, Tel. +49 551 999-500,
Fax +49 551 999-50111, E-mail journalsproduction@hogrefe.com

Subscriptions Hogrefe Publishing, Herbert-Quandt-Strasse 4, D-37081 Gttingen, Germany, Tel. +49 551 50688-900,
Fax +49 551 50688-998

Advertising/Inserts Melanie Beck, Hogrefe Publishing, Merkelstr. 3, D-37085 Gttingen, Germany, Tel. +49 551 999-500,
Fax +49 551 999-50111, E-mail marketing@hogrefe.com

ISSN ISSN-L 1015-5759, ISSN-Print 1015-5759, ISSN-Online 2151-2426

Copyright 2013 Hogrefe Publishing. This journal as well as the individual contributions and illustrations contained within
Information it are protected under international copyright law. No part of this publication may be reproduced, stored in a retrieval
system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, microfilming, recording
or otherwise, without prior written permission from the publisher. All rights, including translation rights, reserved.

Publication Published in 4 issues per annual volume.

Subscription prices Annual subscription, Institutions (2013): e284.00, US $384.00, 228.00

Annual subscription, Individuals (2013): e142.00, US $192.00, 114.00
Postage and handling: e12.00, US $16.00, 10.00
Special rates: IAAP/Colegio Oficial de Psiclogos members: e85.00, US $114.00, 74.00 (+ e12.00, US
$16.00, 10.00 postage and handling); EAPA members: Included in membership
Single issues: e70.00, US $98.00, 56.00 (+ postage and handling)

Payment Payment may be made by check, international money order, or credit card, to Hogrefe Publishing, Merkelstr. 3,
D-37085 Gttingen, Germany, or, for North American customers, to Hogrefe Publishing, 38 Chauncy Street,
Suite 1002, Boston, MA 02111, USA.

Electronic Full Text The full text of the European Journal of Psychological Assessment is available online at www.psyjournals.com
and in PsycARTICLES.

Abstracting/Indexing The journal is abstracted/indexed in Current Contents / Social & Behavioral Sciences (CC/S&BS), Social Sciences
Services Citation Index (SSCI), Social SciSearch, PsycINFO, Psychological Abstracts, PSYNDEX, ERIH, and Scopus.
Impact Factor (2011): 2.529.

2013 Hogrefe Publishing

 european journal of
psychological
assessment
Volume 29, No. 3, 2013

Official Organ of the

European Association of Psychological Assessment
European Journal of Psychological Assessment 2013;V ol. 29(3) 2013 Hogrefe Publishing

Contents

Editorial Getting Entangled in the Nomological Net: Thoughts on Validity and Conceptual
Overlap
Matthias Ziegler, Tom Booth, and Doreen Bensch 157

Original Articles Framing Work Competencies Through Personality Traits: The Big Five
Competencies Grid
Chiara Consiglio, Guido Alessandri, Laura Borgogni, and Ronald F. Piccolo 162
Development and Validation of the German Test for (Highly) Intelligent Kids
T(H)INK
Tanja Gabriele Baudson and Franzis Preckel 171
The MMPI-2-Restructured Form and the Standard MMPI-2 Clinical Scales
in Relation to DSM-IV
Paul T. van der Heijden, Jos I. M. Egger, Gina M. P. Rossi, Gitte Grundel,
and Jan J. L. Derksen 182
The Factor Structure and Psychometric Properties of the Greek Version
of the Eating Disorders Examination Questionnaire (EDE-Q)
Theodoros Giovazolias, Ioannis Tsaousis, and Christina Vallianatou 189
Self-Efficacy in Regulating Positive and Negative Emotions: A Validation
Study in Germany
Catherine Gunzenhauser, Tobias Heikamp, Maria Gerbino, Guido Alessandri,
Antje von Suchodoletz, Laura Di Giunta, Gian Vittorio Caprara,
and Gisela Trommsdorff 197
Reliability and Validity of the German Version of the Young Schema
Questionnaire Short Form 3 (YSQ-S3)
Levente Kriston, Janina Schfer, Gitta A. Jacob, Martin Hrter,
and Lars P. Hlzel 205
Validation of the German Version of the Exercise Dependence Scale
Astrid Mller, Laurence Claes, Dirk Smits, Olaf Gefeller, Anja Hilbert,
Antonia Herberg, Verena Mller, Dirk Hofmeister, and Martina de Zwaan 213
Age Differences and Measurement Invariance of Working Memory in
5- to 12-Year-Old Children
K. Michalczyk, N. Malstdt, M. Worgt, T. Knen, and M. Hasselhorn 220

European Journal of Psychological Assessment 2013; Vol. 29(3) 2013 Hogrefe Publishing
 M.
European Journal of Psychological Assessment Ziegler
2013;
2013 et29(3):157161
Vol. al.:Publishing
Hogrefe Editorial

Editorial
Getting Entangled in the Nomological Net
Thoughts on Validity and Conceptual Overlap

Matthias Ziegler1, Tom Booth2, and Doreen Bensch1

1
Humboldt-Universitt zu Berlin, Germany
2
Centre for Cognitive Ageing and Cognitive Epidemiology,
Department of Psychology, The University of Edinburgh, UK

Psychological research strongly relies heavily on tests and ported by evidence of convergent and discriminant validation.
questionnaires to measure constructs and traits. Thus, tests (p. 100)
and questionnaires are not only in high demand, but are Within this Editorial we would like to outline some prob-
constantly being developed anew. Likewise, researchers lems related to construct validity and suggest some re-
have suggested many new traits or constructs. Such sug- search lines to solve these. Schweizer (2012) already dis-
gestions then cause a wave of test and questionnaire devel- cussed problems with convergent validity at length in an
opment. This investment of time and research resources is editorial in this journal. Therefore, we will try to enlarge
necessary to ensure high-quality measurement tools that on the focus and include some additional issues we deem
can be trusted by other researchers and practitioners as well important as well.
to assess the intended trait or construct.
For this reason, journals such as the European Journal
of Psychological Assessment publish studies evaluating the
psychometric properties of such new measurement tools.
Typically, such evaluation studies include some estimate of
The Idea of Convergent and
reliability (Schweizer, 2011) and concentrate on demon- Discriminant Validity as Proposed by
strating the validity of the score derived from the new Campbell and Fiske (1959)
measure. A look at the history of this journal reveals that
the number of publications that apply some form of factor
Campbell and Fiske (1959) started their seminal paper by
analysis has risen from around 28% in the 1990s to 40%
pointing out four aspects important to a validation process.
since the year 2000 (Alonso-Arbiol & van de Vijver, 2010).
They stated, first, that a convergent validity necessarily re-
From this one can assume that the factorial validity of the
quires independent measurement procedures, i.e., it is nec-
published measurement tools has been a central theme of
essary to apply different measurement approaches (e.g., pa-
published research. Factorial validity is of course an ex-
per-pencil and observation). Their second demand is that,
tremely important issue and provides information neces-
besides convergent validity-related evidence, discriminant
sary to all scoring procedures. Since it consists of trait
validity-related evidence is also required. Only then does
scores (in their various forms) which are most commonly
a full picture of validity emerge. The third point Campbell
used in applied studies, factorial validity should not be ne-
and Fiske made is that each measure includes variance due
glected. However, while a newly devised measurement tool
to a trait and variance due to method. Without disentan-
may demonstrate factorial validity and produce reliable test
gling these different variance sources, validity estimates for
scores, its utility in the field is far from assured. Construct
a test score might be inflated. Finally, in their fourth point
validity-related evidence is still necessary to ensure that the
Campbell and Fiske stated that, in order to achieve these
new measure truly captures the trait it was intended to cap-
goals, it is necessary to employ more than one method and
ture. Campbell and Fiske (1959) asserted this as follows:
to assess more than one trait. The approach they suggest
We believe that before one can test the relationships between the multitrait-multimethod matrix (MTMM) allows all of
a specific trait and other traits, one must have some confidence these aspects to be part of an analysis. Such a matrix sum-
in ones measures of that trait. Such confidence can be sup- marizes the correlations computed based on data for sever-

2013 Hogrefe Publishing European Journal of Psychological Assessment 2013; Vol. 29(3):157161
DOI: 10.1027/1015-5759/a000173
158 M. Ziegler et al.: Editorial
al traits, all assessed with the same methods. Importantly,
it should be done with more than one method. Within the
matrix Campbell and Fiske differentiate reliability diago-
nals, validity diagonals, heterotrait-monomethod triangles,
and heterotrait-heteromethod triangles. Moreover, by spec-
ifying the relationships between validity diagonals and tri-
angles as well as between the correlational patterns within
the triangles, Campbell and Fiske defined what evidence is
needed to speak of convergent and discriminant validity.
Schweizer (2012) outlined some of the problems around
this approach, which we do not need to repeat here.
Two Implications From Campbell and
Fiskes MTMM Approach
Issue 1: Selecting Traits for the Study of
Convergent and Discriminant Validity
We want to focus on two important issues. The first is im-
portant to discriminant validity-related evidence. In select-
ing discriminant traits, Campbell and Fiske emphasized the
importance of providing a definition as well as positioning
the discriminant trait within a nomological net of the trait
to be measured by the new instrument. Such a framework
provides the necessary depth of information to select ap-
propriate discriminant traits. Surprisingly, validity studies
often include correlations with numerous different opera-
tionalizations of the same trait, suggesting convergent va-
lidity-related evidence. When it comes to discriminant va-
lidity-related evidence, sometimes no clear underlying ra-
tionale for selecting exactly these traits becomes obvious,
making it look arbitrary. However, as Campbell and Fiske
already pointed out:
When a dimension of personality is hypothesized, when a con-
struct is proposed, the proponent invariably has in mind dis-
tinctions between the new dimension and other constructs al-
ready in use. One cannot define without implying distinctions,
and the verification of these distinctions is an important part
of the validational process. (p. 84)
Following this statement and the demands for selecting
discriminant traits, it seems necessary to again recall the
requirement to clearly define the trait to be measured, em-
bed it in a nomological net, and base the selection of dis-
criminant traits on this network. This way, obtaining dis-
criminant validity-related evidence is more difficult but
more informative. It is necessary to show that a new meas-
ure assessing a trait can be distinguished from an existing
measure, capturing a (closely) related trait. Nevertheless,
these findings tell us a lot more about discriminant valid-
ity than do correlations with measures assessing very dis-
tant traits.
European Journal of Psychological Assessment 2013; Vol. 29(3):157161
Visualizing the Nomological Net of Personality
Pace and Brannick (2010) conducted a bare bones meta-
analysis (corrections only for sampling error) for different
Big Five questionnaires. The underlying assumption here
was that all questionnaires should basically capture the
same trait. Pace and Brannick concluded:
Convergent validities were lower than expected, indicating
substantial differences among tests. Such a result begs for an
explanation of the differences among tests as well as a consid-
eration of the implications of such differences for theory and
practice. (p. 674)
In fact, the largest overall convergent correlation was found
for Extraversion at .56, whereas the estimated overall reliabil-
ity of Extraversion measures was .83. Thus, even in the best
case, about 50% of the instruments reliable variance is not
shared but rather unique to the specific questionnaires.
What the Pace and Brannick (2010) study highlights is
what has become known as the jingle-jangle fallacy, name-
ly, that scales with the same name may measure different
things, and that scales with a different name may measure the
same thing. Here we demonstrate the utility of network dia-
grams (see Epskamp, Cramer, Waldorp, Schmittmann, &
Borsboom, 2012) in representing cross-sectional association
matrices to visualize the jingle-jangle within personality
inventories and to highlight the challenge of selecting traits
in discriminant and convergent validity studies.
Figure 1 shows a network representation of the corre-
lation matrix between 113 personality facet scale scores
from the NEO-PI-R, HEXACO, 6FPQ, 16PF, MPQ, and
JPI derived from the Eugene-Springfield Community
Sample (Goldberg, 2005). Correlations are based on a
sample of 459 participants for whom complete data were
available. Within the figure, each facet scale score is a
node (circle) and the magnitude of the correlations be-
tween them is depicted as an edge (line). The thickness of
the edge represents the magnitude of the associations. For
clarity, associations less than r = .35 have been sup-
pressed.
Marked in gray are three clusters and two pairs of facets
that represent a series of situations with respect to the jin-
gle-jangle fallacy and new test construction. First, consider
the two pairs of scales on the right hand side of Figure 1.
The two nodes labeled TR are the facet scales of Tradition-
alism from the MPQ and JPI. As might be expected from
two scales that share a label, they are highly associated (r
= .77). However, the two nodes labeled CR and IN are the
Creativity facet of the HEXACO and the Innovation scale
of the CPI, respectively. Despite being labeled differently,
the pairwise association between these scales is nearly
identical (r = .76) to that of the Traditionalism scales. Thus,
the question is whether this correlation is evidence for con-
vergent or discriminant validity?
This same question arises when we consider both
broader clusters of traits within the nomological net and
traits that may be considered to be among the most highly
2013 Hogrefe Publishing
 M. Ziegler et al.: Editorial 159

Figure 1. A network diagram of the correlations between 113 facet scales from the NEO-PI-R, HEXACO, 6FPQ, 16PF,
MPQ and JPI derived from the Eugene-Springfield Community Sample (n = 459). The diagram was constructed using the
qgraph package in R (Epskamp, Cramer, Waldrop, Schmittmann & Borsboom, 2012). The graph uses the spring option
and produces a diagram in which the length of the edges is dependent on the weight (correlation) between nodes. This has
the visual effect of drawing more closely associated nodes together in the graph layout.

researched in the field. For example, consider the cluster
of nodes at the bottom center of Figure 1. Two nodes are
the Social Boldness (SB; r = .70) facets of the 16PF and
HEXACO. The remaining three nodes, which share equiv-
alent associations (mean r = .72; range = .61 to .79) with
the other nodes in the node cluster, are the Exhibition facet
of the 6FPQ (EX), Social Potency facet of the MPQ (SP),
and the Social Confidence facet of the JPI (SC). If we
regard the magnitude of associations between the two So-
cial Boldness scales as being indicative of their conver-
gent validity, then we have four different labels for the
same construct within this single cluster. Next, consider
the cluster of four nodes to the left of Figure 1, which
represent the Anxiety (AX) facet from the NEO-PI-R, JPI,
and HEXACO, and the Stress Reaction (SR) facet of the
MPQ. The situation within this cluster is the same as that
found for the sociability scales. The Anxiety scales have
a mean correlation of .68, whereas the average correlation
of the Stress Reaction scale with the three Anxiety scales
is .72.
Finally, consider the cluster of nodes at the top of Fig-
ure 1. Two are the Order (OR) facets of the 6FPQ and
NEO-PI-R, two are the Perfectionism (PF) facets of the
16PF and HEXACO, and two are the Organization (OG)
facets of the JPI and HEXACO. Of note here is that, while
most nodes are quite highly related something we may
expect as they can be argued to all cluster under some
(perhaps higher-order) Conscientiousness factor the two
Perfectionism scales have a notably different pattern of
associations with other related scales, despite sharing a
facet label. As such, when selecting a Perfectionism scale
from an extant inventory to study the convergent or dis-
criminant validity of a new measure, our choice of com-
parison Perfectionism scale may have profound implica-
tions for whether we consider our new scale to be distinct
or not.
A cursory glance at the rest of the network graph shown
in Figure 1 highlights many other areas of local clustering
not emphasized here. Thus, when researchers follow
Campbell and Fiskes guidelines and select measures that
purportedly capture the same trait to ascertain discrimi-
nant validity, they might be in for a surprise: The pattern
of convergent and discriminant correlations may not be as
expected. Test constructors have to be careful when se-

2013 Hogrefe Publishing European Journal of Psychological Assessment 2013; Vol. 29(3):157161
160 M. Ziegler et al.: Editorial
lecting convergent measures and ensure the highest pos-
sible conceptual and statistical overlap. Again, this judg-
ment requires a clearly defined construct embedded in a
clearly defined nomological net. Network visualizations
of the nomological net of personality facets may greatly
aid such decisions during scale development.
Possible Reasons for the Jingle-Jangle
Reasons for the low convergent validities Pace and Bran-
nick provided in their paper are item context (e.g., general
context or work specific context), breadth of the instru-
ment, and test family. The latter refers to the differences
between instruments from the NEO family and those from
the BFI family (see also Miller, Gaughan, Maples, &
Price, 2011). However, the first as well as the second rea-
son stated bear further implications for assessment-orient-
ed research. It is a well-documented fact that changing the
context of an item, for example, by adding in school
changes, (mostly) improves test-criterion correlations.
Reasons for this might be found within the ideas of Bruns-
wiks lens model (see also Miller et al., 2011). More im-
portant here though is the question how this added piece
of information might change construct validity of the
measurement tool used. Thus, we need empirical research
to investigate these effects.
The second reason for the low convergent correlations was
breadth of the measurement tool. It is no new insight that most
traits can be described as being hierarchically organized: Be-
low a rather abstract domain there are narrower facets. As
before, there is evidence suggesting that such facets improve
test-criterion correlations (Brunswik, 1955). However, for
most traits there is no common agreement about the number
and nature of such facets. Pace and Brannick stated:
Recognition of the facets measured by tests may lead toward
understanding similarities and differences among personality
tests, and perhaps the nature of any differential prediction by
tests. (p. 675)
Issue 2: The Issue of Method Variance
The second issue we want to raise with regard to Campbell
and Fiske is method variance. Campbell and Fiske (1959)
wrote:
The interpretation of the validity diagonal in an absolute fash-
ion requires the fortunate coincidence of both an independence
of traits and an independence of methods, represented by zero
values in the heterotrait-heteromethod triangles. . . . In prac-
tice, perhaps all that can be hoped for is evidence for relative
validity, that is, for common variance specific to a trait, above
and beyond shared method variance. (p. 84)
This pessimistic conclusion can be mitigated today. There
are different methodological approaches to modeling all
kinds of method effects (e.g., Eid, Lischetzke, Nussbeck,
European Journal of Psychological Assessment 2013; Vol. 29(3):157161
& Trierweiler, 2003; Podsakoff, MacKenzie, Lee, & Pod-
sakoff, 2003). Despite these new modeling techniques,
Campbell and Fiskes remark should call our attention to
the fact that we still do not know enough about the nature
of method variance. Oftentimes method variance is per-
ceived as variance due to the administration mode (e.g.,
paper-pencil). However, method variance could also be so-
cial desirability (Ziegler & Bhner, 2009), response sets or
styles (Wetzel, Carstensen, & Bhnke, 2013), or acquies-
cence (Rammstedt & Kemper, 2011), to name just three
examples. All of these terms are well known. However,
with the possible exception of social desirability (Paulhus,
2002; Ziegler, MacCann, & Roberts, 2011), elaborated the-
ories of such method variance producing phenomena is
scarce. Thus, researchers with an interest in psychological
assessment should strengthen their efforts to shed light on
method variance producing phenomena like social desir-
ability, response sets and styles, or acquiescence.
Conclusion
In this Editorial we wanted to raise awareness concerning
some of the problems we believe to have been identified
regarding convergent and discriminant validation efforts.
Summarizing the thoughts outlined above, let us stress
three aspects that papers reporting validation studies
should follow: (1) The trait to be measured should be
clearly defined and embedded within a nomological net-
work. (2) Besides convergent validity, discriminant valid-
ity is important in order to gain a more complete picture
of the validity of an instrument. To this end, different traits
have to be assessed with different methods. The nomolog-
ical network should guide the selection of the discriminant
trait(s). (3) Effects of method variance should be modeled.
Moreover, this Editorial also revealed the need for more
research in the areas of method variance producing phe-
nomena (e.g., acquiescence, response sets and styles, and
social desirability), effects of item context (e.g., items spe-
cifically phrased for school or work context), and the facet
structure underlying and defining broad domains.
We want to end this Editorial with a quote from Camp-
bell and Fiske (1959), which in our opinion is as true today
as it was in 1959:
The test constructor is asked to generate from his literary con-
ception or private construct not one operational embodiment,
but two or more, each as different in research vehicle as pos-
sible. Furthermore, he is asked to make explicit the distinction
between his new variable and other variables, distinctions
which are almost certainly implied in his literary definition. In
his very first validational efforts, before he ever rushes into
print, he is asked to apply the several methods and several traits
jointly. His literary definition, his conception, is now best rep-
resented in what his independent measures of the trait hold
distinctively in common. (p. 101)
2013 Hogrefe Publishing
 M. Ziegler et al.: Editorial 161

Schweizer, K. (2012). On issues of validity and especially on the

References misery of convergent validity. European Journal of Psychologi-
cal Assessment, 28, 249254. doi 10.1027/1015-5759/a000156
Alonso-Arbiol, I., & van de Vijver, F. J. R. (2010). A historical Wetzel, E., Carstensen, C. H., & Bhnke, J. R. (2013). Consisten-
analysis of the European Journal of Psychological Assess- cy of extreme response style and nonextreme response style
ment, 26, 238247. doi 10.1027/1015-5759/a000032 across traits. Journal of Research in Personality, 47, 178189.
Brunswik, E. (1955). Representative design and probabilistic the- doi 10.1016/j.jrp.2012.10.010
ory in a functional psychology. Psychological Review, 62, Ziegler, M., & Bhner, M. (2009). Modeling socially desirable
193217. responding and its effects. Educational and Psychological
Campbell, D. T., & Fiske, D. W. (1959). Convergent and discrim- Measurement, 69, 548565.
inant validation by the multitrait-multimethod matrix. Psycho- Ziegler, M., MacCann, C., & Roberts, R. (Eds.). (2011). New per-
logical Bulletin, 56, 81105. spectives on faking in personality assessments. New York: Ox-
Eid, M., Lischetzke, T., Nussbeck, F. W., & Trierweiler, L. I. ford University Press.
(2003). Separating trait effects from trait-specific method ef-
fects in multitrait-multimethod models: A multiple-indicator
CT-C(M-1) model. Psychological Methods, 8, 3860. doi
10.1037/1082-989x.8.1.38 Matthias Ziegler
Epskamp, S., Cramer, A. O. J., Waldorp, L. J., Schmittmann,
V. D., & Borsboom, D. (2012). Qgraph: Network visualiza- Institut fr Psychologie
tions of relationships in psychometric data. Journal of Statis- Humboldt University Berlin
tical Software, 48, 118. Rudower Chaussee 18
Goldberg, L. R. (2005). The Eugene-Springfield community sam- 12489 Berlin
ple: Information available from the research participants Germany
(Vol. 45, 1, Technical Report). Eugene, OR: Oregon Research Tel. +49 30 2093-9447
Institute. Fax +49 30 2093-9361
Miller, J. D., Gaughan, E. T., Maples, J., & Price, J. (2011). A E-mail zieglema@hu-berlin.de
comparison of agreeableness scores from the Big Five In-
ventory and the NEO PI-R: Consequences for the study of
narcissism and psychopathy. Assessment, 18, 335339. doi
10.1177/1073191111411671 Doreen Bensch
Pace, V. L., & Brannick, M. T. (2010). How similar are personal- Institut fr Psychologie
ity scales of the same construct? A meta-analytic investiga- Humboldt University Berlin
tion. Personality and Individual Differences, 49(7), 669676. Rudower Chaussee 18
doi 10.1016/j.paid.2010.06.014 12489 Berlin
Paulhus, D. L. (2002). Socially desirable responding: The evolu- Germany
tion of a construct. In H. I. Braun, D. N. Jackson, & D. E. Wiley Tel. +49 30 2093 9447
(Eds.), The role of constructs in psychological and educational Fax +49 30 2093 9361
measurement (pp. 4969): Mahwah, NJ: Erlbaum. E-mail benschd@cms.hu-berlin.de
Podsakoff, P. M., MacKenzie, S. B., Lee, J. Y., & Podsakoff, N. P.
(2003). Common method biases in behavioral research: A crit-
ical review of the literature and recommended remedies. Jour-
nal of Applied Psychology, 88, 879903. Tom Booth
Rammstedt, B., & Kemper, C. J. (2011). Measurement equiva-
lence of the Big Five: Shedding further light on potential caus- Centre for Cognitive Ageing and Cognitive Epidemiology
es of the educational bias. Journal of Research in Personality, Department of Psychology
45, 121125. The University of Edinburgh
Schweizer, K. (2011). On the changing role of Cronbachs in Edinburgh EH8 9AD
the evaluation of the quality of a measure. European Journal United Kingdom
of Psychological Assessment, 27, 143144. doi 10.1027/1015- Tel. +44 131 650-8405
5759/a000069 E-mail tom.booth@ed.ac.uk

2013 Hogrefe Publishing European Journal of Psychological Assessment 2013; Vol. 29(3):157161
 C. Co nsiglio
European et al.:Psychological
Journalof Work Competencies and
Assessment the Five-Factor
2013;
2012 Vol.
Hogrefe Mod el
29(3):162170
Publishing

Original Article

Framing Work Competencies

Through Personality Traits
The Big Five Competencies Grid
Chiara Consiglio1, Guido Alessandri1, Laura Borgogni1, and Ronald F. Piccolo2
1
Department of Psychology, Sapienza, University of Rome, Italy, 2Rollins College, Winter Park, FL, USA

Abstract. The present study investigated the construct validity and reliability of the Big Five Competencies grid (BFC grid), an instrument
framed and validated within a well-established conceptual model, the Five Factor Model (FFM) of personality. A first sample of 1,307
employees (65% males) in a variety of job types completed the BFC grid and a list of Big Five prototypical adjectives. Exploratory
structural equation modeling attested a six-factor competencies structure corresponding to four broad traits (Extraversion, Agreeableness,
Emotional stability, Openness) and two facets of conscientiousness. The six scales showed adequate reliability and dominant associations
with the corresponding personality traits. In a second sample of 150 employees (87% males), the six-factor structure was confirmed
together with convergent validity between self- and other-ratings of work competencies. Implications and suggestions for future results
are discussed.

Keywords: competencies, ESEM, Big Five traits, BFC grid

In most modern organizational settings, which tend to be or-
ganic, fluid, and characterized by near constant change, de-
scriptions for preferred employee behavior (i.e., job descrip-
tions) have evolved to capture a broad range of diverse func-
tional activities. Employees are expected to be flexible in
their work activities, not limited in terms of their contribu-
tions to organizational functioning. Traditional human re-
source management (HRM) systems, however, apply a
functional perspective that relies on context-specific skills
based on traditional job-based management of activities,
which often fail to adequately capture the broad array of skills
and behaviors that employees bring to modern organizations.
As an alternative, we assert that organizations benefit from
employee-assessment systems based on work competencies,
which are more dynamic, flexible, and generalizable than
context-dependent functional skills (Schippmann et al.,
2000). Competencies are expressed in behaviors that reflect
how individuals perform successfully in a variety of circum-
stances, in order to capture skills that are transferable and
more indicative of work activity in contemporary organiza-
tions. Competency models, by identifying the specific di-
mensions linked to successful performance applicable to dif-
ferent roles and situations (Bartram, 2005), provide a map of
behaviors that is valued and rewarded within organizations.
A competency approach to evaluating personnel has be-
come increasingly popular in the last several years, such that
approximately 7580% of companies use competency-based
HRM systems (Becker, Huselid, & Ulrich, 2001). However,
despite the great popularity and apparent utility of a compe-
tency-based approach, there are at least two main concerns
with application and measurement of these models: (1) the
lack of a widely accepted conceptual definition and interpre-
tation of competency; and (2) the need for reliable and valid
competency-based models and applications.
Over the years, a number of scholars have proposed varied
definitions of the word competency (Spencer & Spencer,
1993), some that may or may not include different individual
characteristics causally related to successful performance.
Consequently, the majority of competency models used in
organizations are expressed as a miscellanea of indistinct in-
dividual descriptors from varied constructs such as traits, mo-
tivations, abilities, skills, knowledge, and behaviors (Markus,
Cooper-Thomas, & Allpress, 2005), whereas they should re-
fer exclusively to observable work behaviors (Bartram,
2005). In addition, competency models are seldom validated,
increasing the risk of bias and evaluation errors (Markus et
al., 2005). Hence, practitioners have routinely called for reli-
able and validated competency tools so critical for the effec-
tiveness of HRM programs.
The use of a competency-based approach implies the
choice between specific and universal models. Many HR
departments seem to prefer generic behavioral models
which are simpler, user-friendly, and allow for comparisons
across roles. However, most of these models lack a valid
conceptual framework that might identify which compe-
tencies should be assessed and developed (Hayes, Rose-

European Journal of Psychological Assessment 2013; Vol. 29(3):162170 2012 Hogrefe Publishing
DOI: 10.1027/1015-5759/a000139
 C. Consiglio et al.: Work Competencies and the Five-Factor Model
Quirie, & Allison, 2005; for a notable exception see Bar-
tram, 2005).
In that vein, we believe one such conceptual framework
is the Five Factors Model of personality (FFM), which rep-
resents a universal and parsimonious model for personality
description. The FFM may provide a relevant source of
universal work competencies that is able to describe the
main behavioral domains related to job performance. Since
competencies are primarily predicated in traits (Bartram,
2005; Kanfer & Ackerman, 2005), we propose in the pre-
sent study that a comprehensive and well-validated person-
ality description model, namely the FFM, represents a con-
sistent and sound frame of reference for rooting a compe-
tency model applicable across diverse roles and
organizations.
Hence, the purpose of our study is fourfold:
1. To present the Big Five Competencies grid (BFC grid)
for assessing work competencies. This grid is framed
within a valid personality model (the FFM);
2. To conduct a preliminary test of the construct validity
and reliability of the BFC grid;
3. To examine relationships between each of the five per-
sonality traits and the competencies assessed with the
BFC grid;
4. To examine the convergent validity between self and
other ratings of the BFC grid.
Traits and Competencies
Personality traits are pervasive and enduring styles of
thought, emotional expression, and behavior that account for
individual differences (McCrae & Costa, 1997). Traits repre-
sent stable individual proclivities acting as potentialities in-
forming individual functioning. The FFM created at the con-
vergence of lexical and questionnaire approaches to the study
of personality, represents a framework of individual differ-
ences that is extensively validated across languages and cul-
tures (McCrae & Costa, 1997). The model provides a consen-
sual, objective, quantifiable description of essential surface
tendencies that capture the between-person uniqueness. Spe-
cifically, the FFM provides a comprehensive, economical,
comprehensible, and reasonably sufficient framework for or-
ganizing individual differences in personality disposition
(Caprara & Cervone, 2000, p. 72). The FFM include Extra-
version, Agreeableness, Emotional Stability, Conscientious-
ness, and Openness (see Caprara & Cervone, 2000, for a com-
plete description of the FFM). Consistent with the original
lexical hypothesis of personality description (Goldberg,
1992), the FFM was originally formed using an adjectival
descriptors approach to measuring traits, which has demon-
strated high levels of scale reliability and construct validity
(Barbaranelli, Caprara, & Steca, 2002).
The FFM has been largely employed in organizational
settings, and is widely regarded as occupationally relevant
given its consistent relationship to effective work behaviors
(e.g., Barrick & Mount, 1991). Conscientiousness is the
2012 Hogrefe Publishing
163
most reliable predictor of job performance across occupa-
tions, organizations and countries, while different combi-
nations of traits have yielded strong validities for specific
occupations (Barrick, Mount, & Judge, 2001; Ones, Dil-
chert, Viswesvaran, & Judge, 2007). A similar organiza-
tional approach was taken by the NEO Job Profiler (Costa,
McCrae, & Kay, 1995), which was designed to identify the
different job requirements defined in terms of traits that
contribute to performance in specific jobs.
In our view, broad personality dimensions may serve as
general frame to organize sets of effective work behaviors,
namely generic competencies which may apply to all pro-
fessions. If traits portray an individuals behavioral patterns
in both work and nonwork related situations, then compe-
tencies, being rooted in traits (Kanfer & Ackerman, 2005;
Spencer & Spencer, 1993), set the limits of how individual
potentialities turn into effective organizational behaviors.
The Big Five Competencies Grid
Given that personality is the most important source of com-
petent behavior, we used the FFM as the theoretical frame-
work for generating the BFC grid. Because research attests
that all five traits predict work performance across different
professional groups (Ones et al., 2007), we assert that the
basic five traits may represent relevant sources of five dis-
tinct sets of effective work behaviors, each one operation-
alized as a generic work competency, namely:
1. Proactivity (i.e., starting and rapidly carrying on ones
own work also through fast decision making) related to
Extraversion;
2. Teamwork (i.e., effectively cooperating with colleagues,
supporting them and integrate the contribution of differ-
ent people) related to Agreeableness;
3. Innovation (i.e., handling work problems and situations
in innovative and creative manners) pertaining to Open-
ness;
4. Emotion management (i.e., effectively facing work
stress, and emotionally effortful situations, such as neg-
ative feedback or pressure on deadlines) related to Emo-
tional Stability;
5. Accomplishment (i.e., organizing work activities and
persisting over obstacles and impediments to get goals)
related to Conscientiousness.
The present article presents two studies. The first, conduct-
ed on a large sample of employees from different positions
and organizational settings, explores the factorial validity
of the hypothesized five competencies and its relationship
with traits, measured by adjectives. We do so by testing two
hypotheses:
Hypothesis 1: The five identified competencies (namely,
Proactivity, Teamwork, Innovation, Emotion manage-
ment, and Accomplishment) represent five different but
correlated latent factors, hence each item will load on its
corresponding factor.
European Journal of Psychological Assessment 2013; Vol. 29(3):162170
164 C. Consiglio et al.: Work Competencies and the Five-Factor Model
Hypothesis 2: Each of the five competency scales is pri-
marily related with the corresponding personality trait
measured by prototypical adjectives.
The second study, carried out on an independent work sam-
ple, replicates the results of the first study concerning the
factorial validity of the BFC grid and examines the conver-
gent validity between self and other ratings (colleagues)
assessment of work competencies. Competency tools are
mainly used in multisource evaluations systems which in-
volve different informants (e.g., the job incumbent, super-
visor, colleagues). Hence, in order to test whether the BFC
grid is a valid measure of both self and others ratings of
competencies, we test the following hypothesis:
Hypothesis 3: Each competency of the BFC grid will
demonstrate an adequate level of convergent validity be-
tween self and other ratings.
Study 1
Method
Participants and Procedure
The research involved a sample of 1,307 employees (35%
females) from a cross-section of organizations (52% from
the public sector). Job positions include a wide range of
occupational roles (28% were clerks, 32% were profession-
als, 29% were blue-collars). The majority of these employ-
ees (44%) had worked between 7 and 10 years within the
same organization. The employees completed the question-
naire anonymously. In order to reduce sequence effects,
presentation of the measures of interest was counterbal-
anced such that half of the respondents had a survey with
items from the BFC grid first and adjectives from the FFM
second; while half had a survey with the FFM model first,
BFC grid second. We measured the two scales using dif-
ferent items and anchors in order to reduce common meth-
od bias.
Measures
The BFC Grid
Initially, 90 items were derived from well-known existing
competency models (Bartram, 2005; Spencer & Spencer,
1993) together with a list of competency repertoires used
in performance appraisal systems of different organiza-
tions. Only observable work behaviors formulated in ac-
tion terms were selected, worded in the first person. Based
on content analysis, three independent judges assigned
each item to one of the five competency categories, or to a
sixth category if an item was deemed unfit for one of the
five categories. The 40 items conjointly assigned by all the
three judges to one of the five competencies were selected
European Journal of Psychological Assessment 2013; Vol. 29(3):162170
to form the final version of the BFC grid comprising the
following scales:
Proactivity, measured with 6 items (e.g., At work I ini-
tiate, rather than wait for instructions); Teamwork, mea-
sured by 8 items (e.g., I help colleagues that are in trou-
bles);
Innovation, measured by 7 items (e.g., I adopt innova-
tive approaches when faced with problems);
Emotion management, measured by 8 items (e.g., I
keep calm even in presence of a negative work cli-
mate);
Accomplishment, measured by 11 items (e.g., I guaran-
tee the accomplishment of the goals assigned to me).
Each item was evaluated on a frequency scale ranging from
1 (= never) to 7 (= always). This choice was made because
behavioral frequency scales to assess competencies, as
compared to other judgment scales, are considered more
evidence based, acceptable, reliable and at the same time
independent from the job content (Grote, 1996).
Personality
Each participant gave self-descriptions by responding to a
list of 25 prototypical adjective markers of the FFM in the
Italian lexicon (Barbaranelli et al., 2002), indicating the ex-
tent to which each adjective was characteristic of his or her
personality (1 = not at all, 5 = very much). The list included
five markers for each dimension: Extraversion (e.g., active,
dynamic); Agreeableness (e.g., cordial, unselfish); Consci-
entiousness (e.g., reliable, scrupulous); Emotional Stability
(e.g., level-headed, relaxed), and Openness (e.g., sharp, up-
to-date). The coefficients were .85, .70, .77, .81, and .83,
respectively.
Statistical Analyses
When the number of factors is expected to be known but
the specific pattern of salient and nonsalient loadings is not,
the exclusive pursuit of strictly exploratory or strictly con-
firmatory models is discouraged (Bollen, 1989). Alterna-
tively, partially confirmative models (Jreskog & Sorbom,
1979) are recommended. Accordingly, using MPlus (ver-
sion 5.2, Muthn & Muthn, 2008), we performed an ex-
ploratory structural equation modeling (ESEM) procedure,
which provides parameter estimates, standard errors, and
goodness-of-fit statistics typically associated with factor
analysis. Unlike CFA, in which each indicator is forced to
load on factor and to have zero loadings on all other factors,
ESEM estimates all possible factor loadings subject to con-
straints, allowing the model to be identified (Asparouhov
& Muthn, 2009). This approach is preferred when (1)
items pertaining to the same dimensions may differ broadly
in content and yield significant loadings on secondary fac-
tors (as may be the case in this study), and when (2) a sim-
ple structure (i.e., solution with the highest number of un-
2012 Hogrefe Publishing
 C. Consiglio et al.: Work Competencies and the Five-Factor Model 165

ambiguous marker variables per factor and the fewest num-
ber of variables with meaningful loadings, > .40, on more
than one factor), rather than an independent clusters model
(ICM) (i.e., items have significant loadings on a factor and
zero loading on all other factors), is hypothesized.
As we expected positive relationships among different
competencies, we used an oblique Geomin rotation with =
.05 as recommended by Asparouhov and Muthn (2009). Be-
cause multivariate normality cannot be retained in our data
(skewness normalized estimate = 74.98, p < .001; kurtosis
normalized estimate = 1011.20, p < .001), we used a Robust
Maximum likelihood estimator (MLR), which takes the non-
normal distribution of the data into account. For the evalua-
tions of covariance structure models, we used the goodness
of fit supplemented by RMSEA, SRMR, and comparative fit
indexes (CFI). Given that few applications of ESEM have
systematically addressed the appropriateness of traditional
indices of model fit (Marsh et al., 2009), in absence of well-
established standards, we evaluated models utilizing the
same recommended standards for traditional structural equa-
tion model analysis (e.g., RMSEA values < .08,
SRMR < .06, CFI > .90 for the CFI, see Kline, 2005). With
the aim of reliably comparing the alternative models, we
adopted the Akaikes information index (AIC), which takes
into account not only model fit but also model parsimony
(Akaike, 1974): The lower the AIC index, the better is the
goodness of fit. Moreover, in the evaluation of our models
fit, we also considered: (1) parameter estimates in relation to
a priori hypotheses, (2) findings from previous research, (3)
generative theory, and (4) common sense, as recommended
by Marsh and colleagues (2009).
We also computed factor determinacies (i.e., correlations
between item score estimates and their respective factors).
These are crucial in the evaluation of factor analytic findings.
A highly indeterminate factor can produce radically different
factor scores that are nonetheless equally consistent with the
obtained factor loadings. Relations among the FFM person-
ality traits and competencies were examined using zero order
and partial correlations (i.e., by computing the partial corre-
lation between each competence and its related trait, control-
ling for the other four traits).
Results
Means (M), standard deviations (SD), skewness, and kur-
tosis were calculated for each of the 40 items in this study.
Means of the items ranged from 4.76 (item 5) to 6.16 (item
31), with an overall item mean (M) of 5.21 and standard
deviation (SD) of 0.57. The scores revealed a reasonably
normal distribution according to skewness 0.83 (SD =
0.32), and kurtosis 0.69 (SD = 1.01). With the exception of
three items (19, 25, and 36), none of the items had skewness
or kurtosis greater than 1.
ESEM Analysis1
The five-factor solution showed a good data fit, (556) =
1592.35, p < .01, CFI = .918, RSMEA = .042 (90% CI =
.040 .045), SRMR = .029, AIC = 123300,96, and ac-
counted for 42.17% of variance. An inspection of the ro-
tated matrix revealed that the fifth factor was comprised
of only four of the 11 hypothesized items for Accomplish-
ment (the remaining seven were sparse across factors).
Thus, we proceeded further in extracting a six-factor so-
lution, which showed an adequate fit to the data, (522) =
1327,70, p < .01, CFI = .936, RSMEA = .039 (90% CI =
.036 .041), SRMR = .025, AIC = 123104,31, and ac-
counted for 54% of the variance. This solution provided
a better fit than the five-factor solution (34) = 264.65,
p < .01, and showed a lower AIC2. Each factor was mean-
ingfully and readily interpretable, with the Accomplish-
ment items (6 and 5, respectively) loading on two different
factors (Table 1).
The first factor (27% variance explained; labeled Pro-
cess Management) was comprised of six items from the
original Accomplishment scale. These six items referred
to behaviors concerning planning, organization and time-
liness of the work processes (e.g., In my job, I regularly
organize my activities). The second factor (7% variance
explained) was comprised of the six items aimed at meas-
uring Proactivity; the third factor (5% variance ex-
plained) by the eight items aimed at measuring Emotion
Management; the fourth factor (4% variance explained)
by the eight items aimed at measuring Teamwork; the
fifth factor (4% variance explained) by the seven items
aimed to measure Innovation. Finally, the sixth factor
(3% variance explained) was comprised of the five Ac-
complishment items that referred to behaviors con-
cerning goals attainment through effort and persistence
(e.g., I guarantee the accomplishment of the goals as-
signed).
All primary loadings were greater than .40 (M = .50;
SD = 0.09); all secondary loadings were lower than .29
(M = .04, SD = 0.08)3. Correlations between factors

1 All models tested differed only by the number of factors they implied. All were congeneric models, with no correlations among uniqueness
allowed.
2 As the number of factors in the final model was different from what hypothesized, we performed a Parallel analysis (Horn, 1965), in order
to further corroborated this unexpected result. Only six eigenvalues of the unreduced and unrotated correlation matrix (i.e., 10.69, 2.34, 1.62,
1.45, 1.35, 1.24, 1.05, 1.03, .95, .92, .88, .85, .80, .77, .76), resulted greater in value than the corresponding eigenvalue (at the 95th percentile)
associated of random data (i.e., 1.38, 1.34, 1.28, 1.25, 1.23, 1.21, 1.19, 1.18, 1.16, 1.14, 1.13, 1.11, 1.10, 1.08, 1.08). Accordingly, the
extraction of six factors seemed justified.
3 Following a recommendation by Asparouhov and Muthn (2009), we also performed a Target rotation. The results further corroborated
those presented in Table 1. When target-rotation was used, all items showed a substantial loading on the intended factors of the six-factor

2012 Hogrefe Publishing European Journal of Psychological Assessment 2013; Vol. 29(3):162170
166 C. Consiglio et al.: Work Competencies and the Five-Factor Model

Table 1. The ESEM solution for the best fitting model

F1 F2 F3 F4 F5 F6
Process management
Item1 .62** .10* .06 .00 .07 .07
Item2 .55** .06 .05 .09 .19* .05
Item3 .41** .15* .17 .00 .02 .07
Item4 .40** .18* .08 .03 .11* .05
Item5 .52** .02 .02 .04 .23* .11*
Item6 .51** .04 .04 .05 .11* .06
Proactivity
Item7 .15** .43** .02 .13** .05 .01
Item8 .07 .46** .06 .05 .18** .11*
Item9 .05 .66** .09 .12** .02 .13**
Item10 .08 .55** .00 .01 .01 .05
Item11 .17** .59** .03 .03 .06 .11*
Item12 .05 .76** .12* .02 .03 .09
Emotion management
Item13 .06 .16** .11* .42** .08 .04
Item14 .03 .07 .15** .44* .09 .00
Item15 .12** .02 .18** .40** .04 .01
Item16 .05 .05 .14** .61** .04 .01
Item17 .01 .02 .04 .49** .06 .26**
Item18 .04 .28** .01 .49** .00 .11*
Item19 .03 .11* .19** .45** .03 .13**
Item20 .04 .05 .00 .42** .19** .01
Teamwork
Item21 .02 .15** .46** .04 .03 .21**
Item22 .01 .08 .49** .00 .03 .26**
Item23 .09 .01 .44** .10* .05 .14**
Item24 .02 .11* .68** .06 .03 .04
Item25 .02 .01 .54** .06 .22** .01
Item26 .10* .04 .42** .08 .08 .21**
Item27 .05 .04 .41** .21** .12** .23**
Item28 .03 .00 .41** .15** .04 .18**
Innovation
Item29 .04 .08 .04 .00 .69** .03
Item30 .12** .00 .01 .01 .52** .10*
Item31 .00 .26** .01 .02 .56** .03
Item32 .01 .08 .05 .16** .52** .09
Item33 .07 .04 .01 .05 .46** .07
Item34 .10* .13** .05 .14** .42** .09
Item35 .09 .17 .07 .01 .49** .15**
Accomplishment
Item36 .20** .04 .07 .08 .05 .57**
Item37 .13** .06 .03 .03 .03 .45**
Item38 .12* .00 .06 .03 .01 .48**
Item39 .06 .11* .15** .04 .03 .43**
Item40 .06 .11* .15** .04 .03 .43**

European Journal of Psychological Assessment 2013; Vol. 29(3):162170 2012 Hogrefe Publishing
 C. Consiglio et al.: Work Competencies and the Five-Factor Model 167

Table 1 (continued)

F1 F2 F3 F4 F5 F6
1 2 3 4 5 6
1. Process management (.74)
2. Proactivity .49** (.71)
3. Emotion management .49** .35** (.79)
4. Teamwork .41** .40** .33** (.78)
5. Innovation .36** .51** .28** .37** (.76)
6. Accomplishment .25** .26** .21** .10* .24** (.76)
Notes. *p < .05, **p < .01. Primary loadings are in bold. Reliabilities are on the diagonal within brackets.

Table 2. Correlations between competencies and the Big Five personality traits
PM EM TM IN AC PR
r rp r rp r rp r rp r rp r rp
A .21** .08* .29** .06 .63** .37* .18** .01 .22** .09* .06 .02
E .36** .10* .26** .14** .21** .08* .27** .19* .31** .10* .38** .18**
C .60** .49** .29** .10* .31** .16* .28** .02 .53** .43** .23** .11**
S .08 .06 .45** .26** .21** .02 .11** .03 .06 .09* .03 .04
O .02 .05 .04 .11** .07** .01 .59** .35** .01 .04 .06 .08
Notes. R = Pearson 0 order correlation; rp = partial correlation. Numbers in bold indicate hypothesized competenciesBig five relationships.
A = Agreeableness; E = Energy; C = Conscientiousness; S = Emotional stability; O = Openness; PM = Process Management; EM = Emotion
Management; TM = Teamwork; IN = Innovation; AC = Accomplishment; PR = Proactivity.

ranged from .08 to .51 (M = .34, SD = 0.13) indicating a
good degree of discrimination among the extracted fac-
tors4. Moreover, all items showed a satisfying degree of
factor simplicity (e.g., the ratio between the percentage
of variance shared with its factor and the percentage of
variance shared with all other factors). Factor score de-
terminacies proved to be adequately high (from .80 to
.93) as were reliabilities for the unit weighted scales (see
Table 1). Finally, according to the Harman (1967) single
factor test, we extracted a one-factor solution to deter-
mine whether the majority of the item variances-covari-
ances can be accounted for by one general factor that cap-
tures the influence of method bias on observed item co-
variances. This solution explained 24% of variance and
showed a poor data fit: (702) = 3988.10, p < .01, CFI =
.739, RSMEA = .067 (90% CI = .065.069), SRMR =
.059.
Relationships with the Big Five
All six factors of the competencies scale were strongly asso-
ciated with individuals personality traits (Table 2). Accord-
ing to partial correlations, Teamwork was primarily associat-
ed with Agreeableness, Proactivity was primarily associated
with Extraversion, Process Management and Accomplish-
ment were both strongly associated with Conscientiousness,
Emotion Management had dominant correlations with Emo-
tional Stability. Finally, Innovation was primarily related to
Openness. We further investigated relationships between
competencies and personality traits using ESEM. First, we
established a measurement model for the BFC grid and the
Big Five traits. In particular, the five traits were modeled as
single-indicator latent variables. Then, this measurement
model for the five traits was combined with the six factors
model of the BFC grid. In this measurement model, we freely
estimated the correlations between latent variables (i.e., be-
tween all the six competencies and all the traits). This model
showed an adequate fit (687) = 1706.56, p < .01, CFI =
.928, RSMEA = .038 (90% CI = .036 .040), SRMR = .026.
Then, the correlation of each competency with the conceptu-
ally most closely related trait was fixed to one. Fixing the
correlation between (1) Process Management and Conscien-
tiousness ((1) = 220.61, p < .01), (2) Emotion Manage-
ment and Emotional Stability ((1) = 401.61, p < .01); (3)
Teamwork and Agreeableness ((1) = 200.61, p < .01); (4)
Innovation and Openness ((1) = 196.51, p < .01); (5) Ac-
complishment and Conscientiousness ((1) = 356.61, p <
.01); and (6) Proactivity and Extraversion ((1) = 391.03,
p < .01) resulted in a significant decrease in fit. Accordingly,
competencies can be considered empirically distinct from
personality.

solution (M = .51; SD = 0.07). Furthermore, we found a fully comparable pattern of small secondary loadings (M = .03, SD = 0.09).
4 To better understand the instrument functioning, we compared the interscale correlations with the interfactor correlations, by computing the
individuals mean scores for each dimensions. In general, factor correlation resulted lower (M = .07; SD = 0.04) than observed scores
correlations.

2012 Hogrefe Publishing European Journal of Psychological Assessment 2013; Vol. 29(3):162170
168 C. Consiglio et al.: Work Competencies and the Five-Factor Model

Figure 1. CTCM model of self- and coworker-evaluated competencies. Note. All coefficients are significant at p < .05,
except those in italics. Self = self-evaluations; COW = coworker evaluations.

Study 2
Participants and Procedure
The participants of this study were 150 employees working
for a national mail-delivery company. The mean age of par-
ticipants was 34.62 (SD = 9.99); 87% were men. The ma-
jority (64%) had worked between 10 and 15 years in the
same organization. For each participant, we obtained a peer
rating from one of his/her colleagues on the dimensions of
interest. Workers were separated to avoid possible data ma-
nipulations (e.g., impression management, socially desir-
able responding).
Measures
Competencies were assessed using the same 40 items of
Study 1. Reliability coefficients for each of the 6 subscales
were adequate with s ranging from .75 (Proactivity) to
.85 (Teamwork). These same items were worded in the
third person for the coworker measure of participants
competencies. Again, reliability was good with s ranging
from .79 (Accomplishment) to .92 (Emotion manage-
ment).
Results
As a preliminary step, we ran two ESEM analyses, one for
self-ratings and one for coworker evaluations of competen-
cies. With regard to self-rated competencies, goodness of
fit indexes attested a good fit for the six-factor solution
(522) = 890.88, p < .01, CFI = .943, RSMEA = .036
(90% CI = .026 .071), SRMR = .029. All items showed
significant primary loading (M = .55; SD = 0.05) on their
intended factors, whereas the secondary loadings were gen-
erally low (M = .04, SD = 0.08). Similarly, the ESEM six-
factor solution provided a good fit to the data for cowork-
er-rated competencies, (522) = 771.90, p < .01, CFI =
.951, RSMEA = .031 (90% CI = .023 .081), SRMR =
.0225. Primary loadings were high (M = .53; SD = 0.06),
and secondary loadings were small (M = .05, SD = 0.11).
To evaluate interrater agreement, we suited the correlat-
ed traits correlated methods (CTCM) approach using con-
firmatory factor analysis (CFA), and the estimated factor
scores for self and co-worker rated competencies6. This
framework treats each informant (self and coworker) as a
different method in order to decompose the observed score
variance into two orthogonal components: the method (un-
substantive or nuisance), and the construct (or substantive)
variance. It also allows the latent method factors to be cor-
related across informants. This model fit our data well
(27) = 21.42, p = .77, CFI = 1.00, RMSEA = .00 (.00,

5 The five-factor solution, was also tested for the self-ratings ((556) = 990.10, p < .01, CFI = .891, RSMEA = .041 (90% CI = .043 .111),
SRMR = .038), and coworker ratings ((556) = 891.10, p < .01, CFI = .902, RSMEA = .039 (90% CI = .029 .093), SRMR = .032) BFC
grid. In both cases the six-factor solution fitted better, as revealed by a significant partial test for both self (34) = 99.22, p < .01, and
coworker rating of BFC grid (34) = 99.0, p < .01.
6 The correlation matrix for this analysis may be obtained upon request by the corresponding author.

European Journal of Psychological Assessment 2013; Vol. 29(3):162170 2012 Hogrefe Publishing
 C. Consiglio et al.: Work Competencies and the Five-Factor Model
.052), SRMR = .059. Standardized parameter estimates are
presented in Figure 1. Factor loadings were all statistically
significant.
Overall, convergent validity was found for all compe-
tencies. Moreover, all competencies were substantively
correlated. Higher latent correlations were observed be-
tween Teamwork and Process Management, Teamwork
and Emotion Management, Emotion Management, and
Proactivity.
Discussion and Implications
Results support the factorial validity of the BFC grid pro-
viding evidence of a six-competency model, namely Ac-
complishment, Process Management, Innovation, Emotion
Management, Teamwork, and Proactivity. Contrary to our
expectations, six and not five competencies emerged from
the ESEM analysis with two primary and differentiated fac-
tors associated with conscientiousness. Process Manage-
ment is likely associated with the dependability facet of
conscientiousness, defined as the tendency to be reliable,
meet attendance standards and deadlines, avoid risks, and
refrain from impulsiveness; such aspects correspond to the
inhibitory aspects of conscientiousness (Moon, 2001). Dif-
ferently, Accomplishment is likely derived from the active
aspect of conscientiousness, ascribable to the achievement-
orientation facet, defined as the tendency to set challenging
goals, work diligently, and strive to improve performance.
It is well known that conscientiousness is the most impor-
tant and generalizable predictor of job performance (Bar-
rick et al., 2001). Moreover among its facets, dependability
and achievement orientation have been identified as the
two dimensions more closely associated with job perfor-
mance (Ones at al., 2007).
A satisfactory level of internal consistency was observed
for the model as each item was highly correlated with its
respective scale. As predicted, personality was strongly cor-
related with perceived competencies. Specifically, in Study
1, we were able to identify strong convergence between each
individual competency and the hypothesized FFM dimen-
sion, thus providing support for the theoretical associations
between each competency and the corresponding trait. More-
over, the distinction between each of the six competencies
and its corresponding personality factors was also demon-
strated. In Study 2, we confirmed the psychometric properties
of the BFC grid and its convergent validity for the two differ-
ent informants (self and peer). Hence, competencies and traits
represent different but related constructs.
Taken together, these results attest to the high psycho-
metric validity of this instrument offering evidence for in-
ternal consistency, factorial validity and convergent valid-
ity in relation to traits and sources of evaluations.
One direct implication of this study is the development
of a valid and reliable tool able to assess relevant work
behaviors (competencies) related to personality traits
2012 Hogrefe Publishing
169
(FFM) that tap different aspects of performance. Our pur-
pose was to test a competency model of generic work be-
haviors, in order to focus on general areas that can be rel-
evant across different jobs (clerks, blue-collars, and profes-
sionals), across private and public organizations, and across
different raters. In that vein, we assert that BFC grid can be
used in performance appraisal systems to evaluate different
occupational roles, covering all relevant personality areas
for performance and giving a contribution in reducing eval-
uation errors and with all scales operationalized at the be-
havioral level. Moreover, the BFC grid offers the opportu-
nity to integrate the performance appraisal system with oth-
er tools used in HR processes, such as selection and
potential evaluation, all framed within the same conceptual
model, namely the FFM. Lastly, the attested convergent
validity among different sources of evaluation can be con-
sidered an encouraging result given the fact that one of the
possible uses of the BFC grid is related to multisource per-
formance appraisal systems and 360 feedback.
Concerning the limitations of this study, a potential
weakness is that, in the first study, traits and competencies
were assessed by self-report, which exposes the observed
results to social desirability. As all items are positively
keyed and competency behaviors are all socially desirable,
people could have altered their responses regarding the oc-
currence of effective work behaviors. Even if it has been
demonstrated that social desirability does not destroy the
construct validity of the five factors tools (Ones & Viswes-
varan, 1998), future research should add to the BFC grid a
social desirability scale in order to account for the potential
risk of positive distortion. That being said, our second study
overcomes this limitation by integrating self-assessment
with evaluations of competencies made by other raters. An-
other limitation is related to the lack of an objective meas-
ure of performance. However, all the BFC items were de-
rived from well-known competency model and from exist-
ing performance appraisal systems of different
organizations. By the way, some authors argue that other
rated work behaviors may constitute an external perfor-
mance criterion depending on the reliability of the col-
leagues. In fact, colleagues ratings are routinely used to
obtain evaluations of contextual performance (Nielsen,
Hrivnak, & Shaw, 2009) and of job performance (Barrick
et al., 2001; Piccolo, Greenbaum, den Hartog, & Folger,
2010). Future studies should explore the specific contribu-
tion and incremental validity of each competency in pre-
dicting objective performance criteria beyond traits. More-
over, the role of alternative predictors of job performance,
such as mental ability, and its relationship with the six com-
petencies should also be explored. Finally, the results of
Study 1 revealed the existence of some low to moderate
secondary loadings in our model; hence factor scores
would yield a more accurate representation of the con-
structs of the BFC grid. In this regard, future research
should further evaluate the validity of factor scores as com-
pared to unit weighted scores of competencies in predicting
job performance.
European Journal of Psychological Assessment 2013; Vol. 29(3):162170
170 C. Consiglio et al.: Work Competencies and the Five-Factor Model
In conclusion, organizations would benefit from includ-
ing the BFC grid in their performance appraisal systems in
order to assess and develop generic work competencies
with a theoretically sound instrument that has applicability
to a variety of jobs and organizations.
References
Akaike, H. (1974). A new look at the statistical model identifica-
tion. IEEE Transactions on Automatic Control, 19, 716723.
Asparouhov, T., & Muthn, B. (2009). Exploratory structural equa-
tion modeling. Structural Equation Modeling, 16, 397438.
Barbaranelli, C., Caprara, G. V., & Steca, P. (2002). BFA. Big Five
Adjectives, manual. Firenze: Giunti OS.
Barrick, M. R., & Mount, M. K. (1991). The Big Five personality
dimensions and job performance: A meta-analysis. Personnel
Psychology, 44, 126.
Barrick, M. R., Mount, M. K., & Judge, T. A. (2001). Personality
and job performance at the beginning of the new millennium:
What do we know and where do we go next? International
Journal of Selection and Assessment, 9, 930.
Bartram, D. (2005). The great eight competencies: A criterion-
centric approach to validation. Journal of Applied Psychology,
90, 11851203.
Becker, B. E., Huselid, M. A., & Ulrich, D. (2001). The HR Score-
card: Linking people, strategy, and performance. Boston, MD:
Harvard Business School Press.
Bollen, K. A. (1989). Structural equations with latent variables.
New York: Wiley.
Caprara, G. V., & Cervone, D. (2000). Personality. Determinants,
dynamics, and potentials. Cambridge, UK: Cambridge Uni-
versity Press.
Costa, P. T., McCrae, R. R., & Kay, G. (1995). Persons, places,
and personality: Career assessment using the revised NEO per-
sonality inventory, Journal of Career Assessment, 3, 123139.
Goldberg, L. R. (1992). The development of the markers of the
big five factor structure. Psychological Assessment, 4, 2642.
Grote, R. C. (1996). The complete guide to performance apprais-
al. New York: Amacom.
Harman, H. H. (1967). Modern factor analysis. Chicago, IL: Uni-
versity of Chicago Press.
Hayes, J., Rose-Quirie, A., & Allison, C. V. (2005). Senior man-
agers perceptions of the competencies they require for effec-
tive performance; implications for training and development.
Personnel Review, 29, 92105.
Horn, J. L. (1965). A rationale and test for the number of factors
in factor analysis. Psychometrika, 30, 179185.
Jreskog, K. G., & Srbom, D. (1979). Advances in factor anal-
ysis and structural equation models. Cambridge, MA: Abt
Books.
Kanfer, R., & Ackerman, P. L. (2005). Work competence: A per-
European Journal of Psychological Assessment 2013; Vol. 29(3):162170
son-oriented perspective. In A. J. Elliot & C. S. Dweck (Eds.),
Handbook of competence and motivation (pp. 336353). New
York: Guilford.
Kline, R. B. (2005). Principles and practices of structural equa-
tion modeling. New York: Guilford.
Markus, L. H., Cooper-Thomas, H. D., & Allpress, K. N. (2005).
Confounded by Competencies. An evaluation and use of com-
petency models. New Zealand Journal of Psychology, 34,
117126.
Marsh, H. W., Muthn, B., Asparouhov, T., Ldtke, O., Robitzsch,
A., Morin, A. J. S., & Trautwein, U. (2009). Exploratory struc-
tural equation modeling, integrating CFA and EFA: Applica-
tion to students evaluations of university teaching. Structural
Equation Modeling, 16, 439476.
McCrae, R. R., & Costa, P. T. (1997). Personality trait structure
as a human universal. American Psychologist, 52, 509516.
Moon, H. (2001). The two faces of conscientiousness: Duty and
achievement striving in escalation of commitment dilemmas.
Journal of Applied Psychology, 86, 533540.
Muthn, L. K., & Muthn, B. (2008). MPlus users guide. Los
Angeles, CA: Muthn & Muthn.
Nielsen T. M., Hrivnak G. A., & Shaw M. (2009). Organizational
citizenship behavior and performance: A meta-analysis of
group-level research. Small Group Research, 40, 555577.
Ones, D. S., Dilchert, S., Viswesvaran, C., & Judge T. A. (2007).
In support of personality assessment in organizational settings.
Personnel Psychology, 60, 9951027.
Ones, D. S., & Viswesvaran, C. (1998). The effects of social de-
sirability and faking on personality and integrity assessment
for personnel selection. Human Performance, 11, 245269.
Piccolo, R. F., Greenbaum, R., den Hartog, D., & Folger, R.
(2010). The relationship between ethical leadership and core
job characteristics. Journal of Organizational Behavior, 31,
259278.
Schippmann, J. S., Ash, R. A., Battista, M. A., Carr, L., Eyde,
L. D., Hesket, B., . . . Sanchez, J. I. (2000). The practice of
competency modeling. Personnel Psychology, 53, 703740.
Spencer, L. M., & Spencer, S. M. (1993). Competence at work:
Models for superior performance. New York: Wiley.
Published online: May 11, 2012
Chiara Consiglio
Department of Psychology
Sapienza, University of Rome
Via dei Marsi 78
00185 Rome
Italy
Tel./Fax +39 64991-7723
E-mail chiara.consiglio@uniroma1.it
2012 Hogrefe Publishing
 T. G. Bau dson & F. Preckel: T(H)INK:
European Development
Journalof and
Psychological Validation
Assessment of anVol.
2013;
2012 Intelligence
Hogrefe Test
29(3):171181
Publishing

Original Article

Development and Validation

of the German Test for (Highly)
Intelligent Kids T(H)INK
Tanja Gabriele Baudson and Franzis Preckel
Department of Psychology, Giftedness Research and Education, University of Trier, Trier, Germany

Abstract. Background: Currently, no recently normed group intelligence test for primary school children is available in Germany to
cover verbal, numerical, and figural reasoning abilities. Furthermore, no test for this age group is difficult enough to be able to differentiate
especially at the upper end of the intelligence distribution. Aims: The Test for (Highly) Intelligent Kids T(H)INK was developed to fill
this gap. Methods: A total of 1,629 children from grades 1 to 3 of primary school took the 36-item test (one version per class level).
Results: At all three class levels, the test proved sufficiently reliable (Cronbachs = .73.80 for the full scale). Retest reliabilities after
1 year, assessed for a random subsample, ranged from .61 to .73. Factorial validity was established by exploratory and confirmatory
factor analyses, with a g factor explaining 57.6163.67% of the variance. Furthermore, convergent validity (correlations with a nonverbal
intelligence test, teacher and parent ratings, grades, and the probability for the child to be nominated for a gifted and talented program)
was satisfying, as was the predictive validity for average marks a year later. Discussion: Implications of the findings and ideas for future
research and development of the test are discussed.

Keywords: intelligence, intelligence test, cognitive ability, cognitive assessment, primary school, giftedness

Although the conception of cognitive ability and its develop-
ment is still a matter of debate, intelligence can be considered
one of the best predictors of academic and professional at-
tainment, explaining a larger amount of variance than all oth-
er predictors (e.g., Gottfredson, 1997; Neisser et al., 1996). It
can be assessed objectively, reliably, and validly by psycho-
metric tests.1 In children, intelligence tests have proved their
usefulness in developmental and educational counseling and
interventions (e.g., identification of underachievement or
specific learning disabilities). Anastasi and Urbina (1997)
outlined the advantages of individual testing (e.g., closer rap-
port between child and administrator, less boredom or frus-
tration due to more flexible and adaptive procedures). These
issues are of particular importance when testing children
(e.g., Flanagan & Kaufman, 2009). However, group intelli-
gence tests have specific advantages: They allow for an eco-
nomic assessment of childrens ability level (e.g., Ford &
Dahinten, 2005) and are therefore highly suitable for research
or screening purposes. In general, the choice of the assess-
ment method depends on its purpose.
In Germany, no recently normed group intelligence test
for primary school children comprising crucial content do-
mains is currently available. Furthermore, no test for this
age group is difficult enough to allow for differentiation at
the right tail of the intelligence distribution. The newly de-
veloped and validated group intelligence test presented
here aims to fill this gap.
Structure and Development of Childrens
Intelligence
The early observation that different cognitive abilities tend
to correlate (Spearman, 1904) led to the now well-support-
ed assumption of an underlying general cognitive ability,
or g (Carroll, 1993). g has consistently been found to ac-
count for the greatest share of interindividual differences
in cognitive ability (Jensen, 1998). As to intellectual devel-
opment, investment theory (Cattell, 1963; revision Schwei-
zer & Koch, 2001) posits that, during childhood and ado-
lescence, fluid intelligence (Gf, the innate ability to solve
novel problems, e.g., by drawing inferences) is invested
into the acquisition of crystallized intelligence (Gc, the

1 Other factors influencing success (e.g., other ability domains, or person or environmental factors moderating the relationship between ability
and its successful actualization, in itself a fuzzy term), have been addressed, for instance, in the Munich Model of Giftedness (e.g., Heller
& Perleth, 2008), which conceives of ability as a multidimensional construct requiring beneficial person and environmental conditions to
fully develop. (The authors thank an anonymous reviewer for bringing forward this complementary aspect.)

2012 Hogrefe Publishing European Journal of Psychological Assessment 2013; Vol. 29(3):171181
DOI: 10.1027/1015-5759/a000142
172 T. G. Baudson & F. Preckel: T(H)INK: Development and Validation of an Intelligence Test
knowledge that has been gained, e.g., Horn, 1991). Because
children have had fewer opportunities to acquire knowl-
edge and because schools tend to standardize the growth
of the knowledge base during the time of schooling
(Schweizer & Koch, 2001, p. 78), Gf can be assumed to be
the best estimate for childrens g.2 Both Gf-Gc and g factor
theory can be subsumed under the widely acknowledged
CHC model of intelligence (McGrew, 2005).
Measurement of Fluid Intelligence in
Children
To assess Gf in children, one must take verbal (VA), numeri-
cal (NA), and figural abilities (FA) into account (Marshalek,
Lohman, & Snow, 1983); especially reasoning in the three
domains is the best indicator of Gf and g (Lohman, 2002). VA
and NA are valid predictors of scholastic and professional
achievement (Wai, Lubinski, & Benbow, 2005), whereas FA
are far less important in curriculum development (Wai, Lu-
binski, & Benbow, 2009) and, hence, to academic success.
However, uneven ability profiles in favor of FA have even
reported to be detrimental (Smedler & Trestad, 1996) or
related to underachievement (Gohm, Humphreys, & Yao,
1998), underlining the importance of their early identifica-
tion. FA also play a role in many disciplines of the fields of
science, technology, engineering and mathematics (STEM).
Furthermore, FA tests use nonverbal materials and may there-
fore be a better estimate of g for children with verbal difficul-
ties (e.g., nonnative or otherwise disadvantaged speakers),
which is also one reason why they have frequently been used
in so-called culture-fair testing (CFT, see Ortiz & Dynda,
2005, for a critical discussion). Often, such abstract materials
are equally unfamiliar to all testtakers, reducing potential in-
fluences of Gc (as recommended by Beauducel & Kersting,
2002). Reasoning in all three domains verbal, numerical,
and figural should thus be considered in the assessment of
Gf in children.
Rationale for Developing a New Test
Several German tests have been proposed to assess primary
school childrens cognitive abilities, e.g., the WISC-IV as the
most frequently used individual test (Petermann & Peter-
mann, 2011); the CFT 1 (Cattell, Wei, & Osterland, 1997)
and the CFT 20-R (Wei, 2008) are German adaptations of
Cattells Culture Fair Tests and are group tests that assess
figural abilities; the KFT 13 (Heller & Geisler, 1983) and
the KFT 412+R (Heller & Perleth, 2000; the German equiv-
alents to Lohmans CogAT) and the MHBT-P (Heller & Per-
leth, 2007) derived from the latter as multidimensional as-
sessment procedures to be used from year 4 and 3 on, respec-
tively. All tests have their shortcomings, though:
2 Results from a recent study (Schweizer, Troche, & Rammsayer, 2011) examining university students suggest this applies to older populations
as well.
European Journal of Psychological Assessment 2013; Vol. 29(3):171181
1. They may not include all the content areas described
above (verbal, numerical, and figural, e.g., CFT 13 and
CFT 20-R),
2. They may provide outdated norms (e.g., KFT 13),
3. They may allow for individual testing only (e.g., WISC-
IV),
4. They can be used from year 3 on only (e.g., MHBT-P,
KFT 412 +, CFT 20-R), or
5. They do not have sufficiently high ceilings to allow for
differentiated assessment at the right tail of the IQ dis-
tribution (applies to all tests mentioned except for the
MHBT-P).
The Test for (Highly) Intelligent Kids T(H)INK was de-
veloped to fill this gap: T(H)INK targets Gf by tapping ver-
bal, numerical, and figural reasoning, accounting for the
theoretical and empirical findings outlined above.
Test Development: Task Selection,
Construction Principles, and Pilot Testing
Besides representing the three targeted content areas, test
tasks should be suitable for group testing (which excluded,
e.g., working memory tasks such as the WISC digit span
subtest). Because not all pupils are able to read and write
fluently by the end of the first grade, mainly multiple-choice
items (paper and pencil) using images or numbers as solu-
tions were used. For the numerical and figural domain, many
such tasks exist (e.g., number series or figural matrices) and
could thus be adapted to the requirements of the present test
(see Materials section for a detailed description of the final
tasks). However, only few adequate verbal tasks were avail-
able, and only for children in grade 3 and above. Thus, new
types of items were developed for the present test. The con-
struction principles will be described in the following.
Development of Verbal Tasks: Construction
Principles
Carroll (1993) differentiates two dimensions of verbal abil-
ities: (1) receptive vs. productive skills, and (2) oral vs.
written language. During early primary school, children do
not yet master writing, which excluded written production.
Because the test was intended as a group test, oral produc-
tion had to be excluded as well. Therefore, T(H)INK fo-
cuses on receptive skills, using a written instruction (to be
read aloud to the younger pupils) and pictures.
Language is structured into different interacting levels
(Weinert, 2010). T(H)INK consists of a grammatical under-
standing and a vocabulary part, thus including both syntactic
(deep structure) and semantic (surface structure) elements.
Grammatical understanding was designed to tap verbal rea-
2012 Hogrefe Publishing
 T. G. Baudson & F. Preckel: T(H)INK: Development and Validation of an Intelligence Test
soning as one aspect of Gf. The child has to decode which
picture represents a given sentence consisting of familiar
words, but varying in syntactic complexity. Item construction
followed theoretical and empirical findings about language
development, especially acquisition of grammatical struc-
tures. Understanding of surface structures (semantics) always
precedes the development of increasingly complex deep
structures (syntax/morphology; Grimm & Schler, 1991).
For instance, negative clauses are more complex than affir-
mative clauses; compound clauses (two joint main clauses)
more so than single main clauses; and neither-nor construc-
tions, combining negation and compound clauses, represent
an even higher difficulty level. Vocabulary, on the other hand,
is one of the most important predictors of reading acquisition
and, as such, of scholastic achievement (e.g., Cunningham &
Stanovich, 1997). Although vocabulary can be assumed to
represent Gc rather than Gf, it was included as an important
predictor of both understanding of grammatical structures
and later achievement (Grimm & Schler, 1991). Items were
derived from two sources: first, the most comprehensive em-
pirical study providing extensive information on childrens
active use of words (Pregel & Rickheit, 1987) as a proxy for
receptive (passive) vocabulary in the sense of Carroll (1993);
and second, by informally asking parents of primary-school
children about words their children had recently learned and
used (productive, or active vocabulary). Common group vo-
cabulary tests use written words; yet even if all answer op-
tions are read aloud, it may still be difficult for younger chil-
dren to keep all solutions in mind, due to limited working
memory capacities. T(H)INK therefore uses pictures (of con-
crete words mostly, as it is difficult to represent abstract terms
graphically). Altogether, 12 words of increasing difficulty
were thus selected.
Pilot Testing
We assessed the understanding of instructions with 16 pupils
from year 1 (Y1). The items were piloted in two phases. First,
a pool of 45 items was administered to pupils from Y1
through Y4 at two schools (Ns for Y1/Y2/Y3/Y4:
31/68/39/49). Variance, difficulty, and corrected item-total
correlations were evaluated, whereupon items were dropped
(11), modified (1), or newly designed (10 verbal, 14 numeri-
cal, 4 figural). The four resulting test versions for Y1 to Y4
(36 items each) were then administered at another school (Ns
for Y1/Y2/Y3/Y4: 18/12/16/12) to fine-tune instruction
wording and timing. Unlike in other group intelligence tests,
time limits were avoided. Schweizer and Koch (2001) as-
sume untimed testing to be better suited to assess the limits
of reasoning abilities (or, in other terms, capacity). This re-
quires tasks of sufficient difficulty, which was the case with
the present test. In primary-school pupils in particular, the
time required to accomplish a task may not be the best indi-
cator of level of ability (Lavergne & Vigneau, 1997). One
possible reason are changes in metacognitive development
occurring during childhood, especially the acquisition and
2012 Hogrefe Publishing
173
use of strategies (Krajewski, Kron, & Schneider, 2004; Lockl
& Schneider, 2010). A more efficient strategy usually results
in time and resource gains only after it has been automatized.
During this automatization phase, however, children may ac-
tually require more time than with their prior (inferior) strat-
egy. Because many such metacognitive abilities are acquired
during primary-school age, the test was thus constructed as a
power test (150250% of the time the slowest child had need-
ed in pilot testing) so that each child had the chance to work
on every single task (see Materials section for a detailed test
description). Finally, although the test was constructed ac-
cording to classical test theory, we did not want to preclude
the possibility for item-response theoretic models (see Dis-
cussion section), which is preferably applied to unspeeded
tests (de Ayala, 2009).
Method
Participants and Procedure
Sample
Data collection was carried out in two waves. In Phase I, data
were gathered during the last 8 weeks of the school year (May
to early July 2009) at 20 primary schools in Rhineland-Pala-
tinate. In Phase I, only pupils from Y1 (35 classes), Y2 (36
classes), and Y3 (36 classes) took part. Phase II took place 1
year later (June 2010) at two out of the 20 schools; 5 first, 4
second, 5 third, and 4 fourth classes participated. Data from
both phases were collapsed for students attending the same
year. Thus, e.g., Y1 students who were tested again 1 year
later entered both the Y1 and the Y2 sample. For the factor
analyses, where data from different class levels were related
to each other, the second cohort was excluded from the Y23
subsamples to prevent effects of repeated measurements of
the same child. Six schools were located in communities with
more than 5,000 inhabitants (urban), yielding slightly less
than half the sample (46.3348.14%) for Y1 to Y3. This is
comparable to overall Rhineland-Palatinate, where 45.1% of
the population live in communities of 5,000 or fewer inhab-
itants (Statistisches Bundesamt, 2010). The final sample con-
sisted of 40 classes from Y1 (N = 533; 52.2% girls, 7.3%
nonnative speakers [NN]), 40 from Y2 (N = 531; 49.7% girls,
9.4% NN), 41 from Y3 (N = 565; 54.7% girls, 7.8% NN), and
4 from Y4 (N = 76; 46.1% girls, 9.2% NN). For a part of the
sample, parent reports of the childrens age were available
(Y1: 7 years 3 months, SD = 4.98 months, N = 444; Y2: 8
years 4 months, SD = 5.34 months, N = 481; Y3: 9 years 5
months, SD = 5.45 months, N = 497). The percentage of non-
German native speakers was about representative for Rhine-
land-Palatinate (overall rate: 7.6%; Statistisches Landesamt
Rheinland-Pfalz, 2011). However, because the goal of the
present study was to develop and validate the test and not yet
to standardize it, drawing a representative sample was not one
of our primary issues.
European Journal of Psychological Assessment 2013; Vol. 29(3):171181
174 T. G. Baudson & F. Preckel: T(H)INK: Development and Validation of an Intelligence Test
Procedure
Prior to testing, the study was approved by the School Super-
visory Board and the Rhineland-Palatinate data protection
commissioner; parental approval was obtained. In all parts of
the test, students participated voluntarily and were rewarded
with a small toy. Children were tested in their classrooms. The
main examiner and his/her assistant (in Y1 or if else deemed
necessary) presented themselves, handed out the test book-
lets, explained the tasks to come and the procedure. Instruc-
tions and items were also shown on overhead transparencies.
Each subtest was preceded by an example, which was solved
in the group and then explained again to ensure all children
had understood the idea. In Y1 and Y2, all text-based items
were read aloud. The test took about 90 min in Y1 (including
two breaks), 75 min in Y2 (one break), and 45/40 min in
Y3/Y4 (no break required). In Phase II, children were tested
with an additional nonverbal IQ test (Y1/Y2: CFT1, Cattell
et al., 1997; administered 12 days later; Y3/Y4: CFT 20-R,
Wei, 2008; administered after a break). Both CFT versions
are partial adaptations of Cattells Culture Fair Test and meas-
ure fluid intelligence using figural tasks. The tests were ad-
ministered under speeded power conditions, as outlined in the
manual.
Materials
T(H)INK
The final test consisted of four versions (Y1Y4), each
comprising the 36 items with the best item characteristics
for this class level from the pilot phase (12 figural, 12 ver-
bal, and 12 numerical items), with a partial overlap between
the four class levels (see Appendix for an overview of the
items shared by the versions for Y1Y3; taken together,
T(H)INK 14 comprises 64 different items). The test con-
sisted of 8 subtests (3 figural [F], 2 verbal [V], and 3 nu-
merical [N]) presented in the following sequence:
1. F-Picture concepts (similar to the German WISC-IV;
Petermann & Petermann, 2011): 23 pictures are pre-
sented in 23 rows. Subjects are to select one item per
row so that all selected items go together.
2. F-Graphical series completion (similar to the Charkow
tasks used in the Berlin Test of Intelligence Structure
BIS-4; Jger, S, & Beauducel, 1997): A series of sim-
ple graphics following a rule (e.g., a zig-zag line alter-
nately going up and down) is to be continued by two
items.
3. F-Matrices: Subjects are to select the solution out of 5
distractors which completes a 2 2 matrix.
4. V-Understanding of grammar: One out of 45 images
corresponding to a complex sentence (e.g., The triangle
3 We opted for the CFT a nonverbal test because it measures Gf (as intended with T(H)INK) but focuses on a different content factor. Both
the CFT 13 and the CFT 20-R are shorter than other group tests (e.g., the KFT suite), thus providing less strain, and are easy to administer
to groups. In addition, the CFT 20-R provides recent norms.
European Journal of Psychological Assessment 2013; Vol. 29(3):171181
is not white, but in front of the square) has to be iden-
tified.
5. V-Vocabulary: Subjects are to identify an object de-
scribed in the instruction out of 5 pictures (e.g., Which
one is the reptile? Solution: alligator).
6. N-Arithmetic problems: Complex arithmetic problems
including conversions between scales (e.g., from min-
utes to hours) are to be solved.
7. N-Number series: The missing item from a number se-
ries is to be identified.
8. N-Routes on a map (in the style of SAGES-2; Johnsen
& Corn, 2001): A route is described by the instructor
and has to be pursued on a simplified map by the subject,
who then computes the length of the way.
Culture-Fair Tests
Both the CFT 1 (Culture-Fair Test, Scale 1; Cattell et al.,
1997; age- and class-based standardized IQ norms sampled
in 1995) and the CFT 20R (Wei, 2008; age- and class-
based norms sampled in 2003/2004), as partial adaptations
of Cattells original tests, measure fluid intelligence with 5
or 4 types of figural tasks, respectively (CFT 1: substitu-
tions, labyrinths, classifications, similarities, matrices;
CFT 20R: series, classifications, matrices, topologies).
Items are presented under speeded power conditions in a
multiple-choice format (paper and pencil). In the present
study, the complete CFT 1 and the short form of the CFT
20-R (part 1, 4 subtests) were administered.3
Teacher and Parent Questionnaires
Teacher and parent ratings of cognitive abilities, creativity,
motivation, social behavior, and work attitude were assessed
with a self-developed questionnaire based on existing mea-
sures (e.g., Gifted Rating Scale, School Form; Pfeiffer & Ja-
rosevich, 2003; Child Behavior Check List 418/Teachers
Report Form; Achenbach & Rescorla, 2001; Fragebogen zur
Erfassung emotionaler und sozialer Schulerfahrungen
FEESS; Rauer & Schuck, 2003, 2004). Teachers rated each
child on 19 items (7-point Likert scale from 0 = shows this
characteristic less than all other pupils in class to 6 = shows
this characteristic more than all other pupils in class). The
teacher rating scale consisted of 6 reliable scales (sample
items and Cronbachs ranges for Y1 to Y3 in brackets):
cognitive ability (understands new contents quickly, 6
items; = .96.97); creativity (has many ideas, 3 items;
= 92 in all cases), motivation (tries to solve really difficult
tasks, 3 items; = .85.86), social behavior (gets along
well with his/her classmates, 2 items; = .77.80), and
work attitude (works in a neat, clean and organized way, 2
2012 Hogrefe Publishing
 T. G. Baudson & F. Preckel: T(H)INK: Development and Validation of an Intelligence Test
items; = .78.83). Scale sums were averaged if at least 80%
of the items had been answered and were then z-standardized
for each class. Furthermore, the probability (0100%) with
which the teacher would recommend the child for a gifted and
talented program was assessed.
The parent rating scale comprised 34 Likert-type items
(ranging from 0 = is not at all true for my child to 5 = is
perfectly true for my child). Only those items teacher and
parent questionnaire had in common were considered for the
present study. As above, averaged scale sums were used if at
least 80% of the items had been answered. Except for social
behavior, standardized alphas were acceptable across all age
groups (cognitive ability: .88.91; creativity: .77.79; moti-
vation: .70.73; social behavior: .36.55; work attitude:
.71.80). Furthermore, parents were asked about demograph-
ical data, their childs age and scholastic achievement. Be-
cause no marks are given in year 1 of the German school
system, the Y1 pupils parents were asked to make an estima-
tion of their childs achievement. For the parent ratings, only
questionnaires that had been answered by mothers, fathers,
or both were considered, because these groups did not differ
with respect to their assessment. Other raters judgment (e.g.,
grandparents, older siblings, foster parents, or neighbors; Y1:
N = 16; Y2: N = 15; Y3: N = 13) differed significantly,
though; these cases were therefore excluded.
Data Analyses
Except for the multiple group comparisons, which used data
from Phase I only, the following analyses are based on the
complete sample, including data from both Phase I and II.
Incomplete data sets and other problematic cases (e.g., obvi-
ous noncompliance to instructions) were eliminated; the
number of invalid cases thus determined for Y1/Y2/Y3 were
32 (5.7%)/23 (4.2%)/19 (3.3%). Prior to analyzes, data were
parceled by averaging the raw score sums for each content
area (figural, verbal, numerical). A content-based differenti-
ation at a medium aggregation level makes sense from both
theoretical and practical points of view. Theoretically (e.g., in
the BIS, Jger, 1984; or in the Radex model, Marshalek et al.,
1983) and empirically (e.g., considering findings by Benbow
and colleagues), the three ability domains are well estab-
lished. Practically speaking, greater measurement error is to
be expected with the younger children, caused by factors such
as unfamiliarity with test-like situations, fluctuating atten-
tion, test anxiety, etc. (Ford & Dahinten, 2005); their results
are therefore less reliable than those of older children. To
minimize measurement error, it makes sense to aggregate
data at a level the researcher considers meaningful (see Little,
Cunningham, Shahar, & Widaman, 2002, for a critical dis-
cussion of parceling). The procedure has also been used in
other intelligence tests, e.g., the SAGES-2 (Johnsen & Corn,
2001).
First, the T(H)INK full scale sums were examined for dis-
tribution and mean differences between sex and language
group using t-tests. Second, item parcels (averaged scale
2012 Hogrefe Publishing
175
sums for figural, verbal, and numerical reasoning) were ana-
lyzed for difficulty and corrected item-total correlations ac-
cording to classical test theory. Third, internal consistency of
the total scale was examined (Cronbachs ). Test-retest reli-
ability was established by correlating pupils results with
what they achieved 1 year later. In a fourth step, extensive
validity examinations were conducted. Factorial validity was
assessed using both exploratory (EFA) and confirmatory fac-
tor analyses (CFA). Construct validity was established by
correlations with teacher and parent ratings, school marks,
and the nonverbal IQ measures. Because the German grading
scale ranges from 1 = high distinction to 6 = insufficient,
marks were inverted such that high values indicate higher
achievement. All data were analyzed using PASW 18.0; the
CFA was computed with AMOS 18.0.
Results
Group Comparisons
Regarding full scale sums, sex had an influence in Y2 only
(Mgirls = 18.61, SD = 5.66; Mboys = 19.93, SD = 5.73;
t(529) = 2.67, p < .01, Cohens d = .23). Significant dif-
ferences between language groups (German native speak-
ers/bilingual pupils with German as a first language [GB]
versus nonnative speakers [NN]) were observed in Y1 to
Y3 (Y1: MGB = 17.51, SD = 4.85; MNN = 15.28, SD = 4.50;
t(527) = 2.78, p = .006, Cohens d = .48; Y2: MGB = 19.61,
SD = 5.73; MNN = 16.10, SD = 4.64; t(529) = 4.19, p <
.001, Cohens d = .67; Y3: MGB = 21.40, SD = 5.09; MNN =
17.68, SD = 5.40; t(560) = 4.63, p < .001; Cohens d = .71).
Table 1. Difficulties, standard deviations, and corrected
item-total correlations
Year N Parcel p (SD) ritc Percentiles (25/50/75)
1 533 F .39 (0.18) .47 14/17/21
V .57 (0.15) .36
N .49 (0.20) .51
Mean .48 .45
2 531 F .48 (0.20) .53 15/20/24
V .54 (0.18) .50
N .58 (0.22) .59
Mean .54 .54
3 565 F .49 (0.18) .46 18/22/25
V .59 (0.17) .46
N .68 (0.21) .55
Mean .59 .49
Note. Difficulties and item-total correlations were computed based on
the content parcels (averaged sums of the three content areas figural
(F), verbal (V), and numerical (N) ability). p = difficulty, SD = stand-
ard deviation, ritc = corrected item-total correlation. Averaged difficul-
ties and item-total correlations for each year are given in the bottom
lines of each respective year (in italics). The last column shows the
number of items children at the respective percentile level had solved
correctly. No child at any class level attained the maximum of 36.
European Journal of Psychological Assessment 2013; Vol. 29(3):171181
176 T. G. Baudson & F. Preckel: T(H)INK: Development and Validation of an Intelligence Test
Item Analysis
Item difficulties and corrected item-total correlations for
Y1 to Y3 (range and mean values across the three content
parcels) are reported in Table 1. Data for individual items
are given in the Appendix.
Reliability
Cronbachs was acceptable for all three full scales (36
items each; Y1/Y2/Y3: Cronbachs = .73/.80/.76). Sta-
bility indices across 1 year were satisfactory at all class
levels (Y1/Y2: r = .73, N = 51; Y2/3: r = .70, N = 66;
Y3/Y4: r = .61, N = 64; all ps < .001).
Validity
Factorial Validity
Factorial validity of the g-factor model was established us-
ing EFA and CFA. To identify g, we performed an unrotat-
ed principal components analysis using the three content
parcels (figural, verbal, and numerical reasoning) for Y1 to
Y3, respectively. Data were adequate for factor analysis (all
KMOs > .60, all ps/Bartletts test of sphericity < .01; Hair,
Anderson, Tatham, & Black, 1998). One factor was extract-
ed in all grades, explaining between 57.61 and 63.67% of
the total variance. At all three class levels, component load-
ings and communalities exceeded .67 (highest for numeri-
cal and lowest for verbal tasks; see Table 2).
Table 2. Component loadings (CL), communalities (h2), ei-
genvalues (EV), and percent of variance explained
one-factor solution
CL h2 % variance
Year F V N F V N Eigenvalues explained
1 .78 .68 .81 .61 .46 .66 1.73 57.61
2 .79 .77 .83 .62 .59 .70 1.91 63.67
3 .75 .75 .83 .57 .56 .68 1.81 60.39
Notes. Method of extraction: principal components analysis, unrotat-
ed; one g factor extracted.
To check whether the g-factor model would hold simulta-
neously at all class levels, we conducted a CFA (multiple
group comparison, MGC) for Y1 to Y3. Only data from
Phase 1 were considered (Y1/Y2/Y3: N = 486/474/476). In
this analysis also, the g-factor model was tested using the
averaged content parcels (figural/FA, verbal/VA, and numer-
ical ability/NA) as indicators. Because content areas were
represented by different items in the three class levels, par-
celing had the further advantage that it allowed for compari-
sons despite these differences. The variance of g was set to 1
across all three groups. Each content parcel was assigned the
same weight across Y1Y3 (i.e., weights for VA in Y1, Y2,
and Y3 were equated; same for NA and FA), but not within
European Journal of Psychological Assessment 2013; Vol. 29(3):171181
a year (i.e., VA, NA, and FA were not assigned the same
weights in Y1; same for Y2 and Y3). In addition, error vari-
ances were equated within each year (i.e., error variances for
VA, NA, and FA were the set to be equal in Y1; same for Y2
and Y3), but not between years (i.e., error variances for VA
in Y1, Y2, and Y3 were not equated; same for NA and FA).
This makes sense from a developmental point of view, as
factors external to the test itself have a stronger influence in
younger children; yet there was no reason to assume different
error variances for the three content areas within a year. The
MGC model showed excellent fit (/df = .73, p = .73; CFI =
1.00; RMSEA = .00, 90% confidence interval [.00; .02];
SRMR = .01). Again, standardized regression weights were
highest for NA and lowest for VA for Y1 to Y3 (s for
Y1/Y2/Y3: VA .60/.58/.59, NA .76/.75/.76, FA .65/.64/.65).
Construct Validity
Teacher and parent ratings of cognitive abilities, creativity,
motivation, social behavior, and work attitude were used as
criteria for construct validity; so was the probability
(0100%) with which the teacher would recommend the
child for a gifted and talented program. All correlations are
detailed in Table 3. For teachers, intelligence was related to
all domains; for parents, only to achievement-related do-
mains. In both cases, the highest correlations were found for
cognitive ability ratings. Convergent validity was also estab-
lished by correlations between T(H)INK and CFT results
from Phase II (Y1: r = .44, N = 40; Y2: r = .45, N = 54; Y3:
r = .61, N = 89; all ps < .01). Because the CFT measures
nonverbal reasoning only, the results can be considered sat-
isfactory. Correlations with marks for mathematics (rs for
Y1/Y2/Y3: .36/.52/.62, all ps < .001) and German (rs for
Y1/Y2/Y3: .31/.38/.49, all ps < .001) yield further evidence
for the tests concurrent validity, while correlations with
sports as a nonintellectual subject corroborate its discriminant
Table 3. Correlations between T(H)INK and Teacher (TR)
and Parent Ratings (PR)
Year 1 Year 2 Year 3
Domain TR PR TR PR TR PR
COG .54*** .27*** .56*** .39*** .60*** .41***
CREA .45*** .11* .48*** .25*** .53*** .18***
MOT .46*** .12* .47*** .28*** .51*** .24***
SOC .19*** .04ns .19*** .00ns .16*** .06ns
WORK .36*** .15** .31*** .11* .40*** .25***
N (listwise) 508 435 492 458 485 482
G&T program .47*** n/a .48*** n/a .44*** n/a
N 464 n/a 489 n/a 462 n/a
Notes. Teachers and parents rated childrens abilities in the following
domains: cognitive abilities (COG), creativity (CREA), motivation
(MOT), social behavior (SOC), and work attitude (WORK). Teacher
ratings were z-standardized classwise. G&T program = probability
with which a teacher would recommend a child for a gifted and tal-
ented program. n/a = not applicable. *p < .05, **p < .01, ***p < .001.
2012 Hogrefe Publishing
 T. G. Baudson & F. Preckel: T(H)INK: Development and Validation of an Intelligence Test
validity (rs for Y1/Y2/Y3: .05/.11/.06, p < .05 in Y2 only;
in Y1/Y2/Y3, N = 383/440/464 parents provided information
on all 3 subjects). Finally, predictive validity was established
by correlating cognitive abilities from Phase I to GPA from
Phase II (Y1/Y2: r = .48, N = 47; Y2/3: r = .56, N = 65;
Y3/Y4: r = .64, N = 64; all ps < .01).
Discussion
The present study yields first evidence for the validity of a
new intelligence test assessing Gf in primary school children
across figural, verbal, and numerical reasoning as central
content domains of intelligence. EFA and CFA results docu-
ment its factorial validity. A g-factor model was identified to
represent cognitive ability well, which is in line with promi-
nent models of intelligence structure (e.g., Jensen, 1998; Mc-
Grew, 2005). A multiple-group comparison across class lev-
els using CFA points to a comparable intelligence structure
from Y1 to Y3 as assessed by the new test T(H)INK. Internal
consistencies and retest reliabilities are acceptable to good.
The test is of medium difficulty for Y1Y3; yet for Y1, a
sufficiently high ceiling can be safely assumed, thus allowing
for differentiated gifted identification. Correlations with
teacher/parent ratings, marks, and a nonverbal Gf test support
the concurrent and divergent validity of the test: Those con-
structs that can be considered conceptually closest to Gf (e.g.,
teacher and parent ratings of cognitive ability; marks for
math) showed the highest correlations, whereas more remote
constructs (e.g., social behavior or marks for sports) showed
substantially lower correlations. The test predicted students
GPA 1 year later well.
T(H)INK seems to work well as a group intelligence test.
Whether its results are sufficient for individual case diagnos-
tics as well remains to be examined. This question has hardly
been addressed empirically, and, to our knowledge, no avail-
able group intelligence test provides differentiated norms for
individual vs. group testing. In practice, group-based norms
are often uncritically transferred to individual testing, al-
though the different situations are likely to elicit different
responses as well (e.g., Kaplan & Saccuzzo, 2008), especially
in younger children. Which type of test is suited also depends
on the goals of the diagnostics. For screening and research
purposes, group tests like T(H)INK are well suited; for more
complex issues such as placement decisions or interventions,
individual testing may be the better alternative.
An interesting secondary finding of our study was that
intelligence is related not only to teacher ratings of the childs
cognitive abilities, but almost as strongly to their ratings of
creativity, motivation, and work attitude. Although meta-an-
alytic studies suggest that, for instance, divergent thinking
and reasoning are only slightly related (r = .17; Kim, 2005),
intelligence and creativity (and, furthermore, motivation and
work attitude) seem to be conceptually closer to teachers than
such empirical studies suggest. Teachers are trained to assess
students achievement not their potential. Achievement, in
2012 Hogrefe Publishing
177
turn, results from all these abilities combined; a recent study
(Baudson, 2011) found that teachers implicit theories com-
prise an intellect/achievement dimension integrating sev-
eral of the characteristics described in the teacher rating
scales. Alternatively, it is conceivable that the impression a
teacher has of a childs intellectual ability impacts his/her
judgment of other characteristics as well (halo effect).
A more differentiated view was observed in parents,
where test results correlated highest with ratings of the
childs cognitive abilities. Parents may thus perceive their
children in a more differentiated way than teachers do, a
route possibly worth exploring. In this context, the low re-
liability of the social behavior scale (parent rating) needs
to be considered. Low internal consistency indicates a het-
erogeneous construct, which might explain the low corre-
lations as well. Despite higher reliability, however, teacher
ratings on this scale did not show much higher correlations.
In the light of the possibly more differentiated view parents
have on their children, it is conceivable that this is the case
for social behavior as well. The two aspects addressed by
the scale (behavior toward peers vs. adults) may thus be
perceived as more distinct by parents than by teachers.
Limitations
Internal consistencies are acceptable for a group intelligence
test but might be improved by replacing a few bad apples.
Test items had been selected based on pilot test results; it was
not foreseeable some would not pass the main test. However,
parceling the items by content groups for the reasons outlined
above (Little et al., 2002) buffered some of the shortcomings.
This issue will be addressed in future versions.
In addition, the validity of the teacher and parent rating
scales needs to be further investigated. While being of high
face validity and modeled on existing scales, correlations
of the newly developed scales with other questionnaires
and objective measures need to be assessed. However,
much evidence speaks for the overall validity of the test.
Thus, despite these minor shortcomings, T(H)INK can be
considered a valid indicator of childrens cognitive ability.
Future Research
Efforts to remedy the above shortcomings are being under-
taken. Recently, new and more difficult items have been
pilot tested in the context of a large-scale study in Baden-
Wrttemberg, proving that, even in its preliminary version,
T(H)INK can attract interest from the scientific communi-
ty. The aim of more difficult items is to avoid possible ceil-
ing effects beyond Y 1, especially because the test is to be
validated with gifted children. Clinical validation samples
may include children with learning disabilities or specific
learning issues (e.g., dyslexia).
Results for a Y 4 version are promising, too: A prelim-
inary version currently being extended has been success-
European Journal of Psychological Assessment 2013; Vol. 29(3):171181
178 T. G. Baudson & F. Preckel: T(H)INK: Development and Validation of an Intelligence Test
fully tested with a sample of N = 76, confirming the validity
results for Y1Y3 presented here.
Beyond the figural CFT tasks, correlations between
T(H)INK and specific numerical or verbal tests should be
examined. Although T(H)INK was developed to measure
Gf, tests assessing verbal abilities in particular might yield
evidence for its validity, considering the large impact of this
domain during primary school age in particular.
Further sampling variables (e.g., parent SES) should be
considered as well. Furthermore, the representative yet over-
all small quota of non-German native speakers is a possible
issue, which may be overcome by oversampling underrepre-
sented groups (e.g., Reynolds & Kamphaus, 2003). Consid-
ering the significant differences between German and non-
German native speakers, separate norms might make sense,
although the practice has not remained undisputed. In this
line, future studies should also pay attention to how long a
student has been learning German. For instance, in the BIS-
HB norming sample (Jger et al., 2006), group differences
were no longer observable only after students had learned
German for more than 8 years. This points out the importance
of attending to heterogeneity in groups that are less homoge-
neous than they appear at first sight.
A particularly important aim is to extend the data base
to provide norm data. T(H)INK has been successfully test-
ed only in Rhineland-Palatinate; a standardization sample
(this was not the goal of the present study), which is cur-
rently being planned, will include other German Federal
states and other German-speaking countries (Austria, Swit-
zerland). The standardization will be performed using
T(H)INK-2, the second version being based on the findings
presented here as well as on successfully piloted new items.
Up to now, data have only been analyzed according to clas-
sical test theory, calling for application of item response theory
models. Also, possibilities of test equating using shared items
to make test scores comparable across different versions are to
be explored, with a special focus on longitudinal research.
Considering the good retest reliabilities and the comparable
factor structure across age groups, we assume T(H)INK will
be a promising route for economic testing of childrens cogni-
tive abilities in cross-sectional and longitudinal research.
Acknowledgments
The present research was funded by the Rhineland-Palatin-
ate Ministry for Education, Science, Youth, and Culture,
Germany. All statements expressed in this paper are the
authors own and do not reflect the official opinions or pol-
icies of the authors host affiliations.
The authors wish to thank all participating students,
teachers, and parents as well as the following student as-
sistants for their invaluable help in administering the tests
and digitizing the data (in alphabetical order): Max Auers-
wald, Hannah Frh, Johannes Gorges, Sven Gorholt, Lilia
Held, Caroline Irmisch, Dorothee Katzenberger, Elisabeth
Meier, Sarah Muno, and Vibeke Nordstrand.
European Journal of Psychological Assessment 2013; Vol. 29(3):171181
References
Achenbach, T. M., & Rescorla, L. A. (2001). Manual for the ASEBA
School-Age Forms and Profiles. Burlington, VT: University of
Vermont.
Anastasi, A., & Urbina, S. (1997). Psychological testing (7th ed.).
Upper Saddle River, NJ: Prentice Hall.
Baudson, T. G. (2011). The (mis-) measure of childrens cognitive
abilities. Unpublished doctoral dissertation, University of Trier,
Trier.
Beauducel, A., & Kersting, M. (2002). Fluid and crystallized intel-
ligence and the Berlin Model of Intelligence Structure (BIS).
European Journal of Psychological Assessment, 18, 97112.
Carroll, J. B. (1993). Human cognitive abilities: A survey of factor-
analytic studies. New York: Cambridge University Press.
Cattell, R. B. (1963). Theory of fluid and crystallized intelligence: A
critical experiment. Journal of Educational Psychology, 54, 122.
Cattell, R. B., Wei, R. H., & Osterland, J. (1997). Grundintelligenz-
test Skala 1 CFT 1 [Test of Basic Intelligence, Scale 1] (5th ed.).
Gttingen: Hogrefe.
Cunningham, A. E., & Stanovich, K. E. (1997). Early reading acqui-
sition and its relation to reading experience and ability 10 years
later. Developmental Psychology, 33, 934945.
de Ayala, R. J. (2009). The theory and practice of item response
theory. New York: Guilford.
Flanagan, D. P., & Kaufman, A. S. (2009). Essentials of WISC-IV
assessment (2nd ed.). Hoboken, NJ: Wiley.
Ford, L., & Dahinten, V. S. (2005). Use of intelligence tests in the
assessment of preschoolers. In D. P. Flanagan & P. L. Harrison
(Eds.), Contemporary intellectual assessment: Theories, tests,
and issues (2nd ed., pp. 487503). New York: Guilford.
Gohm, C. L., Humphreys, L. G., & Yao, G. (1998). Underachieve-
ment among spatially gifted students. American Educational Re-
search Journal, 35, 515531.
Gottfredson, L. S. (1997). Mainstream science on intelligence: An
editorial with 52 signatories, history, and bibliography. Intelli-
gence, 24, 1323. [Reprinted from The Wall Street Journal, De-
cember 13, 1994.]
Grimm, H., & Schler, H. (1991). Heidelberger Sprachentwick-
lungstest H-S-E-T II. Handbuch [Heidelberg Test of Language
Development. Manual] (2nd ed.). Gttingen: Hogrefe.
Hair, J. F., Anderson, R. E., Tatham, R. L., & Black, W. C. (1998).
Multivariate data analysis (5th ed.). Upper Saddle River, NJ:
Prentice Hall.
Heller, K. A., & Geisler, H. J. (1983). KFT 13. Kognitiver Fhig-
keitstest fr 1. bis 3. Klassen [Cognitive Ability Test for Grades
1 to 3]. Gttingen: Hogrefe.
Heller, K. A., & Perleth, C. (2000). KFT 412+R. Kognitiver Fhig-
keitstest fr 4. bis 12. Klassen, Revision [Cognitive Ability Test
for Grades 4 to 12, revised]. Gttingen: Hogrefe.
Heller, K. A., & Perleth, C. (2007). MHBT-P. Mnchner Hochbega-
bungstestbatterie fr die Primarstufe [Munich Test Battery for
the assessment of giftedness Primary school level]. Gttingen:
Hogrefe.
Horn, J. L. (1991). Measurement of intellectual capabilities: A re-
view of theory. In K. S. McGrew, J. K. Werder, & R. W. Wood-
cock (Eds.), WJ-R technical manual (pp. 197232). Chicago, IL:
Riverside.
Jger, A. O. (1984). Intelligenzstrukturforschung: Konkurrierende
Modelle, neue Entwicklungen, Perspektiven [Research on intel-
2012 Hogrefe Publishing
 T. G. Baudson & F. Preckel: T(H)INK: Development and Validation of an Intelligence Test
ligence structure: Competing models, recent developments, per-
spectives]. Psychologische Rundschau, 35, 2135.
Jger, A. O., Holling, H., Preckel, F., Schulze, R., Vock, M., S,
H.-M., & Beauducel, A. (2006). Berliner Intelligenzstruktur-Test
fr Jugendliche: Begabungs- und Hochbegabungsdiagnostik
(BIS-HB) [Berlin Test of Intelligence Structure for youth: As-
sessment of giftedness and talent]. Gttingen: Hogrefe.
Jger, A. O., S, H.-M., & Beauducel, A. (1997). Berliner Intelli-
genzstruktur-Test. BIS-Test, Form 4 [Berlin Test of Intelligence
Structure, BIS test, form 4]. Gttingen: Hogrefe.
Jensen, A. R. (1998). The g factor: The science of mental ability.
Westport, CT: Praeger.
Johnsen, S. K., & Corn, A. L. (2001). SAGES2. Screening assess-
ment for gifted elementary and middle school students (2nd ed.).
Austin, TX: ProEd.
Kaplan, R. M., & Saccuzzo, D. P. (2008). Psychological testing: Prin-
ciples, applications, and issues. Stanford, CT: Cengage Learning.
Kim, K. H. (2005). Can only intelligent people be creative? A meta-
analysis. The Journal of Secondary Gifted Education, 16, 5766.
Krajewski, K., Kron, V., & Schneider, W. (2004). Entwicklungsver-
nderungen des strategischen Gedchtnisses beim bergang
vom Kindergarten in die Grundschule [Developmental changes
in strategic memory during the transition from kindergarten to
primary school]. Zeitschrift fr Entwicklungspsychologie und
Pdagogische Psychologie, 36, 4758.
Lavergne, C., & Vigneau, F. (1997). Response speed on aptitude
tests as an index of intellectual performance: A developmental per-
spective. Personality and Individual Differences, 23, 283290.
Little, T. D., Cunningham, W. A., Shahar, G., & Widaman, K. F.
(2002). To parcel or not to parcel: Exploring the question, weigh-
ing the merits. Structural Equation Modeling, 9, 151173.
Lockl, K., & Schneider, W. (2010). Kognitive Grundlagen: Denken,
Gedchtnis und Metakognition [Cognitive foundations: Think-
ing, memory, and metacognition]. In E. Walther, F. Preckel, & S.
Mecklenbruker (Eds.), Befragung von Kindern und Jugendli-
chen (pp. 7195). Gttingen: Hogrefe.
Lohman, D. F. (2002). Fluid intelligence, inductive reasoning, and
working memory: Where the theory of multiple intelligences falls
short. Paper presented at the annual meeting of the American
Educational Research Association in Seattle, WA.
Marshalek, B., Lohman, D. F., & Snow, R. E. (1983). The complex-
ity continuum in the radex and hierarchical models of intelli-
gence. Intelligence, 7, 107127.
McGrew, K. S. (2005). The Cattell-Horn-Carroll theory of cognitive
abilities. Past, present and future. In D. P. Flanagan, & P. L. Har-
rison (Eds.), Contemporary intellectual assessment: Theories,
tests, and issues (2nd ed., pp. 136181). New York: Guilford.
Neisser, U., Boodoo, G., Bouchard, T. J., Jr., Wade Boykin, A., Bro-
dy, N., Ceci, S. J., . . . Urbina, S. (1996). Intelligence: Knowns
and unknowns. American Psychologist, 51, 77101.
Ortiz, S. O., & Dynda, A. M. (2005). Use of intelligence tests with
culturally and linguistically diverse populations. In D. P. Flan-
agan & P. L. Harrison (Eds.), Contemporary intellectual assess-
ment (pp. 545556). New York: Guilford.
Petermann, F., & Petermann, U. (2011). WISC-IV. (German trans-
lation and adaptation of the WISC-IV by David Wechsler; 4th
ed.). Frankfurt a. M., Germany: Pearson.
Pfeiffer, S. I., & Jarosevich, T. (2003). Gifted Rating Scales School
form. San Antonio, TX: Harcourt Assessment.
Pregel, D., & Rickheit, G. (1987). Der Wortschatz im Grundschul-
alter [Primary school childrens vocabulary]. Hildesheim: Olms.
2012 Hogrefe Publishing
179
Rauer, W., & Schuck, K.-D. (2003). Fragebogen zur Erfassung
emotionaler und sozialer Schulerfahrungen von Grundschulkin-
dern dritter und vierter Klassen FEESS 34 [Assessment of
third- and fourth-grade pupils emotional and social school expe-
riences]. Gttingen: Hogrefe.
Rauer, W., & Schuck, K.-D. (2004). Fragebogen zur Erfassung
emotionaler und sozialer Schulerfahrungen von Grundschulkin-
dern erster und zweiter Klassen FEESS 12 [Assessment of
first- and second-grade pupils emotional and social school ex-
periences]. Gttingen: Hogrefe.
Reynolds, C. R., & Kamphaus, R. W. (2003). Handbook of psycho-
logical and educational assessment of children. New York: Guil-
ford.
Schweizer, K., & Koch, W. (2001). A revision of Cattells invest-
ment theory: Cognitive properties influencing learning. Learning
and Individual Differences, 13, 5782.
Schweizer, K., Troche, S., & Rammsayer, T. H. (2011). On the spe-
cial relationship between fluid and general intelligence: New ev-
idence obtained by considering the position effect. Personality
and Individual Differences, 50, 12491254.
Smedler, A.-C., & Trestad, B. (1996). Verbal intelligence: A key to
basic skills? Educational Studies, 22, 343356.
Spearman, C. (1904). General intelligence objectively determined
and measured. American Journal of Psychology, 15, 201293.
Statistisches Bundesamt/Federal Statistical Office. (2010). Statisti-
cal yearbook 2010 for the Federal Republic of Germany includ-
ing international tables. Wiesbaden: Author.
Statistisches Landesamt Rheinland-Pfalz/Statistical Office of
Rhineland-Palatinate. (2011). Statistische Berichte: Haushalte
und Familie 2010 [Statistical reports: Households and families].
Bad Ems: Author.
Wai, J., Lubinski, D., & Benbow, C. P. (2005). Creativity and occu-
pational accomplishments among intellectually precocious
youth: An age 13 to age 33 longitudinal study. Journal of Edu-
cational Psychology, 94, 785794. doi 10.1037/0022-0663.97.3.
484
Wai, J., Lubinski, D., & Benbow, C. P. (2009). Spatial ability for
STEM domains: Aligning over 50 years of cumulative psycho-
logical knowledge solidifies its importance. Journal of Educa-
tional Psychology, 101, 817835. doi 10.1037/a0016127
Weinert, S. (2010). Erfassung sprachlicher Fhigkeiten [Assessment
of verbal abilities]. In E. Walther, F. Preckel, & S. Mecklenbru-
ker (Eds.), Befragung von Kindern und Jugendlichen (pp. 227
262). Gttingen: Hogrefe.
Wei, R. H. (2008). Grundintelligenztest Skala 2 Revision (CFT
20-R) [Test of Basic Intelligence, Scale 2 Revised]. Gttingen:
Hogrefe.
Published online: May 11, 2012
Tanja Gabriele Baudson
Department of Psychology, Gifted Research and Education
Universittsring 15
54286 Trier
Germany
Tel. +49 651 201-4558 / Fax +49 651 201-4578
E-mail baudson@uni-trier.de
European Journal of Psychological Assessment 2013; Vol. 29(3):171181
180 T. G. Baudson & F. Preckel: T(H)INK: Development and Validation of an Intelligence Test

Appendix

Item characteristics and overview of overlapping items for year 13

Figural abilities
Grade level
1 (N = 533) 2 (N = 531) 3 (N = 565)
Item p SD ritc p SD ritc p SD ritc
PC1 .53 0.50 .20 .73 0.44 .20 .72 0.45 .21
PC2 .15 0.36 .12
PC3 .08 0.27 .14 .16 0.37 .22 .24 0.42 .25
PC4 .21 0.41 .14 .24 0.43 .06 .26 0.44 .11
PC5 .29 0.46 .31 .45 0.50 .35
PC 4 4 4
GSC1 .56 0.50 .31
GSC2 .33 0.47 .30 .60 0.49 .34 .64 0.48 .22
GSC3 .39 0.49 .26 .58 0.49 .39
GSC4 .26 0.44 .35 .51 0.50 .36 .50 0.50 .30
GSC5 .66 0.48 .36 .79 0.41 .37
GSC6 .11 0.32 .20
GSC 4 4 4
MA1 .55 0.50 .15
MA2 .53 0.50 .17 .76 0.42 .18
MA3 .54 0.50 .37 .71 0.45 .36
MA4 .50 0.50 .20 .54 0.50 .26 .62 0.49 .20
MA5 .48 0.50 .30
MA6 .24 0.43 .12 .21 0.41 .01
MA7 .58 0.49 .23
MA 4 4 4

Verbal abilities
Grade level
1 (N = 533) 2 (N = 531) 3 (N = 565)
Item p SD ritc p SD ritc p SD ritc
UG1 .85 0.36 .19
UG2 .77 0.42 .19 .86 0.35 .23
UG3 .63 0.48 .29 .72 0.45 .29 .77 0.42 .25
UG4 .43 0.50 .31 .60 0.49 .34 .70 0.46 .29
UG5 .24 0.43 .24 .42 0.49 .33 .35 0.48 .23
UG6 .11 0.31 .03
UG7 .89 0.31 .31
UG8 .84 0.37 .38
UG 6 3 6
VO1 .89 0.31 .11
VO2 .61 0.49 .08 .69 0.46 .14
VO3 .90 0.29 .18 .94 0.22 .24
VO4 .83 0.37 .22
VO5 .27 0.45 .01 .36 0.48 .18 .29 0.46 .16
VO6 .24 0.43 .06 .31 0.46 .14 .33 0.47 .10
VO7 .86 0.35 .18

European Journal of Psychological Assessment 2013; Vol. 29(3):171181 2012 Hogrefe Publishing
 T. G. Baudson & F. Preckel: T(H)INK: Development and Validation of an Intelligence Test 181

VO8 .44 0.50 .28 .55 0.50 .26

VO9 .55 0.50 .15 .64 0.48 .01
VO10 .40 0.49 .25
VO11 .24 0.43 .23 .38 0.48 .21
VO12 .51 0.50 .08
VO 6 9 6

Numerical abilities
Grade level
1 (N = 533) 2 (N = 531) 3 (N = 565)
Item p SD ritc p SD ritc p SD ritc
AP1 .40 0.49 .26 .74 0.44 .31 .76 0.42 .36
AP2 .15 0.36 .25 .31 0.46 .28 .28 0.45 .25
AP3 .30 0.46 .12
AP4 .19 0.38 .22 .49 0.50 .44
AP5 .63 0.48 .36
AP6 .76 0.42 .45
AP7 .70 0.46 .43
AP 4 4 4
NS1 .68 0.47 .37
NS2 .59 0.49 .41
NS3 .44 0.50 .11 .64 0.48 .21
NS4 .36 0.48 .37 .78 0.41 .44 .87 0.34 .27
NS5 .75 0.43 .42 .87 0.34 .32
NS6 .41 0.49 .34 .50 0.50 .29
NS7 .17 0.37 .12
NS 4 4 4
RM1 .73 0.45 .40 .85 0.35 .34 .90 0.30 .29
RM2 .80 0.40 .41
RM3 .50 0.50 .31 .70 0.46 .29 .66 0.48 .33
RM4 .75 0.44 .28
RM5 .57 0.50 .49 .56 0.50 .44
RM6 .60 0.49 .34 .61 0.49 .32
RM 4 4 4
Notes. p = item difficulty, SD = standard deviation, ritc = corrected item-total correlation. The sequence of the subtests corresponds to their
sequence in the test. Abbreviations of subtests: PC = Picture Concepts; GSC = Graphical Series Completion; MA = Matrices; UG = Under-
standing of Grammar; VO = Vocabulary; AP = Arithmetic Problems; NS = Number Series; RM = Routes on a Map (see text for detailed
descriptions). The sum sign () indicates the number of items a particular subtest includes in the respective version.

2012 Hogrefe Publishing European Journal of Psychological Assessment 2013; Vol. 29(3):171181
 P. T. van derEuropean
Heijden Journalof
et al.: Evaluation of theAssessment
Psychological MMPI-2 RC Scales
2013;
2012 Vol.with
Hogrefe the SCID
29(3):182188
Publishing

Original Article

The MMPI-2-Restructured Form and

the Standard MMPI-2 Clinical Scales
in Relation to DSM-IV
Paul T. van der Heijden1, Jos I. M. Egger2,3,4, Gina M. P. Rossi5,
Gitte Grundel6, and Jan J. L. Derksen7
1
Centre for Adolescent Psychiatry, Reinier van Arkel Mental Health Institute, s-Hertogenbosch, The
Netherlands, 2Behavioral Science Institute and Donders Institute for Brain, Cognition and Behavior,
Centre for Cognition, Radboud University Nijmegen, The Netherlands, 3Centre of Excellence for
Neuropsychiatry, Vincent van Gogh Institute for Psychiatry, Venray, The Netherlands, 4Pompe Institute
for Forensic Psychiatry, Forum GGZ, Nijmegen, The Netherlands, 5Faculty of Psychology and
Educational Sciences, Department of Clinical and Life Span Psychology, Vrije Universiteit Brussel,
Belgium, 6Centre for Adolescent Psychiatry, Reinier van Arkel Mental Health Institute,
s-Hertogenbosch, The Netherlands, 7Behavioral Science Institute, Radboud University Nijmegen, The
Netherlands

Abstract. In a Dutch sample of psychiatric outpatients (N = 94), we linked the Minnesota Multiphasic Personality Inventory2 (MMPI-2;
Butcher et al., 2001) Clinical scales and MMPI-2-Restructured Form (MMPI-2-RF; Ben-Porath & Tellegen, 2008) Higher-Order (H-O)
scales, Restructured Clinical (RC) scales and Specific Problem scales to the Structured Clinical Interview for the Diagnostic and Statis-
tical Manual of Mental Disorders (DSM-IV, 4th ed.; American Psychiatric Association, 1994) by First, Spitzer, Gibbon, and Williams
(1995). Overall, the H-O scales, RC scales, and Specific Problem scales display meaningful relationships to Axis I and Axis II disorders
conceptualized by the DSM-IV. In addition, the RC scales demonstrate a moderate improvement in validity over the standard Clinical
scales. Theoretical and clinical implications are considered.

Keywords: MMPI-2-RF, MMPI-2, international adaptation, predictive validity, DSM-IV

Within the field of clinical psychology, the Minnesota Mul-
tiphasic Personality Inventory-2 (MMPI-2; Butcher et al.,
2001) is worldwide the most widely used and researched
self-report measure of psychopathology and personality
characteristics (Camara, Nathan, & Puente, 2000). The
most recent update in the long history of the MMPI-2 is the
development of the MMPI-2 Restructured Form (MMPI-
2-RF; Ben-Porath & Tellegen, 2008). The MMPI-2-RF,
with its RC scales and three H-O scales based on the RC
scales as its core components, is shorter than the MMPI-2
(i.e., 338 items versus 567 items) and contains one new and
seven revised Validity scales, 23 Specific Problem Scales,
two Interest Scales, and five revised PSY-5 scales. The cur-
rent investigation presents a predictive validity study of the
H-O scales, RC scales, and Specific Problem scales with
respect to the Structured Clinical Interview for the Diag-
nostic and Statistical Manual of Mental Disorders (DSM-
IV, 4th ed.; American Psychiatric Association, 1994) Axis
I and Axis II disorders by First, Spitzer, Gibbon, and Wil-
liams (1995) and First, Spitzer, Gibbon, Williams, and Ben-
jamin (1997; e.g., SCID-I and SCID-II, respectively).
The RC scales were developed by Tellegen et al., (2003)
to solve some major interpretive difficulties of the MMPI
Clinical scales. Among these difficulties are the high inter-
correlations due to item overlap and common variance re-
lated to psychopathology or emotional distress in general
(but not to specific diagnoses) (Helmes & Reddon, 1993).
Also, the Clinical scale item pool contains a vast amount
of information regarding a broad range of psychopatholo-
gy. In developing the RC scales, Tellegen et al. (2003)
wanted to preserve this valuable information of the Clinical
scales and simultaneously improve their uniqueness and
discriminant validity.
Construction of the RC scales was performed in four
steps. First, the conceptualization and labeling of the com-
mon distress factor in the Clinical scales was guided by the

European Journal of Psychological Assessment 2013; Vol. 29(3):182188 2012 Hogrefe Publishing
DOI: 10.1027/1015-5759/a000140
 P. T. van der Heijden et al.: Evaluation of the MMPI-2 RC Scales with the SCID
two-factor model of mood and affect (Tellegen, Watson, &
Clark, 1999). This distress factor was labeled Demoraliza-
tion and was constructed through factor analysis of items
of Scale 2 (Depression) and Scale 7 (Psychasthenia), seeing
that these scales have the most demoralization-related
items. Then, these demoralization items were added to each
Clinical scale, and a combined item factor analysis was
performed, which for each Clinical scale yielded a demor-
alization factor and a distinctive factor representing the
core element of that particular Clinical scale. Items that
defined these core elements of each Clinical scale were
then correlated to the entire MMPI-2 item pool. Items from
the MMPI-2 item pool with unique and strong correlations
were added to these core elements to enhance internal con-
sistency and content representativeness of the RC scales.
Finally, the scales were further improved by correlating
them with relevant extra test criteria and by removing items
that displayed poor validity. No item overlap between any
RC scales was allowed. Nine RC scales were developed in
this manner; no RC scales were developed for Clinical
scale 5 and 0 as they are not measures of psychopathology.
The restructuring of RC3 (Cynicism) was a bit different: A
separate factor marked as navet was identified as the dis-
tinctive core component of Scale 3 (Hysteria) because
items reflecting somatic complaints were already assigned
to RC1 (Somatic Complaints). Items reflecting navet
were then scored inversely, and the final scale RC3 was
named Cynicism, which also contains items of the original
Scale 6 (Paranoia).
Specific Problem scales were developed to measure
distinctive components from the Clinical scales that were
not captured by the RC scales, in addition to some facets
of the RC scales that require separate assessment for
example, substance abuse, which is part of RC4 (Antiso-
cial Behavior; Ben-Porath & Tellegen, 2008).
Factor analysis of the RC scales revealed three under-
lying dimensions representing internalizing problems,
externalizing problems, and thought disorders. These
three dimensions were the primary markers of three
Higher Order (H-O) scales Emotional/Internalizing Dys-
function (EID) [based mainly on the items from Demor-
alization (RCd), Low Positive Emotions (RC2) and Dys-
functional Negative Emotions (RC7)], Behavioral/Exter-
nalizing Dysfunction (BXD) [based on items from
Antisocial Behavior (RC4) and Hypomanic Activation
(RC9)] and Thought Dysfunction (THD) [consisting of
items from the Ideas of Persecution (RC6) and Aberrant
Experiences (RC8) scales]. These dimensions parallel the
classical MMPI code types 2/7, 4/9, and 6/8 and are fairly
consistent with the higher-order factors found among
common mental disorders (Markon, 2010).
Simms, Casillas, Clark, Watson, and Doebbeling
(2005) compared the RC scales with the Clinical scales
with respect to the Schedule for Nonadaptive and Adap-
tive Personality (SNAP; Clark, 1993) and the SCID in a
sample of psychology clinic clients (N = 285) and a sam-
ple of military veterans (N = 567). They found that RCd,
2012 Hogrefe Publishing
183
RC1, RC2, and RC7 were related to current and lifetime
diagnosis of depression from the SCID, with a medium
effect size (i.e., .30 r < .50; Cohen, 1992). However,
depressive disorders were better predicted by Scale 2
than by RC2. Ratings of anxiety disorders were correlat-
ed most strongly with the restructured and traditional
versions of Scales 1, 2, 7, and 8 as well as with RCd.
Anxiety disorders were better predicted by Scale 7 than
by RC7. RC1 and Scale 1 displayed equal relations to
ratings of somatoform disorders, whereas substance
abuse disorders were better predicted by RC4 than by
Scale 4. Simms and colleagues did not consider the
MMPI-2-RF H-O scales, nor did they consider SCID-II
classifications for personality disorders (PDs).
The study by Simms and colleagues (2005) is the only
one so far to relate the RC scales to the SCID-I. To our
knowledge, no study has investigated the relationship be-
tween the RC scales and the SCID-II. This is remarkable
because the SCID-I is commonly seen as the golden
standard in the assessment of clinical diagnoses (Shear
et al., 2000). The SCID-I is therefore a valuable instru-
ment for exploring correlates of the RC scales, although
the RC scales were not explicitly designed to predict clin-
ical diagnoses (as the Clinical scales were). Arbisi, Sell-
bom, and Ben-Porath (2008) suggest the use of structured
diagnostic interviews in validation studies to allow for an
investigation of the diagnostic implications of elevated
scores on the RC scales. In addition, the SCID-II could
be considered an appropriate measure for the evaluation
of PDs because personality-disordered patients often lack
self-insight (Westen, 1997), and a diagnostic interview
provides the possibility to ask for elaborations or con-
crete examples for clarification.
Presently, there is paucity in research on the RC scales
outside the United States, while crosscultural compari-
sons are needed to investigate whether U. S. validation
studies on the RC scales may be generalized to other
(Western) countries. For all of these reasons, we decided
to investigate the relationships of the H-O scales, RC
scales, Specific Problem scales, and standard Clinical
scales with the SCID-I and SCID-II. Based on prior re-
search (e.g., Simms et al., 2005), we hypothesized that
overall the RC scales and Clinical scales would demon-
strate comparable convergent and divergent validity with
respect to the SCID-I. Based on research with the SNAP
(Simms et al., 2005), we hypothesized that the RC scales
would demonstrate better validity with respect to the
SCID-II than their Clinical scale counterparts. In addi-
tion, we hypothesized that, for several diagnostic criteria,
the Specific Problem scales would demonstrate strong
correlation [e.g., Stress/Worry (STW) and Anxiety
(AXY) for the anxiety-related disorders, Substance
Abuse (SUB) for substance use disorders and the Inter-
personal Scales for the schizoid, histrionic, narcissistic,
avoidant, and dependent PDs; Tellegen & Ben-Porath,
2008].
European Journal of Psychological Assessment 2013; Vol. 29(3):182188
184 P. T. van der Heijden et al.: Evaluation of the MMPI-2 RC Scales with the SCID
Method
Participants
All participants (N = 101) were recruited in an outpatient
mental health institute in The Netherlands. Data were col-
lected as part of the clinical routine, and the confidentiality
of participants identities was maintained throughout the
study process. Participants were excluded from the study
who had MMPI-2 profiles that met the following criteria:
Cannot Say raw scores 30, VRIN and TRIN T score 80
and L > 80 (Butcher et al., 2001); or MMPI-2-RF profiles
that met the following criteria: Cannot Say raw scores 15,
VRIN-r and TRIN-r T score 80, Fp-r 100 and L-r 80
(Ben-Porath & Tellegen, 2008). Seven participants were
excluded based on these criteria, one of whom demonstrat-
ed VRIN = 80 but no significant elevations on the F scale
or Clinical scales. The other six excluded participants dis-
played high F scores (103 F T score 120) and Fp scores
(102 Fp T score 117) and met criteria for several PDs
(M = 3, range 28) including borderline PD. The final sam-
ple consisted of 94 participants (46 men and 48 women)
with a mean age of 36.6 years at day of testing (SD = 8.99;
range 2157 years).
Measurements
The Dutch language version of the MMPI-2 was translated
and standardized for Belgium and the Netherlands by Derk-
sen, De Mey, Sloore, and Hellenbosch (1993). Detailed in-
formation about the translation process and the psychomet-
ric properties of the MMPI-2 and MMPI-2-RF scales in the
Dutch normative sample is provided by Derksen et al.
(1993) and Van der Heijden, Egger, and Derksen (2008,
2010). No commercial version of the Dutch MMPI-2-RF
is available yet, and the Dutch language version of the
MMPI-2-RF is available only for scientific research and
validation studies. We used uncorrected raw scores in the
analyses.
The SCID-I (First, Spitzer, Gibbon, & Williams, 1995;
Groenestijn, Akkerhuis, Kupka, Schneider, & Nolen, 1998)
and SCID-II (First, Spitzer, Gibbon, Williams, & Benja-
min, 1997; Weertman, Arntz, & Kerkhofs, 2000) are wide-
ly used structured clinical interviews to measure clinical
syndromes (CSs) and PDs based on criteria from the DSM-
IV. Three interviewers accounted for all the interviews in
the current research. These interviewers were adequately
trained and experienced in administering the interview to
guarantee good interrater reliability. When interviewers are
sufficiently trained, the SCID-I and SCID-II showed excel-
lent interrater reliability and test-retest reliabilities in di-
verse studies (Weertman, Arntz, Dreessen, Van Velzen, &
Vertommen, 2003; Zanarini et al., 2000).
In the analyses, we reduced all SCID-I classifications to
four broad disorders in line with Simms and colleagues
European Journal of Psychological Assessment 2013; Vol. 29(3):182188
(2005). Participants were classified with a depressive dis-
order if they met criteria for one or more depression-related
disorders (except bipolar disorders), an anxiety disorder if
they met criteria for any of the anxiety disorders, and with
a somatoform disorder if they met criteria for any of the
somatoform disorders or for fibromyalgia, irritable bowel
syndrome or chronic fatigue syndrome (Wesley & White,
2004). A category substance use disorder (SUD) was made
for people with alcohol or drug abuse and/or dependency.
We did not create a category for psychotic disorders as the
frequency of this type of disorders was too small in our
outpatient sample. Only current diagnoses were used in the
analyses. For the PDs we calculated number of criteria met,
instead of diagnoses.
Statistical Analyses
First, we calculated that a minimum of 85 participants was
needed to realize a power of .80, based on = .05 and given
an expected medium effect size (Cohen, 1992; Simms et
al., 2005). Reliability coefficients and intercorrelations
were calculated for the MMPI-2 and MMPI-2-RF scales.
In addition, we investigated the diagnostic overlap in the
current sample. Then, zero order correlations were calcu-
lated for the H-O scales, RC scales, Specific Problem
scales, and standard Clinical scales with the SCID-I and II
classifications. We calculated Fishers Z to test the differ-
ences in correlations between the Clinical scales and the
RC scales with the SCID-I and II; Cohens q was calculated
as effect size for these comparisons. Because of the large
number of comparisons and Z-tests, we set the p-value for
significance conservatively to .01 for all the analyses.
Results
Detailed information about reliability statistics of the H-O
scales, RC scales, Specific Problem scales and Clinical
scales, intercorrelations among RC scales and Clinical
scales, and differences in average T-scores for RC scales
and corresponding Clinical scales are available on request
from the first author. In terms of DSM-IV, there was con-
siderable diagnostic overlap in the current sample. For ex-
ample, the mean number of current Axis I disorders was
2.88 (SD = 2.45). Of the 50 participants with depression-
related disorders, 35 (70%) also met criteria for at least one
of the anxiety disorders. On the other hand, of the 54 pa-
tients with anxiety-related disorders, 35 (65%) patients also
met criteria for one of the depression-related disorders. The
mean number of PDs in the current sample was 1.54 (SD =
1.68; range 07). The mean number of PDs for participants
who met criteria of at least one PD (N = 58) was 2.5 (range
17). The avoidant and borderline PD had the highest fre-
quencies in the sample (i.e., 26 and 23, respectively).
Table 1 presents the bivariate correlations for H-O scales,
2012 Hogrefe Publishing
 P. T. van der Heijden et al.: Evaluation of the MMPI-2 RC Scales with the SCID 185

Table 1. Correlations for HO scales, RC scales, and clinical scales with clinical syndromes and personality disorders
(N = 94)
MMPI-2/MMPI-2-RF SCID-I classifications SCID-II classifications for personality disorders
HO scales, RC scales and Clinical Depr Anx Som SUD Avoid DepenObs Pass Depr Par Szt Szd His Nar Bor Asp
scales
Emotional/internalizing dysfunction .51 .44 .27 .04 .54 .37 .31 .24 .66 .37 .28 .10 .07 .00 .44 .02
(EID)
Thought dysfunction (THD) .05 .10 .13 .04 .11 .18 .09 .21 .29 .44 .35 .08 .02 .15 .36 .16
Behavioral/externalizing dysfunction .10 .01 .12 .21 .10 .07 .06 .27 .04 .31 .13 .11 .02 .16 .37 .65
(BXD)
Demoralization (RCd) .50 .41 .23 .03 .54 .38 .25 .22 .62 .35 .29 .07 .13 .03 .43 .02
Somatic complaints (RC1) .13 .25 .33 .06 .21 .29 .14 .19 .41 .43 .19 .04 .19 .03 .38 .06
Scale 1: Hypochondriasis (Hs) .22 .32 .35 .04 .25 .33 .14 .21 .46 .49 .22 .09 .17 .01 .45 .08
Low positive emotions (RC2) .47 .30 .16 .01 .44 .26 .17 .04 .40 .10 .01 .18 .05 .04 .16 .16
Scale 2: Depression (D) .52 .45 .29 .02 .44 .33 .14 .16 .54 .27 .11 .01 .18 .03 .35 .17
Cynicism (RC3) .08 .20 .14 .03 .08 .01 .16 .33 .23 .39 .21 .18 .15 .22 .29 .20
Scale 3: Hysteria (Hy) .29 .19 .28 .11 .03 .23 .08 .02 .24 .16 .04 .21 .26 .05 .28 .07
Antisocial behavior (RC4) .07 .07 .21 .30 .02 .07 .09 .31 .16 .41 .22 .06 .00 .01 .45 .58
Scale 4: Psychopathic deviate (Pd) .35 .30 .29 .11 .11 .17 .14 .40 .45 .47 .22 .09 .08 .04 .46 .19
Ideas of persecution (RC6) .02 .16 .18 .10 .09 .11 .03 .24 .28 .44 .27 .13 .01 .18 .25 .03
Scale 6: Paranoia (Pa) .33 .25 .09 .16 .19 .22 .14 .23 .42 .41 .19 .15 .04 .25 .32 .03
Dysfunctional negative emotions .24 .37 .23 .04 .41 .36 .19 .27 .56 .51 .31 .03 .11 .06 .52 .16
(RC7)
Scale 7: Psychasthenia (Pt) .40 .37 .27 .03 .44 .37 .26 .25 .61 .47 .29 .01 .17 .01 .54 .12
Aberrant experiences (RC8) .01 .07 .17 .06 .13 .27 .14 .17 .31 .40 .37 .01 .07 .08 .50 .21
Schizophrenia (Sc) .30 .32 .27 .01 .36 .34 .27 .33 .53 .53 .35 .07 .17 .03 .54 .18
Hypomanic activation (RC9) .17 .03 .01 .06 .19 .02 .23 .36 .02 .41 .18 .08 .16 .20 .40 .47
Scale 9: Hypomania (Ma) .10 .10 .06 .00 .23 .00 .10 .28 .03 .38 .14 .02 .19 .17 .40 .32
Notes. Depr = Depression; Anx = Anxiety; Som = Somatoform; SUD = Substance Use disorder; Avoid = Avoidant PD; Depen = Dependent
PD; Obs = Obsessive-Compulsive PD; Pass = Passive Aggressive PD; Depr = Depressive PD; Par = Paranoid PD; Szt = Schizotypal PD; Szd =
Schizoid PD; His = Histrionic PD; Nar = Narcissistic PD; Bor = Borderline PD; Asp = Antisocial PD (Personality Disorder). Correlations with
a strong effect size (i.e., r = .50) are in bold, correlations with a medium effect size are in italics. rs = .27 are significant at p < .05.

RC scales, and corresponding Clinical scales with SCID-I
and II classifications. Table 2 provides correlations for the
Specific Problem scales with the same criteria. As Table 1
indicates, the RC scales demonstrate convergent and discrim-
inant validity in relation to the SCID-I and II classifications:
RCd (Demoralization) has a strong correlation with depres-
sive disorders and a medium correlation with anxiety-related
disorders. RC2 (Low Positive Emotions) demonstrates the
strongest relationship with depressive disorders, whereas
RC7 (Dysfunctional Negative Emotions) demonstrates its
strongest relationship with anxiety-related disorders. RC1
(Somatic Complaints) displays a medium relationship with
somatoform disorder (r = .33), and RC4 (Antisocial Behav-
ior) is clearly related to SUD (i.e., medium effect, r = .30).
The RC scales also exemplify meaningful relationships with
the PDs. For example, RC6 (Ideas of Persecution) shows a
unique and medium correlation with the paranoid PD, and
RC4 (Antisocial Behavior) demonstrates a strong correlation
with the antisocial PD.
Concerning the Specific Problem scales, Anxiety
(AXY) demonstrated a small correlation (r = .24) with the
anxiety-related disorders, whereas Stress/Worry (STW)
displayed a medium correlation (r = .43) with these disor-
ders. As hypothesized, Substance Abuse (SUB) showed a
medium correlation with SUD. Juvenile Conduct Problems
(JCP) demonstrated a strong relationship with antisocial
PD and Suicidal/Death ideation with borderline PD. Shy-
ness (SHY) was strongly correlated with the avoidant PD
and Activation (ACT; heightened excitation and energy
level) with borderline PD. Disaffiliativeness (DSF; dislik-
ing people and being around them) showed medium corre-
lations with depressive, schizoid and avoidant PD. The In-
terpersonal Scales were not related to the histrionic and
narcissistic PDs.
We did compare correlations for RC scales and Clinical
scales with DSM-IV Axis I and Axis II disorders. Here, we
found some significant differences in correlations, but none
of these differences reached a large effect size (i.e., Cohens
q = .50). For example, RC4 (Antisocial Behavior) dis-
played a stronger correlation with antisocial PD than Scale
4 did (i.e., Z = 3.10; p < .01; Cohens q = .46). Similarly,
RC3 (Cynicism) demonstrates stronger relationships with

2012 Hogrefe Publishing European Journal of Psychological Assessment 2013; Vol. 29(3):182188
186 P. T. van der Heijden et al.: Evaluation of the MMPI-2 RC Scales with the SCID

Table 2. Correlations for Specific Problem scales with clinical syndromes and personality disorders (N = 94)
SCID-I classifications SCID-II classifications for personality disorders
MMPI-2-RF Specific Problem scales Depr Anx Som SUD Avoid Depen Obs Pass Depr Par Szt Szd His Nar Bor Asp

Somatic/Cognitive and Internalizing Scales

Malaise (MLS) .42 .33 .27 .06 .25 .31 .18 .22 .52 .44 .16 .04 .15 .05 .50 .12

Gastrointestinal complaints (GIC) .16 .27 .41 .10 .25 .24 .16 .15 .37 .30 .21 .07 .04 .04 .32 .17

Head pain complaints (HPC) .15 .21 .26 .02 .13 .31 .08 .09 .33 .31 .22 .07 .19 .03 .32 .06

Neurological complaints (NUC) .01 .15 .19 .06 .10 .15 .06 .24 .27 .38 .07 .03 .20 .12 .30 .03

Cognitive complaints (COG) .34 .33 .31 .06 .35 .38 .28 .21 .48 .32 .24 .07 .20 .01 .45 .11

Suicidal/death ideation (SUI) .23 .07 .05 .09 .03 .23 .24 .18 .16 .29 .29 .01 .06 .11 .50 .21

Helplessness/hopelessness (HLP) .41 .26 .17 .03 .46 .35 .22 .21 .53 .29 .30 .13 .03 .07 .35 .02

Self-doubt (SFD) .29 .31 .21 .03 .48 .28 .20 .08 .57 .21 .27 .00 .00 .14 .29 .08

Inefficacy (NFC) .19 .26 .18 .08 .47 .36 .25 .19 .45 .37 .33 .04 .05 .08 .41 .17

Stress/worry (STW) .38 .43 .24 .01 .39 .32 .23 .28 .60 .41 .25 .08 .19 .07 .49 .11

Anxiety (AXY) .15 .24 .25 .06 .22 .26 .01 .19 .34 .40 .26 .09 .25 .02 .47 .17

Anger proneness (ANP) .18 .13 .21 .05 .01 .11 .35 .41 .36 .37 .07 .01 .15 .04 .42 .22

Behavior-restricting fears (BRF) .06 .17 .15 .04 .19 .26 .15 .10 .26 .22 .23 .01 .21 .02 .31 .12

Multiple specific fears (MSF) .001 .20 .10 .06 .19 .17 .09 .02 .21 .20 .18 .14 .04 .24 .21 .12

Externalizing and Interpersonal Scales

schizoid PD than Scale 3 (Z = 2.67; p < .01; Cohens q =
.40). We also compared our correlation table (RC scales
with Axis I classifications) with correlations found by
Simms and colleagues (2005) in a sample with 564 military
veterans (88% male). None of 36 comparisons demonstrat-
ed a significant difference (p < .01).
Discussion
As hypothesized, the RC scales demonstrate convergent
and discriminant validity with respect to the SCID-I and II.
For example, our findings concerning the relationships be-
tween RC2 and RC7 and the DSM-IV measures for anxiety
and depression were in line with Tellegens model (1985).
According to this model, low positive emotions (reflected
by RC2) are distinctive for depressive disorders, whereas
high negative affectivity (i.e., RC7) is related to anxiety.
The common variance that is often seen in depression, and
anxiety can be considered as a higher order factor concep-
tualized as demoralization (Tellegen et al., 2003). In our
sample, RC2 (Low Positive Emotions) had a stronger rela-
tionship with depression than RC7 (Dysfunctional Nega-
tive Emotions), whereas for anxiety the opposite was true.
RCd (Demoralization) has equal correlations with both
anxiety and depression.
RC3 (Cynicism) displayed medium correlations with the
paranoid and passive-aggressive PDs. High scores on RC3
reflect cynical beliefs about others, believing others look
out only for their own interests, and being distrustful of
others, which are indeed core beliefs of the paranoid and
passive aggressive PDs (Millon & Davis, 1996). RC9 (Hy-
pomanic Activation) exemplifies medium-size correlations
with the paranoid, borderline, antisocial, and passive ag-
gressive PDs. Elevated scores on RC9 reflect aggression,
poor impulse control, mood instability, sensation seeking,
and other forms of undercontrolled behavior (Ben-Porath
& Tellegen, 2008). However, high scores on RC9 are also
associated with narcissistic personality features, although
the correlation of RC9 with the narcissistic PD is small in
the current sample. This may be due to the fact that the

European Journal of Psychological Assessment 2013; Vol. 29(3):182188 2012 Hogrefe Publishing
 P. T. van der Heijden et al.: Evaluation of the MMPI-2 RC Scales with the SCID
criteria for the narcissistic PD from the SCID-II are very
obvious and transparent and therefore not easy to admit in
an interview. In our sample only 4 persons met the criteria
for a narcissistic PD. Simms et al. (2005) found a strong
correlation for RC9 with the narcissistic PD in the SNAP.
The Specific Problem scales showed meaningful rela-
tionships with the DSM-IV classifications. The internaliz-
ing scales were related primarily to the internalizing disor-
ders, whereas the externalizing scales were related primar-
ily to SUD and antisocial PD. Contrary to our hypotheses,
we didnt find significant correlations for the Interpersonal
scales and the narcissistic and histrionic PDs, probably be-
cause of low frequencies of these disorders in the current
sample. Anxiety (AXY; pervasive anxiety, frights, frequent
nightmares) showed only a small correlation with anxiety-
related disorders, whereas it showed medium correlations
with depressive, borderline, and paranoid PD. A possible
explanation might be that the anxiety-related disorders in
the current sample is a broad dimension, whereas the AXY
scale is a narrow defined construct.
Our hypotheses for comparisons of both sets of scales
with the DSM-IV disorders were only partially con-
firmed. RC scales measuring externalizing behaviors and
psychotic symptoms demonstrate some improvement in
convergent and discriminant validity over their Clinical
scale counterparts. For example, RC4, RC6, and RC8 are
less saturated with depression-related variance and there-
fore display better divergent validity with respect to de-
pression-related disorders. However, for RC scales meas-
uring internalizing pathology this is not the case; these
RC scales display no improvement compared to the Clin-
ical scales with respect to internalizing disorders. That
being said, the RC scales definitely do not perform worse
than their Clinical scale counterparts with respect to the
SCID, although these scales were not designed to predict
these diagnoses. Moreover, the RC scales have fewer
items and thus could be seen as more efficient. Overall,
we agree with Simms and colleagues (2005), who stated
that the RC scales appear to represent a modest psycho-
metric improvement over the standard clinical scales
(p. 357). Interestingly, our findings were very much in
correspondence with findings by Simms and colleagues,
offering a first indication for the generalizability of U.S.
validation studies to the Dutch language version of the
MMPI-2.
Potential limitations of the current study must also be
considered. First of all, we had to reduce the DSM-IV
Axis I disorder to four broad categories (without a cate-
gory with psychotic disorders) because of the relative
small sample and the low frequency of some severe psy-
chiatric disorders. In addition, the SCID classifications
were not mutually exclusive; there is considerable diag-
nostic overlap in the current sample. This limits our abil-
ity to draw inferences about discriminant validity. On the
other hand, one should have access to an overwhelming
big sample with an extensive variety of problems and dis-
orders to capture all 374 classifications in DSM-IV
2012 Hogrefe Publishing
187
(APA, 1994). Also, comorbidity is widespread among
common mental disorders (Krueger, 1999). Moreover,
the categories we made are the most common ones in
mental health practice, and we calculated that our sample
should be big enough to reach a power of .80. Further-
more, we included patients with a variety of Axis I and
Axis II disorders, which reduces the possibility to restrict
range effects. Another limitation might be that we did not
consider some of the MMPI-2 content and PSY-5 scales
that show considerable conceptual overlap with the RC
scales (Nichols, 2006; Van der Heijden et al., 2008). That
would make a fairer comparison if one wants to evaluate
possible redundancy of the RC scales (Nichols, 2006).
Yet, these particular scales do not appear in the MMPI-
2-RF so the possible redundancy is not an issue for cli-
nicians who solely use the new MMPI-2-RF scales.
An interesting direction for future research could be
the connections of the MMPI-2-RF conceptualizations
with current proposals for the DSM-5 (personality) dis-
orders. Furthermore, the treatment utility (Nelson-Gray,
2003) of the MMPI-2-RF scales should be considered in
future research. Ultimately, clinicians are most interested
in the way these scales can add in the prediction of rele-
vant issues such as treatment response and treatment out-
come.
As mentioned above, there is no commercial version
of the MMPI-2-RF available yet in the Netherlands, but
publication is planned. In the United States, both MMPI-
2 versions are available at the same time. So which one
to chose? As far as the current research is concerned, the
H-O scales, RC scales, and Specific Problem scales show
meaningful correlations with the SCID-I and II classifi-
cations, although they were not explicitly designed to
predict these diagnoses. Moreover, the RC scales show a
moderate improvement compared to the more compre-
hensive Clinical scales. The RC scales show the same
reliability statistics, less intercorrelations (Tellegen et al.,
2003; Van der Heijden et al., 2008), and slightly better
discriminant validity than the Clinical scales. In addition,
the MMPI-2-RF has 40% less items. The RC scales pro-
vide an efficient and homogeneous measure, whereas the
Clinical scales may show more syndrome fidelity and can
rest on an enormous empirical tradition. However, new
research on the MMPI-2 is becoming more and more re-
lated to the RC scales and H-O scales instead of the Clin-
ical scales. So the development of an equally impressive
and extensive research base for the MMPI-2-RF scales to
draw upon during test interpretation seems just a matter
of time. That makes the MMPI-2-RF more promising for
the future.
Acknowledgments
We thank Ben Hogerheijde, Ilvy Mayen, Marcel van de
Langenberg, and Shakira Cornelissen for their participation
in this research project.
European Journal of Psychological Assessment 2013; Vol. 29(3):182188
188 P. T. van der Heijden et al.: Evaluation of the MMPI-2 RC Scales with the SCID
References
American Psychiatric Association. (1994). Diagnostic and statis-
tical manual of mental disorders (4th ed.). Washington, DC:
American Psychiatric Association.
Arbisi, P. A., Sellbom, M., & Ben-Porath, Y. S. (2008). Empirical
correlates of the MMPI-2 restructured clinical (RC) scales in
psychiatric inpatients. Journal of Personality Assessment, 90,
122128.
Ben-Porath, Y. S., & Tellegen, A. (2008). MMPI-2-RF users
guide for reports. Minneapolis, MN: University of Minnesota
Press.
Butcher, J. N., Graham, J. R., Ben-Porath, Y. S., Tellegen, A.,
Dahlstrom, W. G., & Kaemmer, B. (2001). Minnesota Multi-
phasic Personality Inventory-2: Manual for administration
and scoring (revised ed.). Minneapolis, MN: University of
Minnesota Press.
Camara, W. J., Nathan, J. S., & Puente, A. E. (2000). Psycholog-
ical test usage: Implications in professional psychology. Pro-
fessional Psychology: Research and Practice, 31, 141154.
Clark, L. A. (1993). Schedule for nonadaptive and adaptive per-
sonality: Manual for administration, scoring, and interpreta-
tion. Minneapolis, MN: University of Minnesota Press.
Cohen, J. (1992). Quantitative methods in Psychology. A Power
Primer. Psychological Bulletin, 112, 155159.
Derksen, J., De Mey, H., Sloore, H., & Hellenbosch, G. (1993).
MMPI-2: Handleiding bij afname, scoring en interpretative
[Minnesota Multiphasic Personality Inventory-2: MMPI-2:
Manual for administration, scoring, and interpretation]. Nijme-
gen, The Netherlands: PEN Test Publishers.
First, M. B., Spitzer, R. L., Gibbon, M., & Williams, G. B. W. (1995).
Structured clinical interview for DSM-IV axis I disorders pa-
tient edition (SCID-I/P), Version 2.0. New York: Biometrics Re-
search Department, New York State Psychiatric Institute.
First, M. B., Spitzer, R. L., Gibbon, M., Williams, G. B. W., &
Benjamin, L. (1997). Structured clinical interview for DSM-IV
axis II personality disorders (SCID-II). Washington, DC:
American Psychiatric Press.
Groenestijn, M. A. C., van, Akkerhuis, G. W., Kupka, R. W.,
Schneider, N., & Nolen, W. A. (1998). SCID I Gestructureerd
Klinisch Interview voor DSM-IV As-I Stoornissen [Structured
clinical interview for DSM-IV Axis I disorders]. Lisse: Swets
& Zeitlinger.
Helmes, E., & Reddon, J. R. (1993). A perspective on develop-
ments in assessing psychopathology: A critical review of the
MMPI and MMPI-2. Psychological Bulletin, 113, 453471.
Krueger, R. F. (1999). The structure of common mental disorders.
Archives of General Psychiatry, 56, 921926
Markon, K. E. (2010). Modeling psychopathology structure: A
symptom-level analysis of Axis I and Axis II disorders. Psy-
chological Medicine, 40, 273288.
Millon, T., & Davis, R. D. (1996). Disorders of personality: DSM-
IV and beyond (2nd ed.). New York: Wiley.
Nelson-Gray, R. O. (2003). Treatment utility of psychological as-
sessment. Psychological Assessment, 15, 521531.
Nichols, D. S. (2006). The trials of separating bath water from
baby. A review and critique of the MMPI-2 Restructured Clin-
ical Scales. Journal of Personality Assessment, 87, 121138.
Shear, M. K., Greeno, C., Kang, J., Ludewig, D., Frank, E.,
Swartz, H. A., & Hanekamp, M. (2000). Diagnosis of nonpsy-
European Journal of Psychological Assessment 2013; Vol. 29(3):182188
chotic patients in community clinics. American Journal of Psy-
chiatry, 157, 581587.
Simms, L. J., Casillas, A., Clark, L. A., Watson, D., & Doebbeling,
B. N. (2005). Psychometric evaluation of the restructured clin-
ical scales of the MMPI-2. Psychological Assessment, 17,
345358.
Tellegen, A. (1985). Structures of mood and personality and their
relevance to assessing anxiety, with an emphasis on self-re-
port. In A. H. Tuma & J. D. Maser (Eds.), Anxiety and the anx-
iety disorders (pp. 681706). Hillsdale, NJ: Erlbaum.
Tellegen, A., & Ben-Porath, Y. S. (2008). MMPI-2-RF Technical
manual. Minneapolis, MN: University of Minnesota Press.
Tellegen, A., Ben-Porath, Y. S., McNulty, J. L., Arbisi, P., Gra-
ham, J. R., & Kaemmer, B. (2003). MMPI-2 restructured clin-
ical (RC) scales: Development, validation and interpretation.
Minneapolis, MN: University of Minnesota Press.
Tellegen, A., Watson, D., & Clark, L. A. (1999). On the dimen-
sional and hierarchical structure of affect. Psychological Sci-
ence, 10, 297303.
Van der Heijden, P. T., Egger, J. I. M., & Derksen, J. J. L. (2008).
Psychometric evaluation of the MMPI-2 restructured clinical
scales in two Dutch samples. Journal of Personality Assess-
ment, 90, 456464.
Van der Heijden, P. T., Egger, J. I. M., & Derksen, J. J. L. (2010).
Comparability of scores on the MMPI-2-RF scales generated
with the MMPI-2 and MMPI-2-RF Booklets. Journal of Per-
sonality Assessment, 93, 254259.
Weertman, A., Arntz, A., Dreessen, L., Van Velzen, C., & Ver-
tommen, S. (2003). Short-interval test-retest interrater reliabil-
ity of the Dutch version of the Structured Clinical Interview
for DSM-IV Personality Disorders (SCID-II). Journal of Per-
sonality Disorders, 17, 562567.
Weertman, A., Arntz, A., & Kerkhofs, M. (2000). Handleiding
Gestructureerd Klinisch Interview voor DSM-IV As-II Per-
soonlijkheidsstoornissen [Manual structured clinical interview
for DSM personality disorders]. Lisse, The Netherlands: Swets
Test Publishers.
Wessely, S., & White, P. D. (2004). There is only one functional
somatic syndrome. British Journal of Psychiatry, 185, 9596
Westen, D. (1997). Divergences between clinical and research
methods for assessing personality disorders: Implications for
research and the evolution of Axis II. American Journal of
Psychiatry, 154, 895903.
Zanarini, M. C., Skodol, A. E., Bender, D., Dolan, R., Sanislow,
C., & Schaefer, E. (2000). The Collaborative Longitudinal Per-
sonality Disorders Study: Reliability of axis I and II diagnoses.
Journal of Personality Disorders, 14, 291299.
Published online: May 11, 2012
Paul T. van der Heijden
Reinier van Arkel Groep
P.O. Box 70058
5201 DZ s-Hertogenbosch
The Netherlands
Tel. +31 73 658-6651
Fax +31 73 658-6648
E-mail p.t.vander.heijden@rvagroep.nl
2012 Hogrefe Publishing
 T. Giovazolias
European Journalof Psychological et al.:
Assessment The Vol.
2013;
2012 EDE -Q Publishing
Hogrefe in Greece
29(3):189196

Original Article

The Factor Structure and

Psychometric Properties of the Greek
Version of the Eating Disorders
Examination Questionnaire (EDE-Q)
Theodoros Giovazolias1, Ioannis Tsaousis1, and Christina Vallianatou2
1
Department of Psychology, School of Social Sciences, University of Crete, Crete, Greece
2
Hellenic Air Force Academy, Athens, Greece

Abstract. This study investigates the latent structure and the psychometric characteristics of the attitudinal subscales of the Greek
translation of the EDE-Q (G-EDE-Q) using two samples. The first sample consisted of 500 university female students and tested the
latent structure of the G-EDE-Q. The second sample consisted of 164 female psychology students and examined the internal consistency
as well as the concurrent, convergent, and discriminant validity of the G-EDE-Q. The results from confirmatory factor analysis (CFA)
showed that a three-factor solution fit the data better than the other models examined, providing evidence to contradict the originally
proposed structure of the instrument. The results support both, the internal consistency as well as the concurrent, convergent, and dis-
criminant validity of the G-EDE-Q global scale and its subscales. The discussion focuses on the critical review of previous statistical
procedures employed and the clinical and research implications of the present findings.

Keywords: EDE-Q, CFA, eating disorders, factorial validity, Greece

Eating disorders (EDs), a growing field of research and
practice in a number of countries across the world, are het-
erogeneous in manifestation as well as in background.
Therefore, the careful psychometric assessment thereof is
essential to diagnosis, case formulation, and treatment
planning. It is generally recognized that the best method of
assessing and diagnosing EDs is the clinical interview con-
ducted by trained clinicians. The Eating Disorders Exami-
nation (EDE; Cooper & Fairburn, 1987), a semistructured,
investigator-based clinical interview, is widely considered
the method of choice for that purpose (Wilson, 1993), al-
though it has certain disadvantages (i.e., is time consuming,
requires special training, and is not cost effective). For
these reasons, self-report questionnaires are often used as
an alternative method of evaluation.
Several self-report instruments are utilized for the iden-
tification of early signs of eating disturbances and likewise
the tendency to develop eating psychopathology. One of
the most widely used is the Eating Disorders Examination-
Questionnaire (EDE-Q; Fairburn & Beglin, 1994), derived
from the EDE interview. The EDE-Q provides frequency
data on key behavioral aspects of eating disorder psycho-
pathology and includes four attitudinal subscales measur-
ing dietary restraint, eating concerns, shape concerns and
weight concerns. Additionally, it yields a global score
which is the average of the four subscale scores. Previous
research demonstrated the EDE-Q global scale and sub-
scales have sound psychometric properties (Hrabosky et
al., 2008; Luce & Crowther, 1999; Peterson et al., 2007).
The increase in the prevalence of eating disorders
across cultures points to the need for thorough investiga-
tion in the different cultural contexts. A growing body of
research indicates that the nature of eating disorders may
be culturally bound. It has been argued that different eth-
nic populations vary in their attitudes toward eating and
body disturbances (Diaz, Mainous, & Pope, 2007). In
Mediterranean cultures, for example, the changing role
of women and the transition to a less patriarchal society
may have contributed to a differing manifestation of par-
ticular aspects of eating disorders. Therefore, using mea-
sures developed in Anglo-Saxon populations may need
careful examination before being administered in a dif-
ferent ethnic group.
A considerable amount of evidence suggests that eating
disorders have their onset in late adolescence or early adult-
hood. Mond, Hay, Rodgers, and Owen (2006) provided

2012 Hogrefe Publishing European Journal of Psychological Assessment 2013; Vol. 29(3):189196
DOI: 10.1027/1015-5759/a000138
190 T. Giovazolias et al.: The EDE-Q in Greece
comprehensive normative data for the EDE-Q among
young adult women, based on a large and representative
Australian sample, showing the high rates of eating disor-
ders among this specific group. Additionally, Luce, Crow-
ther and Pole (2008) using the EDE-Q in a group of college
women in the United States, aged 1825 years, presented
similar norms. Further, the EDE-Q works better than most
self-report measures when used to identify cases of clini-
cally significant eating disorders among young adult wom-
en in the community (Mond, Hay, Rodgers, Owen, & Beu-
mon, 2004).
So far, the EDE-Q has been translated into German,
Swedish, Spanish, and Norwegian, but only for the first
three versions is there any evidence regarding its factor
structure. Hilbert, Tuschen-Caffier, Karwautz, Niederho-
fer, and Munsch (2007) examined the German version of
the EDE-Q in a mixed sample of clinical, subclinical, and
nonclinical individuals. Using exploratory factor analysis
(EFA), they retained only 17 of the 22 items and suggested
a three-factor solution with the Weight and Shape Concern
items merging in one factor (apart from reaction to pre-
scribed weighing), 4 items from Eating Concern, 1 item
from Restraint (empty stomach) and 2 items from Weight
Concern (preoccupation with shape/weight, reaction to
prescribed weighing) forming a second factor, and 4 items
from the Restraint subscale (except for empty stomach)
loading on a third factor. Peterson et al. (2007), using EFA
in a sample of Swedish bulimic women, proposed an alter-
native three-factor solution, in which the first factor was a
combination of the Weight and Shape subscales (except for
preoccupation with weight and shape), the second factor
comprised of all the Eating Concern subscale items along
with empty stomach and preoccupation with weight and
shape, and the third factor included all items from the Re-
straint subscale, apart from empty stomach.
One of the main criticisms of the above-mentioned stud-
ies is that they adopted EFA as a method for investigating
the factor structure of EDE-Q, although EFA is used mainly
to describe, summarize, or reduce data and make them
more easily understood. This is why it is often considered
to be more appropriate in the early stages of scale develop-
ment (Hurley et al., 1997). On the other hand, confirmatory
factor analysis (CFA), as opposed to EFA, seems to be more
appropriate in the investigation of the crossvalidation of the
latent structure of a scale, since it allows the researcher,
after specifying an a priori model, to optimally match the
observed and theoretical factor structures for a given data
set in order to determine the goodness of fit of the prede-
termined factor model.
To date, only two studies have employed CFA in the
investigation of the latent structure of the EDE-Q. In the
first study, Hrabosky et al. (2008) performed a CFA on a
reduced version of the EDE-Q (12 items) derived from the
use of an EFA and suggested the following four latent fac-
tors: Dietary Restraint, Eating Disturbance, Appearance
Concerns, and Shape/Weight Overvaluation. In the second
study, Villarroel, Panelo, Portell, and Raich (2009), found
European Journal of Psychological Assessment 2013; Vol. 29(3):189196
that the four-factor model, as proposed by Fairburn and
Beglin (1994), provided a good fit to their data. This is the
only study so far that has provided evidence regarding the
robustness of Fairburn and Beglins original four-factor
model in a different cultural context.
Given the contradictory results, the main purpose of this
study was to use CFA to investigate the latent structure of
the four attitudinal subscales of the Greek translation of the
EDE-Q. The different models under investigation were as
follows:
1. A one-factor model that assumes that a single latent fac-
tor underlies all the EDE-Q items;
2. The original four-factor model proposed by Fairburn and
Beglin (1994);
3. Hilberts three-factor model;
4. Petersons three-factor.
Another aim of the study was to examine the psychometric
properties of the G-EDE-Q. For that purpose, we examined
the internal reliability and the item-to-scale homogeneity
as well as the concurrent, the convergent, the divergent, and
the discriminant validity of the G-EDE-Q global scale and
subscales.
Given that previous research has confirmed that eating
disorders are associated with individual characteristics,
personality traits as well as self-esteem were used to inves-
tigate the convergent validity of the G-EDE-Q. For exam-
ple, specific personality traits such as neuroticism and con-
scientiousness have been found to correlate with eating dis-
order symptoms (Diaz-Marza, Carrasco, & Saiz, 2000;
Ghaderi & Scott, 2000; MacLaren & Best, 2009). Further,
with regard to potential risk factors, self-esteem has been
identified as an important etiological aspect in reviews of
the relevant literature (e.g., Jacobi, Hayward, de Zwaan,
Kraemer, & Argas, 2004; Stice, 2002) and has been shown
to remain stable over time in nonclinical samples of under-
graduate women (Berg, Frazier, & Sherr 2009). On the oth-
er hand, since previous studies showed no relationship be-
tween eating disorder symptoms and extraversion and
openness to experience dimensions of personality (e.g.,
Ghaderi & Scott, 2000), nonsignificant correlations among
these variables could be considered as evidence for the di-
vergent validity of the G-EDE-Q.
Finally, since the majority of the previous studies inves-
tigated the psychometric properties of the EDE-Q using
clinical samples, we chose to test its psychometric sound-
ness in a nonclinical sample of undergraduate women, col-
lege women being acknowledged as a particular high-risk
group for developing eating disturbances (Berg et al.,
2009). Researchers have suggested that the unique social
and academic stressors associated with the university envi-
ronment may place female students at increased risk for
developing disordered eating habits (Compas, Wagner, Sla-
vin, & Vannatta, 1986). Therefore, careful assessment of
eating attitudes and behaviors in these populations seems
to become of great importance.
2012 Hogrefe Publishing
 T. Giovazolias et al.: The EDE-Q in Greece
Method
Participants
Two samples were used in this study. The first sample con-
sisted of 500 undergraduate female students used to ex-
amine the latent structure of the G-EDE-Q. The mean age
of the sample was 20.55 years (range 1744, SD = 3.27);
145 (29%) were 1st-year students, 105 (21%) were 2nd-
year students, 81 (16.2%) were 3rd-year students, and 164
(32.8%) were 4th-year or higher students. Regarding field
of study, 359 (71.8%) were studying psychology, 106
(21.2%) were studying sociology, and 32 (6.4%) were
studying literature; 6 (0.6%) participants did not report
their field of study. Finally, with regard to place of resi-
dence, 190 (38%) were living in cities with population
over 150,000, 162 (32.4%) were living in cities with pop-
ulation up to 100,000, 72 (14.4%) were living in small
towns up to 10,000 and 76 (15.5%) were living in villages.
The mean body mass index (BMI) for this sample was
21.72 (SD = 3.75), where 26.7% had a BMI below 20 and
11.2% had a BMI above 25. Using a cutoff score of 4
(Fairburn & Beglin, 1994), 8% of the participants ob-
tained scores in the clinically significant range on the
Global EDE-Q, 6.4% on the Restraint subscale, 2% on the
Eating Concern, 16.6% on the Shape Concern and 13.4%
on the Weight Concern subscale, respectively.
The second sample consisted of 164 undergraduate fe-
male psychology students used to examine the psycho-
metric quality of the G-EDE-Q. The mean age was 20.90
years (range 1845, SD = 3.29); 46 (28%) were 1st-year
students, 50 (30.5%) were 2nd-year students, 24 (32%)
were 3rd-year students, and 44 (26.8%) were 4th-year or
higher students. The majority of the participants lived in
urban areas: 54 (32.9%) were living in cities with popu-
lation over 150,000, 52 (31.7%) were living in cities with
population up to 150,000, 25 (15.2%) were living
in small towns up to 10,000, and 33 (20.1%) were living
in villages. The BMI for this second sample was 22.83
(SD = 4.38), where 17.1% had a BMI below 20 and
17.7% above 25. Regarding other clinical characteristics,
10.4% scored in the clinically significant range on the
Global EDE-Q, 9.8% on the Restraint, 4.3% on the Eat-
ing Concern, 18.9% on the Shape Concern and 18.9% on
the Weight Concern subscales.
Measures
Eating Disorder Examination-Questionnaire
(EDE-Q; Fairburn & Beglin, 1994)
The EDE-Q is a 36-item self-report instrument assessing be-
havioral and attitudinal aspects of eating psychopathology.
Responses to the questionnaire provide four subscale scores
2012 Hogrefe Publishing
191
based only on the 22 attitudinal items. The four subscales are
Restraint, Eating Concern, Shape Concern, and Weight Con-
cern. An overall Global score is also obtained by summing
the four subscale scores. The EDE-Q was translated into
Greek in three consecutive stages. During the first stage, four
bilingual psychologists, experts in eating disorders, individ-
ually translated the measure from English into Greek to ob-
tain linguistic equivalence. In the second stage, a committee
consisting of the same four translators examined the four
translated versions, discussed any discrepancies, misunder-
standings, mistranslations, or inaccuracies that emerged, and
made proper adjustments to ensure conceptual equivalence.
Finally, a pilot study (n = 15) was carried out to determine
whether both the original and the translated versions were
equivalent using as criteria the percent of missing data, floor
and ceiling effects, and alpha reliabilities. The results from
this analysis revealed that none of the adopted criteria re-
vealed substantial differences between the two versions, sug-
gesting that the Greek and the original versions of EDE-Q
were indeed equivalent.
Eating Attitudes Test (EAT-26; Garner & Garfinkel,
1979)
The EAT-26 is a widely used self-report questionnaire that
assesses behaviors related to disturbed eating and attitudes
relating to body weight. It comprises 26 questions that refer
to thoughts, feelings, and behaviors; these are organized
into three subscales (Diet, Bulimia and Food Preoccupa-
tion, and Oral Control) as well as a global scale. In our
study, the reliability of EAT-26 Global scale was consid-
ered satisfactory ( = .85). The EAT-26 has been standard-
ized in Greek (Douka, Grammatopoulou, Skordilis, &
Koutsouki, 2009) and possesses excellent psychometric
properties for the Greek population.
Body Shape Questionnaire (BSQ-34; Cooper,
Taylor, Cooper, & Fairburn, 1987)
The BSQ-34 is a 34-item self-report questionnaire measur-
ing body dissatisfaction, including preoccupation with and
distress about body shape. Using a 6-point scale (never to
always) it provides a global score in which high scores in-
dicate higher levels of body dissatisfaction. The reliability
of the BSQ-34 was found to be = 97 in our study. The
measure has now been adapted to the Greek language (Kos-
kina & Giovazolias, 2010) and demonstrates acceptable
psychometric properties with the Greek population.
The Big Five Inventory (BFI; Benet-Martinez &
John, 1998)
The BFI is a 44-item self-report measure designed to meas-
ure the Big Five dimensions (i.e., Extraversion, Neuroti-
European Journal of Psychological Assessment 2013; Vol. 29(3):189196
192 T. Giovazolias et al.: The EDE-Q in Greece
Table 1. Goodness-of-fit indices of the hypothesized latent-factor models of the G-EDE-Q (N = 500)
Model GFI
Model I (one-factor) 10979.30 .97
Model II (Fairburn & Beglin, 1994) Model is not admissible
Model III (Hilbert et al., 2007) 5604.15 .98
Model IV (Peterson et al., 2007) 7572.98 .98
Notes. GFI = goodness-of-fit index; AGFI = adjusted goodness-of-fit index; NFI = normed fit index; SRMR = standardized root-mean square
residual; AIC = Akaike information criterion.
cism, Openness to experience, Agreeableness, and Consci-
entiousness) using 5-point ratings (1 = strongly disagree to
5 = strongly agree). In previous research, its domain scales
showed high reliability, clear factor structure, strong con-
vergence with longer Big Five measures, and substantial
self-peer agreement (Benet-Martinez & John, 1998). Alpha
reliabilities for this study ranged between = .70 and .74.
The Rosenberg Self-Esteem Scale (RSE; Rosenberg,
1965)
This 10-item self-report instrument purports to assess self-
esteem using a four-point scale ranging from 1 to 4 (strong-
ly agree to strongly disagree). The RSE is coded so that a
high score indicates low self-esteem. Alpha reliability for
this study was = .80.
Procedure
The participants filled out the questionnaires during an or-
dinary lecture. Participation in the study was voluntary, and
participants were informed that all data were held confi-
dential. After completing a consent form, students filled out
the questionnaires in the presence of the researchers, who
were available for any clarifications needed. For the first
sample there was no payment or other incentive to com-
plete the study, whereas participants of the second sample
received course credit for their participation.
Results
In order to study the latent structure of the G-EDE-Q, we
specified and estimated CFA alternative models using
AMOS 6 (Arbuckle, 2005). First, we examined whether the
assumption of multivariate normality of the data was satis-
fied. Using the SPSS macro developed by DeCarlo (1997),
we found that the majority of the skewness and kurtosis
values were significant (p < .05) and the total Mardias co-
efficient was 257.53; these results suggest deviations from
normality. For that reason, we decided to use the unweight-
ed least square (ULS) method of estimation, since it is less
sensitive to data that deviate from normality (Hu & Bentler,
European Journal of Psychological Assessment 2013; Vol. 29(3):189196
AGFI NFI SRMR AIC
.96 .96 .09 11067.30
.97 .97 .07 5678.15
.98 .97 .07 7666.98
1999). In addition, because of the particular method of es-
timation adopted, the GFI, AGFI, NFI, and SRMR good-
ness-of-fit indices were used. Generally, for GFI, AGFI,
and NFI a value close to 0.95 indicates a model with a good
fit; for the SRMR values less than .08 indicate good fit (Hu
& Bentler, 1999). Table 1 presents the fit statistics for the
different G-EDE-Q proposed models.
The results from the analysis showed that model II, the
original four-factor model, yielded a nonpositive definite
matrix solution, which means that some variance estimates
are negative, or that some exogenous variables have an es-
timated covariance matrix that is not positive definite.
Thus, this model was not acceptable. Regarding model I,
although some of the fit indices were within the acceptable
range, some others were not ( = 1079.30, GFI = .97,
AGFI = .96, NFI = .96, SRMR = .09), suggesting an only
marginally acceptable model. For model III ( = 5604.15,
GFI = .98, AGFI = .97, NFI = .97, SRMR = .07) and model
IV ( = 7572.98, GFI = .98, AGFI = .98, NFI = .97,
SRMR = .07), all fit indices were within the standard rec-
ommended range, suggesting that both fit the data well.
In order to decide which of the three models yielding an
acceptable fit better represents the underlying structure of the
G-EDE-Q, we used the Akaike information criterion (AIC),
the appropriate goodness-of-fit index for choosing between
competing statistical models. The AIC index showed that
model III (i.e., Hilberts three-factor model), better reproduced
the data than model IV (i.e., Petersons, three-factor model)
and model I (i.e., one-factor model) (the AICs were 5,678.15,
7,666.98 and 11,067.30, respectively). However, one should
bear in mind that Hilberts model contains fewer items than
the original version (17 rather than 22), which means that the
estimated parameters in Hilberts model are fewer, and that the
chi-square and AIC values are inevitably lower compared to
a model that contains all the items (and of course more esti-
mated parameters), such as Petersons model, which contains
all 22 items. Thus, although AIC statistic suggests that Hil-
berts model better fits the data, Petersons model seems more
inclusive and more cohesive, since it is closer to the original
version. For that reason it seems more preferable.
Internal Consistency of the G-EDE-Q
To evaluate the internal reliability and item-to-scale ho-
mogeneity of the G-EDE-Q, we calculated alpha coeffi-
2012 Hogrefe Publishing
 T. Giovazolias et al.: The EDE-Q in Greece 193

Table 2. Pearsons correlations between G-EDE-Q and validity measures (N = 164)

Measures Mean SD 1 2 3 4
1 Restraint 6.68 6.16 1 .54** .63** .76**
2 Eating concern 6.63 8.31 1 .72** .85**
3 Shape and weight concern 24.39 18.03 1 .96**
4 Global EDE-Q 37.70 29.04 1
5 EAT-26 Global Scale 13.79 10.47 .45** .65** .62** .67**
6 Dieting 7.23 7.20 .57** .65** .70** .74**
7 Bulimia and food preoccupation 2.57 3.17 .20** .58** .52** .53**
8 BSQ Global Scale 85.38 34.90 .58 .72** .89** .88**
9 Global Self-Esteem 19.59 5.52 .10 .31** .37** .34**
10 Extraversion 26.40 4.55 .05 .07 .15 .13
11 Neuroticism 25.93 5.46 .15 .16 .30** .27**
12 Openness to experience 36.24 6.22 .01 .06 .08 .07
13 Agreeableness 34.73 5.71 .12 .25** .17 .21*
14 Conscientiousness 31.32 5.64 .05 .25** .17 .17
Note. *p < .05, **p < .001.

cients and mean interitem correlations. The results
showed that Cronbachs coefficients were .92 for the
G-EDE-Q global scale, .78 for the Restraint subscale, .74
for the Eating Concern subscale, and .91 for the combined
Shape and Weight Concern subscale, indicating adequate
internal consistency for all G-EDE-Q subscales. The mean
interitem correlation coefficients for the G-EDE-Q global
scale was .42, while the corresponding values for Re-
strain, Eating Concern, and Shape and Weight Concern
subscales were .47, .29, and .49, respectively. All values
were within the acceptable recommendable range for mul-
tifactor scales (Briggs & Cheek, 1986).
Concurrent Validity of the G-EDE-Q
Concurrent validity was assessed by correlating the G-
EDE-Q Global scale and subscales with two instruments:
the EAT-26, a measure assessing behavioral and attitudi-
nal aspects of eating psychopathology; and the BSQ, a
measure assessing dissatisfaction with shape and preoccu-
pation with body shape. The results from the analysis (Ta-
ble 2, upper part) showed that G-EDE-Q is strongly cor-
related with both measures, providing evidence to support
the concurrent validity of the Greek translation of the
EDE-Q. Particularly, the majority of the G-EDE-Q global
scale and subscales were positively correlated with all
EAT-26 scales with correlations ranging from .45 to .74.
Only the Restraint subscale was weakly correlated with
the Bulimia and Food Preoccupation scale (r = .20). Ad-
ditionally, G-EDE-Q global scale and subscales were pos-
itively and highly correlated with the global BSQ scale,
with correlations ranging from .58 to .89 (Table 2, upper
part).
Convergent and Divergent Validity of the
G-EDE-Q
Since previous research showed that personality character-
istics (Ghaderi, & Scott, 2000; MacLaren & Best, 2009)
and self-esteem (Stice, 2002) are to some extent related to
eating behavior, we expected that the G-EDE-Q global
scale and subscales would be related to both constructs.
The results in Table 2 (lower part) showed that G-EDE-Q
global scale was positively associated with Neuroticism
(r = .27) and negatively related to Agreeableness (r =
.21). Eating Concern subscale was negatively related to
Agreeableness (r = .25) and Conscientiousness (r = .25).
Shape and Weight Concern subscale was positively related
to Neuroticism (r = .30). The vast majority of these corre-
lations were significant (p < .001), although weak in mag-
nitude. Restraint subscale was not significantly related to
any personality dimension. Finally, Global Self-esteem
scale was positively correlated with Global G-EDE-Q (r =
.34) as well as with Eating Concern (r = .31) and Shape and
Weight Concern (r = .37) subscales. Restraint subscale was
not significantly related to self-esteem. All of these corre-
lations were significant (p < .001), although moderate in
magnitude.
Discriminant Validity of the G-EDE-Q
Discriminant validity indicates the ability of a scale to dis-
tinguish different groups of individuals. An instrument for
assessing the severity of a psychopathological behavior
such as eating psychopathology should clearly be able to
distinguish between individuals with and without the ten-
dency to develop eating disordered attitudes. For that rea-

2012 Hogrefe Publishing European Journal of Psychological Assessment 2013; Vol. 29(3):189196
194 T. Giovazolias et al.: The EDE-Q in Greece
son, two separate logistic regression analyses were per-
formed to investigate whether individual G-EDE-Q scores
could predict the risk to develop eating disordered atti-
tudes. The variable to be predicted was the tendency to
develop a type of eating disorder (yes vs. no) as indicated
by EAT-26 global score (> 20). According to the literature,
respondents who score exactly at or above the cutoff score
of 20 belong to the group of people who has the tendency
to develop disordered eating (Garner, Olmsted, Bohr, &
Garfinkel, 1982)
In the first analysis, the Global G-EDE-Q scale was en-
tered as predictor variable. A significant regression model
was produced using a forced entry method of estimation
[model (1) = 55.93; p = .000]. This model correctly pre-
dicts the overall presence or absence of disturbed eating
attitudes in 86.6% of the cases. Test sensitivity and speci-
ficity of this regression model were 51.5% and 94.7%, re-
spectively, while predictive values positive and negative
were 71% and 89%, respectively. In the second analysis,
Restraint, Eating Concern, and Shape and Weight Concern
were entered as predictor variables. A test of the full model
with the three predictors was statistically reliable, (3) =
59.71, p < .001, indicating that, as a set, the predictors re-
liably distinguished between high-risk (HR) and no-risk
(NR) individuals in developing eating disorder problems.
The variance in the HR status accounted for was large, with
Nagelkerke R2 = .48. Prediction was good, with 54.5% of
high-risk individuals and 94.7% of no-risk individuals cor-
rectly being predicted, for an overall success rate of 86.6%,
while predictive values positive and negative were 72%
and 89%, respectively. Table 3 shows the regression coef-
ficient, Wald statistic, and odds ratio for each of the three
predictors. According to the Wald criterion, only Shape and
Weight Concern score predicted HR status. This finding
implies that the odds ratio of this predictor (1.10) shows a
great change in the likelihood of the individual developing
eating disorder problems on the basis of one unit change in
this variable (i.e., Shape and Weight Concern scale).
Table 3. Logistic regression predicting presence of disor-
dered eating attitudes (N = 164)
Scales B SE Odds ratio Wald statistic
(95% CI)
Restraint .001 .05 1.00 (0.911.09) .001
Eating concern .030 .03 1.03 (0.961.10) .760
Weight and .094 .02 1.10 (1.051.15) 17.52**
shape concern
Global EDE-Q .060 .01 1.06 (1.041.08) 32.97**
Note. **p < .01.
Discussion
This study investigates the factor structure of the Greek
version of the Eating Disorders Examination Questionnaire
(G-EDE-Q) in a nonclinical sample, using the statistical
European Journal of Psychological Assessment 2013; Vol. 29(3):189196
technique of CFA. This statistical technique appears to be
more appropriate and robust than EFA, which has been
used in most of the previous studies investigating the fac-
torial structure of the EDE-Q in different crosscultural set-
tings. The results from the CFA suggested that the original
four-factor model (Fairburn & Beglin, 1994) was not ac-
ceptable, since the correlation matrix was not positively
definite. This could be either because the model was wrong
or because the sample size was too small (Jreskog & Sr-
bom, 1996). However, in our study, the sample size was
adequate, since according to Muthn and Muthn (2002)
the minimum sample size for nonnormally distributed data
should be at least 315.
The analysis also revealed that both Hilbert et al.s
(2007) and Peterson et al.s (2007) three-factor models had
acceptable fit indices, suggesting that the Shape and Weight
Concern subscales should be considered as a single factor,
instead of two separate factors as they have been introduced
in the original four-factor model. This result is in line with
findings from other studies in eating disorders, which al-
ready consider weight and shape concern as a single com-
ponent and relate it to different manifestations of eating
disturbances (e.g., Allen, Byrne, McLean, & Davis, 2008).
The present study provides further evidence and supports
this perspective empirically, since, according to the logistic
regression results, the combined scale of Shape and Weight
Concern is the only subscale that can successfully discrim-
inate between high- and low-risk individuals. A possible
theoretical explanation for the emergence of shape and
weight concern as a single factor is that individuals emit-
ting concerns about their shape are likely to present also
weight-related concerns, as a means of controlling changes
in their body figure.
Although Hilberts and Petersons three-factor solutions
provide acceptable data fit, the latter seems more prefera-
ble, since it is closer to Fairburn and Beglins original mod-
el. Hilberts model appears important apart from the fact
that contains fewer items, as a result of the inappropriate
use of EFA, and because many items are misplaced com-
pared to Fairburn and Beglins (1994) and Petersons et al.
(1997) models. Finally, the one-factor model was also mar-
ginally fit the data. This was not an unexpected result, since
the Global EDE-Q factor has been often used as a broad
indicator of eating disturbances (Masheb & Grilo, 2008).
Another aim of this study was to investigate the internal
consistency of the G-EDE-Q global scale and the four sub-
scales. The alpha coefficients obtained are within the min-
imum acceptable range ( > .70), and support the findings
from previous studies (Hilbert et al., 2007; Luce & Crow-
ther, 1999; Peterson et al., 2007). Furthermore, the homo-
geneity of the global scale and the three subscales was con-
firmed according to the results from the mean interitem
coefficients.
The results of the present study add to existing evidence
for the concurrent, convergent, and divergent validity of the
translated version of the scale. In terms of concurrent va-
lidity, it was found that there is a substantial relationship
2012 Hogrefe Publishing
 T. Giovazolias et al.: The EDE-Q in Greece
among two measures of eating psychopathology (EAT-26
and BSQ) and G-EDE-Q subscales. Furthermore, in line
with previous findings (e.g., Diaz-Marza et al., 2000; Gha-
deri & Scott, 2000; MacLaren & Best, 2009), it was found
that there is a relationship between attitudinal aspects of
eating disorders and specific personality traits, such as
Neuroticism (positive) and Agreeableness and Conscien-
tiousness (negative), all of which support the convergent
validity of the scale. The insignificant relationship between
Extraversion and Openness with most of the G-EDE-Q
scales found in this study provide evidence regarding the
divergent validity of the measure, supporting previous find-
ings (e.g., Ghaderi & Scott, 2000). Finally, the positive re-
lationship found between self-esteem and G-EDE-Q Glob-
al scale and subscales was also in line with findings from
previous studies, which demonstrated that self-esteem is an
important etiological factor in eating disorders (e.g., Stice,
2002), since it has been shown to remain stable over time
in nonclinical samples of undergraduate women (Berg et
al., 2009).
An important limitation of the present study is the nature
of the sample used (nonclinical student sample). One could
argue that EDE-Q is a clinical instrument and therefore
should be used solely with clinical samples. However,
based on the continuum hypothesis proposed by Nylander
(1971) and as evidenced by its expansion by Mintz and
Betz (1988), eating disturbances occur along a continuum,
constituting three different groups (asymptomatic, sympto-
matic, and eating disordered). In that sense, eating disor-
ders occur when people simply display extreme manifesta-
tions of common behavioral, attitudinal, and psychological
dimensions, so that group differences are a matter of degree
and not kind. Gleaves, Lowe, Green, Cororve, and Wil-
liams (2000) also concluded that eating disturbances ap-
pear to fall along a continuum of symptom severity, lending
partial support to the continuum model. To this end, it has
been suggested that research should be directed toward the
investigation of the psychometric properties of existing
measures in nonclinical samples (Engelsen & Laberg,
2001). Moreover, female university students were reported
to present high rates of eating disorders and eating disorder
symptoms (Berg et al., 2009; Compas et al., 1986). Future
research should investigate the factorial invariance of the
translated version in a more diverse population in terms of
age, educational level, and socioeconomic background,
since student samples are quite homogeneous. Finally, an-
other limitation concerns the use of EAT-26 as the only
criterion for discriminating participants with or without
eating disorders: Future research should employ a more
clinical methodology (e.g., clinical interview) to identify
clinically diagnosed cases.
The present study showed that the three-factor version
of the G-EDE-Q may be utilized as a useful measure in the
assessment of attitudinal features of disturbed eating atti-
tudes in female university students. Given the fact that col-
lege women are a high-risk group for eating disorders and
present high rates of subclinical eating disturbances, the
2012 Hogrefe Publishing
195
particular questionnaire may prove an important screening
tool for university counselors and/or other clinicians in
similar contexts. Additionally, the availability of normative
data for community samples would be an advantage for
using the G-EDE-Q in primary-care settings, where practi-
tioners (e.g., psychologists, counselors) may be facilitated
to detect cases of eating disturbances and thus choose an
appropriate referral route.
References
Allen, K. L., Byrne, S. M., McLean, N. J., & Davis, E. A. (2008).
Overconcern with weight and shape is not the same as body
dissatisfaction: Evidence from a prospective study of preado-
lescent boys and girls. Body Image, 5, 261270.
Arbuckle, J. L. (2005). AMOS 6.0. users guide. Spring House,
PA: Amos Development Corporation.
Benet-Martinez, V., & John, O. P. (1998). Los Cinco Grandes
across cultures and ethnic groups: Multitrait-multimethod
analyses of the Big Five in Spanish and English. Journal of
Personality and Social Psychology, 75, 729750.
Berg, K. C., Frazier, P., & Sherr, L. (2009). Change in eating dis-
order attitudes and behavior in college women: Prevalence and
predictors. Eating Behaviors, 10, 137142.
Briggs, S. R., & Cheek, J. M. (1986). The role of factor analysis
in the development and evaluation of personality scales. Jour-
nal of Personality, 54, 106148.
Compas, B. E., Wagner, B. M., Slavin, L. A., & Vannatta, K.
(1986). A prospective study of life events, social support, and
psychological symptomatology during the transition from high
school to college. American Journal of Community Psycholo-
gy, 14, 241257.
Cooper, P. J., Taylor, M. J., Cooper, Z., & Fairburn, C. G. (1986).
The development and validation of the Body Shape Question-
naire. International Journal of Eating Disorders, 6, 485494.
Cooper, Z., & Fairburn, C. G. (1987). The Eating Disorders Ex-
amination: A semistructured interview for the assessment of
the specific psychopathology of eating disorders. International
Journal of Eating Disorders, 6, 18.
DeCarlo, L. T. (1997). On the meaning and use of kurtosis. Psy-
chological Methods, 2, 292307.
Diaz, V. A., Mainous III, A. G., & Pope, C. (2007). Cultural con-
flicts in the weight loss experience of overweight Latinos. In-
ternational Journal of Obesity, 31, 328333.
Diaz-Marsa, M., Carrasco, J. L., & Saiz, J. (2000). A study of
temperament and personality in anorexia nervosa and bulimia
nervosa. Journal of Personality Disorders, 14, 352359.
Douka, A., Grammatopoulou, E., Skordilis, E., & Koutsouki, D.
(2009). Factor analysis and cut-off score of the 26-item eating
attitudes test in a Greek sample. Biology of Exercise, 5, 5167.
Engelsen, B. K., & Laberg, J. K. (2001). A comparison of three
questionnaires (EAT-12, EDI, and EDE-Q) for assessment of
eating problems in healthy female adolescents. Nordic Journal
of Psychiatry, 55, 129135.
Fairburn, C. G., & Beglin, S. J. (1994). Assessment of eating dis-
orders: Interview or self-report questionnaire? International
Journal of Eating Disorders, 16, 363370.
Garner, D. M., & Garfinkel, P. E. (1979). The Eating Attitudes
European Journal of Psychological Assessment 2013; Vol. 29(3):189196
196 T. Giovazolias et al.: The EDE-Q in Greece
Test: An index of the symptoms of anorexia nervosa. Psycho-
logical Medicine, 9, 273279.
Garner, D. M., Olmsted, M. P., Bohr, Y., & Garfinkel, P. E. (1982).
The Eating Attitudes Test: Psychometric features and clinical
correlates. Psychological Medicine, 12, 871878.
Ghaderi, A., & Scott, B. (2000). The big five and eating disorders:
A prospective study in the general population. European Jour-
nal of Personality, 14, 311323.
Gleaves, D. H., Lowe, M. R., Green, B. A., Cororve, M. B., & Wil-
liams, T. L. (2000). Do anorexia and bulimia nervosa occur on
a continuum? A taxometric analysis. Behavior Therapy, 31,
195219.
Hilbert, A., Tuschen-Caffier, B., Karwautz, A., Niederhofer, H.,
& Munsch, S. (2007). Eating disorder examination-question-
naire: Evaluation der deutschsprachigen bersetzung. Diag-
nostica, 53, 144154.
Hrabosky, J. I., White, M. A., Masheb, R. M., Rothschild, B. S.,
Burke-Martindale, C. H., & Grilo, C. M. (2008). Psychometric
evaluation of the eating disorder examination-questionnaire
for bariatric surgery candidates. Obesity, 16, 763769.
Hu, L., & Bentler, P. M. (1999). Cut off criteria for fit indexes in
covariance structure analysis: Conventional criteria versus
new alternatives. Structural Equation Modeling, 6, 155.
Hurley, A. E., Scandura, T. A., Schriesheim, C. A., Brannick, M. T.,
Seers, A., Vandenberg, R. J., & Williams, L. J. (1997). Explorato-
ry and confirmatory factor analysis: Guidelines, issues, and alter-
natives. Journal of Organizational Behavior, 18, 667683.
Jacobi, C., Hayward, C., de Zwaan, M., Kraemer, H. C., & Argas,
W. S. (2004). Coming to terms with risk factors for eating dis-
orders: Application of risk terminology and suggestions for a
general taxonomy. Psychological Bulletin, 130, 1965.
Jreskog, K. G., & Srbom, D. (1996). Lisrel 8: Structural equa-
tion modeling with the SIMPLIS command language. Chicago,
IL: Scientific Software International.
Koskina, N., & Giovazolias, T. (2010). The effect of attachment
insecurity in the development of eating disturbances across
gender: The role of body dissatisfaction. Journal of Psychol-
ogy, 144, 449471.
Luce, K. H., & Crowther, J. H. (1999). The reliability of the eating
disorder examination-Self-report questionnaire version (EDE-
Q). International Journal of Eating Disorders, 25, 349351.
Luce, K. H., Crowther, J. H., & Pole, M. (2008). Eating Disorder
Examination Questionnaire (EDE-Q): Norms for undergraduate
women. International Journal of Eating Disorders, 41, 273276.
MacLaren, V. V., & Best, L. A. (2009). Female students disor-
dered eating and the big five personality facets. Eating Behav-
iors, 10, 192195.
Masheb, R. M., & Grilo, C. M. (2008). Prognostic significance of
two subcategorization methods for the treatment of binge eat-
ing disorder: Negative affect and overvaluation predict, but do
European Journal of Psychological Assessment 2013; Vol. 29(3):189196
not moderate, specific outcomes. Behavior Research and Ther-
apy, 46, 428437.
Mintz, L. B., & Betz, N. E. (1988). Prevalence and correlates of
eating disordered behavior among undergraduate women.
Journal of Counseling Psychology, 35, 463471.
Mond, J. M., Hay, P. J., Rodgers, B., & Owen, C. (2006). Eating
Disorder Examination Questionnaire (EDE-Q): Norms for
young adult women. Behavior Research and Therapy, 44,
5362.
Mond, J. M., Hay, P. J., Rodgers, B., Owen, C., & Beumont, P. J.
(2004). Validity of the Eating Disorder Examination Question-
naire (EDE-Q) in screening for eating disorders in community
samples. Behavior Research and Therapy, 42, 551567.
Muthn, L. K., & Muthn, B. O. (2002). Teacherss corner: How
to use a Monte Carlo study to decide on sample size and de-
termine power. Structural Equation Modeling, 9, 599620.
Nylander, J. (1971). The feeling of being fat and dieting in a
school population: Epidemiological interview investigation.
Acta Sociomedica Scandinavica, 3, 1726.
Peterson, C. B., Crosby, R. D., Wonderlich, S. A., Joiner, T., Crow,
S. J., Mitchell, J. E., . . . le Grange, D. (2007). Psychometric
properties of the Eating Disorder Evaluation-Questionnaire:
Factor structure and internal consistency. International Jour-
nal of Eating Disorders, 40, 386389.
Rosenberg, M. (1965). Society and the adolescent self-image.
Princeton, NJ: Princeton University Press.
Stice, E. (2002). Risk and maintenance factors for eating pathol-
ogy: A meta-analytic review. Psychological Bulletin, 128,
825848.
Villarroel, A. M., Panelo, E., Portell, M., & Raich, R. M. (2009).
Screening for eating disorders in undergraduate women: Norms
and validity of the Spanish version of the Eating Disorder Exam-
ination Questionnaire (EDE-Q). Journal of Psychopathology
and Behavioral Assessment. doi 10.1007/s10862-009-9177-6
Wilson, G. T. (1993). Assessment of binge eating. In C. G. Fair-
burn & G. T. Wilson (Eds.), Binge eating: Nature, assessment,
and treatment (12th ed., pp. 227249). New York: Guilford.
Published online: May 11, 2012
Theodoros Giovazolias
Department of Psychology
School of Social Sciences
University of Crete
Crete, 74100
Greece
Tel. +30 28 3-107-7529
Fax +30 28 3-107-7578
E-mail giovazolias@psy.soc.uoc.gr
2012 Hogrefe Publishing
 C.Journalof
European Gunzenh auser et al.: Self-Efficacy
Psychological in
Assessment2013;Regulating
2012 Vol.
Hogrefe Emotions
29(3):197204
Publishing

Original Article

Self-Efficacy in Regulating
Positive and Negative Emotions
A Validation Study in Germany
Catherine Gunzenhauser1, Tobias Heikamp2, Maria Gerbino3,
Guido Alessandri3, Antje von Suchodoletz1, Laura Di Giunta3,
Gian Vittorio Caprara3, and Gisela Trommsdorff2
1
University of Freiburg, Germany, 2University of Konstanz, Germany,
3
University of Rome La Sapienza, Italy

Abstract. Perceived self-efficacy in emotion regulation facilitates various aspects of psychosocial adjustment. The Regulatory Emotional
Self-Efficacy scale (RESE) by Caprara and Gerbino (2001) measures perceived capabilities to express positive emotions (POS) and to
manage negative emotions, namely, despondency/distress (DES) and anger/irritation (ANG). The present research investigated the va-
lidity of the RESE scale in Germany. Study 1 investigated the factor structure and convergent validity of the scale in a sample of university
students. In order to test the generalizability of findings from Study 1, in Study 2 we studied the factor structure, cross-gender invariance,
and convergent validity of a slightly revised version of the scale in a sample of parents. The previously found factor structure was
successfully replicated in both samples. Partial invariance on the scalar level was confirmed across gender. All self-efficacy subscales
were positively correlated with life satisfaction and with reappraisal (a cognitive emotion regulation strategy). Suppression, a strategy of
regulating emotional expression, was negatively related to POS. Findings suggest that the RESE scale is a valid instrument to assess
emotion regulation self-efficacy in German-speaking samples.

Keywords: self-efficacy beliefs, emotion regulation, measurement invariance

Self-efficacy beliefs are a central component of human
agency. They are defined as perceived capabilities to act in
the way necessary to attain desired outcomes in specific
situations (Bandura, 1997). Self-efficacy beliefs increase
the ability and flexibility to adjust to social and situational
demands by motivating people to set higher goals, to invest
greater effort, and to persist when facing difficulties (see
Bandura, 1997). Domain-specific self-efficacy beliefs are
also the basis for successful emotion regulation (Caprara et
al., 2008; Caprara & Gerbino, 2001).
Regulation of Positive and Negative
Emotions
Emotion regulation refers to attempts individuals make to
influence which emotions they have, when they have them,
and how these emotions are experienced and expressed
(Gross, Richards, & John, 2006, p. 14). The regulation of
positive emotions and the regulation of negative emotions
are distinctly related to emotional and social adjustment.
For instance, experience and expression of positive emo-
tions are associated with rewarding social relationships,
health, and occupational success (Davidov & Grusec,
2006; Lyubomirsky, King, & Diener, 2005). Difficulties in
downregulating strong negative emotions are associated
with problematic interpersonal behavior (Eisenberg et al.,
2001).
The Regulatory Emotional Self-Efficacy Scale
The Regulatory Emotional Self-Efficacy scale (RESE;
Caprara et al., 2008; Caprara & Gerbino, 2001) was de-
veloped to assess self-efficacy beliefs in the domain of
emotion regulation. The RESE scale assesses self-efficacy
in expressing positive emotions (POS) and self-efficacy in
managing negative emotions (NEG). POS is defined as
the perceived capability to experience and to allow one-
self to express positive emotions such as joy, enthusiasm
and pride in response to success or pleasant events (Ca-
prara et al., 2008, p. 228). NEG refers to the perceived
capability to ameliorate negative emotional states once
they are aroused in response to adversity or frustrating

2012 Hogrefe Publishing European Journal of Psychological Assessment 2013; Vol. 29(3):197204
DOI: 10.1027/1015-5759/a000151
198 C. Gunzenhauser et al.: Self-Efficacy in Regulating Emotions
events and to avoid being overcome by emotions such as
anger, irritation, despondency, and discouragement (Ca-
prara et al., 2008, p. 228). POS constitutes a first-order
factor, whereas NEG constitutes a second-order factor
represented by two first-order-factors: self-efficacy in
managing despondency/distress (DES) and self-efficacy in
managing anger/irritation (ANG). Confirmatory factor
analyses have supported this structure in Italian, Bolivian,
and U. S. samples (Caprara et al., 2008). Moreover, Ca-
prara et al. (2008) documented positive associations of the
RESE subscales with indicators of well-being in an Italian
sample.
Importance of Cross-Gender Invariance
In general, women engage in emotion regulation more fre-
quently than men and use a greater variety of emotion-reg-
ulation strategies (Nolen-Hoeksema & Aldao, 2011; Tam-
res, Janicki, & Helgeson, 2002). At the same time, women
are more susceptible to depressive symptoms, whereas
men tend to be more likely to show aggressive reactions
to emotional arousal (Knight, Guthrie, Page, & Fabes,
2002; Nolen-Hoeksema & Aldao, 2011). Emotion regula-
tion has been linked to gender differences in both depres-
sive symptoms and aggressive reactions (Knight et al.,
2002; Nolen-Hoeksema & Aldao, 2011; Sontag & Graber,
2010). For example, Nolen-Hoeksema and Aldao (2011)
found that maladaptive emotion-regulation strategies,
which are used more frequently by women (Tamres et al.,
2002), are associated with depression across gender. Son-
tag and Graber (2010) found that emotional disengage-
ment was related to aggression in boys but not in girls.
Thus, cross-gender invariance of emotion-related meas-
urement scales has relevance for further research in the
areas of developmental and clinical psychology. The study
by Caprara et al. (2008) revealed measurement invariance
of RESE across gender in Italian, Bolivian, and U. S. sam-
ples.
Study Aims
The goals of the present research were
1. to examine the reliability and the factor structure of the
RESE scale in a German-speaking sample,
2. to further investigate the convergent validity of the Ger-
man scale, and
3. to examine the cross-gender invariance of the German
scale.
Study 1
In Study 1, we studied the factor structure of a preliminary
version of the German RESE scale and replicated the mul-
European Journal of Psychological Assessment 2013; Vol. 29(3):197204
tidimensional latent structure of the RESE scale. Addi-
tionally, Study 1 examined convergent validity by inves-
tigating the relations between self-efficacy beliefs in reg-
ulating emotions, life satisfaction, and two emotion-regu-
lation strategies. Life satisfaction is defined as a
conscious cognitive judgment of ones life in which the
criteria for judgment are up to the person (Pavot & Die-
ner, 1993, p. 164). RESE beliefs have been shown in other
countries to be positively correlated with life satisfaction
(Caprara & Steca, 2005). Therefore, we assumed that all
RESE subscales would be positively related to life satis-
faction. We further examined the relations of RESE with
two specific emotion-regulation strategies, reappraisal
and suppression (Gross & John, 2003). Reappraisal refers
to a cognitive reframing of an emotion-eliciting event; it
modulates emotional experience, which is mirrored in
emotional expression. For example, a woman who re-
ceives an ugly necklace for her birthday could reappraise
the situation by focusing on how sweet it is that the giver
tried so hard to please her. This reappraisal would make
her feel and look happier. Individuals skilled at reappraisal
can be expected to find themselves capable of downregu-
lating negative emotions and of pointing out the positive
features of a situation to themselves. Thus, we assumed
positive correlations of reappraisal with POS, DES, and
ANG. Suppression is an attempt to hide external signs of
emotion, without changing the emotional experience. For
instance, another woman who receives very beautiful ear-
rings for her birthday might restrain herself from jumping
for joy because she feels it might be inappropriate or em-
barrassing. She might still feel happy, but her suppression
strategy would make her seem less so. In contrast to the
woman in our example, individuals with high self-efficacy
in expressing positive emotions feel capable to express
their positive feelings. Thus, we assumed that POS should
be negatively associated with suppression. Since DES and
ANG emphasize the management of emotional experience
rather than emotional expression, we did not expect DES
and ANG to be associated with suppression.
Materials and Methods
Participants
Participants were 192 male and 307 female undergraduate
students from southern Germany. Students were between
18 and 26 years old (M = 21.44, SD = 1.46). Participants
could participate in a lottery of EUR 20 gift certificates or
receive course credit.
Measures
Regulatory Emotional Self-Efficacy Scale
Study 1 used a version of the RESE scale that consists of
12 items (Caprara et al., 2008). POS, DES, and ANG were
2012 Hogrefe Publishing
 C. Gunzenhauser et al.: Self-Efficacy in Regulating Emotions 199

measured with four items each. The scale was translated Missing Data
from English by native German speakers fluent in both lan-
guages. Participants rated their self-efficacy beliefs on a The rate of missing data was below 1% for each variable.
5-point Likert scale from 1 (= not at all well) to 5 (= very However, missing data were handled with MLR estimation
well). Due to a translation error, the word upset had been in the confirmatory factor analyses. Before creating mean
translated into the German word for sad in one ANG item scores for each scale to investigate convergent validity,
(How well can you avoid getting upset when others keep missing values were replaced using the regression proce-
giving you a hard time?). Thus, the translated item was dure in SPSS (IBM SPSS Statistics, 2010).
not adequate in terms of content validity to measure self-
efficacy in managing anger. This item was excluded from
the analyses. Results
Means and standard deviations for POS, DES, and ANG
Emotion Regulation Strategies were 4.16 (0.71), 3.09 (0.66), and 3.04 (0.70), respective-
ly. Cronbachs s were (POS) = .76, (DES) = .65 and
Reappraisal (six items) and suppression (four items) were
(ANG) = .50, while reliability for the summated scale
assessed using the Emotion Regulation Questionnaire
of self-efficacy in managing negative emotions was
(ERQ; Gross & John, 2003). All items were translated from
= .72.
English to German by a psychologist fluent in both lan-
guages. Participants rated their agreement on a 7-point Lik-
ert scale from 1 (= strongly disagree) to 7 (= strongly Confirmatory Factor Analyses
agree). Reliability and validity of the ERQ have been dem-
onstrated in several samples of college students (Gross & Three models were tested. Model 1 was a one-factor model.
John, 2003). Cronbachs s were .80 for reappraisal and Model 2 was a two-factor oblique model with NEG and
.78 for suppression. POS. Model 3 was a model which treated NEG as a sec-
ond-order factor represented by ANG and DES. As nega-
Life Satisfaction tive emotions encompass both high-activation negative
emotions and low-activation negative emotions (Lang,
Life satisfaction was assessed using a German version of 1995), loadings of DES and ANG on NEG were con-
the Satisfaction with Life Scale (SWLS; Slva, Baumann, strained to be equal. For information on the anchor items
& Lettner, 1995) by Diener, Emmons, Larsen, and Griffin used see Table 3.
(1985). The unidimensional nature, reliability, and validity Model 1 did not fit the data well (see Table 1). Model
of the SWLS have been confirmed in several studies (see 2 and Model 3 both showed a satisfying fit. However, the
Pavot & Diener, 1993). Participants answered five items on Satorra-Bentler test comparing Model 2 and Model 3
a 7-point Likert scale ranging from 1 (= strongly disagree) was significant, indicating that Model 3 provided a better
to 7 (= strongly agree). Cronbachs was. 81. fit to the data than Model 2, SB-(1) = 3.95, p < .05.
A modification index of 35.32 suggested a covariance be-
tween the errors of items 3 and 4. As those items both
Analytic Approach refer to the open expression of joy unrelated to gratifica-
tion due to personal achievement, this covariance was
specified (Model 3a). Standardized factor loadings of the
We conducted confirmatory factor analyses using Mplus
11 items on their respective factors ranged between .30
5.21 (Muthn & Muthn, 2009). Since the estimates of
and .82.
multivariate skewness (Srivastavas b1p = 79.35 (11); p <
.001) and kurtosis (Mardias b2p = 161.46, p < .001) re-
vealed that the data deviated from multivariate normal dis- Table 1. Fit indices of the confirmatory factor analyses of
tribution, we employed maximum-likelihood estimation the RESE scale
with restricted standard errors (MLR) and used Satorra- Model SB- dfa CFI RMSEA 90% CI SRMR
Bentler (S.-B.) scaled difference tests to compare nested Study 1
models (Satorra, 2000). Global model fit was evaluated us-
Model 1 595.49 44 .42 .16 [.15, .17] .14
ing root mean square error of approximation (RMSEA),
comparative fit index (CFI) and standardized root-mean- Model 2 135.26 43 .90 .07 [.05, .08] .05
square residual (SRMR). Models were accepted when Model 3 130.15 42 .91 .07 [.05, .08] .05
RMSEA was lower than .08 (Browne & Cudeck, 1993). Model 3a 96.67 41 .94 .05 [.04, .07] .05
For SRMR, a cutoff value of < .08 was used (Kelloway, Study 2
1998). Following OBoyle and Williams (2011), we used a Model 3a 54.66 32 .96 .05 [.03, .08] .06
two-stage cutoff criterion for CFI, with CFI > .90 indicating Note. aEach model indicated a significant with p < .001 due to the
an acceptable fit, and CFI > .95 indicating a good fit. large sample size.

2012 Hogrefe Publishing European Journal of Psychological Assessment 2013; Vol. 29(3):197204
200 C. Gunzenhauser et al.: Self-Efficacy in Regulating Emotions

Table 2. Correlations of POS, DES, and ANG (log-trans- in a German-speaking sample. Positive correlations of all
formed scores) with Life Satisfaction, Self-Es- RESE subscales with life satisfaction and reappraisal as
teem, and Emotion Regulation Strategies (male well as the negative correlation between POS and suppres-
and female participants combined) sion were in line with our expectations. Unexpectedly,
however, ANG was positively correlated with suppres-
Indicator M SD POS DES ANG
sion. Moreover, internal consistencies were not entirely
Life satisfaction
satisfying. In particular, the reliability of the ANG sub-
Study 1 4.92 1.06 .36** .41** .20** scale was low.
Study 2 7.51 1.67 .21** .24** .14*
Reappraisal
Study 1 4.55 0.98 .19** .24** .16**
Study 2 4.45 1.16 .26** .14* .22** Study 2
Suppression
Study 1 3.28 1.27 .38** .03 .16** In Study 2, self-efficacy beliefs in regulating emotions
Study 2 3.07 1.10 .23** .03 .07 were assessed in parents as a part of a larger study on child
Notes. Pearsons correlation coefficients. *p < .05, **p < .01. development. Here, we used a revised version of the Ger-
man RESE scale (RESE-R) for two reasons. First, given
Convergent Validity the unsatisfactory reliabilities found in Study 1, the Ger-
man wording of ANG and DES items had to be revised.
For an evaluation of the convergent validity in terms of In contrast to Study 1, items were translated from Italian
relations to other constructs, we correlated POS, DES, into German. Second, recent research had led to a revision
ANG, and NEG with life satisfaction, reappraisal, and sup- of the original RESE scale (Caprara, Di Giunta, Pastorelli,
pression. Log transformation was performed on all RESE & Eisenberg, 2011). The revised RESE scale comprises
subscales to approach normal distribution. Results are pre- 10 of the 12 items of the original scale (POS: 4 items,
sented in Table 2. DES: 3 items, ANG: 3 items), which we included in the
German RESE-R (see Table 3). We aimed to document
that improved item wording would provide satisfactory
Conclusion reliabilities, while preserving the factor structure and the
correlations to life satisfaction and emotion-regulation
In Study 1 the factor structure previously found in Italy, strategies. Moreover, we aimed to test for cross-gender
Bolivia, and the United States was successfully replicated invariance of the revised scale.

Table 3. Scale items of the German RESE-R (German text in italics)

Scale How well can you . . ./Wie gut gelingt es Ihnen, . . .
Self-efficacy in ex- (1) . . . rejoice over your successes? / . . . sich ber Ihre Erfolge zu freuen?
pressing positive (2) . . . feel gratified over achieving what you set out to do? / . . . zufrieden zu sein, wenn Sie das erreicht haben, was Sie
emotions sich vorgenommen hatten?
(3) . . . express joy when good things happen to you? / . . . Freude zu zeigen, wenn Ihnen etwas Gutes passiert?
(4) . . . express enjoyment freely at parties? / . . . auf Partys offen zu zeigen, dass Sie Spa haben?
Self-efficacy in (1) . . . keep from getting dejected when you are lonely? / . . . sich vor Niedergeschlagenheit zu schtzen, wenn die Perso-
regulating despon- nen, die Ihnen wichtig sind, Ihnen nicht beistehen knnen, wenn Sie sie brauchen?
dency/distress (2) . . . keep from getting discouraged in the face of difficulties? / . . . sich von Schwierigkeiten nicht entmutigen zu las-
sen?
(3) . . . keep from getting discouraged by strong criticism? / . . . nicht den Mut zu verlieren, wenn Sie stark kritisiert wer-
den?
Self-efficacy in (1) . . . avoid flying off the handle when you get angry? / . . . zu verhindern, dass Sie die Kontrolle ber Ihr Handeln ver-
regulating anger/ lieren, wenn Sie wtend sind?
irritation (2) . . . get over irritation quickly for wrongs you have experienced? / . . . ber rger hinwegzukommen, wenn Ihnen Un-
recht getan wurde?
(3) . . . avoid getting upset when others keep giving you a hard time?/. . . Ihre Wut im Zaum zu halten, wenn andere Men-
schen Ihnen das Leben schwer machen?
Notes. German items refer to items used in Study 2 (German RESE-R). English items refer to corresponding items published by Caprara et al.
(2008). Anchor items in Study 2 were items 1 of each subscale. In Study 1, ANG and DES subscales contained one additional item respectively
(ANG: manage negative feelings when reprimanded by your parents or significant others; DES: reduce your upset when you dont get the
appreciation you feel you deserve?). Anchor items in Study 1 were items 1 of the POS and DES subscales and the ANG item not used in
Study 2.

European Journal of Psychological Assessment 2013; Vol. 29(3):197204 2012 Hogrefe Publishing
 C. Gunzenhauser et al.: Self-Efficacy in Regulating Emotions
Materials and Method
Participants
Participants were N = 144 mothers and N = 120 fathers
(including 118 couples) of preschool children in Southern
Germany. Participants were between 22 and 59 years old
(women: MAge = 38, SD = 5.44; men: MAge = 41, SD = 5.43;
based on information on 138 women and 117 men). Some
52% of the women and 64% of the men held at least a
college degree (based on information on 126 women and
110 men). Participants received a EUR 5 gift certificate
from a book store.
Measures
Self-Efficacy in Regulating Emotions
POS items were adopted from Study 1. DES and ANG
items were translated from Italian to German by a native
German speaker trained in psychology and backtranslated
by a professional translator.
Life Satisfaction
A single-item measure was preferred because participants
were required to complete several questionnaires for the
larger study. Therefore, life satisfaction was assessed with
the item How satisfied are you currently with your life as
a whole? rated on an 11-point Likert scale from 0 (= totally
dissatisfied) to 10 (= totally satisfied). This item has proved
to be a valid measure (Fujita & Diener, 2005).
Emotion Regulation Strategies
Study 2 used a recently validated German version of the
ERQ (Abler and Kessler, 2009). Again, participants used a
7-point Likert scale from 1 (= strongly disagree) to 7 (=
strongly agree). Cronbachs s were .82 for reappraisal and
.61 for suppression.
Analytic Approach
We tested the best-fitting model from Study 1 (i.e., model
3a) with the data from our total sample. Again, we used
MLR estimation to account for multivariate nonnormality
(skewness: Srivastavas b1p = 27.48 (10), p < .001; kurtosis
(Mardias b2p = 131.08, p < .001). Since our sample con-
tained parent couples, we used the clustering procedure
available in Mplus 5.21 (Muthn & Muthn, 2009). Cross-
gender measurement invariance was examined using mul-
tigroup analyses. Following the stepwise procedure sug-
gested by Chen, Sousa, and West (2005), we tested a series
of nested models to assess configural, metric, and scalar
invariance. According to Steenkamp and Baumgartner
(1998), at least one item besides the anchor item needs to
be invariant on each factor to allow for comparisons of
2012 Hogrefe Publishing
201
means between groups (partial invariance). Evaluation of
model fit and treatment of missing data followed the pro-
cedures described in Study 1. The rate of missing data was
below 4% per variable.
Results
Means and standard deviations for DES, ANG, and POS
were 3.40 (0.68), 3.15 (0.75) and 4.25 (0.58), respectively.
The internal consistencies of the summated scales were
(POS) = .79, (DES) = .69, (ANG) = .68, and (NEG)
(6 items) = .72.
Factor Structure and Cross-Gender Invariance
Model 3a from Study 1 represented a good model-data fit
(see Table 1 for fit indices and Table 3 for information on
anchor items). We then tested this model in a multiple-
group approach with gender as a group variable. Fit indices
were acceptable, SB- (64) = 100.12, CFI = .94, RMSEA
= .07, 90% CI [.04, .09], SRMR = .07. Thus, configural
invariance was confirmed. When item loadings were con-
strained to be equal across gender, the change in Satorra-
Bentler was not significant, SB- (7) = 13.28, p = .07.
Also, after imposing additional equality constraints for the
loadings of ANG and DES on the second-order factor
NEG, the model fit did not change significantly. Next, item
intercepts were constrained to be equal across gender. Here,
the test indicated that the model fit decreased significantly,
SB- (7) = 18.85, p < .01. When equality constraints
were lifted from item POS/3, the change in became non-
significant, SB- (6) = 11.26, p = .08. Last, equality con-
straints were imposed on the intercepts of ANG and DES,
which led to a significant change in model fit, SB- (2)
= 23.85, p < .001. When we tested partial scalar invariance
on this level by lifting the equality constraint from the in-
tercept of DES, the change in became nonsignificant,
SB- (1) = 1.47, p = .23. The fit indices for the second-
order partial scalar invariance model were SB- (79) =
127.70, CFI = .92, RMSEA = .07, 90% CI [.05, .09], and
SRMR = .11.
Convergent Validity
As in Study 1, we correlated log-transformed RESE-R sub-
scales with life satisfaction, reappraisal, and suppression.
The results are shown in Table 2. Correlation coefficients
refer to the total sample.
Conclusion
Study 2 examined the German RESE-R in a nonstudent
sample. The factor structure found in Study 1 was success-
European Journal of Psychological Assessment 2013; Vol. 29(3):197204
202 C. Gunzenhauser et al.: Self-Efficacy in Regulating Emotions
fully replicated. Internal consistencies were satisfying for
all subscales. Full scalar invariance was supported for the
ANG and DES factors. POS and the second-order factor
NEG were found to be partially invariant. In line with our
suggestions, we found positive associations of the RESE
subscales with life satisfaction and reappraisal. Moreover,
POS was negatively related to suppression.
Discussion
The goals of this research were to investigate the reliability,
factor structure, and convergent validity of the German
RESE scale, and to examine cross-gender invariance of the
scale for German speakers. In Studies 1 and 2, CFAs re-
vealed that self-efficacy in regulating emotions can be
meaningfully described as consisting of two factors, self-
efficacy in managing negative emotions (NEG) and self-
efficacy in expressing positive emotions (POS). NEG was
represented by two first order factors: self-efficacy in man-
aging despondency/distress (DES) and self-efficacy in
managing anger/irritation (ANG). These findings are con-
sistent with those reported by Caprara et al. (2008) for Bo-
livian, U. S., and Italian samples. For statistical reasons, a
model containing POS, DES, and ANG as three correlated
factors would provide a similar fit as the model with a sec-
ond-order factor NEG. However, the latter model includes
the theoretically meaningful distinction between positive
and negative emotions which is in line with previous theo-
rizing and findings (Caprara et al., 2008). Internal consis-
tencies could be substantially improved by the revised Ger-
man RESE scale (RESE-R) used in Study 2. They were
comparable to the internal consistencies of the Italian,
Spanish, and English versions of the scales, which range
from .64 to .85 (Caprara et al., 2008). Partial scalar cross-
gender invariance of the German RESE-R was demonstrat-
ed in Study 2. Equality constraints could be maintained for
all items of the DES and ANG subscales, and for all but
one item of the POS subscale. According to Steenkamp and
Baumgartner (1998), cross-gender comparisons of mean
values of the POS, DES, and ANG factors are thus mean-
ingful. Nevertheless, caution is necessary because estimat-
ed factor mean differences may differ depending on the
anchor indicators chosen for the factor models (Vanden-
berg, 2002). Convergent validity of the RESE scale was
investigated and confirmed in both studies. The significant
positive correlations of POS, DES, and ANG with life sat-
isfaction and reappraisal were consistent with our hypoth-
eses. Furthermore, the suggested negative correlation be-
tween POS and suppression was supported by the data. In
Study 1, we found an unexpected positive correlation be-
tween ANG and suppression. This might be due to the item
that was excluded in Study 2 (managing negative feelings
when reprimanded by parents), which may be related to
suppression of anger in asymmetric relationships. Howev-
er, considering the low internal consistency of ANG in
European Journal of Psychological Assessment 2013; Vol. 29(3):197204
Study 1, caution is warranted when interpreting this result.
In Study 2, the association between ANG and suppression
was not significant.
Limitations and Next Steps
Whereas participants in Study 1 as well as participants in
previous studies in other countries (Caprara et al., 2008)
were young adults in their early twenties, Study 2 covered
a sample of adults who were the parents of preschool chil-
dren. Basically, the general factor structure and the pattern
of correlations found in Study 1 were replicated in Study
2. These findings indicate that the results concerning relia-
bilities and cross-gender invariance of the German RESE-
R found in Study 2 might also apply to a population of
younger adults. However, future studies should further ex-
plore the validity of the German RESE-R across samples
varying in age and demographic background. In particular,
caution is necessary when administering the scale to per-
sons from different cultural backgrounds (Trommsdorff &
Rothbaum, 2008). Moreover, the present research did not
uncover the sources of the multivariate skewness of the
RESE item distributions. Further studies should investigate
whether scores might be biased by social desirability. Fi-
nally, even though the factor structure found in Italy, Bo-
livia, and the United States was replicated, systematic
crosscountry investigations are needed.
Conclusions
Findings suggest that the RESE scale is a valid instrument
to assess self-efficacy in expressing positive emotions and
regulating negative ones in both men and women in Ger-
man-speaking samples. As the revised version of the Ger-
man scale showed better internal consistencies than the ver-
sion of the scale used in Study 1, we recommend using the
German RESE-R.
Acknowledgments
The authors contributions are acknowledged in alphabet-
ical order: Development of the study design and selection
of instruments were carried out by Gian Vittorio Caprara,
Maria Gerbino, Catherine Gunzenhauser, Tobias Heikamp,
Antje von Suchodoletz, and Gisela Trommsdorff. Data col-
lection was conducted by Catherine Gunzenhauser, Tobias
Heikamp, and Antje von Suchodoletz. Statistical analyses
were performed by Guido Alessandri and Catherine Gun-
zenhauser. The article was composed and revised by Guido
Alessandri, Gian Vittorio Caprara, Laura Di Giunta, Maria
Gerbino, Catherine Gunzenhauser, Tobias Heikamp, Antje
von Suchodoletz, and Gisela Trommsdorff. Study 1 was
2012 Hogrefe Publishing
 C. Gunzenhauser et al.: Self-Efficacy in Regulating Emotions
financed by a grant from the German Research Foundation
(DFG) as a part of the project Developmental Conditions
of Intentionality (Principal Investigator: Prof. Dr. Gisela
Trommsdorff, DFG GZ, TR 169/14-2) within the interdis-
ciplinary research group Limits of Intentionality (DFG
Research Unit 582) at the University of Konstanz, Germa-
ny. Study 2 was financed by grants given to the Research
Group The Empirics of Education: Economic and Behav-
ioral Perspectives in the context of the German Initiative
of Excellence at the University of Freiburg, Germany. We
thank Eva Grunenberg, Natalie Hoffmann, and Antonia
Skock for their great help with data collection and data
preparation.
References
Abler, B., & Kessler, H. (2009). Emotion Regulation Question-
naire Eine deutschsprachige Fassung des ERQ von Gross
und John [Emotion Regulation Questionnaire A German ver-
sion of the ERQ by Gross and John]. Diagnostica, 55,
144152. doi 10.1026/00121924.55.3.144
Bandura, A. (1997). Self-efficacy: The exercise of control. New
York: Freeman.
Browne, M. W., & Cudeck, R. (1993). Alternative ways of assess-
ing model fit. In K. A. Bollen & S. J. Long (Eds.), Testing
structural equation models (pp. 136162). Newbury Park, CA:
Sage. doi 10.1177/0049124192021002005
Caprara, G. V., Di Giunta, L., Eisenberg, N., Gerbino, M., Pasto-
relli C., & Tramontano, C. (2008). Assessing regulatory emo-
tional self-efficacy in three countries. Psychological Assess-
ment, 20, 227237. doi 10.1037/1040-3590.20.3.227
Caprara, G. V., Di Giunta, L., Pastorelli, C., & Eisenberg, N.
(2011). Mastery of negative affect: The hierarchical structure
of self-efficacy beliefs related to discrete negative emotions.
Manuscript submitted for publication.
Caprara, G. V., & Gerbino, M. (2001). Autoefficacia Emotiva Per-
cepita: La capacit di regolare laffetivit negative e di esprimere
quella positive [Affective perceived self-efficacy: The perceived
capacity to regulate negative affect and to express positive af-
fect]. In G. V. Caprara (Ed.), La valutazione dellautoefficacia
(pp. 3550). Trento, Italy: Edizioni Erickson.
Caprara, G., & Steca, P. (2005). Self-efficacy beliefs as determi-
nants of prosocial behavior conductive to life satisfaction
across ages. Journal of Social and Clinical Psychology, 24,
191217. doi 10.1521/jscp.24.2.191.62271
Chen, F. F., Sousa, K. H., & West, S. G. (2005). Testing measure-
ment invariance of second-order factor models. Structural Equa-
tion Modeling, 12, 471492. doi 10.1207/s15328007sem1203_7
Davidov, M., & Grusec, J. E. (2006). Untangling the links of pa-
rental responsiveness to distress and warmth to child out-
comes. Child Development, 77, 4458. doi 10.1111/j.1467-
8624.2006.00855.x
Diener, E., Emmons, R. A., Larsen, R. J., & Griffin, S. (1985). The
Satisfaction with Life Scale. Journal of Personality Assess-
ment, 49, 7175. doi 10.1207/s15327752jpa4901_13
Eisenberg, N., Cumberland, A., Spinrad, T. L., Fabes, R. A., Shep-
ard, S. A., Reiser, M., . . . Guthrie, I. K. (2001). The relations
of regulation and emotionality to childrens externalizing and
2012 Hogrefe Publishing
203
internalizing problem behavior. Child Development, 72,
11121134. doi 10.1111/1467-8624.00337
Fujita, F., & Diener, E. (2005). Life satisfaction set point: Stability
and change. Journal of Personality and Social Psychology, 88,
158164. doi 10.1037/00223514.88.1.158.
Gross, J. J., & John, O. P. (2003). Individual differences in two
emotion regulation processes: Implications for affect, relation-
ships, and well-being. Journal of Personality and Social Psy-
chology, 85, 348362. doi 10.1037/0022-3514.85.2.348
Gross, J. J., Richards, J. M., & John, O. P. (2006). Emotion regu-
lation in everyday life. In D. K. Snyder, J. Simpson, & J. N.
Hughes (Eds.), Emotion regulation in couples and families:
Pathways to dysfunction and health (pp. 1335). Washington,
DC: American Psychological Association. doi 10.1037/11468-
001037/11468-001
IBM SPSS Statistics. (2010). SPSS Statistics version 19.0 [Com-
puter software]. Somers, NY: IBM Corporation.
Kelloway, E. K. (1998). Using LISREL for structural equation
modeling: A researchers guide. Thousand Oaks, CA: Sage.
Knight, G. P., Guthrie, I. K., Page, M. C., & Fabes, R. A. (2002).
Emotional arousal and gender differences in aggression: A
meta-analysis. Aggressive Behavior, 28, 366393. doi 10.
1002/ab.80011
Lang, P. L. (1995). The emotion probe. American Psychologist,
50, 372385. doi 10.1037//0003-066X.50.5.372
Lyubomirsky, S., King, L., & Diener, E. (2005). The benefits of
frequent positive affect: Does happiness lead to success? Psy-
chological Bulletin, 131, 803855. doi 10.1037/0033-
2909.131.6.803
Muthn, B., & Muthn, L. (2009). Mplus (Version 5.21) [Com-
puter software]. Los Angeles, CA: Authors.
Nolen-Hoeksema, S., & Aldao, A. (2011). Gender and age differ-
ences in emotion-regulation strategies and their relationship to
depressive symptoms. Personality and Individual Differences,
51, 704708. doi 10.1016/j.paid.2011.06.012
OBoyle, E. H., & Williams, L. J. (2011). Decomposing model fit:
Measurement vs. theory in organizational research using latent
variables. Journal of Applied Psychology, 96, 112. doi
10.1037/a0020539
Pavot, W., & Diener, E. (1993). Review of the Satisfaction with
Life Scale. Psychological Assessment, 5, 164172. doi
10.1037/1040-3590.5.2.164
Satorra, A. (2000). Scaled and adjusted restricted tests in multi-
sample analysis of moment structures. In R. D. H. Heijmans,
D. S. G. Pollock, & A. Satorra (Eds.), Innovations in multivar-
iate statistical analysis: A Festschrift for Heinz Neudecker
(pp. 233247). London: Kluwer. doi 10.2139/ssrn.189431
Slva, M., Baumann, U., & Lettner, K. (1995). Wohlbefinden:
Definitionen, Operationalisierungen, empirische Befunde
[Well-being: Definitions, operationalizations, empirical re-
sults]. Zeitschrift fr Gesundheitspsychologie, 3, 292309. doi
10.1024/1421-0185/a000003
Sontag, L. M., & Graber, J. A. (2010). Coping with perceived peer
stress: Gender-specific and common pathways to symptoms of
psychopathology. Developmental Psychology, 46, 16051620.
doi 10.1037/a0020617
Steenkamp, J. E. N., & Baumgartner, H. (1998). Assessing meas-
urement invariance in cross-national consumer research. Jour-
nal of Consumer Research, 25, 7890. doi 10.1086/209528
Tamres, L. K., Janicki, D., & Helgeson, V. S. (2002). Sex differ-
ences in coping behavior: A meta-analytic review and an ex-
European Journal of Psychological Assessment 2013; Vol. 29(3):197204
204 C. Gunzenhauser et al.: Self-Efficacy in Regulating Emotions

amination of relative coping. Personality and Social Psychol- Published online: July 20, 2012
ogy Review, 6, 230. doi 10.1207/S15327957PSPR0601_1
Trommsdorff, G., & Rothbaum, F. (2008). Development of
emotion regulation in cultural context. In M. Vandekerckhove, Catherine Gunzenhauser
C. v. Scheve, S. Ismer, S. Jung, & S. Kronast (Eds.), Regulating
emotions: Culture, social necessity, and biological inheritance Research Group The Empirics of Education
(pp. 85120). Malden, MA: Blackwell. doi 10.1002/ University of Freiburg
9781444301786.ch4 Starkenstr. 44
Vandenberg, R. J. (2002). Toward a further understanding of im- 79085 Freiburg
provement in measurement invariance methods and proce- Germany
dures. Organizational Research Methods, 5, 139158. doi Tel. +49 761 2039-7589
10.1177/1094428102005002001 Fax +49 761 2039-7592
E-mail catherine.gunzenhauser@psychologie.uni-freiburg.de

European Journal of Psychological Assessment 2013; Vol. 29(3):197204 2012 Hogrefe Publishing
 L. Kriston
European et al.:
Journalof Reliability and
Psychological Validity
Assessment of theVol.
2013;
2012 German
Hogrefe YSQ-S3
29(3):205212
Publishing

Original Article

Reliability and Validity of the

German Version of the
Young Schema Questionnaire
Short Form 3 (YSQ-S3)
Levente Kriston1, Janina Schfer2, Gitta A. Jacob2,3,
Martin Hrter1, and Lars P. Hlzel2
1
Department of Medical Psychology, University Medical Center Hamburg-Eppendorf, Hamburg,
Germany, 2Department of Psychiatry and Psychotherapy, University Medical Center Freiburg, Freiburg
i. Br., Germany, 3Institute of Psychology, University of Freiburg, Freiburg i. Br., Germany

Abstract. To date, the psychometric properties of the German version of the Young Schema Questionnaire Short Form 3 (YSQ-S3)
have not yet been examined. We investigated the reliability and validity of the German version of the YSQ-S3. In a community sample
of 1,150 participants and a clinical sample of 30 psychiatric inpatients, we performed reliability analysis, confirmatory factor analysis,
and correlation analyses, and tested for group differences using analysis of variance. The YSQ-S3 proved to be reliable and corresponded
to the theoretically proposed 18-dimensional structure. Schema scores were positively associated with measures of psychopathology and
personality disorder, indicating convergent validity. Furthermore, the YSQ-S3 differentiated between participant subgroups defined by
level of health-care utilization, supporting discriminant validity. We conclude that the YSQ-S3 is a psychometrically sound instrument
that can be used in German-speaking countries in research on early maladaptive schemas. Further research is necessary particularly in
larger clinical samples.

Keywords: early maladaptive schema, Young Schema Questionnaire, factor structure, reliability, validity

Introduction EMS are usually assessed with self-report question-

Schema therapy is a psychotherapy approach combining
traditional cognitive-behavioral therapy with elements of
psychodynamic approaches, Gestalt therapy, and humanis-
tic therapies (Young, Klosko, & Weishaar, 2003). It recent-
ly gained increased attention, since outcome studies dem-
onstrated its efficacy and effectiveness in patients with per-
sonality disorders, mainly borderline personality disorder
(Farrell, Shaw, & Webber, 2009; Giessen-Bloo et al., 2006;
Gude & Hoffart, 2008; Nadort et al., 2009; Nordahl &
Nysaeter, 2005).
Central to schema therapy is the concept of early mal-
adaptive schemas (EMS), thought to develop in childhood
when essential needs of the child remain unmet. EMS com-
prise feelings, cognitions, bodily sensations, and coping be-
haviors (Young et al., 2003). Young et al. defined 18 sche-
mas, clustered in five schema domains, each of which is
considered to be connected with specific unmet childhood
needs (see Table 1).
naires, mainly the Young Schema Questionnaire (YSQ).
The original version of this instrument was developed by
Young (1990) to assess 16 schemas. It consists of 205
items, and it was shown to be reliable and valid in large
clinical and student samples (Schmidt, Joiner, Young, &
Telch, 1995). Based on the findings of Schmidt et al.
(1995), Young and Brown (1994) developed a short form
of the YSQ comprising five items for each of the 16 EMS.
The psychometric properties of various translations of both
YSQ versions have been investigated in several studies
(overview in Oei & Baranoff, 2007). In their review, Oei
and Baranoff (2007) conclude that internal consistency,
test-retest reliability, convergent validity, and discriminant
validity of the YSQ are sufficient to warrant the use of the
YSQ in research settings. However, since findings on the
factorial structure are conflicting, the relevance of further
research was highlighted (Oei & Baranoff, 2007).
The short form of the YSQ has been revised repeatedly.
In its latest form, the YSQ-S3, it comprises 90 items as-

2012 Hogrefe Publishing European Journal of Psychological Assessment 2013; Vol. 29(3):205212
DOI: 10.1027/1015-5759/a000143
206 L. Kriston et al.: Reliability and Validity of the German YSQ-S3

sessing 18 schemas with five items each (Young, 2005). To Instruments

date, four studies have tested the psychometric qualities of
the YSQ-S3 in four different languages (Portuguese: Rijo The YSQ-S3 (Young, 2006) is a self-report instrument. Peo-
& Gouveia, 2008; Finnish: Saariaho, Saariaho, Karila, & ple are asked to describe themselves by rating descriptive
Joukamaa, 2009; Turkish: Soygt, Karaosmano6lu, & Ca- statements through a 6-step Likert-type response format
kir, 2009; Romanian: Trip, 2006). All studies found good ranging from completely untrue of me to describes me per-
discriminant validity with regard to group differences (pa- fectly. Higher values indicate a stronger presence of the re-
tient groups always scored higher than psychologically spective schema. The YSQ-S3 assesses 18 EMS (see Table
healthy groups) and correlations with other measures of 1) with five items per scale, resulting in a total of 90 items.
psychopathology, but results regarding the factorial struc-
ture were less conclusive. Trip (2006) found high internal
consistency of all subscales in a Romanian version. Rijo Table 1. Schemas, schema domains, and associated needs
and Gouveia (2008) confirmed the proposed factorial struc- Schema domains Associated needs Schemas
ture in a Portuguese sample by confirmatory factor analy- Disconnection Safe attachment, Emotional deprivation
sis; however, six items had to be removed due to substantial acceptance, nurtur- Abandonment/Instability
crossloadings. With a Turkish version, Soygt et al. (2009) ing, protection Mistrust/Abuse
found 14 of the proposed 18 factors in a principal compo- Social isolation/Alienation
nents analysis. Saariaho et al. (2009) tested a Finnish trans- Defectiveness/Shame
lation of the YSQ-S3 and verified the 18-factor structure Impaired autono- Autonomy, compe- Failure to achieve
using confirmatory factor analysis. Furthermore, internal my and achieve- tency, identity Dependence/Incompetence
ment Vulnerability to harm or ill-
consistency of all subscales was high. Putting together the ness
findings from theses studies suggests that the YSQ-S3 is a Enmeshment/Undeveloped
psychometrically sound instrument. self
So far, the German translation of the YSQ-S3 (Young, Impaired limits Realistic limits and Entitlement/Grandiosity
2006; German translation: Berbalk, Grutschpalk, Parfy, & self-control Insufficient self-control
Zarbock) has yet to be tested for psychometric properties. Other-directed- Free expression of Subjugation
An increasing number of both schema therapy studies in ness needs and emotions Self-sacrifice
the German-speaking region and international schema ther- Approval-seeking
apy studies with German-speaking centers, however, call Exaggerated vigi- Spontaneity and Emotional inhibition
for a psychometric test of the German YSQ-S3. With this lance and inhibi- play Unrelenting standards
tion Negativity/Pessimism
study we examined the reliability as well as factorial, con- Punitiveness
vergent, and discriminant validity of the German YSQ-S3.

A short-form of the Symptom Checklist 90-R (SCL-90-R;

Materials and Methods
Sampling Procedure
Cross-sectional data were collected in a convenience com-
munity sample and a clinical sample. Data in the commu-
nity sample were collected via internet from July 14 to Au-
gust 13, 2009. The survey link was distributed via email to
psychology faculties at German universities, several inter-
net forums, and the of authors surroundings. Participants
did not get incentives for study participation, and no per-
sonal data (e.g., IP-address) were recorded.
Data from the clinical sample were collected among in-
patients of the psychiatric Department of a university med-
ical center between August 14 and September 25, 2009.
Patients gave written informed consent and were then
asked to fill out a paper-and-pencil version of the instru-
ments used in the online survey. Completed questionnaires
were collected in envelopes that were closed by the patients
themselves to ensure anonymity.
The study protocol was approved by the local ethics
committee.
Derogatis, 1992), the SCL-K-9 (Klaghofer & Brhler, 2001),
was administered as a global measure of psychological dis-
tress. Participants rated the intensity of psychopathological
symptoms within the past week on nine 5-step Likert-scaled
items. The SCL-K9 measures a single dimension of general
symptom severity and correlates with the General Severity
Index (GSI) of the SCL-90-R to a degree of 0.93 with a Cron-
bachs of 0.87 (Klaghofer & Brhler, 2001).
Furthermore, a brief self-rating instrument to assess
symptoms of personality disorders was used. The Standard-
ized Assessment of Personality Abbreviated Scale (SA-
PAS; Moran et al., 2003) contains eight items with a binary
response format. Germans, van Heck, Moran, and Hodia-
mont (2008) used it as a self-rating instrument and found a
good test-retest reliability of 0.89 and a Cronbachs of
0.45. Discriminant validity was sufficient with a correct
classification rate of 81% in subjects with and without per-
sonality disorders.
Additionally, self-reported demographic (sex, age,
family status, level of education, employment status) and
clinical data (self-reported diagnosis of a mental disorder
and past health care utilization due to a mental disorder)
were collected.

European Journal of Psychological Assessment 2013; Vol. 29(3):205212 2012 Hogrefe Publishing
 L. Kriston et al.: Reliability and Validity of the German YSQ-S3 207

Statistical Analysis rather than the correlation matrix because the latter is like-
ly to lead to statistical and generalizability problems (e.g.,
Due to the limited size of the clinical sample, analysis of Cudeck, 1989; Hair, Black, Babin, & Anderson, 2009).
reliability as well as factorial and convergent validity was Although the responses to the items of the YSQ-S3 should
calculated only with data from the community sample. be considered ordered categorical rather than continuous,
Reliability was estimated by calculating item discrimina- we applied the maximum likelihood estimator, since it
tion statistics and Cronbachs for each scale. Factorial was shown to be robust in similar situations. Furthermore,
validity was analyzed with confirmatory factor analysis. alternative estimation methods would have required a
The predefined structure consisted of 18 oblique (corre- considerably larger sample size (e.g., Green, Akey, Flem-
lated) factors (representing schemas) and uncorrelated er- ing, Hershberger, & Marquis, 1997; Muthn & Kaplan,
rors corresponding to Youngs model. The covariance ma- 1985; Tabachnick & Fidell, 2007). Local fit was investi-
trix of the items was analyzed by means of maximum like- gated by examining factor loadings, factor reliabilities,
lihood estimation. We preferred to analyze the covariance average extracted variance in items, and congruence (cor-
relation) between factor scores and corresponding scale
Table 2. Sample characteristics sum scores. According to expert recommendations (e.g.,
Hair et al., 2009; Kline, 2010; Schweizer, 2010), we as-
Community Clinical sessed global model fit using the discrepancy statistic,
sample sample the normed statistic, the Bentler comparative fit index
n = 1,150 n = 30 (CFI), the root mean square error of approximation
Sex; n (%) (RMSEA), and the standardized root mean square residual
female 899 (78.2) 19 (67.9) (SRMR). A more detailed evaluation of the latent factorial
male 251 (21.8) 9 (32.1) structure of the YSQ-S3 was reported by Kriston, Schfer,
Age; years Hrter, and Hlzel (2010). Convergent validity was esti-
mean (SD) 29.3 (10.1) 40.5 (11.4)
mated via Pearson correlation coefficients between sche-
ma scales and general distress (SCL-K-9) as well as symp-
Family status; n (%)
toms of personality disorders (SAPAS), respectively.
single 896 (77.9) 17 (56.7) Discriminant validity was examined by comparing
married 188 (16.3) 9 (30.0) schema scale scores of the clinical and the community
separated 15 (1.3) 2 (6.7) sample and by comparing subjects with different intensi-
divorced 48 (4.2) 2 (6.7) ty of health care utilization within the community sample
widowed 3 (0.3) 0 (0) (former counseling, former outpatient treatment, former
Level of education; n (%) inpatient treatment due to a mental disorder). Mean sche-
ma scale scores were compared across groups by univar-
low 29 (2.5) 5 (16.7)
iate analysis of variance (ANOVA). All analyses were
middle 223 (19.4) 12 (40.0)
performed using SPSS 15.0 (SPSS Inc., Chicago, IL) and
high 898 (78.1) 13 (43.3) Mplus 5.1 (Muthn & Muthn, 1998).
Occupational status; n (%)
employed 451 (39.2) 9 (33.3)
homemaker 23 (2.0) 1 (3.7)
pensioner 23 (2.0) 5 (18.5) Results
student 557 (48.4) 3 (11.1)
unemployed 35 (3.0) 9 (33.3)
Participant Characteristics
other 61 (5.3) 0 (0) Data were collected from 1,150 online survey participants
Lifetime diagnosis of a mental health disorder; n (%) (community sample) and 30 psychiatric patients (clinical
yes 244 (21.2) sample). Sample characteristics are given in Table 2.
no 906 (78.8) Community sample participants were mostly female,
Health care utilization due to mental health disorder; n (%)a rather young, and highly educated. One fifth reported a life-
none 755 (65.7) time diagnosis of a mental disorder. The most frequent en-
counseling 220 (19.1)
tries were affective disorder (n = 134), anxiety disorder (n
= 43), eating disorder (n = 32), and borderline personality
outpatient treatment 229 (19.9)
disorder (n = 25).
inpatient treatment 81 (7.0) The majority of the clinical sample was female. Their
Notes. Missing data were not present in the community sample. Valid mean age was 40 years. Most patients had a medium to high
sample size varies between 27 and 30 in the clinical sample because
of missing data (valid percentages are reported). Data on lifetime level of education and around one third were employed.
mental health-disorder diagnosis and health-care utilization were not The reasons for inpatient treatment (including comorbid
collected in the clinical sample. amultiple selections possible. diseases) were affective disorder (n = 17), psychotic disor-

2012 Hogrefe Publishing European Journal of Psychological Assessment 2013; Vol. 29(3):205212
208 L. Kriston et al.: Reliability and Validity of the German YSQ-S3

Table 3. Results of the reliability analysis

Corrected item-total correlations
Scale (schema) 1st item 2nd item 3rd item 4th item 5th item
Emotional deprivation 0.70 0.74 0.70 0.80 0.60 0.88
Abandonment/Instability 0.67 0.71 0.76 0.63 0.67 0.87
Mistrust/Abuse 0.52 0.62 0.66 0.72 0.58 0.82
Social isolation/Alienation 0.77 0.61 0.82 0.79 0.81 0.90
Defectiveness/Shame 0.75 0.84 0.79 0.79 0.74 0.91
Failure to achieve 0.73 0.50 0.79 0.78 0.74 0.87
Dependence/Incompetence 0.63 0.61 0.58 0.55 0.71 0.82
Vulnerability to harm or illness 0.61 0.70 0.44 0.47 0.54 0.77
Enmeshment/Undeveloped self 0.57 0.57 0.43 0.60 0.59 0.78
Subjugation 0.57 0.66 0.53 0.61 0.68 0.82
Self-sacrifice 0.53 0.51 0.67 0.55 0.63 0.80
Emotional inhibition 0.71 0.73 0.58 0.68 0.77 0.87
Unrelenting standards 0.52 0.57 0.51 0.54 0.36 0.74
Entitlement/Grandiosity 0.34 0.55 0.30 0.51 0.43 0.67
Insufficient self-control 0.61 0.57 0.52 0.61 0.56 0.80
Approval-seeking 0.42 0.62 0.62 0.60 0.58 0.79
Negativity/Pessimism 0.71 0.72 0.71 0.58 0.65 0.86
Punitiveness 0.54 0.44 0.45 0.47 0.39 0.70
Note. refers to Cronbachs .

der (n = 6), anxiety disorder (n = 4), borderline personality
disorder (n = 4), obsessive-compulsive disorder (n = 2),
posttraumatic stress disorder (n = 2), attention deficit
hyperactivity disorder (n = 2), substance dependence (n =
1), and somatoform disorder (n = 1).
Reliability
Results of the reliability analysis are shown in Table 3. Item
discriminations (corrected item-total correlations) exceed-
ed the recommended threshold of 0.40 in all but four cases
(one item of the scale Unrelenting Standards, two items of
the scale Entitlement/Grandiosity, and one item of the scale
Punitiveness). Internal consistency of 17 of the 18 scales
was sufficient (Cronbachs > 0.70). Internal consistency
of the scale Entitlement/Grandiosity was 0.67.
Factorial Validity
Factorial validity results are reported in Table 4. Loadings
of all but three items exceeded the required threshold of
0.40 (one item of the scale Entitlement/Grandiosity, two
items of the scale Punitiveness). Factor reliability was sat-
isfactory (> 0.70) in all scales except Entitlement/Grandi-
osity. Average extracted variance failed to reach the de-
sired threshold of 0.50 but was still above 0.40 for the
scales Vulnerability to Harm or Illness, Enmeshment/Un-
developed Self, Self-Sacrifice, Unrelenting Standards, In-
sufficient Self-Control, and Approval-Seeking. Average
extracted variance in items of the scales Entitle-
ment/Grandiosity and Punitiveness was substantially low-
er than required (0.34 and 0.37, respectively). Factor-scale
congruence (correlation between factor scores and corre-
sponding scale sum scores) was high (at least 0.95) for 17
scales. Congruence of the scale Punitiveness yielded 0.85.
All factors were positively and statistically significantly
(p < .001) interrelated, with correlations ranging from
0.21 to 0.87.
The discrepancy test indicated a statistically signif-
icant misfit between defined and observed variance-co-
variance matrix (2 = 11,516.1; df = 3,762; p < .001). The
normed statistic (3.06) marginally exceeded the
threshold for an acceptable fit (below 3.00). The CFI
(0.847) missed the required threshold for an acceptable
fit (above 0.90). The RMSEA (0.042; 95% CI =
0.0410.043) was less than 0.050, indicating a good fit.
The SRMR (0.060) stayed below 0.100 signifying an ac-
ceptable model fit.
Convergent Validity
Pearson correlation coefficients (r) between YSQ-S3 scales
and general psychopathology as measured by the SCL-K-9
ranged from 0.28 (Scale Entitlement/Grandiosity) to 0.65
(scales Abandonment/Instability and Negativity/Pessi-
mism) as reported in Table 4. Twelve of the 18 scales
showed a medium or strong association (r > 0.5) with the
SCL-K-9.
Associations of the YSQ-S3 scales with symptoms of

European Journal of Psychological Assessment 2013; Vol. 29(3):205212 2012 Hogrefe Publishing
 L. Kriston et al.: Reliability and Validity of the German YSQ-S3 209

Table 4. Results of the factorial and convergent validity analysis

Factor loadings Factor Variance Factor Correlation with
Scale (schema) 1st item 2nd item 3rd item 4th item 5th item reliability extracted congru- SCL-K-9 SAPAS
ence
Emotional deprivation 0.74 0.81 0.78 0.87 0.64 0.88 0.62 0.98 0.50 0.48
Abandonment/Instability 0.70 0.74 0.85 0.69 0.77 0.87 0.61 0.98 0.65 0.56
Mistrust/Abuse 0.58 0.73 0.76 0.78 0.63 0.83 0.51 0.95 0.60 0.59
Social isolation/Alienation 0.81 0.65 0.86 0.86 0.86 0.91 0.70 0.99 0.57 0.51
Defectiveness/Shame 0.80 0.87 0.84 0.84 0.80 0.92 0.72 0.99 0.59 0.54
Failure to achieve 0.78 0.58 0.85 0.85 0.80 0.89 0.65 0.99 0.55 0.41
Dependence/Incompetence 0.69 0.69 0.66 0.62 0.81 0.83 0.54 0.95 0.61 0.51
Vulnerability to harm or illness 0.79 0.82 0.48 0.56 0.60 0.78 0.47 0.95 0.60 0.47
Enmeshment/Undeveloped self 0.65 0.62 0.51 0.65 0.76 0.78 0.47 0.95 0.50 0.45
Subjugation 0.70 0.73 0.55 0.68 0.76 0.82 0.53 0.95 0.57 0.50
Self-sacrifice 0.60 0.57 0.79 0.62 0.73 0.80 0.48 0.98 0.34 0.22
Emotional inhibition 0.77 0.80 0.67 0.75 0.81 0.87 0.62 0.99 0.44 0.42
Unrelenting standards 0.64 0.66 0.61 0.67 0.45 0.74 0.42 0.96 0.37 0.41
Entitlement/Grandiosity 0.44 0.72 0.38 0.67 0.56 0.69 0.34 0.96 0.28 0.28
Insufficient self-control 0.67 0.68 0.61 0.67 0.67 0.80 0.49 0.97 0.51 0.40
Approval-seeking 0.48 0.72 0.79 0.65 0.63 0.79 0.46 0.96 0.38 0.34
Negativity/Pessimism 0.79 0.80 0.78 0.64 0.70 0.86 0.61 0.97 0.65 0.57
Punitiveness 0.72 0.57 0.34 0.39 0.68 0.67 0.37 0.85 0.42 0.39
Notes. All reported parameters are statistically significantly different from zero at p < .001. SCL-K-9 = Nine-item short-form of the Symptom
Checklist 90-R (SCL-90-R). SAPAS = Standardized Assessment of Personality Abbreviated Scale.

Figure 1. Mean schema profiles of subgroups according to health-care utilization. The overall difference between group
means in one-factorial analysis of variance is statistically significant for each scale at p < .001.

2012 Hogrefe Publishing European Journal of Psychological Assessment 2013; Vol. 29(3):205212
210 L. Kriston et al.: Reliability and Validity of the German YSQ-S3
personality disorder as measured by the SAPAS were (with
the exception of the scale Unrelenting Standards) weaker
than associations with psychopathological symptoms, but
still statistically significant (see Table 4). Seven YSQ-S3
scales showed an at least medium-sized association (r >
0.5) with the SAPAS, with the scales Mistrust/Abuse, Neg-
ativity/Pessimism, and Abandonment/Instability yielding
the strongest interrelations.
Discriminant Validity
Statistically significant (p < .001) group differences in all
YSQ-S3 scales were found for groups defined by health-
care utilization (see Figure 1). Most schemas were most
pronounced in the clinical sample, followed by former
inpatients from the community sample. Similarly, out-
patients scored lower than inpatients but higher than sub-
jects reporting counseling sessions. Participants reporting
no mental-health related health-care utilization scored
lowest on all scales. Overall differences were most pro-
nounced for schemas in the domains Disconnection and
Impaired Autonomy and Achievement (see Table 1).
Discussion
The present study revealed strong empirical support for
the psychometric soundness of the German version of the
YSQ-S3. The instrument proved to be reliable and showed
acceptable factorial validity. Schema scores were posi-
tively associated with measures of psychopathology and
personality disorder, indicating convergent validity. The
YSQ-S3 differentiated between subgroups with different
levels of health-care utilization, supporting discriminant
validity. Although the scales Entitlement/Grandiosity and
Punitiveness showed some problems with regard to reli-
ability and factorial validity, we still consider them suffi-
ciently good to be used without serious revision.
All YSQ-S3 scales are associated with self-rated gen-
eral psychopathology, personality disorder severity, and
health care utilization. Furthermore, YSQ-S3 scales are
highly interrelated. These results raise questions regard-
ing the specificity of the schema constructs, i.e., whether
different schemas can indeed be regarded as different
constructs (Kriston et al., 2010), and/or whether they can
be grouped in a hierarchical structure (Rijo & Gouveia,
2008). Schema theory suggests five higher-order schema
domains (see Table 1; Young et al., 2003), however, the
validity of schema domains is currently being discussed
(Lockwood, 2011).
At first glance, global fit indexes are inconclusive in
confirmatory factor analysis. While some (discrepancy
chi-square test, normed , CFI) indicate poor fit, others
(RMSEA, SRMR) support the hypothesized structure.
However, Hair et al. (2009) note that the discrepancy
European Journal of Psychological Assessment 2013; Vol. 29(3):205212
test is oversensitive (i.e., almost always statistically sig-
nificant) in sample sizes over 300, and that the CFI tends
to indicate poorer fit with increasing model complexity.
Schweizer (2010) also points out that the recommended
cutoff limits for the normed statistic may need to be
enlarged in large samples. Since our study was conducted
with a large sample size (more than 1,000 participants)
and a rather complex model (18 correlated factors), we
relied on RMSEA and SRMR more strongly than on the
test and the CFI. But, it should be noted that our con-
firmatory factor-analytic model support is still slightly
weaker than in the study by Rijo and Gouveia (2008) and
considerably inferior to the findings reported by Saariaho
et al. (2009).
Surprisingly, YSQ-S3 scores are seemingly more
strongly related to general psychopathology than to per-
sonality disorder symptoms. Based on this finding one
might argue that the association between personality dis-
order features and reported schema scores are moderated
by schema modes that may weaken the intercorrelation
when avoidant/detached coping styles are present. How-
ever, Lobbestael, van Vreeswijk, and Arntz (2008) argue
that schema modes, particularly the so-called detached
protector mode, which covers avoidant/detached coping,
can reflect general psychopathology. Further investiga-
tion of the multiple associations between personality psy-
chopathology, global psychopathology, schemas, and
schema modes are clearly necessary. However, a detailed
clinical discussion of this issue would go beyond the
scope of the present psychometric study. It should also
be noted that the instrument used for assessing personal-
ity disorder symptoms (SAPAS) uses binary items that
have higher response thresholds than items of the SCL.
This is likely to lead to a decreased variance of the SA-
PAS score, which in turn may deteriorate the Pearson cor-
relation coefficients. In summary, no definite conclusion
on the comparison of these interrelations should be
drawn.
A major limitation of our study is our investigating a
convenience community sample. The online survey for-
mat allowed us to collect complete data from a large sam-
ple, but it did not allow for controlling the selection of
participants. However, a substantial proportion of the
community sample reported current or earlier health-care
treatment, and variance in YSQ-S3 scores was satisfying-
ly high. Demographic and clinical characteristics ranged
broadly, leading to a heterogeneous sample without a
clear population to which the findings could be general-
ized. Although we examined an additional clinical sam-
ple, its size was rather small because of limited resources.
Thus, further investigations in large clinical samples on
the psychometric properties of the YSQ-S3 are necessary.
To our knowledge this study is the first to test the Ger-
man version of the YSQ-S3. Largely in agreement with
previous findings in other countries and with former ver-
sions of the instrument, we conclude that the YSQ-S3
2012 Hogrefe Publishing
 L. Kriston et al.: Reliability and Validity of the German YSQ-S3
provides a valid and reliable measurement of Youngs
early maladaptive schemas.
Acknowledgments
We thank all persons who participated in the study either
as inpatients or by filling out the online survey. We are
grateful to the two anonymous reviewers for their helpful
comments on a previous version of the manuscript.
References
Cudeck, R. (1989). Analysis of correlation matrices using covari-
ance structure models. Psychological Bulletin, 105, 317327.
Derogatis, L. R. (1992). The SCL-90-R, administration, scoring
and procedures manual (2nd ed.). Towson, MD: Clinical Psy-
chometric Research.
Farrell, J. M., Shaw, I. A., & Webber, M. A. (2009). A schema-fo-
cused approach to group psychotherapy for outpatients with
borderline personality disorder: A randomized controlled trial.
Journal of Behavior Therapy and Experimental Psychiatry, 40,
317328.
Germans, S., van Heck, G. L., Moran, P., & Hodiamont, P. P. G.
(2008). The self-report Standardized Assessment of Personal-
ity-Abbreviated Scale: Preliminary results of a brief screening
test for personality disorders. Personality and Mental Health,
2, 7076.
Giessen-Bloo, J., van Dyck, R., Spinhoven, P., van Tilburg, W.,
Dirksen, C., van Asselt, T., . . . Arntz, A. (2006). Outpatient
psychotherapy for borderline personality disorder: random-
ized trial of schema-focused therapy vs. transference-fo-
cused psychotherapy. Archives of General Psychiatry, 63,
649658.
Green, S. B., Akey, T. M., Fleming, K. K., Hershberger, S. L., &
Marquis, J. G. (1997). Effect of the number of scale points
on chi-square fit indices in confirmatory factor analysis.
Structural Equation Modeling: A Multidisciplinary Journal,
4, 108120.
Gude, T., & Hoffart, A. (2008). Change in interpersonal problems
after cognitive agoraphobia and schema-focused therapy ver-
sus psychodynamic treatment as usual of inpatients with ago-
raphobia and cluster C personality disorders. Scandinavian
Journal of Psychology, 49, 195199.
Hair, J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (2009).
Multivariate data analysis (7th ed.). Upper Saddle River, NJ:
Prentice Hall.
Klaghofer, R., & Brhler, E. (2001). Konstruktion und teststatis-
tische Prfung einer Kurzform der SCL-90-R [Construction
and psychometric evaluation of a short-form of the SCL-90-
R]. Zeitschrift fr Klinische Psychologie, Psychiatrie und Psy-
chotherapie, 49, 115124.
Kline, R. B. (2010). Principles and practices of structural equa-
tion modeling (3rd ed.). New York: Guilford.
Kriston, L., Schfer, J., Hrter, M., & Hlzel, L. (2010, July). All
the same? Factorial structure of the Young Schema Question-
naire Short Form 3 (YSQ-S3). Paper presented at the 4th
2012 Hogrefe Publishing
211
Annual International Society of Schema Therapy Meeting,
Berlin, Germany.
Lobbestael, J., van Vreeswijk, M., & Arntz, A. (2008). An empir-
ical test of schema mode conceptualizations in personality dis-
orders. Behavior Research and Therapy, 46, 854860.
Lockwood, G. (2011, February 12). Do schema domains exist? [On-
line blog]. Retrieved from http://www.isst-online.com/node/226
Moran, P., Leese, M., Lee, T., Walters, P., Thornicroft, G., &
Mann, A. (2003). Standardized Assessment of Personality
Abbreviated Scale (SAPAS): Preliminary validation of a brief
screen for personality disorder. British Journal of Psychiatry,
183, 228232.
Muthn, B., & Kaplan, D. (1985). A comparison of some meth-
odologies for the factor analysis of nonnormal Likert variables.
British Journal of Mathematical and Statistical Psychology,
38, 171189.
Muthn, L. K., & Muthn, B. O. (1998). Mplus users guide (6th
ed.). Los Angeles, CA: Muthn & Muthn.
Nadort, M., Arntz, A., Smit, J. H., Giessen-Bloo, J., Eikelenboom,
M., Spinhoven, P., . . . van Dyck, R. (2009). Implementation
of outpatient schema therapy for borderline personality disor-
der with versus without crisis support by the therapist outside
office hours: A randomized trial. Behavior Research and Ther-
apy, 47, 961973.
Nordahl, H. M., & Nysaeter, T. E. (2005). Schema therapy for pa-
tients with borderline personality disorder: A single case se-
ries. Journal of Behavior Therapy and Experimental Psychia-
try, 36, 254264.
Oei, T. P. S., & Baranoff, J. (2007). Young schema questionnaire:
Review of psychometric and measurement issues. Australian
Journal of Psychology, 59, 7886.
Rijo, D., & Gouveia, P. (2008, October). A confirmatory factor
analysis study of the YSQ-S3 in a large Portuguese sample.
Paper presented at the 3rd Annual International Society of
Schema Therapy Meeting, Coimbra, Portugal.
Saariaho, T., Saariaho, A., Karila, I., & Joukamaa, M. (2009). The
psychometric properties of the Finnish Young Schema Ques-
tionnaire in chronic pain patients and a nonclinical sample.
Journal of Behavior Therapy and Experimental Psychiatry, 40,
158168.
Schmidt, N. B., Joiner, T. E., Young, J. E., & Telch, M. J. (1995).
The Schema Questionnaire: Investigation of psychometric
properties and the hierarchical structure of maladaptive sche-
mas. Cognitive Therapy and Research, 19, 295321.
Schweizer, K. (2010). Some guidelines concerning the modeling
of traits and abilities in test construction. European Journal of
Psychological Assessment, 26, 12.
Soygt, G., Karaosmano6lu, A., & Cakir, Z. (2009). Assessment
of early maladaptive schemas: A psychometric study of the
Turkish young schema questionnaire-short form-3. Trk Psiki-
yatri Dergisi, 20, 7584.
Tabachnick, B. G., & Fidell, L. S. (2007). Using multivariate sta-
tistics (5th ed.) Boston, MA: Pearson.
Trip, S. (2006). The Romanian version of Young Schema Ques-
tionnaire Short Form 3 (YSQ-S3). Journal of Cognitive and
Behavioral Psychotherapies, 6, 173181.
Young, J. E. (1990). Cognitive therapy for personality disorders:
A schema-focused approach. Sarasota, FL: Professional Re-
source Exchange.
Young, J. E. (2005). Young Schema Questionnaire Short Form
3 (YSQ-S3). New York: Cognitive Therapy Center.
European Journal of Psychological Assessment 2013; Vol. 29(3):205212
212 L. Kriston et al.: Reliability and Validity of the German YSQ-S3

Young, J. E. (2006). Young Schema Questionnaire (German ver- Levente Kriston

sion, YSQ-S3) (H. Berbalk, J. Grutschpalk, E. Parfy, & G. Zar-
bock, Trans.). Eckernfrde: Institut fr Schematherapie. Department of Medical Psychology
Young, J. E., & Brown, G. (1994). Young Schema Questionnaire University Medical Center Hamburg-Eppendorf
Short Form. New York: Cognitive Therapy Center. Martinistr. 52
20246 Hamburg
Young, J. E., Klosko, J. S., & Weishaar, M. (2003). Schema ther-
Germany
apy: A practitioners guide. New York: Guilford.
Tel. +49 40 7410-56849
Fax +49 40 7410-54965
Published online: July 20, 2012 E-mail l.kriston@uke.de

European Journal of Psychological Assessment 2013; Vol. 29(3):205212 2012 Hogrefe Publishing
 A. Mller et al.: Validation of the German
European Journalof VersionAssessment
Psychological of the Exercise
2012Dependence
2013; Vol.
Hogrefe Scale
29(3):213219
Publishing

Original Article

Validation of the German Version

of the Exercise Dependence Scale
Astrid Mller1, Laurence Claes2, Dirk Smits2,3, Olaf Gefeller4, Anja Hilbert5,
Antonia Herberg1, Verena Mller1, Dirk Hofmeister6, and Martina de Zwaan1
1
Department of Psychosomatic Medicine and Psychotherapy, Medical School Hannover, Germany,
2
Department of Psychology, University of Leuven, Belgium, 3HUB, Brussels, Belgium, 4Department of
Medical Informatics, Biometry, and Epidemiology, University of Erlangen-Nuremberg, Erlangen,
Germany, 5Integrated Research and Treatment Center (IFB) Adiposity Diseases, University of Leipzig,
Germany, 6Department of Medical Psychology and Medical Sociology, University of Leipzig, Germany

Abstract. This study evaluated the psychometric properties of the German version of the 21-item Exercise Dependence Scale (EDS-G),
which was administered in a large representative German sample (N = 1,611). In order to examine convergent validity, we also asked
participants to answer the Eating Disorder Examination-Questionnaire (EDE-Q). To investigate the discriminant validity, clients of fitness
centers (N = 129), sport students (N = 85), and medical students (N = 129) filled out the EDS-G. In this German population-based sample,
the seven-factor structure suggested by the original authors as well as a higher-order model (7 + 1 factor model) were supported. The
total EDS-G scale showed an excellent internal reliability and was positively related to the EDE-Q total and subscale scores. The EDS-G
scores differentiated between samples with varying degrees of exercise. In summary, the results indicate that the EDS-G is a psychomet-
rically sound and potentially valid brief measure for the assessment of exercise dependence symptoms.

Keywords: exercise dependence, eating disorder symptoms, measurement, validity

Introduction lowing subscales: tolerance, withdrawal effects, continuance,

Exercise dependence is characterized by a preoccupation
with exercise and extreme physical activity; it is relatively
resistant to change and is associated with impaired psycho-
logical, social, physical, and behavioral functioning (Bam-
ber, Cockerill, Rodgers, & Carroll, 2003). Typically, indi-
viduals with exercise dependence continue exercising de-
spite recurrent physical or psychological problems (such as
injury, illness, fatigue, or personal demands) that are likely
to have been caused, or exacerbated, by exercise. Previous
research indicated a link between exercise dependence and
eating disorders since many patients with eating disorders
exercise compulsively in order to control weight and shape
(e.g., Bewell-Weiss & Carter, 2010; Cook & Hausenblas,
2008).
Hausenblas and Symons Downs (2002a) suggested that
exercise dependence should be conceptualized as a maladap-
tive pattern of exercise that causes clinically significant im-
pairment and is operationalized based on three or more diag-
nostic criteria adopted from the DSM-IV criteria for sub-
stance dependence (APA, 1994). Based on these criteria
Hausenblas and Symons Downs (2002c) developed the Ex-
ercise Dependence Scale (EDS-21), which comprises the fol-
lack of control, reductions in other activities, time, and inten-
tion. The final version consists of 21 items to be rated on a
6-point Likert scale ranging from 1 (= never) to 6 (= always),
with three items per subscale (i.e., seven factors or subscales)
(Symons Downs, Hausenblas, & Nigg, 2004). Higher mean
scores indicate a higher incidence of exercise dependence
symptoms. The scale is a widely used instrument (e.g., Cook
& Hausenblas, 2008; Edmunds, Ntoumanis, & Duda, 2006)
and has shown good convergent and discriminant validity as
well as excellent internal and test-retest reliability (e.g., Sy-
mons Downs et al., 2004). To distinguish among individuals
who are at risk, have some symptoms, or have no symptoms
for exercise dependence, the authors described the following
three categories:
1) In order to be classified as at risk for exercise depend-
ence, three or more of the DSM-IV diagnostic criteria
for substance dependence should be present (i.e., a total
score of 15 or more on at least three subscales).
2) Nondependent-symptomatic refers to heightened scores
on an minimum of three EDS-21 subscales (i.e., for all
who have not been classified as at risk for exercise de-
pendence a total score ranging of 7 or more on at least
three subscales).

2012 Hogrefe Publishing European Journal of Psychological Assessment 2013; Vol. 29(3):213219
DOI: 10.1027/1015-5759/a000144
214 A. Mller et al.: Validation of the German Version of the Exercise Dependence Scale
3) Nondependent-asymptomatic is a categorization where-
by one of the three categories obtained using the EDS-21
is generated by a scoring algorithm consisting of flow-
chart decision rules, in which items or combinations of
items determine whether an individual would be classi-
fied in the at-risk, symptomatic, or asymptomatic cate-
gory (Hausenblas & Symons Downs, 2002b).
Allegre and Therme (2008) validated the French version of
the EDS-21 in a sample of 516 half-marathon runners aged
between 17 and 74 years. They found the same 7-factor
structure as the original version. Recently, Sicilia and Gon-
zlez-Cutre (2011) validated the Spanish version in a sam-
ple of 531 sport center users aged between 16 and 60 years.
Their findings supported the first-order seven-factor model
as well as a higher-order factor model (seven first-order
factors and one second-order factor). To date, no validated
German version is available.
The aim of this study was to validate the German ver-
sion of the EDS-21 (EDS-G) in a large representative
German sample. In order to examine convergent validity,
we additionally administered a self-rating instrument for
eating disorder symptoms. Given the association between
eating disorders and excessive exercising, we expected a
significant positive correlation between the EDS-G and
the eating disorder measure. Because the decision rules
for categorizing someone as at risk for exercise depend-
ence proposed by Hausenblas and Symons Downs
(2002b) are rather difficult to handle, we looked for an
alternative way to differentiate individuals who would be
at risk from the nondependent group by using a cut-
off value on the total EDS-G score. We further attempted
to estimate the prevalence of exercise dependence in the
general population based on this cutoff. To investigate
the discriminant validity, we asked clients of fitness cen-
ters, sport students as well as medical students to fill out
the EDS-G. We hypothesized that sport students and cli-
ents of fitness centers would report higher EDS-G scores
than medical students. Additionally, medical students
also answered questions concerning their weekly physi-
cal activity, which we expected to positively correlate
with their EDS-G total scores.
Method
Participants
All individuals participated on a voluntary basis and gave
their written informed consent before entering the study.
The study was approved by the Institutional Ethics Com-
mittee of the University of Erlangen-Nuremberg, Germany.
The population-based survey met the ethical guidelines of
the international Code of Marketing and Social Research
Practice by the International Chamber of Commerce and
the European Society for Opinion and Marketing Research.
European Journal of Psychological Assessment 2013; Vol. 29(3):213219
Population-Based Sample
A random sample of the German general population older than
14 years of age was selected with the assistance of a demo-
graphic consulting company. The sampling procedure was de-
signed to yield random samples representative of the German
population in terms of age, sex, and education. It followed the
establishedguidelinesonhowtoconstructarandompopulation
sample in Germany when no access to a population roster is
possible. This sampling design involves three consecutive
steps:First,agridof258regionalsamplingareaswasrandomly
selected from a roster of such nonoverlapping grids centrally
assembled to enhance representativeness in stratified regional
sampling in Germany. As a second step, a random procedure
to select households of the respective area was implemented
within all sampling areas. In a third and final step, one member
of the selected household fulfilling the inclusion criteria (age
14 or older, able to read and understand the German language)
wassampledrandomlyinaprespecifiedmanner.Afirstattempt
wasmadefor4,091addresses,ofwhich4,069werevalid.Ifthe
person was not at home, a maximum of three attempts was
made to contact him or her. All subjects were visited by a study
assistant,whoinformedthemabouttheinvestigation,obtained
writteninformed consent, and presented them witha question-
nairepackageincludingtheEDS-G.Thesurveywasconducted
between November27andDecember16,2009.A totalsample
of 2,520 individuals from 14 to 95 years of age agreed to par-
ticipate (participation rate:61.9% ofthevalid addresses).Sim-
ilar to the Spanish validation study (Sicilia & Gonzlez-Cutre,
2011) respondents younger than 16 (N = 47) and older than 60
(N= 842)wereexcludedaswedidnotexpectahighprevalence
of excessive exercising in individuals agedbelow 16 years and
above 60 years. Additionally, cases were dropped when data
were missing (N = 20). This provided a final sample of 1,611
individuals for analysis with a mean age of 40.8 years (SD =
12.5, range 1660); 53.6% were female, 58.3% lived with a
partner,and 9% were unemployed. Themean body mass index
(BMI) was 24.9 kg/m2 (SD = 4.3, range 14.255.4).
Additional Samples
The EDS-G was additionally administered to clients of fit-
ness centers (GYM-group), sports students (SP-group),
and medical students (MD-group). The GYM-group in-
cluded 129 individuals with a mean age of 24.5 years (SD
= 8.4, range 1650). The SP-group consisted of 85 sports
students with a mean age of 22.5 years (SD = 1.9, range
1929). In the MD-group there were 129 students with a
mean age of 23.2 years (SD = 2.8, range 2036).
Instruments
The authorized German translation and backtranslation
(Brislin, 1970) of the original American EDS-21 was per-
formed by a licensed translator. The backtranslation was
2012 Hogrefe Publishing
 A. Mller et al.: Validation of the German Version of the Exercise Dependence Scale
verified for discrepancies against the original form by the
American author of the original scale. The EDS-G was an-
swered by the population-based sample as well as by the
other groups.
The validated German version of the Eating Disorder
Examination-Questionnaire (EDE-Q; Hilbert & Tuschen-
Caffier, 2006) was filled out by the population-based sam-
ple. The EDE-Q consists of the four subscales: restraint
(Cronbachs = 0.83 in the present sample), eating concern
( = 0.80), shape concern ( = 0.89), and weight concern
( = 0.80). The 22 items are rated on a 6-point scale ranging
from 1 (no days/not at all) to 5 (every day/markedly).
Data on physical activity were available only from the
medical students. They were asked to indicate the number
of days per week from 0 (no days) to 7 (seven days) with
intensive physical activity.
Data Analysis
As we had a clear hypothesis about the 7-factor structure
of the EDS, we performed confirmatory factor analyses
(CFAs) with LISREL 8.71 (Jreskog & Srbom, 2004). To
test whether the power of the seven-factor model of Hau-
senblas and Symons Downs (2000b) was large enough to
reject a more parsimonious one-factor alternative, we fitted
an additional one-factor model (Bentler, 2007), assuming
one general exercise dependence factor. A second motiva-
tion for the one-factor model was the use of the total EDS-
G score as a measure of exercise dependence, in the ab-
sence of scores on at least one of the seven subscales. For
the 7-factor model, we determined that every item loaded
on its corresponding factor, and latent factors were allowed
to correlate.
The distributions of responses to some items were rather
skewed. Mean skewness for all items equaled 2.40 (SD =
0.44). However, the joint distribution of the items at the
total scale was less skewed (1.88 for the total score). Given
that the data were ordinal and due to the skewness of the
response distribution of several items, a robust weighted
least square estimation method on the polychoric correla-
tions, weighted by the asymptotic variances, was preferred
for estimating the parameters of the CFA models, as sug-
gested by Flora and Curran (2004). To evaluate model fit,
multiple criteria were used: Satorra-Bentler scaled chi-
square values for comparing nested models (2 test), com-
parative fit index (CFI) for fit relative to a null model, the
root mean squared error of approximation (RMSEA), and
the standardized root mean square residual (SRMR) as the
standardized difference between the observed and the pre-
dicted correlation. Stringent cutoff values were used as the
skewness of the response distribution can lead to slightly
overestimated 2 values and fit indices. The criteria for fit
were defined according to Hu and Bentler (1999) as CFI >
0.97, RMSEA < 0.05, and SRMR < 0.09.
A cutoff value for being at risk for exercise dependence
on the total EDS-G score was determined in the popula-
2012 Hogrefe Publishing
215
tion-based sample by using a receiver operating character-
istic (ROC) curve that was based on the diagnostic criteria
for being at risk for exercise dependence reported by Hau-
senblas and Symons Downs (2002b). Congruence of this
new cutoff value and the cutoff algorithm proposed by the
original authors was then assessed using Cohens coeffi-
cient kappa () among clients of fitness centers, sport stu-
dents, and medical students. Cohens coefficient mea-
sures chance corrected agreement between diagnoses. In
nondegenerated situations it ranges from 0 to +1.00, where
< 0.20 indicates only slight agreement, 0.21 < < 0.40
fair agreement, 0.41 < < 0.60 moderate agreement, 0.61
< < 0.80 substantial agreement, and 0.81 < < 1.00 re-
flects almost perfect agreement (Landis & Koch, 1977).
Cronbachs s were calculated in all samples to investi-
gate the internal consistency. Convergent validity was de-
termined by using the Spearman rank correlation coeffi-
cient to calculate the associations between the EDS-G and
the EDE-Q scales in the population-based sample. In the
MD-group the correlation between EDS-G scores and daily
physical activity was investigated.
To explore group differences in terms of age and sex, we
used t-tests for age and tests for sex, respectively. The
Exercise Dependence Scale scores were compared across
samples using one-way analysis of covariance (ANCOVA,
adjusted for age and sex) in order to investigate discrimi-
nant validity. Post hoc tests (Tukey B) were conducted to
evaluate pairwise group differences in age. Overall and
pairwise group differences in the ANCOVA adjusted
means for EDS-G total and subscale scores were also as-
sessed by post hoc tests with a simple Bonferroni correction
to control the type I error across pairwise comparisons.
Results
Factor Structure
Table 1 summarizes the results of the CFAs. The fit of the
one-factor solution was reasonable, but the RSMEA was
somewhat too high. In comparison, the 7-factor model ob-
tained a substantially better fit to the data ( = 1713.72;
Table 1. Indices describing the goodness of fit of the con-
firmatory factor analyses on the items of the Ex-
ercise Dependence Scale (EDS-G) based on the
German population-based sample
df Satorra RMSEA CFI SRMR
Bentler
One-factor model 189 2395.52 0.09 0.99 0.05
Seven-factor model 168 681.80 0.04 0.99 0.02
Seven + one-factor 182 888.37 0.05 0.99 0.03
model
Notes. n = 1,611. CFI = comparative fit index, RMSEA = root mean
squared error of approximation, SRMR = standardized root mean
square residual.
European Journal of Psychological Assessment 2013; Vol. 29(3):213219
216 A. Mller et al.: Validation of the German Version of the Exercise Dependence Scale

df = 21; p < .001). However, all factors were seriously cor- Item English version/German translation Factor
related (correlations ranging from .79 to .94), suggesting a loading
superordinate factor. Therefore, we fitted a CFA with seven 11 I am unable to reduce how often I exercise./Es ge- 0.91
factors, corresponding to Hausenblas and Symons Downs lingt mir nicht, die Hufigkeit meiner sportlichen
Aktivitten zu reduzieren.
(2000b), complemented with a single second-order factor
on which all first-order factors were assumed to load pos- 18 I am unable to reduce how intense I exercise./Es ge- 0.94
lingt mir nicht, die Intensitt meiner sportlichen Ak-
itively. The nature of this higher-order factor may reflect tivitten zu verringern.
the unifying feature that comprehensively defines exercise Reduction in other activities/Verringern anderer Ak-
dependence. The second-order factor explained a large tivitten
amount of the variance in each of the first-level factors (i.e., 5 I would rather exercise than spend time with fami- 0.91
R varies from 0.79 to 0.94). ly/friends./Ich treibe lieber Sport, als Zeit mit mei-
First-factor loadings of the final 7 + 1 factor model are ner Familie/meinen Freunden zu verbringen.
presented in Table 2. Only factor loadings of items to their 12 I think about exercise when I should be concentrat- 0.90
corresponding first-order factor are mentioned (factors are ing on school/work./Ich denke an Sport, wenn ich
printed in italics above the items), as all other first-factor mich eigentlich auf die Schule/Arbeit konzentrieren
loadings are fixed to zero. The loadings of the first-order sollte.
factors to the second order factor ranged from .89 to .96. 19 I choose to exercise so that I can get out of spending 0.88
time with family/friends./Ich entscheide mich dafr,
Sport zu treiben, damit ich keine Zeit mit der Fami-
lie/mit Freunden verbringen muss.
Table 2. Factor loadings of the items of the Exercise De-
Time/Zeit
pendence Scale (EDS-G) on the seven first order
6 I spend a lot of time exercising./Ich verbringe viel 0.92
factors in the German population-based sample
Zeit mit Sport.
Item English version/German translation Factor 13 I spend most of my free time exercising./Ich verbrin- 0.96
loading ge den grten Teil meiner Freizeit mit Sport.
Withdrawal effects/Entzugssymptome 20 A great deal of my time is spent exercising./Ich ver- 0.95
1 I exercise to avoid feeling irritable./Ich treibe Sport, 0.87 bringe den Groteil meiner Zeit mit Sport.
weil ich das Gefhl der Gereiztheit vermeiden mch- Intention effects/Intention
te.
7 I exercise longer than I intend./Ich treibe lnger 0.94
8 I exercise to avoid feeling anxious./Ich treibe Sport, 0.87 Sport als ich mir vorgenommen habe.
um Angstgefhle zu vermeiden.
14 I exercise longer than I expect./Ich treibe lnger 0.96
15 I exercise to avoid feeling tense./Ich treibe Sport, 0.89 Sport als erwartet.
um Spannungsgefhle zu vermeiden.
21 I exercise longer than I plan./Ich treibe lnger Sport 0.96
Continuance/Fortfhren als geplant.
2 I exercise despite recurring physical problems./Ich 0.88 Note. n = 1,611.
treibe Sport, obwohl ich immer wieder krperliche
Beschwerden habe.
9 I exercise when injured./Ich treibe Sport, wenn ich 0.89 Normative Data and Internal Consistency
Verletzungen habe.
16 I exercise despite persistent physical problems./Ich 0.89
Means, standard deviations, and Cronbachs s of the EDS-
bin trotz anhaltender krperlicher Beschwerden G scales can be found in Table 3.
sportlich aktiv.
Tolerance/Toleranzentwicklung
3 I continually increase my exercise intensity to 0.92 Table 3. Means, standard deviations, and Cronbachs for
achieve the desired effects/benefits./Meine sportli- the subscales of the Exercise Dependence Scale
chen Aktivitten werden immer intensiver, damit (EDS-G) in the population-based sample
ich die gewnschten Effekte/Vorteile erziele.
Mean SD Cron-
10 I continually increase my exercise frequency to 0.92
bachs
achieve the desired effects/benefits./Ich treibe im-
mer hufiger Sport, um die gewnschten Effek- EDS Total 32.78 17.29 0.96
te/Vorteile zu erzielen. Withdrawal effects 4.72 2.76 0.81
17 I continually increase my exercise duration to 0.94 Continuance 4.33 2.45 0.83
achieve the desired effects/benefits./Ich verlngere
Tolerance 4.97 3.18 0.89
stndig die Dauer meiner sportlichen Aktivitten,
um die gewnschten Effekte/Vorteile zu erzielen. Lack of control 4.85 3.18 0.88
Loss of control/Kontrollverlust Reduction in other activities 4.09 2.20 0.83
4 I am unable to reduce how long I exercise./Es ge- 0.92 Time 5.12 3.31 0.91
lingt mir nicht, die Dauer meiner sportlichen Aktivi- Intention effects 4.71 3.01 0.93
tten zu verkrzen.
Note. n = 1,611.

European Journal of Psychological Assessment 2013; Vol. 29(3):213219 2012 Hogrefe Publishing
 A. Mller et al.: Validation of the German Version of the Exercise Dependence Scale 217

Table 4. Comparison of Exercise Dependence Scale (EDS-G) scores across samples

Population-based Clients of fitness Sport students (3) Medical Overall group Pairwise group
sample (1) centers (2) students (4) comparison comparisons
N = 1,611 N = 129 N = 85 N = 129
Mean (SD) Mean (SD) Mean (SD) Mean (SD) ANOVA Tukey-B
posthoc test
Age 40.83 (12.46) 25.51 (8.37) 22.46 (1.94) 23.25 (2.82) F(3, 1949) = 211.57* 1 > 2, 4, 3
EDS-subscales ANCOVAa,b ANCOVAa,b
EDS-G Total 32.78 (17.29) 58.17 (17.30) 65.82 (13.70) 43.76 (13.77) F(3, 1946) = 106.12* 3>2>4>1
Withdrawal effects 4.72 (2.76) 7.08 (3.10) 6.96 (2.27) 7.12 (2.99) F(3, 1947) = 32.40* 4,2,3 > 1
Continuance 4.33 (2.45) 7.41 (3.84) 8.73 (3.33) 5.23 (2.49) F(3, 1947) = 84.98* 3>2>4>1
Tolerance 4.97 (3.18) 10.23 (4.01) 10.51 (3.70) 8.01 (3.50) F(3, 1947) = 105.14* 3,2 > 4 > 1
Lack of control 4.85 (3.18) 7.95 (4.16) 8.48 (4.53) 5.89 (3.43) F(3, 1946) = 37.31* 3,2 > 4,1
Reduction in other 4.09 (2.20) 6.08 (2.49) 7.53 (2.63) 4.83 (1.99) F(3, 1947) = 47.75* 3 > 2,4,1
activities
Time 5.12 (3.31) 11.08 (4.06) 14.00 (2.76) 6.83 (2.85) F(3, 1947) = 189.17* 3 > 2 > 4,1
Intention effects 4.71 (3.01) 8.33 (3.91) 9.61 (3.57) 6.11 (2.87) F(3, 1947) = 61.78* 3,2 > 4 > 1
Notes. Means presented in the table are unadjusted means, aadjusted for age and gender, bBonferroni adjusted p = .05/8 = .006. *p < .001.

Cutoff Value for Being at Risk of Exercise
Dependence
The ROC analysis showed that a total EDS-G score above
77 was indicative of exercise dependence risk with a sen-
sitivity of 1 and a specificity of 0.97. Based on this cutoff
value, 56 persons were classified as at risk. By using the
scoring guidelines of Hausenblas and Symons Downs
(2002b), 15 of them (0.09% of the total sample) were also
classified as being at risk for suffering from exercise de-
pendence, and 41 persons were symptomatic. As the EDS-
21 is used as a screening instrument, we decided to include
those who were symptomatic in the ROC analysis in order
to avoid underdiagnosis based on a heightened cutoff.
The analysis of congruence between at-risk diagnoses
for exercise dependence based on the aforementioned
ROC analysis and the algorithm proposed by Hausenblas
and Symons Downs (2002b) revealed kappa coefficients
of 0.73 for the GYM-group, 0.69 for sport students, and
0.66 for medical students.
By using a cutoff EDS-G score above 77, the point preva-
lence of exercise dependence at risk in the present German
representative sample was estimated to be 3.5%. Although
men reported significantly higher EDS-G scores than women
(M = 35.32, SD = 18.92 and M = 30.57, SD = 15.41, F(1, 1609)
= 30.87, p < .001), gender differences in prevalence estimates
for exercise dependence did not reach statistical significance
(4.4% and 2.7%, (1) = 3.64,p = .056, respectively). The high-
est estimated point prevalence was found among sport students
(16.5%) and in the GYM-group (12.4%). In the MD-group the
prevalence rate was substantially lower (1.6%).
Validity
In the population-based sample, the EDS-G total score was
positively, albeit only slightly, correlated with the EDE-Q
total score (r = 0.20, p < .001), and the EDE-Q subscales
restraint (r = 0.21, p < .001), eating concern (r = 0.22, p <
.001), shape concern (r = 0.16, p < .001), and weight con-
cern (r = 0.17, p < .001).
The mean number of days per week with intensive phys-
ical activity among medical students was 2.81 days (SD =
1.76, range 07). This variable was significantly positive
related to EDS-G scores (r = 0.42, p < .01).
As can be seen in Table 4, we found significant age dif-
ferences between groups. Posthoc comparisons showed
that the population-based sample was significantly older
than the three other groups. The samples were also different
with respect to sex ((3) = 54.89, p < .001). The rate of
female participants varied across groups with 75% women
in the MD-group, 54% in the population-based sample,
41% in the SP-group, and 31% in the GYM-group. The
gender differences between medical students and the pop-
ulation-based sample ((1) = 22.02, p < .001) and between
the population-based sample and sport students ((1) =
4.98, p = .026) were significant. In terms of EDS-G scores
analyses revealed significant group differences controlling
for age and sex with sport students reporting the highest
mean scores followed by clients of fitness centers, medical
students, and the population-based sample.
Discussion
In a representative, population-based German sample and
in samples with varying degrees of exercise, the study re-
sults offer support for the EDS-G as a reliable and valid
measure of exercise dependence.
The results of the CFA confirm the 7-factor structure of
the EDS suggested by Hausenblas and Symons Downs
(2002b) in our German population-based sample. This
finding is in line with the results of the French (Allegre &

2012 Hogrefe Publishing European Journal of Psychological Assessment 2013; Vol. 29(3):213219
218 A. Mller et al.: Validation of the German Version of the Exercise Dependence Scale
Therme, 2008) and Spanish (Sicilia & Gonzlez-Cutre,
2011) validation studies. Like the Spanish study, our find-
ings also support a higher-order factor model. Compared
to this study, the correlations between the factors in our
German representative sample were high (present study:
0.79 to 0.94, Spanish study: 0.32 to 0.84). Therefore, one
could argue that the EDS-G is a one-dimensional scale,
which is why we suggest using the total score for the as-
sessment of exercise dependence symptoms.
The total EDS-G scale showed an excellent internal re-
liability in line with previous studies that used the original
EDS-21 (e.g., Symons Downs et al., 2004). Furthermore,
the positive relationship between the total EDS-G score and
the physical activity among medical students offers support
for the validity of the questionnaire. The scores of the eat-
ing disorder symptoms questionnaire (EDE-Q) were posi-
tively, albeit moderately, related to EDS-G scores. In ac-
cordance with previous research concerning the link be-
tween excessive exercising and eating disorder symptoms
we had expected a stronger correlation between the two
scales. Additional analyses by sex, age, and BMI groups
(not reported here but available by request) did not yield
any explanations for the modest correlation.
The between-group comparison of the EDS-G scores
suggests that the scale may differentiate between persons
who regularly exercise and those who exercise less. Unfor-
tunately, except for the MD-group information about phys-
ical activity is lacking, which limits the informational value
of this finding.
Based on a ROC analysis in the German population-
based sample we proposed a cutoff on the total EDS-G
score with high sensitivity and specificity in order to dis-
tinguish people who are at risk for exercise dependence
from nondependent persons. Our findings indicated a sub-
stantial agreement between this EDS-G total score thresh-
old and the cutoff value calculated in accordance with the
scoring guidelines of Hausenblas and Symons Downs
(2002b). Thus, the use of the EDS-G cutoff score indeed
may represent an alternative approach to identify individ-
uals who were at risk for exercise dependence. By using
this cutoff score, we estimated the point prevalence of ex-
ercise dependence in the German population-based sample
to be 3.5%. The prevalence estimates in the general Ger-
man population were closed to those in the group of med-
ical students. Both prevalence rates of exercise dependence
were in accordance with results of previous studies among
U. S. students participating in fitness classes (Symons
Downs et al., 2004). As hypothesized, the highest percent-
age of at-risk individuals was found among sport students
followed by clients of fitness centers. The rates, however,
were much lower than those indicated in a French study
(Lejoyeux, Avril, Richoux, Embouazza, & Nivoli, 2008)
that investigated 300 clients of a Parisian fitness studio by
utilizing a self-designed questionnaire. Lejoyeux et al.
(2008) reported that 42% of clients of the fitness studio
being exercise dependent. This discrepancy could be due
to different screening instruments used.
European Journal of Psychological Assessment 2013; Vol. 29(3):213219
Besides the strengths of our study, some limitations need
to be recalled when interpreting the results of this study. As
mentioned above, a substantial portion of sport students
was at risk for exercise dependence according their EDS-G
scores. It is important to note, however, that the EDS-G
does not facilitate a diagnosis of exercise dependence be-
cause the EDS-G is solely a screening instrument and not
a diagnostic tool. Thus, the data cannot be used to deter-
mine whether or not these at-risk participants clinically rep-
resent a exercise-dependent group. Furthermore, the cur-
rent findings are limited by the fact that participants were
not assessed with an additional clinical interview to con-
firm the classification. Therefore, we are conducting anoth-
er study to investigate the consistency between EDS-G data
and a diagnostic interview for exercise dependence. Anoth-
er shortcoming of the present study is the absence of test-
retest reliability evidence of the EDS-G. The present study
also lacks detailed information about the amount and forms
of exercise (e.g., running, triathlon, weightlifting, cycling,
swimming, team sports) displayed by sport students and
clients of fitness centers. This information could give in-
sight as to whether athletes who engage in specific sport
forms would be more prone to exercise compulsively.
The strengths of the present study include the use of a
large population-based representative sample and the in-
vestigation of additional samples with varying degree of
exercising. Future research should address whether exer-
cise dependence overlaps not only with eating disorder
symptoms, but also with other excessive behaviors such as
compulsive buying or excessive internet use. For example,
Lejoyeux et al. (2008) found that exercise dependence was
strongly associated with fitness-studio clients compulsive
buying. Both exercise dependence and compulsive buying
may be considered socially accepted forms of excessive
behaviors that might explain the link between both diag-
noses. The investigation of possible similar underlying fac-
tors (e.g., personality features) of different behavioral ad-
dictions seems to be a topic worth pursuing.
In conclusion, the current study provides psychometric
support for the German version of the EDS-21. The scale
developed using U. S. subjects successfully can be trans-
ferred to the adult German population. Our findings indi-
cate that the total score reflects a comprehensive feature of
exercise dependence, and that the EDS-G can be used for
screening purposes in a German context as well as for fur-
ther research improving the knowledge about this behav-
ioral excess. In summary, our results demonstrate that the
EDS-G is a brief, psychometrically sound, and valid meas-
ure for the assessment of exercise dependence symptoms.
Acknowledgments
We kindly thank Heather Hausenblas for her support during
the course of this project.
2012 Hogrefe Publishing
 A. Mller et al.: Validation of the German Version of the Exercise Dependence Scale
References
Allegre, B., & Therme, P. (2008). Confirmative study of a French
version of the Exercise Dependence Scale-revised with a
French population. Encephale, 2008, 490495.
American Psychiatric Association. (1994). Diagnostic and statis-
tical manual of mental disorders (4th ed.). Washington, DC:
Author.
Bamber, D. J., Cockerill, I. M., Rodgers, S., & Carroll, D. (2003).
Diagnostic criteria for exercise dependence in women. British
Journal of Sports Medicine, 37, 393400.
Bentler, P. M. (2007). On tests and indices for evaluating structural
models. Personality and Individual Differences, 42, 825829.
Bewell-Weiss, C. V., & Carter, J. C. (2010). Predictors of exces-
sive exercise in anorexia nervosa. Comprehensive Psychiatry,
51, 566571.
Brislin, R. W. (1970). Back translation for cross-sectional re-
search. Journal of Cross-Cultural Psychology, 1, 185216.
Cook, B., & Hausenblas, H. (2008). The role of exercise depend-
ence for the relationship between exercise behavior and eating
pathology: Mediator or moderator? Journal of Health Psychol-
ogy, 13, 495502.
Edmunds, J., Ntoumanis, N., & Duda, J. L. (2006). Examining
exercise dependence symptomatology from a self-determina-
tion perspective. Journal of Health Psychology, 11, 887902.
Flora, D. B., & Curran, P. J. (2004). An empirical evaluation of
alternative methods of estimation for confirmatory factor anal-
ysis with ordinal data. Psychological Methods, 9, 466491.
Hausenblas, H. A., & Symons Downs, D. (2002a). Exercise de-
pendence: A systematic review. Psychology of Sport and Ex-
ercise, 3, 89123.
Hausenblas, H. A., & Symons Downs, D. (2002b). Exercise De-
pendence Scale-21 manual. Retrieved from http://www.per-
sonal.psu.edu/dsd11/EDS/EDS21Manual.pdf
Hausenblas, H. A., & Symons Downs, D. (2002c). How much is
too much? The development and validation of the exercise de-
pendence scale. Psychology and Health, 17, 387404.
Hilbert, A., & Tuschen-Caffier, B. (2006). Eating Disorder Ex-
2012 Hogrefe Publishing
219
amination-Questionnaire. Deutsche bersetzung [Eating Dis-
order Examination-Questionnaire. German translation]. Mn-
ster: Verlag fr Psychotherapie.
Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in
covariance structure analysis: Conventional criteria versus
new alternatives. Structural Equation Modeling, 6, 155.
Jreskog, K., & Srbom, D. (2004). LISREL 8.71 [computer pro-
gram]. Lincolnwood, IL: Scientific Software International.
Landis, J. R., & Koch, G. G. (1977). The measurement of observer
agreement for categorical data. Biometrics, 33, 159174.
Lejoyeux, M., Avril, M., Richoux, C., Embouazza, H., & Nivoli,
F. (2008). Prevalence of exercise dependence and other behav-
ioral addictions among clients of a Parisian fitness room. Com-
prehensive Psychiatry, 49, 353358.
Sicilia, ., & Gonzlez-Cutre, D. (2011). Dependence and phys-
ical exercise: Spanish validation of the Exercise Dependence
Scale-Revised (EDS-R). The Spanish Journal of Psychology,
14, 421431.
Symons Downs, D., Hausenblas, H. A., & Nigg, C. R. (2004).
Factorial validity and psychometric examination of the Exer-
cise Dependence Scale-Revised. Measurement in Physical Ed-
ucation and Exercise Science, 8, 183201.
Published online: July 20, 2012
Astrid Mller
Department of Psychosomatic Medicine and Psychotherapy
Hannover Medical School
Carl-Neuberg-Strae 1
30625 Hannover
Germany
Tel. +49 511 532-9179
Fax +49 511 532-3190
E-mail mueller.astrid@mh-hannover.de
European Journal of Psychological Assessment 2013; Vol. 29(3):213219
 K. Michalczyk et al.:European
Age Differences
Journalofand Measurement
Psychological Invariance
Assessment of Vol.
2013;
2012 Work ingPublishing
Hogrefe Memory
29(3):220229

Original Article

Age Differences and Measurement

Invariance of Working Memory
in 5- to 12-Year-Old Children
K. Michalczyk1,2, N. Malstdt1,2, M. Worgt1,2, T. Knen1,2, and M. Hasselhorn1,2
1
German Institute for International Educational Research (DIPF), Frankfurt a. M., Germany, 2Center for
Individual Development and Adaptive Education of Children at Risk (IDeA), Frankfurt a. M., Germany

Abstract. In this study, we investigated working memory structure and measurement invariance of working memory in children between
5 and 12 years. In a sample of 1,669 children, we administered a battery of 12 computer-based working memory subtests. We found the
same tripartite structure of working memory with a phonological loop, a visual-spatial sketchpad, and a central executive in the age
groups 56 years, 79 years, and 1012 years; interrelations of the latent factors were invariant across the age groups. Only the relation-
ships between the phonological loop and the central executive were weaker in the 5- to 6-year-olds than in the older age groups. The
results corroborate the tripartite working memory model developed by Baddeley (1986) and show that the functional relationships between
working memory components are mostly invariant through childhood. In addition, the findings indicate that future research should focus
on extending the model by a fourth factor representing inhibition.

Keywords: working memory, structure, assessment, children, change, measurement invariance

Introduction than those studying adults. Nevertheless, first findings

In 1986 Baddeley presented one of the most prominent ap-
proaches to working memory research (e.g., Conway, Jarrold,
Kane, Miyake, & Towse, 2007; Jarrold & Towse, 2006). His
model postulates a central executive and two subsystems: the
phonological loop and the visual-spatial sketchpad. The cen-
tral executive works as a flexible master unit that controls and
allocates resources and monitors information processing
across informational domains (see Baddeley, Emslie, Kolod-
ny, & Duncan, 1998; Baddeley & Hitch, 1974). The phono-
logical loop stores verbal information temporarily, whereas
the visual-spatial sketchpad is specialized in processing vis-
ual and spatial information. In adult populations, the compo-
nents of this working memory model have been validated by
a vast body of research including experimental and neuropsy-
chological studies (see Baddeley & Logie, 1999), neuroimag-
ing and neuropsychological studies (e.g., Henson, 2001; Val-
lar & Papagno, 2002). Evidence for this model is also provid-
ed by the individual differences approach, which is
characterized by the use of confirmatory factor analysis (for
a review, see Jarrold & Towse, 2006).
Evidence for the Three-Factor Model in Children
The number of studies using the individual differences ap-
proach to examine samples of children is by far smaller
seem to suggest a multicomponent model that includes the
phonological loop, the visual-spatial sketchpad, and the
central executive as functionally related but separable fac-
tors. For example, findings by Jarvis and Gathercole (2003)
suggest that the visual-spatial sketchpad and the phonolog-
ical loop represent independent entities in 11- and 14-year-
olds and in 5- to 8-year-olds (Pickering, Gathercole, &
Peaker, 1998). In 4- to 5-year-olds, Roebers and Zoelch
(2005) found a clear dissociation of the phonological and
visual-spatial subsystem. In the case of 4-year-olds, how-
ever, the working memory tasks of the phonological loop
and the visual-spatial sketchpad commonly used to assess
working memory subcapacities had to be adapted. As the
authors point out, assessing central executive working
memory capacity before the age of 5 requires the further
development of age-adequate tasks. In confirmatory factor
analyses, this leads to poor model fit and low factor load-
ings. However, with regard to the assessment of the three
subsystems, some findings suggest that a tripartite working
memory structure the phonological loop, the visual-spa-
tial sketchpad, and the central executive exists in young
children. For example, Gathercole, Pickering, Ambridge,
and Wearing (2004) reported data corroborating the as-
sumption of a tripartite structure in children between 6 and
15 years. They point out that this structure might be found
in younger age groups as well. Subsequently, Alloway, Ga-
thercole, and Pickering (2006) indeed found that the tripar-

European Journal of Psychological Assessment 2013; Vol. 29(3):220229 2012 Hogrefe Publishing
DOI: 10.1027/1015-5759/a000149
 K. Michalczyk et al.: Age Differences and Measurement Invariance of Working Memory
tite model well fit the working memory functioning of 4-
to 6-year-olds. Furthermore, Alloway, Gathercole, Willis,
and Adams (2004, see also Alloway et al., 2005) suggested
a model for 4- to 6-year-olds in which the central executive
and the phonological loop represent entities that are sepa-
rable from other cognitive abilities (e.g., nonverbal intelli-
gence, phonological awareness).
Relationship of the Visual-Spatial Sketchpad
and the Central Executive
However, not all findings indicate such clear separability
of the different working memory components. For in-
stance, in 6- and 7-year-olds the visual-spatial sketchpad
was not clearly dissociable from the central executive (Ga-
thercole & Pickering, 2000). Alloway et al. (2006) found
that the correlation of the visual-spatial component was
higher with the central executive in 4- to 6-year-olds than
in older age groups (i.e., 78 and 911 years). Nevertheless,
Gathercole et al. (2004) argue in favor of the separability
of these two constructs in the course of development. This
raises the question whether the visual-spatial component
and the central executive are separate functional working
memory structures in young children.
Functional Differentiation of the Central
Executive
Earlier conceptions of the Baddeley model (e.g., Baddeley,
1986; Baddeley & Hitch, 1974) did not describe the func-
tions of the central executive precisely. However, in later
conceptions (see Baddeley, 1986, 2003) the role of the cen-
tral executive was specified more clearly by postulating
1. processes of focusing attention (switching),
2. coordination of the subsystems,
3. controlling of encoding and decoding strategies, and
4. retrieval of information from long-term memory.
There are, however, alternative or complementary concep-
tions of central executive functioning (e.g., Miyake et al.,
2000; see also St Clair-Thompson & Gathercole, 2006).
Miyake et al. (2000) presented a theoretical framework in
which three executive functions mental set shifting, up-
dating, and inhibition play different roles in executive
tasks. Miyake et al. (2000) showed that these three factors
represent related, but separable constructs. It is important
to note, however, that their theoretical framework was de-
rived from studying adult populations. Nevertheless, in
children, some recent findings suggest that at least inhibi-
tion may represent a factor that can be dissociated from the
central executive: For instance, findings by St Clair-
Thompson and Gathercole (2006) suggested that at least
two executive factors exist in 11- to 12-year-old children,
one being associated with updating functions and the other
2012 Hogrefe Publishing
221
with inhibition. Furthermore, Lambek and Shevlin (2011)
reported that, in children aged 712 years and 1316 years,
verbal working memory, visual-spatial memory, and inhi-
bition represent related but separable constructs, as pro-
posed by Miyake et al. (2000). However, other studies
failed to identify inhibition as a factor separable from work-
ing memory and shifting (e.g., Huizinga, Dolan, & van der
Molen, 2006; Van der Sluis, De Jong, & Van der Leij,
2007). A recent study by Shing, Lindenberger, Diamond,
and Davidson (2010) indicated that, although inhibition
may not be a factor separable from memory maintenance
in children aged 47 years or 79.5 years, in children aged
9.514.5 years it is. Therefore, this finding might indicate
that the differentiation of inhibition emerges in the course
of development. To date, the organization of central exec-
utive processes in young children is far from being fully
understood. Especially, it still remains unclear whether in-
hibition represents a separate component of working mem-
ory in children, and if so, at what age this can be observed.
Developmental Change in the Structure of
the Three-Factor Model
To our knowledge, only few studies have as yet assessed
the change in the structure of working memory in children.
Some first findings, however, provided support for the idea
of a three-factor model as described by Baddeley (1986),
which is mostly invariant across different age groups from
6 to 15 years (Gathercole et al., 2004) or even in younger
children aged 411 years (Alloway et al., 2006). Both stud-
ies found high latent factor correlations of the central ex-
ecutive with the phonological loop and the visual-spatial
sketchpad, whereas the correlations between the subsys-
tems were lower, albeit of similar strength throughout de-
velopment. These findings might be taken as empirical in-
dications of the adequacy of the assumption of an invariant
tripartite working memory structure in the age range 4 to
15 years. However, more evidence is certainly required in
order to corroborate the current findings, especially with
regard to children younger than 6 years.
In sum, the structure of working memory functioning
remains to be further validated in children. The number of
working memory components in children is unclear as is
the extent to which they are correlated. Moreover, whether
the relationship of working memory components, that is,
their functional interdependence, changes over childhood
requires further research. Accordingly, we decided to ad-
dress the following assumptions in more detail:
1. As outlined above, one should expect to find Baddeleys
tripartite structure of working memory (1986) in the age
range between 5 and 12 years (see Alloway et al., 2004,
2005, 2006; Roebers & Zoelch, 2005). A hierarchical
structure across all age groups should be found within
this framework. In particular, the phonological loop is
likely to be only moderately related to the visual-spatial
European Journal of Psychological Assessment 2013; Vol. 29(3):220229
222 K. Michalczyk et al.: Age Differences and Measurement Invariance of Working Memory
sketchpad. The central executive, in contrast, should be
more strongly related to both subsystems and be separa-
ble from the visual-spatial component (e.g., Alloway et
al., 2006; Baddeley, 1986; Gathercole et al., 2004).
2. Although alternative conceptions of working memory in
children exist (see Miyake et al., 2000), we assume Bad-
deleys model to be superior to a single-factor model.
Nevertheless, within the theoretical framework pro-
posed by Miyake et al. (2000), it is conceivable that ad-
ditional dissociable central executive functions can be
differentiated (e.g., Lambek & Shevlin, 2011; St Clair-
Thompson & Gathercole, 2006). Accordingly, a second
aim of the present study was to contribute to the inves-
tigation of whether it is more appropriate to separate in-
hibition from other central executive functions.
3. A third aim of our study is to explore whether both the
structure and the functional interrelations of the phono-
logical loop, the visual-spatial sketchpad, and the central
executive are invariant throughout the course of chil-
drens development (e.g., Alloway et al., 2006; Gather-
cole et al., 2004). We expect the relation of the central
executive and the phonological loop to be weaker in
children younger than 7 years. This assumption is based
on the finding that spontaneous subvocal rehearsal with-
in the phonological loop, which is moderated by the cen-
tral executive, does not reliably occur before the age of
7 (see Gathercole & Hitch, 1993).
Method
Sample/Participants
Data were selected from the norming sample of the Work-
ing Memory Test Battery for Children (AGTB 512; Has-
selhorn et al., 2012). 1,669 children (856 boys and 813
girls) from nursery schools, primary and secondary schools
in Frankfurt/Main, Gttingen, Eichsttt and surrounding
areas in Germany participated in this study. The children
were divided into three age groups: 56 years (n = 284),
79 years (n = 690), and 1012 years (n = 695). These age
ranges were chosen because they mark key stages in young
childrens cognitive development related to working mem-
ory functions. First, rehearsal strategies associated with the
central executive do not reliably occur automatically before
the age of 7 (e.g., Gathercole & Hitch, 1993). Second, ab-
stract self-reflection emerges at the age of approximately
10 years (Hasselhorn, 2008; Piaget, 1971), at which age
children begin to think about their knowledge and their
learning processes. This, in turn, enhances the development
of strategies which relate to central executive functioning.
1 A more detailed description of the working memory tasks is available http://www.idea-frankfurt.eu/journal/ejpa/appendix_M_et_al
European Journal of Psychological Assessment 2013; Vol. 29(3):220229
Procedure
Subjects completed 12 tests from the German Working
Memory Test Battery for Children, 512 years (AGTB
512; Hasselhorn et al., 2012). They were tested individ-
ually in a quiet area of the nursery school or the school
with a break after six tasks, which resulted in approxi-
mately 2 35 minutes of testing.
Working Memory Measures1
Phonological Loop
Four of the working memory subtests assessed the phono-
logical loops efficiency. In the word span forward
(one/three syllables) and the digit span task the child was
required to repeat one/three syllable words or digits in the
presented order. In the nonword repetition task, the child
was required to correctly recall a word-like sound that did
not occur in the German language and has no meaning in
German either (e.g., limparett).
Visual-Spatial Working Memory
Two tasks addressed the visual-spatial working memory
system. In the matrix task, the child was required to recall
black squares in a 4 4 matrix by pointing at a touch-
screen. In the Corsi-block task, a yellow smiley appeared
sequentially over nine empty blocks. The child was then
required to indicate on the touchscreen the serial order of
the blocks in which the smiley had previously appeared.
Central Executive
Six tasks assessed the efficiency of the central executive
and/or inhibition. In the digit span backwards task, the
child was asked to repeat the given sequence of digits in
reverse order. In color span backwards, the child was re-
quired to identify the colored dots presented in reverse
order. In the object span task, the child was required to
say whether the object was edible or nonedible. At the end
of the tasks, the child was asked to recall the items in the
presented order. In the counting span task, the child was
presented a series of pictures depicting circles and asked
to count the number of circles shown as well as to say the
respective number aloud (e.g., four). At the end of the
task, the child was required to recall the numbers she or
he had said aloud (e.g., four, nine). In the go/nogo task,
the child was asked to press a key whenever she or he saw
a specified item (go trial), for example, a yellow balloon.
In a nogo trial, the item was similar but distinct (e.g., a
red balloon). In the Stroop task, a stylized picture of a man
2012 Hogrefe Publishing
 K. Michalczyk et al.: Age Differences and Measurement Invariance of Working Memory 223

or woman was presented on the upper half of the touch-
screen and two other stylized pictures of a man and wom-
an appeared on the lower half. Simultaneously, the child
was given the verbal cue of the word man or woman.
The child was asked to react to the congruent visual stim-
ulus only (man man; woman woman) by tapping onto
the congruent figure in the lower half of the touchscreen
while ignoring the verbal cue.
Stroop task encompassed 24 trials and the number of cor-
rectly repeated congruent trials was scored. Table 1 shows
the retest reliabilities for children aged 56 years, 79
years, and 1012 years (AGTB 512; Hasselhorn et al.,
2012).
Statistical Modeling Procedures

Testing Details and Scoring In order to model the structure of working memory, confir-
matory factor analysis was carried out with Mplus 6.1 (Mu-
Every child was required to complete ten trials of each of thn & Muthn, 19982010) using maximum likelihood
the adaptive tests (i.e., word span tasks, digit span task, estimation with robust standard errors (MLR). Model fit
matrix, Corsi-block, digit span backwards, color span was assessed by the following fit indices:
backwards, counting span, object span). Within each trial 1. The statistic with a nonsignificant value indicating
the length of the list varied, depending on the childs per- an exact model fit. However, the statistic is sensi-
formance. The mean number of correctly repeated items tive to sample size and may therefore become signif-
over these ten trials was then scored. The number of trials icant in large samples despite a good model fit (Kline,
in the nonword repetition task was 24, and the number of 1998),
the correctly repeated trials was scored. In the go/nogo 2. the comparative fit index (CFI; Bentler, 1990) where a
task, the number of trials was 24. As signal detection the- value equal to or greater than .90 represents an adequate
ory applied to the go/nogo task, we calculated the ability fit,
to discriminate (d) over 24 trials according to the proce- 3. the Bayesian information criterion (BIC),
dure described by Stanislaw and Todorov (1999). The 4. the standardized root mean square residual (SRMR),
where a value equal to or lower than .10 indicates a good
Table 1. Retest reliabilities for the age groups 56, 79, and model fit, and
1012 5. the root mean squared error of approximation (RMSEA;
Hu & Bentler, 1999), adjusted for multiple groups.
Retest reliabilitya
Measure 56b 79c 1012d
The RMSEA is less affected by sample size than other fit
Word span forward (one syllable) .54 .69 .77
indices, and values equal to or lower than .08 are accept-
Word span forward (three syllables) .51 .68 .56
able, whereas values equal to or lower than .05 represent
Digit span forward .62 .72 .83 a good fit (McDonald & Ho, 2002). In addition, confi-
Nonword repetition .65 .80 .85 dence intervals can be calculated even if data deviate
Matrix span .35 .63 .34 from normal distribution. Changes in model fit in nested
Corsi-block .61 .62 .54 model comparisons were assessed using a Satorra-Bent-
Digit span backwards .42 .70 .66 ler scaled -difference test ( SB-; Satorra & Bentler,
Color span backwards .37 .57 .66
2001), where a nonsignificant value indicates that a nest-
ed model can describe the data just as well as the com-
Object span .53 .53 .38
parison model. Following recommendations by Cheung
Counting span .68 .60 .59 and Rensvold (2002) measurement invariance was addi-
Go/no-go .46 .37 .40 tionally assessed with the CFI, where values smaller
Stroop .66 .74 .62 than or equal to .01 indicate no difference between the
Subsystem restricted and the unrestricted model, and the null hy-
Phonological loop .82 .86 .90 pothesis of invariance should not be rejected. This indi-
Visual-spatial sketchpad .62 .69 .57 cator is independent of model complexity and size. It
Central executivee .74 .83 .83
does not correlate with overall fit measures.
Notes. Data were drawn from the AGTB 512 norming sample (Has-
selhorn et al., 2012).
a
The test-retest interval was 12 weeks.
b
n = 66; mean age = 6;1; SD = 8 months, range = 5;0 to 6;11 Descriptive Statistics
c
n = 125; mean age = 8;4; SD = 9 months, range = 7;0 to 9;11
d
n = 69; mean age = 11;5; SD = 11 months, range = 10;0 to 12;11
e
In the AGTB 512 norming sample (Hasselhorn et al., 2012) retest The mean scores and standard deviations of each working
reliabilities are shown for the central executive including the go/nogo memory measure are shown as a function of age group in
task and the stroop task. Table 2.

2012 Hogrefe Publishing European Journal of Psychological Assessment 2013; Vol. 29(3):220229
224 K. Michalczyk et al.: Age Differences and Measurement Invariance of Working Memory

Table 2. Means and standard deviations as functions of age Results

group
Age (in years) In the three age groups, distributions of the raw data de-
Measure 56 79 1012 parted from univariate normality distribution in univariate
(n = 284) (n = 690) (n = 695)
skewness and kurtosis. Multivariate skewness and kurto-
sis was also not given. Transforming the raw data to age
Word span forward (one syllable) specific t-values2 showed that normality assumptions
M 3.04 3.56 4.13 were now given in the univariate tests. For the t-values,
SD 0.58 0.68 0.73 Mardias test revealed multivariate normal distribution re-
garding multivariate skewness in the age group 7 to 9
Word span forward (three syllables)
years, and regarding multivariate kurtosis in all three age
M 2.62 2.91 3.20 groups. However, multivariate normal distribution was
SD 0.40 0.44 0.50 not given for the age groups 56 years and 1012 years
Digit span forward as indicated by multivariate skewness. Therefore, while
we continued using t-values, we estimated all model pa-
M 3.22 4.08 4.83
rameters, standard errors, and fit indices using the maxi-
SD 0.72 0.75 0.78 mum likelihood robust rescaling procedure which applies
Nonword repetition when data do not fulfill the criterion of normal distribu-
M 14.69* 14.5* 13.52* tion.
SD 4.44 4.38 4.39
Matrix span
Structural Equation Modeling
M 2.76 4.18 5.57
SD 0.78 1.24 1.37 The Tripartite Model
Corsi-block First, we addressed the question of working memory struc-
M 2.90 3.88 4.71 ture in the different age groups. In accordance with our
SD 0.81 0.82 0.79 assumptions, we used the tripartite model as a starting point
for our analyses. We found that the same working memory
Digit span backward
model provided a good fit for the data in all three different
M 2.13 2.99 3.69 age groups; Figure 1 shows the fit indices, residual vari-
SD 0.62 0.67 0.75 ances, factor loadings, and latent factor correlations for the
Color span backward three-factor model for the age groups 56 years, 79 years,
and 1012 years. It is important to note that the go/nogo
M 1.93 2.71 3.50
task and the Stroop task loaded poorly on the central exec-
SD 0.60 0.79 0.86 utive factor in all age groups. However, the tripartite struc-
Object span ture of working memory turned out to be adequate in all
M 2.06 2.84 3.37 three age groups. In addition, across all three age groups,
correlations of the central executive with the two subsys-
SD 0.59 0.72 0.76
tems were higher than the interrelations between the two
Counting span subsystems.
M 1.80 2.90 3.63
SD 0.63 0.81 0.85
Go/no-go Alternative Models
M 1.22 1.81 2.34 Single-Factor Model
SD 0.84 0.78 0.70
We also tested a single-factor model for the three age
Stroop groups. In the age group 56 years, the model fit was inad-
M 49.99 49.99 50.00 equate with (54, n = 284) = 179.362, p < .01, CFI = .804;
SD 9.94 9.96 9.97 BIC = 24464; RMSEA = .090 (90% CI: .076.105). In the
Note. *The different age groups received wordlists with varying de- age group from 7 to 9 years, the model fit was slightly better
grees of difficulty, which explains the stable level of points received than in the group of younger children with (54, n = 690)
in all age groups. = 318.635, p < .01; CFI = .878; BIC = 58531; RMSEA =

2 The correlations the age range specific t-values of the working memory tasks are available at http://www.idea-frankfurt.eu/journal/ejpa/ap-
pendix_M_et_al

European Journal of Psychological Assessment 2013; Vol. 29(3):220229 2012 Hogrefe Publishing
 K. Michalczyk et al.: Age Differences and Measurement Invariance of Working Memory 225

Table 3. Comparison of fit in the nested models for whole sample

MIa df Compare with df SB- p CFI CFI BIC SRMR RMSEA (90%CI)
COI 153 289.296 .974 141584 .035 .040 (.033.047)
MEI 171 312.312 COI 18 22.92 .19 .973 .001 141473 .047 .039 (.032045)
SC 189 313.263 MEI 18 .64 .99 .976 .003 141340 .047 .034 (.027.041)
SII 213 344.946 SC 24 31.68 .14 .975 .001 141194 .046 .033 (.027.040)
EV 219 352.086 SII 6 6.84 .34 .975 .000 141156 .048 .033 (.027.039)
ECOV_a 224 352.947 EV 5 .86 .97 .976 .001 141119 .048 .032 (.026.038)
ECOV 225 368.498 ECOV_a 1 19.55 .00 .973 .003 141124 .052 .034 (.028.040)
Notes. aMI-Model: COI = all parameters estimated freely (configural invariance); MEI = factor loadings set as equal across groups (metric
invariance); SC = factor loadings and intercepts set as equal across groups (scalar invariance/strong measurement invariance); SII = factor
loadings, intercepts, residuals set as equal across groups (strict invariance); EV = factor loading, intercepts, residuals, and latent factor variances
set as equal across groups; ECOV_a = factor loadings, intercepts, residuals, variances, and covariances set as equal across groups, with covariance
of the phonological loop and the central executive in age group 56 estimated freely; ECOV = factor loadings, intercepts, residuals, and
covariances set as equal across groups.

Figure 1. Three-factor model for the age groups 56, 79, and 1012. Read from left to right the digits represent error
terms, factor loadings, and latent factor correlations, respectively, for the age groups 56/79/1012. The model-fit for
the different age groups was as follows:
56: (51, n = 284) = 64.800, p > .05; CFI = .978; BIC = 24664; RMSEA = .031 (90% CI: 0.052);
79: (51, n = 690) = 123.844, p < .01; CFI = .966; BIC = 58353; RMSEA = .045 (90% CI: .035.056);
1012: (51, n = 695) = 101.471, p < .01; CFI = .980; BIC = 58728; RMSEA = .038 (90% CI: .0270.048).

2012 Hogrefe Publishing European Journal of Psychological Assessment 2013; Vol. 29(3):220229
226 K. Michalczyk et al.: Age Differences and Measurement Invariance of Working Memory
.084 (90% CI: .075.093). The same was true for the age
group 1012 years with (54, n = 695) = 318.296, p < .01;
CFI = .895; BIC = 58925; RMSEA = .084 (90% CI:
.075.093). The three-factor model showed better fit in all
age groups, and furthermore, represented a more parsimo-
nious model than the single-factor model.
A Four-Factor Model: Inhibition as a Factor Separable
from the Central Executive
We additionally tested for a model in which inhibition pro-
cesses were represented by an inhibition factor distin-
guishable from the central executive. This factor com-
prised the Stroop task and the go/nogo task. The model fit
well in all three age groups:
5- to 6-year-olds: (48, n = 284) = 57.311; p > .05; CFI
= .985; BIC = 24373; RMSEA = .026 (90% CI: 0.049);
7-to 9-year-olds: (48, n = 690) = 110.311; p < .01; CFI
= .971; BIC = 58358; RMSEA = .043 (90% CI:
.033.054);
10- to 12-year-olds: (48, n = 690) = 90.189; p < .01;
CFI = .983; BIC = 58736; RMSEA = .036 (90% CI:
.024.047).
In sum, the model that assumes a separate inhibition factor
fitted the data just as well as the model in which inhibition
was part of a common central executive factor. Notably, the
factor loadings of the Stroop and go/nogo-tasks on latent
factor inhibition were low across all age groups (Stroop:
.280 to .396, go/nogo .325 to .407).
Measurement Invariance of the Tripartite Model
In the next step of analysis, we addressed the issue of the
age invariance of the functional interrelations between the
different working memory components across the age
groups 56 years, 79 years, and 1012 years. To do so,
we compared the latent factor correlations across the three
age groups (e.g., is the correlation between the phonolog-
ical loop and the central executive lower in the 56-year-
olds than in the 79-year-olds?). However, this compari-
son requires establishing measurement invariance (MI),
which involves testing for a hierarchy of configural invari-
ance, measurement invariance, scalar invariance, and
strict factor invariance: Configural invariance implies that
the same pattern of nonzero factor loadings fits all age
groups, which constrains construct dimensionality to be
equivalent. Metric invariance requires unstandardized
factor loadings to be invariant across age groups. Scalar
measurement invariance implies that, in addition, specific
intercepts of the indicator variables are equal across
groups. Strict factorial invariance implies that the factor
loadings, intercepts, and the residual variances are equiv-
alent across groups. In a subsequent step, variances can
be set as equal across groups, enabling the restriction of
European Journal of Psychological Assessment 2013; Vol. 29(3):220229
factor covariance. When comparing this sequence of nest-
ed models to one another, no changes in model fit mean
that the more parsimonious model is to be preferred. In
the more parsimonious model, a higher number of param-
eters are fixed, which results in a higher number of de-
grees of freedom (df). Table 3 summarizes the results of
these nested model comparisons.
Comparing the model of metric invariance (MEI) with
that of configural invariance (COI) revealed no reliable
difference between the two models. Therefore, we chose
the more parsimonious model MEI. Then, the model as-
suming scalar invariance (SC) was compared with MEI,
resulting in a preference for the more parsimonious mod-
el SC. In the next step, the model of strict invariance (SII)
proved to be more parsimonious than SC and, in turn, the
model of equal latent factor variances (EV) rather than
SII. This permitted testing for change or invariance in
latent factor covariances: In model ECOV_a all factor
loadings, residuals, intercepts, latent factor variances,
and latent factor covariances were set to be equal (except
for the covariance of the phonological loop and the cen-
tral executive in the age group 56). ECOV_a showed a
more parsimonious fit than EV. In the last step of model
comparisons, a model of equal covariances in all three
age groups (ECOV) was tested against ECOV_a. The re-
strictions in ECOV were the same as in ECOV_a, except
that all latent factor covariances were set as equal across
age groups. ECOV_a was revealed to be the most parsi-
monious model to describe the data. In order to formally
complete the analyses, we tested whether any other mod-
el, in which one latent factor covariance was estimated
freely while the remaining latent factor covariances
were as set equal across the three age groups fitted the
data better than EV. ECOV_a fit the data best.
Discussion
This study investigated the working memory structure and
the development of relations in working memory compo-
nents in 5- to 12-year-old children. Our results revealed a
good fit of a model consisting of the phonological loop,
the visual-spatial sketchpad, and the central executive in
children aged 56 years, 79 years, and 1012 years. This
model showed better fit than a single-factor solution. A
four-factor solution, with inhibition as an entity dissocia-
ble from the central executive, showed a fit comparable
to that of the three-factor solution. However, the validity
of the four-factor solution needs to be interpreted with
caution. Furthermore, our results indicate measurement
invariance of the three-factor model across the age groups.
This finding suggests stable relations of the phonological
loop, the visual-spatial sketchpad, and the central execu-
tive throughout development. Only in the 5- to 6-year-olds
was the correlation of the phonological loop and the cen-
tral executive lower than in the older age groups.
2012 Hogrefe Publishing
 K. Michalczyk et al.: Age Differences and Measurement Invariance of Working Memory
Does Baddeleys Model Hold for Children 5
to 12 Years Old?
Baddeleys model (1986) is characterized by three compo-
nents and their hierarchic interrelation. Our results indicate
that modeling of working memory structure with the pho-
nological loop, the visual-spatial sketchpad, and the central
executive (Baddeley, 1986) is adequate even in the age
range from 5 to 12 years. As such, our findings are in line
with previous studies that found the three-factor model to
hold true in children (e.g., Alloway et al., 2006; Gathercole
et al., 2004).
Hierarchy
Our findings are consistent with the hierarchical concep-
tion of working memory. In this conception, the central ex-
ecutive coordinates the flow of information through work-
ing memory and is responsible for the transmission and
retrieval of information from the slave systems (Baddeley,
1986). In our study, this assumption is corroborated by a
higher latent factor correlation of the central executive with
the two subsystems than the correlation of the latent factors
of the two subsystems. Although this pattern has been re-
ported before (e.g., Alloway et al., 2006; Gathercole et al.,
2004), it still merits more detailed discussion:
Central Executive and the Phonological Loop
As mentioned above, in the present study the relationship
of the phonological loop and the central executive was
weaker in the 5- to 6-year-olds with r = .66 than in the other
age groups with r = .86 (7- to 9-year-olds) and r = .83 (10-
to 12-year-olds). A similar increase of latent factor corre-
lations was reported by Gathercole et al. (2004) where the
phonological loop correlated with the central executive
with r = .73 in 6- to 7-year-olds and r = .74 in 8- to 9-year-
olds, which then increased to r = .92 in 10- to 12-year-olds,
and r = .90 in 13- to 15-year-olds. In our study, this increase
in correlation occurred at an earlier point in time, namely,
after the age of 6 years. This finding corresponds to the
assumption that the spontaneous use of rehearsal strategies
occurs in children from the age of 7 years onward (for a
review, see Gathercole, 1998; Gathercole & Hitch, 1993).
However, an alternative explanation for this finding might
be that children in older age groups process central execu-
tive tasks more efficiently. This would imply that, in older
children, these tasks require less central executive effort,
so that the variance in these tasks appears to be related to
the capacity of the phonological loop to a larger extent.
This, in turn, might be interpreted to support the position
that, in older children, the central executive should be elim-
inated as a separate factor, at least with regard to the tasks
we administered in this study. However, the latent factor of
2012 Hogrefe Publishing
227
the central executive correlates consistently with the visu-
al-spatial sketchpad.
The Central Executive and the Visual-Spatial
Sketchpad
In our study, we found a remarkably high correlation be-
tween the visual-spatial sketchpad and the central execu-
tive. This is in line with previous findings (Alloway et al.,
2006; Gathercole et al., 2004). In these studies, the visual-
spatial sketchpad and the central executive correlated par-
ticularly highly in the younger age groups, in children from
4 to 6 years and from 6 to 7 years. However, our study
differs in two regards with respect to the latent factor cor-
relation of the central executive and the visual-spatial com-
ponent. First, latent factor correlations were lower in our
study: In Alloway et al. (2006) the latent factor correlation
was .97 in the youngest age group, whereas it was .85 in
Gathercole et al. (2004). Second, we could not confirm a
drop in latent factor correlations from the younger to the
older age groups to the same degree as these studies. No-
tably, in our study the retest reliability of the matrix task
was low in the youngest and the oldest age groups, which
weakens factor interpretation. Nevertheless, our findings
suggest a high degree of central executive involvement in
childrens visual-spatial component. It has been argued that
especially young children recruit controlled attention when
they perform visual-spatial tasks (e.g., Cowan, 2005).
However, controlled attention might particularly occur in
dynamic visual-spatial tasks (e.g., Duff & Logie, 1999; Lo-
gie, 1995). This is supported in our study by moderate cor-
relations between the Corsi-block task and the central ex-
ecutive tasks (digit span backwards, color span backwards,
object span, and counting span).
Alternative Conceptions to Baddeleys Model
In adult populations, various conceptions of working mem-
ory exist with regard to cognitive processes and structure;
describing them all would go far beyond the scope of this
study. We therefore limited our study to investigating the
validity of two further models with different factor struc-
tures.
First, we tested for a single-factor model, corresponding
to the idea of a g-factor model in the assessment of intelli-
gence. The results indicated that a multicomponent solu-
tion is superior to a single-factor solution, indicating that a
more differentiated account of working memory structure
is more adequate than a single-factor model. This finding
supports the possibility of functional interrelationships of
latent factors in young children.
Second, against the background of the framework pro-
posed by Miyake et al. (2000) we also evaluated a four-fac-
tor model in which inhibition represented a factor separable
from other central executive processes (see Lambek &
European Journal of Psychological Assessment 2013; Vol. 29(3):220229
228 K. Michalczyk et al.: Age Differences and Measurement Invariance of Working Memory
Shevlin, 2011; Shing et al., 2010; St Clair-Thompson &
Gathercole, 2006). We tested whether inhibition may rep-
resent a factor separable from the central executive and
found a good fit for this four-factor model. However, these
results have to be viewed with caution. On the one hand,
the theoretical framework by Miyake et al. (2000) relates
to adult populations and the frontal lobe, which is generally
associated with central executive functions and continues
to develop into adolescence (Nelson, 1995). As pointed out
above, there are not only few findings regarding the func-
tional differentiation of the central executive based on chil-
dren, but these are also rather diverse (see Lambek & Shev-
lin, 2011). On the other hand, in our study, inhibition was
represented by two tasks only, one of which (go/nogo task)
showed low reliability in all three age groups. Therefore,
although our results might support the view that inhibition
is separable from the central executive, further evidence is
needed to clarify the factor structure of central executive
processes with regard to inhibition.
Does Working Memory Structure Change
During Childhood?
Another main goal of our study was to assess whether the
functional interrelations of the working memory components
are age invariant. In particular, across all age groups, the cen-
tral executive was always strongly related to both the phono-
logical loop and the visual-spatial sketchpad, whereas the
phonological loop was moderately associated with the visu-
al-spatial sketchpad. This finding suggests functional invari-
ance of the three working memory components throughout
development. Only the relationships of the central executive
seem to be less involved in the phonological loop in 5- to
6-year-olds than in 7- to 9-year-olds, and 10- to 12-year-olds,
which, as mentioned above, can be explained with the auto-
mated activation of subvocal rehearsal processes from the age
of 7 years onward. Developmental variance or invariance of
factor structure might also be conceivable for the four-factor
model with inhibition as a separate factor (see Lambek &
Shevlin, 2011; Shing et al., 2010). However, in our study, the
go/nogo-task had low retest reliability, and this task also
showed low factor loadings across all age groups. The inter-
pretation of a separate inhibition factor was thus affected, and
we refrained from testing this four-factor model in nested
model comparison.
Limitations and Outlook
In this study, we addressed the working memory structure
and its invariance, taking Baddeleys (1986) model as a
starting point, and tested alternative conceptions such as a
single-factor model and a model in which inhibition may
represent a separate entity (see Miyake et al., 2000). Nota-
bly, different theoretical accounts on working memory ex-
European Journal of Psychological Assessment 2013; Vol. 29(3):220229
ist beyond the models tested here (for an overview, see Mi-
yake & Shah, 1999; or Conway et al., 2007). In the theo-
retical framework proposed by Miyake et al. (2000), other
possibilities of within-factor differentiation of central ex-
ecutive functions may play a role in children as well. Apart
from inhibition, these may include shifting and updat-
ing. Therefore, it may well be possible that shifting and
updating as central executive functions mark separable but
related factors in children (e.g., Letho, Juujrvi, Kooistra,
& Pulkkinnen, 2003). As in the case of inhibition, investi-
gating the existence of such processes and their relations to
other central executive tasks in young children and the
course of their development certainly presents challenges
for future research.
Acknowledgments
This research was funded by the Hessian Initiative for the
Development of Scientific and Economic Excellence
(LOEWE).
References
Alloway, T. P., Gathercole, S. E., Adams, A.-M., Willis, C., Eaglen,
R., & Lamont, E. (2005). Working memory and phonological
awareness as predictors of progress toward early learning goals
at school entry. British Journal of Developmental Psychology,
23, 417426.
Alloway, T. P., Gathercole, S. E., & Pickering, S. J. (2006). Verbal
and visuospatial short-term and working memory in children:
Are they separable? Child Development, 77, 16981716.
Alloway, T. P., Gathercole, S. E., Willis, C., & Adams, A.-M.
(2004). A structural analysis of working memory and related
cognitive skills in young children. Journal of Experimental Child
Psychology, 87, 85106.
Baddeley, A. (2003). Working memory: Looking back and looking
forward. Nature Reviews. Neuroscience, 4, 829839.
Baddeley, A. D. (1986). Working memory. Oxford, UK: Oxford
University Press.
Baddeley, A. D., Emslie, H., Kolodny, J., & Duncan, J. (1998). Ran-
dom generation and the executive control of working memory.
The Quarterly Journal of Experimental Psychology. A, Human
Experimental Psychology, 51, 819852.
Baddeley, A. D., & Hitch, G. (1974). Working memory. In G. Bower
(Ed.), The psychology of learning and motivation (pp. 4789).
New York, NY: Academic Press.
Baddeley, A. D., & Logie, R. (1999). Working memory: The multi-
ple component model. In A. Miyake & P. Shah (Eds.), Models of
working memory: Mechanisms of active maintenance and exec-
utive control (pp. 2861). Cambridge, UK: Cambridge Universi-
ty Press.
Bentler, P. M. (1990). Comparative fit indexes in structural mod-
els. Psychological Bulletin, 107, 238246.
Cheung, G. W., & Rensvold, R. B. (2002). Evaluating goodness-of-
fit indexes for testing measurement invariance. Structural Equa-
tion Modeling, 9, 233255.
Conway, R. A., Jarrold, C., Kane, M. J., Miyake, A., & Towse, J. N.
2012 Hogrefe Publishing
 K. Michalczyk et al.: Age Differences and Measurement Invariance of Working Memory
(2007). Variation in working memory. New York: Oxford Uni-
versity Press.
Cowan, N. (2005). Working memory capacity. East Sussex, UK:
Psychology Press.
Duff, S. C., & Logie, R. H. (1999). Storage and processing in visuo-
spatial working memory. Scandinavian Journal of Psychology,
40, 251259.
Gathercole, S. E. (1998). The development of memory. Journal of
Child Psychology and Psychiatry, and Allied Disciplines, 39(1),
327.
Gathercole, S. E., & Hitch, G. J. (1993). Developmental changes in
short-term memory: A revised working memory perspective. In
A. Collins, S. E. Gathercole, A. M. Conway, & P. E. Morris
(Eds.), Theories of memory (pp. 189210). Hove, UK: Erlbaum.
Gathercole, S. E., & Pickering, S. J. (2000). Assessment of working
memory in six- and seven-year-old children. Journal of Educa-
tional Psychology, 92, 377390.
Gathercole, S. E., Pickering, S. J., Ambridge, B., & Wearing, H.
(2004). The Structure of working memory from 4 to 15 years of
age. Developmental Psychology, 40, 177190.
Hasselhorn, M. (2008). Competencies for successful learning: De-
velopmental changes and constraints. In J. Hartig, E. Klieme, &
D. Leutner (Eds.), Assessment of competencies in educational
settings (pp. 2343). Gttingen: Hogrefe.
Hasselhorn, M., Schumann-Hengsteler, R., Gronauer, J. C., Grube,
D., Mhler, C., Schmid, I., . . . Zoelch, C. (2012). Arbeitsgedcht-
nistestbatterie fr Kinder von 5 bis 12 Jahren (AGTB 512)
[Working memory test battery for Children, 5 to 12 years]. Gt-
tingen: Hogrefe.
Henson, R. (2001). Neural working memory. In J. Andrade (Ed.),
Working memory in perspective (pp. 151173). Hove, UK: Psy-
chology Press.
Hu, L.-T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in
covariance structure analysis: Conventional criteria versus new
alternatives. Structural Equation Modeling, 6, 155.
Huizinga, M., Dolan, C. V., & van der Molen, M. V. (2006). Age-re-
lated change in executive function: Developmental trends and a
latent variables analysis. Neuropsychologia, 44, 20172036.
Jarrold, C., & Towse, J. N. (2006). Individual differences in working
memory. Neuroscience, 139, 3950.
Jarvis, H. L., & Gathercole, S. E. (2003). Verbal and nonverbal
working memory and achievements on national curriculum tests
at 11 and 14 years of age. Educational and Child Psychology, 20,
123140.
Kline, R. B. (1998). Principles and practice of structural equation
modeling. New York: Guilford.
Lambek, R., & Shevlin, M. (2011). Working memory and response
inhibition in children and adolescents: Age and organization is-
sues. Scandinavian Journal of Psychology, 52, 427432.
Letho, J. E., Juujrvi, P., Kooistra, L., & Pulkkinnen, L. (2003). Di-
mensions of executive functioning: Evidence from children.
British Journal of Developmental Psychology, 21, 5980.
Logie, R. H. (1995). Visuo-spatial working memory. Hove, UK: Erl-
baum.
McDonald, R. P., & Ho, M.-H. R. (2002). Principles and practice in
reporting statistical equation analyses. Psychological Methods,
7(1), 6482.
2012 Hogrefe Publishing
229
Miyake, A., Friedman, N. P., Emerson, M. J., Witzki, A. H., Hower-
ter, A., & Wager, T. D. (2000). The unity and diversity of execu-
tive functions and their contributions to complex frontal lobe
tasks: A latent variable analysis. Cognitive Psychology, 100,
49100.
Miyake, A., & Shah, P. (Eds.). (1999). Models of working memory:
Mechanisms of active maintenance and executive control. New
York: Cambridge University Press.
Muthn, L. K., & Muthn, B. O. (19982010). Mplus users guide
(6th ed.). Los Angeles, CA: Muthn & Muthn.
Nelson, C. A. (1995). The ontogeny of human memory: A cogni-
tive neuroscience perspective. Developmental Psychology, 31,
723738.
Piaget, J. (1971). Biology and knowledge. Chicago, IL: Chicago
University Press.
Pickering, S. J., Gathercole, S. E., & Peaker, M. (1998). Verbal and
visuo-spatial short-term memory in children: Evidence for com-
mon and distinct mechanisms. Memory and Cognition, 26,
11171130.
Roebers, C. M., & Zoelch, C. (2005). Erfassung und Struktur des
phonologischen und visuell-rumlichen Arbeitsgedchtnisses
bei 4-jhrigen Kindern [The assessment and structure of phono-
logical and visual-spatial working memory in 4-year-old chil-
dren]. Zeitschrift fr Entwicklungspsychologie und Pdagogi-
sche Psychologie, 37, 113121.
Satorra, A., & Bentler, P. M. (2001). A scaled difference test sta-
tistic for moment structure analysis. Psychometrica, 66,
507514.
Shing, Y. L., Lindenberger, U., Diamond, A., & Davidson, M. C.
(2010). Memory maintenance and inhibitory control differentiate
from early childhood to adolescence. Developmental Neuropsy-
chology, 35, 679697.
St Clair-Thompson, H. L., & Gathercole, S. E. (2006). Executive
functions and achievements in school: Shifting, updating, inhibi-
tion, and working memory. Quarterly Journal of Experimental
Psychology, 59, 745759.
Stanislaw, H., & Todorov, N. (1999). Calculation of signal detection
theory measures. Behavior Research Methods, Instruments, and
Computers, 31, 137149.
Vallar, G., & Papagno, C. (2002). Neuropsychological impairments
of verbal short-term memory. In M. Kopelman, A. D. Baddeley,
& B. Wilson (Eds.), Handbook of neuropsychology (2nd ed.,
pp. 249270). Chichester, UK: Wiley.
Van der Sluis, S., De Jong, P. F., & Van der Leij, A. (2007). Execu-
tive functioning in children, and its relations with reasoning,
reading, and arithmetic. Intelligence, 35, 427449.
Published online: July 20, 2012
Kurt Michalczyk
German Institute for International Educational Research (DIPF)
Schlossstr. 29
60486 Frankfurt a. M.
Germany
Fax +49 69 24708-216
E-mail michalczyk@dipf.de
European Journal of Psychological Assessment 2013; Vol. 29(3):220229
EAPA APPLICATION FORM
EAPA membership includes a free subscription to the European Journal
of Psychological Assessment. To apply for membership in the EAPA,
please fill out this application form and return it together with your
curriculum vitae to: Valentin Bucik, PhD (EAPA Secretary General),
Dept. of Psychology, Faculty of Arts, University of Ljubljana, Askerceva
2, Ljubljana 1000, Slovenia, fax +386 1 425-9303, E-mail tine.bucik@ff.
uni-lj.si and secretary@eapa-homepage.org

Family name . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

First name . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Affiliation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Address . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

City . . . . . . . . . . . . . . . . Postcode . . . . . . . . . . . . . . . . . . . .

Country . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Phone . . . . . . . . . . . . . . . Fax . . . . . . . . . . . . . . . . . . . . . .

E-mail . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

ANNUAL FEES

EURO 75.00 (US $ 84.00)

IMPORTANT!
3-digit security code in
FORM OF PAYMENT signature field on reverse
of card (VISA/Mastercard)
Credit card or 4 digits on the front
(AmEx)
VISA Mastercard/Eurocard American Express

Number

Expiration date / CVV2/CVC2/CID#

Card holders name . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Signature . . . . . . . . . . . . . . Date . . . . . . . . . . . . . . . . . . . . .

Cheque or postal order

Send a cheque or postal order to the address given above

Signature . . . . . . . . . . . . . . Date . . . . . . . . . . . . . . . . . . . . .
 This issue available for free at www.hogrefe.com
A new direction for the European Psychologist!
Moving into the future as a showcase of relevant and intriguing
research from across Europe as well as from further afield.
Official Organ of the EFPA and supported by
other organizations of psychology in Europe
Table of Contents Issue 1, 2013:
Editorial:
European Psychologist Moves Forward
by A. Grob (Editor-in-Chief) and K. Lavallee (Managing Editor)
Original Articles and Reviews:
Using Personal Cell Phones for Ecological Momentary
Assessment by E. Kuntsche and F. Labhart
Psychological Resilience by D. Fletcher and M. Sarkar
Procrastination by K. B. Klingsieck
Impairments in Mental Model Construction and Benefits
of Defocused Attention
by U. von Hecker, G. Sedek, and A. Brzezicka
Subscribe online at www.hogrefe.com/journals/ep
Call for Papers
The European Psychologist welcomes new submissions
for one of the two sections of the journal:
Original Articles and Reviews / EFPA News and Views
For aims and scope and other details, see the Advice for
Authors at www.hogrefe.com/journals/ep
Editor-in-Chief: Alexander Grob
Managing Editor: Kristen Lavallee
Associate Editors: Rainer Banse, Ulrike Ehlert, Giovanni
Galfano, Katariina Salmela-Aro, Norman Anderson
EFPA News and Views Editor: Veronika A. Poliensk
The European Psychologist is a multidisciplinary journal
that serves as the voice of psychology in Europe, seeking
to integrate across all specializations in psychology and
to provide a general platform for communication and
cooperation among psychologists throughout Europe
and worldwide.
From the Editorial:
We have had a productive year at the editorial offices, chang-
ing direction from mostly empirical articles to nearly exclusively
integrative papers. Starting with the first issue of 2013, expect
to see reviews and synthesizing articles on cutting-edge topics,
current directions in the field, and directions for the broad base
of psychologists in both research and practice.
Alexander Grob and Kristen Lavallee
Attachment the Italian Way
by R. Cassibba, G. Sette, M. J. Bakermans-Kranenburg, and
M. H. van IJzendoorn
Intergenerational Family Relations in Luxembourg
by I. Albert, D. Ferring, and T. Michels
EFPA News and Views:
A Decade of European Congresses of Psychology
by V. Poliensk
News and Announcements From the EFPA Network of
National News Correspondents by V. Poliensk
Meeting Calendar
2013 rates (4 issues per annual volume):
Institutions: US$ 214.00 / 159.00 / 127.00
Individuals: US$ 107.00 / 79.00 / 63.00
Special rate for members of EFPA associations and other supporting
organizations: US$ 68.00 / 49.00 / 39.00
+ p/h US$ 16.00 / 12.00 / 10.00
Subscriptions delivered to the Americas will be charged in US$, in the UK
in , in Europe and RoW in .
ISSN-Print 1016-9040, ISSN-online 1878-531X, ISSN-L 1016-9040
Hogrefe Publishing
38 Chauncy Street, Suite 1002 Boston, MA 02111, USA
Tel: (866) 823 4726 Fax: (617) 354 6875 E-Mail: customerservice@hogrefe-publishing.com
Merkelstr. 3 37085 Gttingen, Germany
Tel: +49 551 99950-0 Fax: +49 551 99950-425 E-Mail: customerservice@hogrefe.de
 New!

Compact, authoritative guidance to effective as-

sessment and treatment of the most common psy-
chological difficulties in children and adolescents
phobia and anxiety disorders
This authoritative but compact text addresses the
psychopathology, assessment, and treatment of
the anxiety disorders and phobias in childhood and
adolescence. These perplexing conditions are the
most prevalent psychological difficulties in young
people and result in considerable impairment and
distress, not only to the child but also to her or his
family. Effective treatments exist, but unfortunate-
ly many of these interventions are either not known
to the practicing professionals or not used by them.
This volume aims to address this gap and to present
these interventions in a clear and straightforward
Amie E. Grills-Taquechel & Thomas H. Ollendick manner.
Phobic and Anxiety Disorders in
Children and Adolescents See sample pages at www.hogrefe.com
Advances in Psychotherapy - Evidence-Based Practice,
Vol. 27
2013, x + 130 pp., ISBN 978-0-88937-339-6
US $29.80 / 19.90 / 24.95

Recent praise:
This informative book describes the various ways that anxiety can be detrimental and interfering for children and
adolescents, and explores the course of anxiety development, methods for its assessment, and considerations in its
treatment. The writing is concise and up-to-date, and guided by evidencebased clinical practice. No hocus pocus, just
well-informed guidance.
Philip C. Kendall, Ph.D., ABPP, Distinguished University Professor, Laura H. Carnell Professor, and Director of the
Child and Adolescent Anxiety Disorders Clinic, Department of Psychology, Temple University, Philadelphia, PA, USA
Anxiety disorders of childhood in a perfect nutshell. If you are looking for a concise, thorough, and bang up-to-date
book, then this is the one!
Dr. Sam Cartwright-Hatton, NIHR Career Development Fellow, University of Sussex, UK
Hogrefe Publishing Hogrefe Publishing Hogrefe Publishing c/o Marston Book Services Ltd
30 Amberwood Parkway Ashland, OH 44805 USA Merkelstr. 3 37085 Gttingen Germany 160 Eastern Ave., Milton Park Abingdon, OX14 4SB UK
Tel: (800) 228-3749 Fax: (419) 281-6883 Tel: +49 551 999 500 Fax: +49 551 999 50 111 Tel: +44 1235 465577 Fax +44 1235 465556
E-Mail: customerservice@hogrefe.com E-Mail: customerservice@hogrefe.de E-mail: direct.orders@marston.co.uk

Order online at www.hogrefe.com

or call toll-free (800) 228-3749 (US only)
 H. Schuler, G. C. Thornton, A. Frintrup, & R. Mueller-Hanson

AMI Achievement Motivation Inventory

Complete Test
Consisting of the manual, a question booklet,
20 response sheets, 20 score profiles (1 pad), and a case.
complete test: order # 05 107 01, US $299.00 / 248.00 / 218.00

Area of Application:
Personnel Selection, Potential Analysis, Professional Counseling,
Personnel Development, Profiling, Psychology of Sports, Personality
Research.
General Aspects:
The Achievement Motivation Inventory (AMI) is a personality inven-
tory designed to measure a broad construct of work-related achieve-
ment motivation. It enables users to test candidates for 17 different
facets of achievement motivation. The AMI is based upon the ac-
cepted theories of the construct, but uniquely it also integrates rel-
evant social motives. Thus, in addition to the traditional scales, e.g.,
Confidence in Success or Persistence, scales such as Dominance or
Status Orientation are also included. The AMI consists of 170 items
to be responded to on a 7-point Likert scale.
Reliability:
Reliability (Cronbachs ) for the total score is = .96, and ranges from = .66 to = .83 for individual scales. Retest reli-
ability is rtt = .94 for the total score (rtt = .71 to rtt = .89 for single scales).
Validity:
Content validity has been demonstrated by extensive research on all major aspects of achievement motivation. Expert
ratings ensured that only relevant aspects are included. Additionally, confirmatory factor analysis shows a good fit of
the model to theory. Construct validity is shown by correlations with related personality scales, e.g., Big Five inventories
(ranging up to r = .72). Criterion-related validity is indicated by prediction of grade point averages in US colleges (r = .22
for the total score and up to r = .29 for single scales) and early academic achievements (ranging between r = .21 to r = .36
on different scales). Studies have shown that there is high social validity in regard to acceptance of the test by examinees.
Norms:
N = 335 US students; N = 410 US working adults; N = 1,267 German students; N = 166 German working adults. Additional
norms for the German original are also
available. Also available separately:
Test Duration: Manual: order # 05 107 02, US $151.00 / 98.00 / 108.00
approx. 30 min. Question booklet: order # 05 107 03, US$39.00 / 28.00 / 28.00
20 response sheets: order # 05 107 04, US $106.00 / 72.00 / 76.00
20 score profiles (= 1 pad of 20): order # 05 107 05, US $22.00 / 14.00 / 16.00
Case: order # 05 107 06, US $53.00 / 38.00 / 38.00

Hogrefe Publishing Hogrefe Publishing Hogrefe Ltd.

30 Amberwood Parkway Merkelstr. 3 37085 Gttingen Burgner House 4630 Kingsgate
Ashland, OH 44805 USA Germany Oxford Business Park South
Tel: (800) 228-3749 Fax: (419) 281-6883 Tel: +49 551 999 500 Fax: +49 551 999 50 111 Oxford OX4 2SU UK
E-Mail: customerservice@hogrefe.com E-Mail: customerservice@hogrefe.de Tel: +44 1865 402900 Fax: +44 1865 402888

Order online at www.hogrefe.com

AMI_Ad_2012.indd 1 22.10.2012 11:38:49

 For further details visit www.hogrefe.com

New!
Thought-provoking contributions from the world of the
Rorschach including a special section on psychoanaly-
sis and the Rorschach.
This latest volume of Rorschachiana well illustrates the
diversity of ideas and the many possibilities that projec-
tive methods can offer. The general section includes a
blend of research papers, innovative methods, and com-
plex theoretical ideas.
Rorschachiana is the official publication of the Interna-
tional Society for the Rorschach and Projective Methods,
now with more than 3,000 members throughout the
world, which for over half a century has played an im-
portant role in supporting the use of the instrument, as
well as helping to develop extensive empirical research
on its effectiveness in specific clinical situations.
Rorschachiana is also available as a journal (ISSN 1192-
5604). The journal subscription includes 2 online issues
Sadegh Nashat (Editors)
per year, access to available back issues, and the book as
an annual print compendium.
Rorschachiana
Journal of the International Society
for the Rorschach
2012, viii + 240 pp.
ISBN 978-0-88937-427-0
US $98.00 / 56.00 / 69.95

Table of contents:
Editorial Original Articles
The Importance of Context by Sadegh Nashat Bridging the Gap: Quantitative Rorschach Approaches to
Original Articles Psychodynamic Constructs by Philip Erdberg
Heres Looking at You: Eye Movement Exploration of Ror- An Epistemological and Methodological Approach to Drives
schach Images by Barry Dauphin and Harold H. Greene and Diffusion of Instincts Through the Clinical Assess-
Human Face Responses in the Rorschach Test: A Reconsid- ment of Suicidal Adolescents: The Contribution of the
eration by Yifat Weinberger and Anne Andronikof Rorschach Test by Dimitra Laimou
Resilience, Mentalization, and the Development Tutor: A Psy- Countertransference in the Rorschach Situation as a Clue to
choanalytic and Projective Approach by Claude de Tychey the Patients Affective Functioning: An Exploratory Study
et al. by Emilie Deschenaux et al.
Selflessness in Anorexia Nervosa as Reflected in the Ror- Changing While Remaining the Same: Self-Representation
schach Comprehensive System by Gili Curiel-Levy et al. Confronted With Aging by Benot Verdon
Using Rorschach CS Narrative Responses of the MOA Scale
Special Section: Psychoanalysis and the Rorschach to Construct and Share Patients Model Scenes by Silvia
Editorial Salcuni et al.
Psychoanalysis and the Rorschach by Justine McCarthy Mentalization and the Rorschach by Adam C. Conklin et al.
Woods and Sadegh Nashat Learning From the Inkblot by Latife Yazigi and Sadegh Nashat
Advice for Authors
Hogrefe Publishing Hogrefe Publishing Hogrefe Publishing c/o Marston Book Services Ltd
30 Amberwood Parkway Ashland, OH 44805 USA Merkelstr. 3 37085 Gttingen Germany PO Box 269 Abingdon, OX14 4YN UK
Tel: (800) 228-3749 Fax: (419) 281-6883 Tel: +49 551 999 500 Fax: +49 551 999 50 111 Tel: +44 1235 465577 Fax +44 1235 465556
E-Mail: customerservice@hogrefe.com E-Mail: customerservice@hogrefe.de E-mail: direct.orders@marston.co.uk

Order online at www.hogrefe.com

or call toll-free (800) 228-3749 (US only)

Rorschachiana_ad_2012.indd 1 18.12.2012 11:32:58

 Instructions to Authors European Journal of Psychological Assessment
The European Journal of Psychological Assessment publishes pa-
pers on all domains of psychological assessment as well as on the as-
sessmentprocess,providedtheyhavenotbeensubmittedforpublication
elsewhere. Of special interest are papers reporting the construction of
new measures or a considerable advancement of an existing measure.
Papers published are original and preferably empirical studies. Review
papersandcasestudieswillalsobeconsiderediftheyresultininnovative
research questions in assessment and decision-making. The Journal
especiallywelcomespapersin which thedatahavebeensampledacross
two or more countries and if applicable with different methods (e.g.,
questionnaire, interview, observation, performance). Test reviews will
not be considered. Papers that propose or discuss alternative measure-
mentmodelsforexistingscalesshouldalsodiscussandpreferablydem-
onstrate the consequences for the construct and criterion validity of the
scales. Papers on psychometric methods will be considered only if they
have a clear bearingonassessmentresearch or practice.Allmanuscripts
will be subject to masked peer review. The Journal also publishes infor-
mation on upcoming meetings, congresses, and other professional
events. The Journal appears four times a year.
Language: All contributions must be in English. It is recom-
mended that authors who are not native speakers of English have
their manuscripts checked by a native-speaker colleague before sub-
mission.
Manuscripts: Authors should follow the Publication Manual of the
American Psychological Association (6th ed.) for general style and
layout. Please arrange and type manuscripts accordingly. Nonstandard
abbreviations should be defined in the text upon first use. Footnotes
should generally be avoided. All manuscripts should be submitted on-
line at http://www.editorialmanager.com/ejpa. Please follow the online
instructions for submission. Should you have any technical queries re-
garding this process, please contact Juliane Munson, Hogrefe Publish-
ing (E-mail juliane.munson@hogrefe.com, Tel. +49 551 99950-422,
Fax +49 551 99950-425). Please direct any editorial questions to the
editorial office, E-mail ejpaeditor@gmail.de. The title page should, in
addition to the title, contain the name and complete address of the cor-
responding author and names and affiliations of all authors. The text
should be written double-spaced with a margin of 2.5 cm (1 inch) on all
sides.
Regular Full-Length Papers may exceed neither 22 manuscript
pages (double-spaced, pitch 12) nor 5,000 words, including figures,
tables, and references. Papers exceeding this limit will immediately
be rejected. They should normally be divided into Summary (not
exceeding 200 words), Keywords (no more than five), Introduction,
Materials and Methods, Results, Discussion, Acknowledgments
(optional), and References.
Statistical and Mathematical Copy: Abbreviations and styling
of statistical and mathematical symbols, Greek letters, etc., as well
as spacing, alignment, punctuation, equations in text, displayed
equations, and tabular presentation or mathematical copy, should
follow the Publication Manual of the APA (Sections 4.414.49).
Figures and Tables: Figures and tables should be numbered
using Arabic numerals. The number of figures and tables should be
kept to a minimum and only be included to facilitate understanding
of the text. The same information should not appear in both a figure
and a table. Each table and figure must be cited in the text and should
be accompanied by a legend on a separate sheet. Please note that
online submission via the Editorial Manager allows text, figures,
and tables to be submitted as separate files. Figures must be supplied
in a form suitable for reproduction: preferably high-resolution bit-
maps (e.g., jpg, 300 dpi) or as vector graphics files. Figures will
normally be reproduced in black and white only. While it is possible
to reproduce color illustrations, authors are reminded that they will
be invoiced for the extra costs involved.
References: References must include all and only works cited in
May 1, 2013. 201 Hogrefe Publishing
the text and are to be arranged alphabetically. Works cited in the text
within the same parentheses should be listed alphabetically. APA style
is to be used for text citations and the reference list throughout (for
details, see Publication Manual of the APA, 6th edtion):
Journal article: Crisp, A.H., Palmer, R.L., & Kalucy, R.S. (1976).
How common is anorexia nervosa? A prevalence study. British
Journal of Psychiatry, 128, 549554.
Entire book: Author (date). Book title. City (State and/or Country if
city is relatively unknown): Publisher.
Article or chapter in edited book: Corbett, J.A., & Turpin, G. (1985).
Tics and Tourettes syndrome. In M. Rutter & L. Hersov (Eds.),
Child and adolescent psychiatry. Modern approaches (2nd ed.,
pp. 516525). Oxford: Blackwell.
Galley Proofs and Offprints: The corresponding author receives
galley proofs of the paper in due course, and these must be sent back to
the publisher by return mail. Changes in the style or content of the
galleys can be made only in exceptional instances. Hogrefe will send
the corresponding author of each accepted paper free of charge an
e-offprint (PDF) of the published version of the paper when it is first
released online. This e-offprint is provided for the authors personal use,
including for sharing with coauthors.
Copyright Agreement: By submitting an article, the author con-
firms and guarantees on behalf of him-/herself and any coauthors that
he or she holds all copyright in and titles to the submitted contribution,
including any figures, photographs, line drawings, plans, maps, sketch-
es, and tables, and that the article and its contents do not infringe in any
way on the rights of third parties. The author indemnifies and holds
harmless the publisher from any third-party claims.
The author agrees, upon acceptance of the article for publication, to
transfer to the publisher the exclusive right to reproduce and distribute
the article and its contents, both physically and in nonphysical, elec-
tronic, or other form, in the journal to which it has been submitted and
in other independent publications, with no limitations on the number of
copies or on the form or the extent of distribution. These rights are
transferred for the duration of copyright as defined by international law.
Furthermore, the author transfers to the publisher the following exclu-
sive rights to the article and its contents:
1. The rights to produce advance copies, reprints or offprints of the
article, in full or in part, to undertake or allow translations into other
languages, to distribute other forms or modified versions of the arti-
cle, and to produce and distribute summaries or abstracts.
2. The rights to microfilm and microfiche editions or similar, to the use
of the article and its contents in videotext, teletext, and similar sys-
tems, to recordings or reproduction using other media, digital or
analog, including electronic, magnetic, and optical media, and in
multimedia form, as well as for public broadcasting in radio, televi-
sion, or other forms of broadcast.
3. The rights to store the article and its contents in machine-readable or
electronic form on all media (such as computer disks, compact disks,
magnetic tape), to store the article and its contents in online databases
belonging to the publisher or to third parties for viewing or for down-
loading by third parties, and to present or reproduce the article or its
contents on visual display screens, monitors, and similar devices,
either directly or via data transmission.
4. The rights to reproduce and distribute the article and its contents by
all other means, including photomechanical and similar processes
(such as photocopying or facsimile), and as part of so-called docu-
ment delivery services.
5. The right to transfer any or all of the rights mentioned in this agree-
ment, as well as the rights retained by the relevant copyright clearing
centers, including the corresponding royalty rights to third parties.
Online Rights for Journal Articles: Guidelines on authors rights to
archive electronic versions of their manuscripts online are given in the
Advice for Authors on the journals web page at www.hogrefe.com.
May 1, 2013. 201 Hogrefe Publishing
 New!
General Aspects:

The FTT is a projective test that is based on 21 drawn cards of

well-known fairy tale characters, for example, Little Red Riding
Hood, Snow White, giant, witch, etc.
In contrast to other thematic tests, the child is not required to
tell a story or complete a story, but to answer directed ques-
tions about the characters and situations in the fairy tales.
There are 30 personality variables in the FTT that are described
and differentiated in detail with user-friendly notes and illus-
trations in the margins. Clear instructions and examples are
given for rating the test as well as quantitatively and qualita-
tively interpreting the variables. The most common defense
mechanisms observed in the FTT are listed with examples. Two
case studies of children from the Indian sample highlight how
to use the test and illustrate the clinical usefulness of this test.
In addition, in order to aid scoring and interpretation, the most
frequently reported answers and the most frequently expressed
variables for each set of cards are listed in the appendix.

Area of Application:
The Fairy Tale Test (FTT) is suitable for children aged 612 years.
Carina Coulacoglou It can be applied in individual therapeutic work, in educational
with co-researchers Nilanjana Sanyal counseling, and in remedial educational work as a basic tool for
and Manisha Dasgupta clinical anamnesis, as a method of assessing personality chang-
es over time, and as an instrument in clinical (therapeutic) and
The Fairy Tale Test cross-cultural research.
2013
Complete test kit consisting of manual, 21 test Validity:
cards, and 10 recording sheets.
Complete test: order # 05 108 01, The validity of the FTT was examined through the application
US $193.00 / 110.00 / 138.00 of factor analyses (first and second order) on the 30 personality
variables.
Also available separately:
Manual: order # 05 108 02, Norms:
US $111.00 / 63.00 / 79.00
Test cards (21 cards): order # 05 108 03, Percentiles and T scores for the FTT variables are listed accord-
US $48.00 / 27.50 / 34.50 ing to gender, age and test region in India based a nonclinical
Recording sheets (10 sheets): order # 05 108 04, sample of Indian children (N = 1,355).
US $34.00 / 19.50 / 24.50
Test Duration:
Approximately 45 minutes test administration time. The time
required for scoring and interpretation lies between 30 to 60
minutes depending on the complexity of the answers and the
experience of the tester.
Hogrefe Publishing Hogrefe Publishing
30 Amberwood Parkway Ashland, OH 44805 USA Merkelstr. 3 37085 Gttingen Germany
Tel: (800) 228-3749 Fax: (419) 281-6883 Tel: +49 551 999 500 Fax: +49 551 999 50 111
E-Mail: customerservice@hogrefe.com E-Mail: customerservice@hogrefe.de

Order online at www.hogrefe.com

or call toll-free (800) 228-3749 (US only)

FTT_ad_2013.indd
View publication stats1 21.01.2013 09:32:47