Вы находитесь на странице: 1из 9

Psychiatry Research 216 (2014) 146154

Contents lists available at ScienceDirect

Psychiatry Research
journal homepage: www.elsevier.com/locate/psychres

Bifactor structural model of symptom checklists: SCL-90-R and Brief

Symptom Inventory (BSI) in a non-clinical community sample
Rbert Urbn a, Bernadette Kun a, Judit Farkas a,b, Borbla Paksi c, Gyngyi Kknyei a,
Zsolt Unoka d, Katalin Felvinczi a, Attila Olh a, Zsolt Demetrovics a,n
Institute of Psychology, Etvs Lornd University, Izabella utca 46, Budapest 1064, Hungary
Doctoral School of Psychology, Etvs Lornd University, Izabella utca 46, Budapest 1064, Hungary
Centre for Behavioural Research, Corvinus University of Budapest, Fvm tr 8., Budapest 1093, Hungary
Department of Psychiatry and Psychotherapy, Semmelweis University, Balassa J. utca 6, Budapest 1083, Hungary

art ic l e i nf o a b s t r a c t

Article history: The Derogatis symptom checklist (SCL-90-R) and its short version, the Brief Symptom Inventory (BSI), are
Received 29 July 2013 widely used instruments, despite the fact that their factor structures were not clearly conrmed.
Received in revised form The goals of this research were to compare four measurement models of these instruments including
11 January 2014
one-factor, nine-factor, a second-ordered factor model and a bifactor model, in addition to testing the
Accepted 16 January 2014
Available online 27 January 2014
gender difference in symptom factors in a community sample. SCL-90-R was assessed in a large
community survey which included 2710 adults who represent the population of Hungary. Statistical
Keywords: analyses included a series of conrmatory factor analyses and multiple indicator multiple cause (MIMIC
Symptom checklist (SCL-90-R) modeling). The responses to items were treated as ordinal scales. The analysis revealed that the bifactor
Brief Symptom Inventory (BSI)
model yielded the closest t in both the full SCL-90-R and BSI; however the nine-factor model also had
Conrmatory factor analysis
an acceptable level of t. As for the gender differences, women scored higher on global severity,
Bifactor model
Gender differences somatization, obsession-compulsion, interpersonal sensitivity, depression and anxiety factors. Men
scored higher on hostility and psychoticism. The bifactor model of symptom checklist supports the
concept of global symptom severity and specic symptom factors. Global symptom severity explains the
large correlations between symptom factors.
& 2014 Elsevier Ireland Ltd. All rights reserved.

1. Introduction concluded that the structural stability of SCL-90-R is weak across

different diagnostic groups, social groups and genders. Further-
The SCL-90-R and its short version, the Brief Symptom Inven- more, the fact that cross-loadings of several items on several
tory (BSI), are widely used instruments to measure self-reported factors are frequently reported, and that the rst unrotated factor
psychological distress and psychopathological symptoms in sam- usually explains disproportionally higher variance than the fol-
ples of psychiatric patients and community non-patients lowing factors, implies that the symptom dimensions might
(Derogatis and Savitz, 2000). These instruments encompass three measure a general distress or discomfort factor and question the
global indices and nine subscales covering clinically relevant multidimensional nature of any version of the instrument.
psychiatric and psychosomatic symptoms (Derogatis, 1983). In the conrmatory factor analysis framework the most fre-
Extensive research effort has been devoted to investigating the quently tested measurement model specied nine correlating
factor structure of both full and briefer versions of symptom rst-order factors; however, the vast majority of psychometric
checklist in different countries and languages; however, there is studies did not support this model and reported its failure to meet
still no agreement regarding the factor structure of SCL-90-R and the conventional criteria for goodness-of-t (Hoffmann and
BSI. Exploratory factor analytical studies have reported various Overall, 1978; Hafkenscheid, 1993). Yet, only a few studies com-
solutions from one to nine factors (Hoffmann and Overall, 1978; pared several measurement models of SCL-90-R (Carpenter and
Brophy et al., 1988; Hafkenscheid, 1993; Holi et al., 1998; Schmitz Hittner, 1995; Vassend and Skrondal, 1999; Schmitz et al., 2000;
et al., 2000; Prunas et al., 2012). In their review, Cyr et al. (1985) Hafkenscheid et al., 2006) and concluded that none of the
proposed models t satisfactorily to the data.
There might be several reasons behind the failure to nd an
Correspondence to: Department of Clinical Psychology and Addiction, Institute
of Psychology, Etvs Lornd University, Izabella utca 46, Budapest 1064, Hungary.
adequate level of t in research on SCL-90-R. Among them, we
Tel.: 36 30 97 610 97; fax: 36 1 461 2697. highlight two possible statistical reasons. The rst is that the vast
E-mail address: demetrovics@t-online.hu (Z. Demetrovics). majority of previous studies applied the maximum likelihood (ML)

0165-1781/$ - see front matter & 2014 Elsevier Ireland Ltd. All rights reserved.
R. Urbn et al. / Psychiatry Research 216 (2014) 146154 147

estimation method which is prone to serious deviation from one-dimensional model and an oblique simple structure model.
multivariate normal distribution. value is inated under the Direct comparisons suggested that the bifactor model slightly
condition of moderate non-normality with values becoming more outperformed the simple structure model in accounting for the
and more inated as non-normality increases, resulting in an internal structure of BSI, which is the brief version of SCL-90-R.
enlarged level of Type I error (Finney and DiStefano, 2006). The
second possible reason is the treatment of responses as a linear 1.1. Gender difference in symptomatology
scale instead of an ordinal scale. This is a frequent practice and is
something of a tradition in psychological and psychiatric research; Women tend to report more somatic and psychological symp-
however, in cases of severe oor and ceiling effect it might be toms than men (Barsky et al., 2001; A, 2007). The gender
misleading when estimating the model t. However, it is well difference in symptomatology is reported in adolescent samples
documented that participants or clients are sometimes confused in as well. Girls reported signicantly higher levels of emotional
using the rating scales (Low, 1988). Using Rasch analysis there is distress, in particular depressed mood and anxiety, than did boys
evidence that although the SCL-90-R response categories advance (Ostrov et al., 1989; Casper et al., 1996). Apart from adolescents,
monotonically from zero (not at all) to four (extremely), the gender difference in psychiatric symptoms and disorders is fre-
patients did not effectively discriminate between two (moder- quently reported in adults as well. In primary care patients, mood,
ately) and three (quite a bit) in their answers (Elliott et al., anxiety, and somatoform disorders and psychiatric comorbidity
2006). In case of non-normally ordered categorical data, maximum were signicantly more common in women than men (Linzer
likelihood-based values are inated and values of comparative et al., 1996). Large epidemiological surveys using diagnostic inter-
t index (CFI) are underestimated (Finney and DiStefano, 2006). If view methods presented evidences that women have a higher
the selection of estimation method ignores the nature of rating prevalence of affective disorders, anxiety disorders and non-
scales then a correctly specied model might not t the data well. affective psychosis than men, while men have higher rates of
The structure and structural stability of SCL-90-R deserves substance use disorders and antisocial personality disorders.
further investigation, thus alternative solutions and approaches Moreover women are more likely to report psychiatric comorbid-
were proposed based on exploratory or theory-based approaches. ities of three or more disorders than men (Kessler et al., 1994;
Recently, item response theory was also applied to investigate the Vicente et al., 2006). The gender difference in symptoms should be
measurement properties of SCL-90-R (Olsen et al., 2004; Elliott similar in the cases of SCL-90-R and BSI; however, it also depends
et al., 2006; Paap et al., 2011). Olsen et al. (2004) presented on the factorial structure of the symptom checklist. For example,
evidence that the items belonging to six subscales formed a strong in the case of a bifactor model it becomes possible to identify
unidimensional scale in a Danish community sample. Elliott et al. gender differences in global severity. Another hypothesis could
(2006) identied one big factor measuring overall clinical distress, also state that gender difference exists only in specic symptoms.
with two small residual subscales measuring depressive motiva- However, in order to test gender differences in psychiatric symp-
tional decit and social distress. In their recent analysis, Paap et al. toms, gender invariance of measurement model should be sup-
(2011) applied a theory-driven item response theory approach and ported. Meaningful comparisons of means and regression
identied 60 items that were clustered in seven scales supporting coefcients can only be made if the measures are comparable
the multidimensionality of SCL-90-R. across different groups such as gender (Chen et al., 2005). There-
There is increasing agreement that psychiatric symptoms and fore the examination of gender differences should be preceded by
disorders maintain hierarchical structure where general or com- the testing of gender invariance in the measurement models.
mon and domain-specic or unique components play important
roles (Watson, 2005; Thomas, 2012). However the size of these
1.2. The goals of the present study
general and specic components might differ markedly across
disorders (Mineka et al., 1998; Watson, 2005). Estimating the
The primary aim of our study was to compare four measure-
bifactor measurement structure is proposed to be an effective
ment models of SCL-90-R and BSI in a large nationally represen-
approach to modeling construct-relevant multidimensionality
tative Hungarian community sample. The secondary aim of the
(Reise, 2012). Similar to the second order factor model, the bifactor
study was to test the gender difference in primary psychiatric
model enables the estimation of specic and general factors
symptoms and global distress.
simultaneously (Gibbons et al., 2007). This measurement model
allows for the indicators of psychological symptoms to load on an
overall primary factor such as global severity and also to have a 2. Methods
secondary loading on a specic dimension of symptoms.
Recently, bifactor model has been applied in several research 2.1. Participants and procedure
investigations on psychopathology including, for example, the
modeling of the structure of the Psychopathy Checklist-Revised SCL-90-R was assessed within the framework of the National Survey on
(Flores-Mendoza et al., 2008), the estimation of the bifactor Addiction Problems in Hungary (NSAPH) (Paksi et al., 2009). In this survey, in
structure of Quality of Life Interview for the Chronically Mentally addition to the assessment of addictive behaviors, also aimed to assess other
symptoms. The research protocol was approved by the Parliamentary Commis-
Ill (Gibbons et al., 2007), the validation of the Inventory of Callous- sioner for Data Protection and Freedom of Information and the University's
Unemotional Traits among young offenders (Kimonis et al., 2008), Institutional Review Board. Data were handled anonymously and on a
the estimation of the measurement model of Beck Depression voluntary basis.
Inventory (Al-Turkait and Ohaeri, 2010), and the estimation of the The target population of the survey was the total population of Hungary
between the ages of 18 and 64 (6,703,854 persons). The sampling frame consisted
bifactor model of ADHD (Martel et al., 2011). Only a few studies
of the whole resident population with a valid address, according to the register of
estimated the bifactor structure of the SCL-90-R (Vassend and the Central Ofce for Administrative and Electronic Public Services on January 1,
Skrondal, 1999) or its abbreviated version, the BSI (Thomas, 2012). 2006 (6,662,587 persons). Data collection was executed on a gross sample of 3183
Vassend and Skrondal (1999) found that the bifactor model people, stratied according to geographical location, degree of urbanization and
and the second-order factor model only marginally increased the age (overall 186 strata) representative of the sampling frame. Participants were
surveyed using the so-called mixed-method via personal visits. Questions regard-
model t compared with the one-dimensional model; however, ing background variables and introductory questions referring to specic disorders
none of these models yielded an adequate level of model were asked in the course of face-to-face interviews, while symptom scales,
t. Thomas (2012) also contrasted the bifactor model against the including SCL-90-R, were applied using self-administered paper-and-pencil
148 R. Urbn et al. / Psychiatry Research 216 (2014) 146154

questionnaires. These questionnaires were returned to the interviewer in a closed In order to quantify the degree of unidimensionality in bifactor models, we
envelope to ensure condentiality. The net sample size was 2710 (response rate: applied the percent of common variance attributable to the general factor through
85.1%). The ratio of samples belonging to each stratum was adjusted to the the use of explained common variance index (ECV; Ten Berge and Soan, 2004;
characteristics of the sampling frame by means of a weighted matrix for each Bentler, 2009). We also used omega and omega hierarchical indices to measure
stratum category. The weights applied have normal distribution (S.D.: 0.228; how precisely a self-reported symptom scale score assesses the combination of
skewness: 0.639; std. error of skewness: 0.047; kurtosis: 2.397; std. error of general and specic constructs and a certain target construct (Brunner et al., 2012).
kurtosis: 0.094). Before testing the gender difference, we needed to examine the gender
invariance of factor structure. Thus we performed a series of analyses to examine
the gender differences in parameters of measurement models including factor
2.2. Instrument loadings and thresholds. In case of WLSMV estimation, comparing the of models
with different level of constrains, we should rely on DIFFTEST procedure provided
2.2.1. Symptom checklist 90-Revised (SCL-90-R) by MPLUS (Asparouhov and Muthn, 2006). However, this procedure is sensitive to
The SCL-90-R is a 90-item self-report symptom inventory designed to reect large sample size; therefore additional goodness-of-t indexes are often used to
psychological symptom patterns of psychiatric and medical patients. Each item of assess model t. Based on simulation studies (Cheung and Rensvold, 2002; Chen,
the questionnaire is rated on a 5-point scale of distress from 0 (not at all) to 4 (very 2007) a difference of larger than 0.01 in the CFI would indicate a meaningful
much). The SCL-90-R consists of the following nine primary symptom dimensions: change in model t for testing measurement invariance. Thus we also examined
somatization (SOM, which reects distress arising from bodily perceptions), the change in goodness-of-t index such as CFI. In order to determine the gender
obsessive-compulsive (O-C, which reects obsessive-compulsive symptoms), inter- differences in factor loadings on the global distress factor, we compared models
personal sensitivity (I-S, which reects feelings of personal inadequacy and with different level of constrains. In the rst step, we estimated the measurement
inferiority in comparison with others), depression (DEP, which reects depressive model separately for men and women. In the second step, we estimated a model in
symptoms, as well as lack of motivation), anxiety (ANX, which reects anxiety both gender together with freely estimated threshold and factor loadings; therefore
symptoms and tension), hostility (HOS, which reects symptoms of negative affect, we assumed only the congural invariance. In the third step, we constrained the
aggression and irritability), phobic anxiety (PHOB, which reects symptoms of factor loadings to be equal in both genders; this is the assumption for metric
persistent fears as responses to specic conditions), paranoid ideation (PAR, which invariance. In the fourth step we constrained the factor loadings and thresholds
reects symptoms of projective thinking, hostility, suspiciousness, fear of loss of being equal in both genders; this is the assumption for scalar invariance. In the nal
autonomy), and psychoticism (PS, which reects a broad range of symptoms from step, we constrained the correlations between specic factors to be gender
mild interpersonal alienation to dramatic evidence of psychosis) (Derogatis, 1983; invariant.
Derogatis and Savitz, 2000). In order to examine the gender difference in reporting psychiatric symptoms in
the bifactor model, we applied multiple indicators and multiple causes (MIMIC)
modeling in which gender is modeled as a predictor variable, and the psychiatric
2.2.2. Brief Symptom Inventory (BSI-53) symptoms and the general distress as latent variables. The MIMIC technique, a
The BSI is the short version of the SCL-R-90 and contains only 53 items specication of structural equation modeling, was chosen for the present study
(Derogatis, 1975; Derogatis and Spencer, 1982). BSI-53 measures the same dimen- because MIMIC models can estimate the effect of indicators on latent variables
sions and the items for each dimension of the BSI-53 were selected based on a when direct effects of grouping variables or other continuous variables on the
factor analysis of the SCL-R-90. latent variables are also included.

2.3. Data analysis strategy

3. Results
We used conrmatory factor analysis with Mplus 6.0 (Muthn and Muthn,
19982007) to estimate the degree of t of previously proposed measurement 3.1. Measurement models for SCL-90-R
models to the data. We tested four measurement models of both SCL-90-R and BSI.
The starting point was a one-dimensional model which is called Model 0. The next
We have tested ve measurement models. The t indices are
model (Model 1) includes nine rst-order factors. In the third model (Model 2) in
addition to the nine rst-order factors we specied one second-order factor
reported in Table 1. The test is signicant in all four models;
representing the global severity dimension and the covariances between factors. however this test is oversensitive to large sample size. In the case
The fourth model (Model 3) includes a bifactor model representing a global of WLSMV estimation the traditional difference test is not
severity dimension on which each item is loaded; nine problem specic factors applicable; thus we performed difftest procedure (Muthn and
on which only the problem specic items are loaded; and the correlations between
Muthn, 19982007) provided by Mplus for similar cases. The
specic factors which were xed to zero. The usual specication of a bifactor model
requires that the specic factors do not correlate with either each other or the bifactor model (Model 3) yielded signicantly closer t to the data
global factor (Reise et al., 2010), despite the fact that covariance between specic than the nine rst-ordered factors model (Model 1; 160.5, d.
factors is identied. Therefore we also estimated the bifactor model with correlat- f. 47, p o0.0001). The bifactor model (Model 3) also produced
ing specic factors (Model 4), but all correlations between each specic factor and closer t than the second-order factor model (Model 2;
the global factor were xed to zero.
Both SCL-90-R and BSI contain items that do not belong to any factors, but they
1697.5, d.f. 74, p o0.0001). The bifactor model had the
are included in the global scale because of the items' clinical relevance. Performing closer t compared to the competing models (one-factor, nine-
conrmatory factor analyses with these items would be misleading because they factor, and second-ordered factor models). We also tested the
are not be associated theoretically with any factors. For this reason we excluded bifactor model with correlating specic factors (Model 4) which
these items when we compared different measurement models.
yielded a signicantly better t to the data than Model 3
Due to severe oor effect in the responses, we treated the items as ordinal
indicators and used the weighted least squares mean and variance adjusted ( 2063, d.f. 36, p o0.0001). Factor loadings are presented
estimation method (WLSMV; Brown, 2006; Finney and DiStefano, 2006). We used in Table 2. In the bifactor model, all items loaded signicantly on
the full information maximum likelihood estimator to deal with missing data global distress factors. The factor loadings ranged from 0.38 to
(Muthn and Muthn, 19982007). In conrmatory factor analysis, a satisfactory 0.82. With the exception of seven items (11, 21, 24, 45, 65, 78, 79),
degree of t requires the CFI and the TuckerLewis index (TLI) to be higher than or
close to 0.95, and the model should be rejected when these indices are less than
all items loaded signicantly on their specic factors. These seven
0.90 (Brown, 2006). The next t index was the root mean square error of items seemed to represent only global distress rather than the
approximation (RMSEA). RMSEA below 0.05 indicates excellent t, a value around specic psychiatric symptoms. We estimated the common var-
0.08 indicates adequate t, and a value above 0.10 indicates poor t. Closeness of iance index in the bifactor model and found that regarding SCL-
model t using RMSEA (CFit of RMSEA) is a statistical test (Brown, 2006), which
90-R the Global severity factor explains 84 % of common variance
evaluates the statistical deviation of RMSEA from the value.05. Non-signicant
probability values (p 40.05) indicate acceptable model t, though some methodol- which supports the presence of a strong global factor. The
ogists would require larger values such as p 40.50 (Brown, 2006). explained common variances of specic factors are also reported
In order to compare the degree of t of the four models, we have performed the in Table 2. Only Somatization scale has relatively large proportion
difftest procedure, which is available in the case of WLSMV estimation in Mplus of variance (5.5%). We also calculated omega and omega hierarch-
program (Muthn and Muthn, 19982007). Difftest procedure is used to obtain a
correct difference test when WLSMV estimator is used because the difference in
ical indices to denote how precisely a self-reported symptom scale
values for two nested models using WLSMV values is not distributed as score assesses the combination of general and specic constructs,
(Muthn and Muthn, 19982007, p. 501). and a certain target construct. To evaluate the measurement
R. Urbn et al. / Psychiatry Research 216 (2014) 146154 149

Table 1
Conrmatory factor analysis of ve measurement models of SCL-90-R.

d.f. CFI TLI RMSEA CFit of


Symptom checklist-90-Revised
Model 0 One-factor model 21,882 3915 0.90 0.90 0.043 1.00
Model 1 Nine rst-ordered factors 15,005 3284 0.93 0.93 0.038 1.00
Model 2 Second-order factor model 16,189 3311 0.93 0.93 0.039 1.00
Model 3 Bifactor model 14,856 3237 0.93 0.93 0.038 1.00
Model 4 Bifactor model with correlating specic 11,539 3201 0.95 0.95 0.032 1.00
Brief Symptom Inventory
Model 0 One-factor model 8627 1127 0.93 0.92 0.051 0.014
Model 1 Nine rst-ordered factors 5825 1091 0.95 0.95 0.041 1.00
Model 2 Second-order factor model 6416 1118 0.95 0.95 0.043 1.00
Model 3 Bifactor model 5573 1078 0.95 0.95 0.041 1.00
Model 4 Bifactor model with correlating specic 4614 1042 0.96 0.96 0.037 1.00
Gender invariance testing of Brief Symptom Inventory (Model 4)
Males separately 2198 1042 0.97 0.97 0.030 1.00
Females separately 2826 1042 0.97 0.96 0.036 1.00
Congural invariance: no equality constrains 5009 2084 0.97 0.97 0.033 1.00
Metric invariance: equality constrains on factor loadings 3776 2182 0.97 0.97 0.024 1.00
Scalar invariance: equality constrains on factor loadings and 4890 2319 0.97 0.97 0.030 1.00

Note: chi-square test statistic; d.f. degrees of freedom; CFI comparative t index; TLI TuckerLewis index; RMSEA root mean square error of approximation;
CFit closeness of t. Direct comparison of values of different models directly from t indices is not possible in case of wlsmv estimation. The only way to compare nested
models is to use difftest procedure (see text).

precision of each subscale in assessing the blend of global distress because CFI and TLI are larger than 0.95 and RMSEA is below 0.05.
and specic symptoms we calculated coefcient omega; and in However, the nine-factor model also has an acceptable degree of
assessing only specic problems or only global distress we t. We also tested the bifactor model with correlating specic
computed coefcient omega hierarchical (for details, see Brunner factors (Model 4) which yielded a signicantly better t to the data
et al., 2012). We report all omega and omega hierarchical coef- than Model 3 ( 887, d.f. 36, p o0.0001). Therefore we use
cients in Table 2. Omega coefcients are above 0.90 in all factors this model (Model 4) in the further analyses. In the bifactor model,
with the exception of Paranoid ideation. Although there is not any all items loaded signicantly on global distress factors (see
clearly dened cut-off for omega hierarchical coefcients, we Table 3). The factor loadings ranged from 0.59 to 0.84. With the
propose that coefcients around 0.10 can be regarded salient; exception of nine items (8, 11, 34, 37, 41, 45, 46, 69, 78 according to
therefore the specic factor explained approximately at least 10% their original numbers), all items loaded signicantly on their
variance of the symptoms score. Somatization, depression, anxiety, specic factors. These nine items seem to represent the global
phobic anxiety hostility and paranoid ideation yielded salient distress more than the specic psychiatric symptoms. One impor-
omega hierarchical coefcients. tant consideration is that interpersonal sensitivity factor does not
The range of correlations between specic factors is between have any signicant factor loadings (items 34, 37, 41, 49) after
0.87 and 0.94, and details are presented in Table 3. The pattern controlling the global distress factor; therefore this factor is not
of correlations demonstrated that somatization, obsessive-com- separated from the global distress in the model.
pulsive, anxiety, phobic anxiety, interpersonal sensitivity and We estimated the explained common variance index in the
depression symptoms correlated largely and positively. On the bifactor model and found that regarding BSI the Global severity
other hand, we identied moderate positive correlations among factor explains 83% of common variance supporting the presence
hostility, paranoid ideation and psychoticism. Correlations of a strong global factor. The explained common variances of
between these two groups of symptoms are usually negative and specic factors are also reported in Table 3. Only Somatization
strong. scale has relatively large proportion of variance. We also reported
omega and omega hierarchical indices in Table 3. All omega
3.2. Measurement models for BSI coefcients are above 0.80. Salient omega hierarchical indices
are present in cases of somatization, depression, hostility, paranoid
We tested the four measurement models in the restricted range ideation and psychoticism.
of items constructing the BSI. The t indices are reported in Correlations between specic factors are presented in Table 3.
Table 1. The test was signicant in all four models in this case The range of correlations between specic factors is between  0.65
as well; however, this test is oversensitive to large sample size. and 0.79. The pattern of correlations demonstrated that somatiza-
Thus we performed the difftest procedure to compare the bifactor tion, obsessive-compulsive, anxiety, phobic anxiety and depression
model with the other models. The model containing nine rst- symptoms correlated largely and positively. On the other hand, we
ordered factors (Model 1) resulted in a signicantly better t to the identied moderate positive correlations among hostility, paranoid
current data than the second-ordered factor model (Model 2; ideation and psychoticism. The correlations between these two
test 559, d.f. 27, p o0.0001). Model 3 (bifactor model) yielded groups of symptoms are usually negative and strong.
signicantly closer t to the data ( test 1058, d.f. 40,
p o0.0001) than Model 2. However, in this case comparison of 3.3. Gender invariance testing and gender differences: BSI
Model 1 with Model 3 computation of was not possible due to
the singular matrix during the computation process. As for other The measurement invariance of BSI was examined in men and
t indices, the bifactor model yielded an excellent degree of t, women by use of multiple group CFA. We estimated the model t
150 R. Urbn et al. / Psychiatry Research 216 (2014) 146154

Table 2
Standardized factor loadings of the bifactor model of SCL-90-R.

Itema (number) SOM O-C I-S DEP ANX HOS PHOB PAR PSY Global

Headaches (1) 0.42 0.38

Faintness (4) 0.52 0.60
Pains in heart/chest (12) 0.56 0.61
Pains in lower back (27) 0.57 0.38
Nausea (40) 0.30 0.68
Soreness of muscles (42) 0.58 0.54
Trouble getting breath (48) 0.46 0.64
Hot/cold spells (49) 0.48 0.59
Numbness (52) 0.42 0.69
Lump in throat (53) 0.31 0.71
Weakness of body (56) 0.54 0.68
Heavy arms/legs (58) 0.58 0.62

Unpleasant thoughts (3) 0.19 0.70

Trouble remembering (9) 0.18 0.62
Worried about sloppiness (10) 0.09 0.70

Feeling blocked (28) 0.41 0.66

Doing things slowly (38) 0.17 0.70
Having to double-check (45) 0.02 0.70
Difculty deciding (46) 0.06 0.69
Mind going blank (51) 0.08 0.81
Trouble concentrating (55) 0.20 0.76
Repeating same actions (65)  0.03 0.77

Feeling critical of others (6) 0.18 0.56

Feeling shy opposite sex (21)  0.02 0.72
Feeling easily hurt (34) 0.22 0.67
Others are unsympathetic (36) 0.35 0.75
People dislike you (37) 0.16 0.71
Feeling inferior to others (41) 0.13 0.77
Uneasy when people are watching you (61) 0.10 0.69
Self-conscious with others (69) 0.09 0.76
Uncomfortable eating/drinking in public (73) 0.24 0.73

Loss of sexual interest (5) 0.19 0.59

Low energy/slow (14) 0.51 0.65
Thoughts of ending life (15) 0.26 0.68
Crying easily (20) 0.33 0.58
Feeling trapped (22) 0.09 0.77

Blaming yourself (26) 0.15 0.74

Feeling lonely (29) 0.28 0.68
Feeling blue (30) 0.41 0.72
Worrying too much (31) 0.35 0.70
No interest in things (32) 0.09 0.73
Hopeless about future (54) 0.23 0.71
Everything is an effort (71) 0.15 0.78
Feeling worthless (79)  0.03 0.79

Nervousness (2) 0.32 0.65

Trembling (17) 0.35 0.73
Suddenly scared (23) 0.18 0.82
Feeling fearful (33) 0.11 0.76
Heart pounding/racing (39) 0.55 0.62
Feeling tense (57) 0.26 0.77
Spells of terror/panic (72) 0.24 0.83
Can't sit still/restless (78)  0.01 0.71
Something bad is going to happen to you (80) 0.08 0.80
Frightening thoughts (86) 0.20 0.73
Easily annoyed (11)  0.03 0.70
Temper outbursts (24) 0.02 0.78
Urges to harm someone (63) 0.56 0.65
Urges to break things (67) 0.53 0.71
Arguing frequently (74) 0.39 0.62
Shouting/throwing (81) 0.44 0.65

Afraid on the street (13) 0.50 0.75

Afraid to go out alone (25) 0.48 0.78
Afraid of public transport (47) 0.51 0.72
Having to avoid things/places/activities (50) 0.17 0.77
Uneasy in crowds (70) 0.20 0.62
Nervous when alone (75) 0.18 0.73
Afraid to faint in public (82) 0.39 0.74

Others are to blame (8) 0.12 0.63

Most people can't be trusted (18) 0.08 0.73
Feeling watched (43) 0.23 0.71
Having beliefs that others do not share (68) 0.43 0.64
R. Urbn et al. / Psychiatry Research 216 (2014) 146154 151

Table 2 (continued )

Itema (number) SOM O-C I-S DEP ANX HOS PHOB PAR PSY Global

Not getting enough credit (76) 0.35 0.67

People will take advantage (83) 0.21 0.70

Someone can control your thoughts (7) 0.05 0.68

Hearing voices (16) 0.12 0.72
Others knowing your private thoughts (35) 0.01 0.55
Thoughts not your own (62) 0.11 0.79
Feeling lonely with others (77) 0.17 0.79
Thoughts about sex that bother you a lot (84) 0.24 0.71
You should be punished for your sins (85) 0.28 0.78
Something is wrong with your body (87)  0.28 0.70
Never feeling close to another person (88) 0.17 0.77
Something is wrong with your mind (90) 0.10 0.81

Explained common variance 6% 1% 1% 2% 1% 2% 2% 1% 1% 84%

Omega 0.95 0.92 0.91 0.94 0.94 0.90 0.93 0.87 0.93 0.99

Omega hierarchicalc 0.37 0.03 0.05 0.09 0.08 0.16 0.17 0.09 0.02 0.97

Correlations between specic factors

Obsessive-compulsive 0.64
Interpersonal sensitivity  0.03 0.19
Depression 0.56 0.74 0.28
Anxiety 0.94 0.62 0.03 0.74
Hostility  0.13  0.33 0.05  0.35  0.26
Phobic anxiety 0.39  0.06  0.67  0.02 0.45  0.07
Paranoid ideation  0.22  0.19 0.71  0.21  0.40 0.35  0.58
Psychoticism  0.64  0.70  0.55  0.82  0.87 0.45  0.09  0.03

Note: SOM: somatization; O-C: obsessive-compulsive; I-S: interpersonal sensitivity; DEP: depression; ANX: anxiety; HOS: hostility; PHOB: phobic anxiety; PAR: paranoid
ideation; PSY: psychoticism.
Boldfaced factor loadings and correlations are signicant at least p o 0.05.
We used the abbreviated version of items instead of the full item similarly to Paap et al. (2011).
Omega refers to the proportion of explained variance in the scale score attributed to the global and specic factors.
Omega hierarchical refers to the proportion of explained variance of the scale score attributed to the specic factor, and italicized loadings are salient ( Z 0.20).

in both genders separately, which yielded an adequate degree of t po 0.001), depression ( 0.26, p o0.001) and anxiety (0.68,
in both groups (see Table 1). Three nested models with increasing po 0.001). Men have higher scores on hostility (  0.20,
constraints were estimated. The t indices are reported in Table 1. po 0.001) and psychoticism ( 0.14, p o0.001). We did not
First, the measurement model was estimated freely in men and nd gender difference in phobic anxiety (  0.07, ns.) and paranoid
women together (congural invariance). This unconstrained solu- ideation ( 0.06, ns.).
tion tted the data satisfactorily supporting the congural invar-
iance of the measurement model. In the second model, the factor
loadings were set as equal between genders (metric invariance). 4. Discussion
According to difftest procedure, the degree of t decreased
signicantly ( 164.5, d.f. 98, p o0.0001), however CFI is Overall, our analysis supported the multidimensional measure-
less than 0.01. Based on the signicance of , the metric ment model of symptom checklist (SCL-90-R); however, we
invariance is not supported; however, CFI index indicated that observed a strong global distress factor and weak specic symp-
factor loadings are invariant between genders. In the third model, tom factors. All multifactorial model tted the present data better
the thresholds were set as equal (scalar invariance). The degree of than the univariate model. Furthermore, a bifactor model yielded
t decreased signicantly ( 859.0, d.f. 196, p o0.0001) but the closest t to data both in the case of SCL-90-R and BSI similarly
CFI index, which is less than 0.01, indicated that thresholds are to the most recent report (Thomas, 2012); however, the original
invariant between genders, therefore we cannot reject the scalar nine-factor measurement model yielded an acceptable level of t
invariance hypothesis. as well in both versions. Our result contradicts several previous
Explained common variances of specic factors and global conrmatory factor analyses (Carpenter and Hittner, 1995;
factor, omega and omega hierarchical coefcients in men and Vassend and Skrondal, 1999; Schmitz et al., 2000; Hafkenscheid
women are reported in Table 4. The explained common variances et al., 2006). The reason for the major difference between the
of global factor and omega coefcients are similar in men and previous and current analyses might be the fact that we applied
women. We observed relatively large differences in several omega estimation method applicable for ordinal scale and non-normal
hierarchical indices; larger proportion of explained variance is distribution of item responses. Besides conrming multidimen-
attributed to specic factors of somatization, depression, hostility, sionality, the present bifactor model also supports the hierarchical
phobic anxiety and psychoticism in women than men. structure of psychiatric symptoms which species the general and
Based on the results supporting the gender invariance of domain-specic components (Watson, 2005; Thomas, 2012).
measurement models, we tested the gender difference in psychia- Although the analysis of general and domain-specic components
tric symptoms and global distress with using a MIMIC model. Men revealed that the general or global distress factor explained the
were coded with 1 and women were coded with 2. Women have vast majority of variance in the present sample, some specic
higher scores on global severity index (0.08, p o0.001), soma- symptom factors such as somatization, depression, hostility phobic
tization ( 0.23, p o0.001), obsession-compulsion ( 0.15, anxiety have also salient contribution in explaining the variance of
152 R. Urbn et al. / Psychiatry Research 216 (2014) 146154

Table 3
Standardized factor loadings of the bifactor model of Brief Symptom Inventory.

Itema (number) SOM O-C I-S DEP ANX HOS PHOB PAR PS Global

Faintness (4) 0.53 0.59

Pains in heart/chest (12) 0.54 0.61
Nausea (40) 0.30 0.68
Trouble getting breath (48) 0.53 0.63
Hot/cold spells (49) 0.50 0.58
Numbness (52) 0.42 0.68
Weakness of body (56) 0.50 0.68

Trouble remembering (9) 0.18 0.63

Feeling blocked (28) 0.31 0.67
Having to double-check (45) 0.00 0.70
Difculty deciding (46) 0.02 0.70
Mind going blank (51) 0.14 0.81
Trouble concentrating (55) 0.17 0.77

Feeling easily hurt (34)  0.07 0.68

People dislike you (37) 0.04 0.71
Feeling inferior to others (41)  0.01 0.78
Self-conscious with others (69) 0.10 0.74

Thoughts of ending life (15) 0.18 0.69

Feeling lonely (29) 0.48 0.67
Feeling blue (30) 0.65 0.72
No interest in things (32) 0.10 0.74
Hopeless about future (54) 0.16 0.73
Feeling worthless (79) 0.06 0.79

Nervousness (2) 0.27 0.65

Suddenly scared (23) 0.20 0.83
Feeling fearful (33) 0.15 0.76
Feeling tense (57) 0.23 0.79
Spells of terror/panic (72) 0.28 0.84
Can't sit still/restless (78)  0.04 0.72

Easily annoyed (11) 0.05 0.70

Temper outbursts (24) 0.06 0.79
Urges to harm someone (63) 0.61 0.64
Urges to break things (67) 0.54 0.70
Arguing frequently (74) 0.35 0.61

Afraid on the street (13) 0.54 0.70

Afraid of public transport (47) 0.56 0.68
Having to avoid things/places/activities (50) 0.24 0.76
Uneasy in crowds (70) 0.24 0.60
Nervous when alone (75) 0.15 0.72

Others are to blame (8) 0.03 0.62

Most people can't be trusted (18) 0.08 0.73
Feeling watched (43) 0.24 0.70
Not getting enough credit (76) 0.38 0.68
People will take advantage (83) 0.25 0.69

Someone can control your thoughts (7) 0.08 0.66

Feeling lonely with others (77) 0.33 0.76
You should be punished for your sins (85) 0.37 0.72
Never feeling close to another person (88) 0.35 0.74
Something is wrong with your mind (90) 0.25 0.79

Explained common variance 0.06 0.01 0.00 0.02 0.01 0.03 0.02 0.01 0.01 0.83
Omega 0.92 0.87 0.82 0.91 0.91 0.89 0.89 0.84 0.89 0.98
Omega hierarchical 0.33 0.03 0.00 0.11 0.05 0.16 0.18 0.06 0.11 0.96

Correlations between specic factors

Obsessive-compulsive 0.73
Interpersonal sensitivity N/A N/A
Depression 0.25 0.36 N/A
Anxiety 0.79 0.27 N/A 0.53
Hostility  0.10  0.27 N/A  0.21  0.32
Phobic anxiety 0.53 0.22 N/A 0.06 0.58  0.04
Paranoid ideation  0.24  0.65 N/A  0.21  0.64 0.23  0.29
Psychoticism  0.08  0.22 N/A 0.09  0.54 0.34 0.20 0.31

Note: SOM: somatization; O-C: obsessive-compulsive; I-S: interpersonal sensitivity; DEP: depression; ANX: anxiety; HOS: hostility; PHOB: phobic anxiety; PAR: paranoid
ideation; PSY: psychoticism.
Boldfaced factor loadings and correlations are signicant at least p o 0.05, and italicized loadings are salient ( Z 0.20).
We used the abbreviated version of items instead of the full item similarly to Paap et al. (2011).
R. Urbn et al. / Psychiatry Research 216 (2014) 146154 153

Table 4
Gender differences in model based reliability indices.


Explained common variance 0.03 0.01 0.01 0.02 0.01 0.02 0.01 0.01 0.01 0.86
Omega 0.93 0.89 0.85 0.91 0.91 0.89 0.88 0.85 0.90 0.98
Omega hierarchical 0.20 0.03 0.07 0.09 0.00 0.17 0.04 0.10 0.05 0.97
Explained common variance 0.06 0.01 0.00 0.03 0.01 0.03 0.03 0.01 0.02 0.80
Omega 0.92 0.87 0.81 0.91 0.91 0.89 0.90 0.84 0.89 0.97
Omega hierarchical 0.35 0.06 0.00 0.14 0.06 0.23 0.23 0.08 0.13 0.95

Note: SOM: somatization; O-C: obsessive-compulsive; I-S: interpersonal sensitivity; DEP: depression; ANX: anxiety; HOS: hostility; PHOB: phobic anxiety; PAR: paranoid
ideation; PSY: psychoticism. Salient omega hierarchical indices ( Z0.10) are in bold.

the scores of these symptom factors. This bifactor model ts well regarding gender difference in somatization (Wool and Barsky, 1994),
with the ndings of high comorbidity among mental disorders in depression (Nolen-Hoeksema et al., 1999), symptoms of anxiety
large scale epidemiological studies (Kessler et al., 1994). Comor- (Pigott, 1999; McLean et al., 2011), hostility (Huesmann and Eron,
bidity among mental disorders is rather a rule than an exception, 1989; Fava et al., 1995), and psychoticism (Golomb et al., 1995).
and symptomatically homogenous groups without symptoms of Our study has several strengths and limitations. Using
other mental disorders are rarely found in empirical data on community-based sampling involves non-clinical samples in
psychopathological symptoms neither in representative nor in which the level of distress and the number of symptoms are lower
clinical population (Krueger et al., 2001; Krueger and Piasecki, than in a clinical sample. Moreover, owing to this sampling, we
2002; Roca et al., 2009). However, besides the global distress could rely only on self-report and had no opportunity to validate
factor, the bifactor model also strengthens the separation of the the responses with additional clinical interviewing. Another lim-
nine self-reported symptom-clusters. Furthermore, the bifactor itation of this study is that the analyses of SCL-90-R and BSI are
model also highlights that a few items may represent the global based on administration of SCL-90-R. External validity of the
severity more than the specic symptoms in a large non-clinical present study is limited to community samples. On the other
sample, although these items should be tested in clinical samples hand, however, SCL-90-R and BSI have never been studied before
as well. on a national representative sample.
A recent study (Reise et al., 2013) proposes that in case of high
explained common variance of the general factor might be an
indicator of unidimensionality of the measure, which raises criti- 5. Conclusion
cism regarding our conclusion about the multidimensionality of
SCL-90-R and BSI-53. Nevertheless, the t indices and examination The present study demonstrated the usefulness and feasibility
of omega hierarchical indices supported our proposition. More- of the administration of SCL-90-R in large scale community studies
over, the advantage of bifactor model is that the strength of global and supported the construct validity of both the SCL-90-R and BSI.
factor and symptom specic factors can be quantied and pre- The best tting model reinforces the use of global severity index
dicted by other variables, which is not possible when using the and the specic symptoms. Our results explain the large correla-
unidimensional model. When the degree of model t allows, the tions between symptom factors, which are represented in the
bifactor model can be used to compare groups with different global severity factor. Latent variables modeling also provides the
distress levels or with different sociocultural backgrounds. Recent statistical opportunity to disentangle the global severity factor and
studies demonstrated that the gender and level of psychological specic symptom factors in relation to covariates and other
distress were related to dimensionality (Paap et al., 2012). Low determinants in further research on non-clinical and clinical
level of psychological distress can support unidimensionality, samples. For example, further research could clarify whether the
while research on samples characterized with high level of correlates of mental health problems such as drug or alcohol use,
psychological distress supports multidimensionality (Paap et al., Internet addiction, or smoking are associated with global severity
2011, 2012). However, these studies used a different statistical factor and/or specic symptom factors.
approach to decide about the dimensionality. Our future goal is to
test the impact of distress on the model t of bifactor model and
model-based reliability indices such as explained common var-
iances, omega hierarchicals in groups characterized with severe
psychological distress. Present work was supported by the European Union and co-
In order to support the construct validity of the symptom check- nanced by the European Social Fund (Grant agreement no.
list, we tested the gender differences in global and specic factors. TAMOP 4.2.1/B-09/1/KMR-2010-0003) and the Hungarian Scienti-
Before examining gender differences we performed the gender c Research Fund Grant 83884 and 109375. Gyngyi Kknyei and
invariance analysis of the bifactor model, and we supported the Zsolt Demetrovics acknowledge nancial support of the Jnos
gender invariance with applying new criteria related to change in a Bolyai Research Fellowship awarded by the Hungarian Academy
goodness of t index. The examination of gender difference revealed of Science.
that on the one hand women scored higher on the global severity
factor, which supports the general view that women tend to report References
more somatic and psychological symptoms than men (Barsky et al.,
2001; A, 2007). On the other hand, women reported higher level A, M., 2007. Gender differences in mental health. Singapore Medical Journal 48,
of somatization, obsession-compulsion, interpersonal sensitivity, 385391.
Al-Turkait, F.A., Ohaeri, J.U., 2010. Dimensional and hierarchical models of depres-
depression, and anxiety, and men reported higher level of hostility sion using the Beck Depression Inventory-II in an Arab college student sample.
and psychoticism. Our results are consistent with previous reports BMC Psychiatry 10, 60.
154 R. Urbn et al. / Psychiatry Research 216 (2014) 146154

Asparouhov, T., Muthn, B., 2006. Robust Chi square difference testing with mean Krueger, R.F., Piasecki, T.M., 2002. Toward a dimensional and psychometrically-
and variance adjusted test statistics. Mplus Web Notes, 10 informed approach to conceptualizing psychopathology. Behaviour Research
Barsky, A.J., Peekna, H.M., Borus, J.F., 2001. Somatic symptom reporting in women and Therapy 40, 485499.
and men. Journal of General Internal Medicine 16, 266275. Linzer, M., Spitzer, R., Kroenke, K., Williams, J.B., Hahn, S., Brody, D., deGruy, F.,
Bentler, P.M., 2009. Alpha, dimension-free, and model-based internal consistency 1996. Gender, quality of life, and mental disorders in primary care: results from
reliability. Psychometrika 74, 137143. the PRIME-MD 1000 study. American Journal of Medicine 101, 526533.
Brophy, C.J., Norvell, N.K., Kiluk, D.J., 1988. An examination of the factor structure Low, G.D., 1988. The semantics of questionnaire rating scales. Evaluation & Research
and convergent and discriminant validity of the SCL-90R in an outpatient clinic in Education 2, 6979.
population. Journal of Personality Assessment 52, 334340. Martel, M.M., Roberts, B., Gremillion, M., von Eye, A., Nigg, J.T., 2011. External
Brown, T.A., 2006. Conrmatory Factor Analysis for Applied Research. The Guilford validation of bifactor model of ADHD: explaining heterogeneity in psychiatric
Press, New York comorbidity, cognitive control, and personality trait proles within DSM-IV
Brunner, M., Nagy, G., Wilhelm, O., 2012. A tutorial on hierarchically structured ADHD. Journal of Abnormal Child Psychology 39, 11111123.
constructs. Jornal of Personality 80, 796846. McLean, C.P., Asnaani, A., Litz, B.T., Hofmann, S.G., 2011. Gender differences in
Carpenter, K.M., Hittner, J.B., 1995. Dimensional characteristics of the SCL-90-R: anxiety disorders: prevalence, course of illness, comorbidity and burden of
evaluation of gender differences in dually diagnosed inpatients. Journal of illness. Journal of Psychiatric Research 45, 10271035.
Clinical Psychology 51, 383390. Mineka, S., Watson, D., Clark, L.A., 1998. Comorbidity of anxiety and unipolar mood
Casper, R.C., Belanoff, J., Offer, D., 1996. Gender differences, but no racial group disorders. Annual Review of Psychology 49, 377412.
differences, in self-reported psychiatric symptoms in adolescents. Journal of the Muthn, L.K., Muthn, B.O., 1998-2007. Mplus User's Guide, 5th ed. Muthn &
American Academy of Child and Adolescent Psychiatry 35, 500508. Muthn, Los Angeles, CA
Chen, F.F., 2007. Sensitivity of goodness of t indices to lack of measurement Nolen-Hoeksema, S., Larson, J., Grayson, C., 1999. Explaining the gender difference
invariance. Structural Equation Modeling 14, 464504. in depressive symptoms. Journal of Personality and Social Psychology 77,
Chen, F.F., Sousa, K.H., West, S.G., 2005. Testing measurement invariance of second- 10611072.
order factor models. Structural Equation Modeling 12, 471492. Olsen, L.R., Mortensen, E.L., Bech, P., 2004. The SCL-90 and SCL-90R versions
Cheung, G.W., Rensvold, R.B., 2002. Evaluating goodness-of-t indexes for testing validated by item response models in a Danish community sample. Acta
measurement invariance. Structural Equation Modeling 9, 233255. Psychiatrica Scandinavica 110, 225229.
Cyr, J.J., McKenna-Foley, J.M., Peacock, E., 1985. Factor structure of the SCL-90-R: is Ostrov, E., Offer, D., Howard, K.I., 1989. Gender differences in adolescent sympto-
there one? Journal of Personality Assessment 49, 571578. matology: a normative study. Journal ofthe American Academy of Child and
Derogatis, L.R., 1975. Brief Symptom Inventory. Clinical Psychometric Research, Adolescent Psychiatry 28, 394398.
Baltimore, MD Paap, M.C., Meijer, R.R., Van Bebber, J., Pedersen, G., Karterud, S., Hellem, F.M.,
Derogatis, L.R., 1983. SCL-90-R: Administration Scoring and Procedures Manual. Haraldsen, I.R., 2011. A study of the dimensionality and measurement precision
Clinical Psychometric Research, Baltimore, MD of the SCL-90-R using item response theory. International Journal of Methods in
Derogatis, L.R., Savitz, K.L., 2000. The SCL-90-R and the Brief Symptom Inventory
Psychiatric Research 20, e39e55.
(BSI) in primary care. In: Maruish, M.E. (Ed.), Handbook of Psychological
Paap, M.C., Meijer, R.R., Cohen-Kettenis, P.T., Richter-Appelt, H., de Cuypere, G.,
Assessment in Primary Care Settings. Lawrence Erlbaum Associates, Mahwah,
Kreukels, B.P., Pedersen, G., Karterud, S., Malt, U.F., Haraldsen, I.R., 2012. Why
pp. 217334
the factorial structure of the SCL-90-R is unstable: comparing patient groups
Derogatis, L.R., Spencer, P.M., 1982. The Brief Symptom Inventory (BSI): Adminis-
with different levels of psychological distress using Mokken Scale Analysis.
tration and Procedures Manual-I. Clinical Psychometric Research, Baltimore,
Psychiatry Research 200, 819826.
Paksi, B., Rzsa, S., Kun, B., Arnold, P., Demetrovics, Z., 2009. A magyar npessg
Elliott, R., Fox, C.M., Beltyukova, S.A., Stone, G.E., Gunderson, J., Zhang, X., 2006.
addiktolgiai problmi: az Orszgos Lakossgi Adatfelvtel az Addiktolgiai
Deconstructing therapy outcome measurement with Rasch analysis of a
Problmkrl (OLAAP) reprezentatv felmrs mdszertana s a minta ler
measure of general clinical distress: the Symptom Checklist-90-Revised.
jellemzi. [Addictive behaviours in Hungary: the methodology and sample
Psychological Assessment 18, 359372.
description of the national survey on addiction problems in Hungary (NSAPH)].
Fava, M., Nolan, S., Kradin, R., Rosenbaum, J., 1995. Gender differences in hostility
Mentlhigin s Pszichoszomatika 10, 273300.
among depressed and medical outpatients. Journal of Nervous and Mental
Pigott, T.A., 1999. Gender differences in the epidemiology and treatment of anxiety
Disease 183, 1014.
Finney, S.J., DiStefano, C., 2006. Nonnormal and categorical data in structural disorders. Journal of Clinical Psychiatry 60 (Suppl. 18), 415.
equation modeling. In: Hancock, G.R., Mueller, R.D. (Eds.), Structural Equation Prunas, A., Sarno, I., Preti, E., Madeddu, F., Perugini, M., 2012. Psychometric
Modeling: A Second Course. Information Age, Greenwich, pp. 269314 properties of the Italian version of the SCL-90-R: a study on a large community
Flores-Mendoza, C., Alvarenga, M.A.S., Herrero, O., Abad, F.J., 2008. Factor structure sample. European Psychiatry 27, 591597.
and behavioural correlates of the Psychopathy Checklist-Revised [PCL-R] in a Reise, S.P., 2012. The rediscovery of bifactor measurement models. Multivariate
Brazilian prisoner sample. Personality and Individual Differences 45, 584590. Behavioral Research 47, 667696.
Gibbons, R.D., Bock, R.D., Hedeker, D., Weiss, D.J., Segawa, E., Bhaumik, D.K., 2007. Reise, S.P., Moore, T.M., Haviland, M.G., 2010. Bifactor models and rotations:
Full-information item bifactor analysis of graded response data. Applied exploring the extent to which multidimensional data yield univocal scale
Psychological Measurement 31, 419. scores. Journal of Personality Assessment 92, 544559.
Golomb, M., Fava, M., Abraham, M., Rosenbaum, J.F., 1995. Gender differences in Reise, S.P., Scheines, R., Widaman, K.F., Haviland, M.G., 2013. Multidimensionality
personality disorders. American Journal of Psychiatry 152, 579582. and structural coefcient bias in structural equation modeling. A bifactor
Hafkenscheid, A., 1993. Psychometric evaluation of the symptom checklist (SCL-90) perspective. Educational and Psychological Measurement 73, 526.
in psychiatric inpatients. Personality and Individual Differences 14, 751756. Roca, M., Gili, M., Garcia-Garcia, M., Salva, J., Vives, M., Garcia Campayo, J., Comas,
Hafkenscheid, A.J., Maassen, G.H., Veeninga, A.T., 2006. The dimensions of the A., 2009. Prevalence and comorbidity of common mental disorders in primary
Dutch SCL-90-R: more than one, but how many? Netherlands Journal of care. Journal of Affective Disorders 119, 5258.
Psychology 63, 2935. Schmitz, N., Hartkamp, N., Kiuse, J., Franke, G.H., Reister, G., Tress, W., 2000. The
Hoffmann, N.G., Overall, P.B., 1978. Factor structure of the SCL-90 in a psychiatric Symptom Check-List-90-R (SCL-90-R): a German validation study. Quality of
population. Journal of Consulting and Clinical Psychology 46, 11871191. Life Research 9, 185193.
Holi, M.M., Sammallahti, P.R., Aalberg, V.A., 1998. A Finnish validation study of the Ten Berge, J.M., Soan, G., 2004. The greatest lower bound to the reliability of a test
SCL-90. Acta Psychiatrica Scandinavica 97, 4246. and the hypothesis of unidimensionality. Psychometrika 69, 613625.
Huesmann, L.R., Eron, L.D., 1989. Individual differences and the trait of aggression. Thomas, M.L., 2012. Rewards of bridging the divide between measurement and
European Journal of Personality 3, 95106. clinical theory: demonstration of a bifactor model for the Brief Symptom
Kessler, R.C., McGonagle, K.A., Zhao, S., Nelson, C.B., Hughes, M., Eshleman, S., Inventory. Psychological Assessment 24, 101113.
Wittchen, H.U., Kendler, K.S., 1994. Lifetime and 12-month prevalence of DSM- Vassend, O., Skrondal, A., 1999. The problem of structural indeterminacy in
III-R psychiatric disorders in the United States. Results from the National multidimensional symptom report instruments. The case of SCL-90-R. Beha-
Comorbidity Survey. Archives of General Psychiatry 51, 819. viour Research and Therapy 37, 685701.
Kimonis, E.R., Frick, P.J., Skeem, J.L., Marsee, M.A., Cruise, K., Munoz, L.C., Aucoin, K.J., Vicente, B., Kohn, R., Rioseco, P., Saldivia, S., Levav, I., Torres, S., 2006. Lifetime and
Morris, A.S., 2008. Assessing callous-unemotional traits in adolescent offen- 12-month prevalence of DSM-III-R disorders in the Chile psychiatric prevalence
ders: validation of the inventory of callous-unemotional traits. International study. American Journal of Psychiatry 163, 13621370.
Journal of Law and Psychiatry 31, 241252. Watson, D., 2005. Rethinking the mood and anxiety disorders: a quantitative
Krueger, R.F., McGue, M., Iacono, W.G., 2001. The higher-order structure of common hierarchical model for DSM-V. Journal of Abnormal Psychology 114, 522536.
DSM mental disorders: internalization externalization and their connections to Wool, C.A., Barsky, A.J., 1994. Do women somatize more than men? Gender
personality. Personality and Individual Differences 30, 12451259. differences in somatization. Psychosomatics 35, 445452.