BRIEF REPORT

The Challenges of Ecological Validity in the Measurement of

Social Perception in Schizophrenia
Anja Vaskinn, PhD,* Mark J. Sergi, PhD, and Michael F. Green, PhD

Abstract: Ecological validity may be substantiated by verisimilitude (the

stimuli and cognitive processing of a test resemble the stimuli and cognitive
processing required in everyday life) and/or veridicality (a test is statistically
associated with everyday life functioning). The Interpersonal Perception
Task uses real-life social behavior as stimuli and has high verisimilitude. We
examined the Interpersonal Perception Task-15 in 72 persons with schizophrenia and 58 healthy controls. The test discriminated well between groups,
but failed to show associations with community functioning in participants
with schizophrenia, indicating poor veridicality. This study shows that the 2
approaches to ecological validity can dissociate. Hence, it is important for
investigators to conduct psychometric investigations when borrowing measures from social psychology for schizophrenia research.
Key Words: Schizophrenia, social cognition, social perception,
psychometrics, ecological validity.
(J Nerv Ment Dis 2009;197: 700 702)

ocial cognition appears closely linked to functional status in

schizophrenia, and may mediate relations between neurocognition and functioning in the illness (Couture et al., 2006; Sergi et al.,
2006; Vaskinn et al., 2008). Despite the accelerating prominence of
social cognition as a construct in the schizophrenia research literature, concern exists regarding the psychometric properties and ecological validity of the measures used to assess social cognition
(Green et al., 2008), especially when these measures are borrowed
from other disciplines.
Ecological validity may be substantiated by verisimilitude
and/or veridicality (Chaytor and Schmitter-Edgecombe, 2003). Verisimilitude indicates that something appears to be true or real.
Based on theoretical knowledge and the judgment of experts, verisimilitude assumes that the stimuli of the measure and the cognitive
processing of these stimuli are both sufficiently similar to the stimuli
and cognitive processing of everyday occurrences of the psychological construct (Franzen and Arnett, 1997). Veridicality uses statistical approaches to associate performance on the measure of interest
with measures of everyday life functioning. Thus, a measure of
social cognition in schizophrenia would show evidence of high
veridicality if it correlated well with measures of functional status.
Social perception is an aspect of social cognition that involves
the interpretation of social cues such as facial expressions, gestures,
and voice tone to form impressions about others such as their

*Clinic for Mental Health, Oslo University Hospital - Aker, Oslo, Norway;
Department of Psychology, California State University, Northridge, CA;
Department of Veterans Affairs VISN 22 Mental Illness, Research, and Clinical
Center (MIRECC), Los Angeles, CA; and Department of Psychiatry and
Biobehavioral Sciences, Geffen School of Medicine, UCLA, Los Angeles, CA.
Supported by grants from the National Institute of Mental Health MH43292 (to
M.F.G.) and the Fulbright Foundation for Educational Exchange (to A.V.).
Reprints: Anja Vaskinn, PhD, Oslo University HospitalAker, Clinic for Mental
Health, Trondheimsvn. 235, 0514 Oslo, Norway. E-mail: anja.vaskinn@medisin.
uio.no.
Copyright 2009 by Lippincott Williams & Wilkins
ISSN: 0022-3018/09/19709-0700
DOI: 10.1097/NMD.0b013e3181b3ae62

700 | www.jonmd.com

relative status, level of intimacy, motives, or veracity. Measures

commonly used to assess social perception in persons with schizophrenia (e.g., Social Cue Recognition Test, Profile of Nonverbal
Sensitivity) typically use stimuli involving restricted information
(e.g., 1 to 3 specific social cues). These measures are psychometrically sound and produce moderate associations with measures of
functional status. For example, Corrigan and Toomey (1995) found
correlations between Social Cue Recognition Test and social problem-solving skills ranging from 0.54 to 0.78. Sergi et al. (2006)
showed associations between Profile of Nonverbal Sensitivity and
functional status ranging from 0.11 to 0.36. Although these measures have veridicality, as illustrated through the moderate associations with functioning, they lack verisimilitude. Simply put, the
stimuli and the cognitive processing needed to complete these tests
are limited compared with the social perception demands commonly
experienced in daily life.
The Interpersonal Perception Task-15 (IPT-15; Costanzo and
Archer, 1993) is a measure of social perception with higher verisimilitude. Whereas other measures of social perception use actors to
display social cues, the videotape clips of the IPT-15 involve real
persons in real situations. While other measures of social perception
use brief exposures of a limited number of social cues, the videotape
clips of the IPT-15 involve highly varied social cues present in
everyday situations. These social cues include facial expressions,
gestures, voice tone, proxemics (spatial distance), haptics (touching), and verbal exchanges connoting subtleties in intimacy and
status. Whereas other measures tend to depict 1 or 2, typically
Caucasian, actors, the IPT-15 involves many people representing
diversity in race, age, and gender.
The aim of the present study was to examine the ecological
validity, specifically the veridicality, of the IPT-15. The IPT-15
clearly has high verisimilitude. However, because the IPT-15 only
involves 15 items, veridicality, the statistically determined aspect of
ecological validity, may be modest.

METHODS
Participants
The sample included 72 outpatients with a diagnosis of
schizophrenia or schizoaffective disorder (SZ) and 58 healthy comparison persons (HC). All participants provided their written informed consent. The SZ group was recruited through the clinics of
the Veterans Affairs Greater Los Angeles Health Care System as
well as local board-and-care facilities. Diagnosis was based on the
Structured Clinical Interview for DSM-IV Axis I Disorders (SCIDFirst et al., 1997). The HC group was recruited by advertisements in
newspapers and on the internet. All potential HC were administered
the SCID and selected sections of the Structured Clinical Interview
for DSM-IV Axis II Disorders (SCID-IFirst et al., 1996). They were
excluded if they had a history of any psychotic disorder, recurrent
depression, bipolar disorder, substance dependence, or if they met
criteria for any of the following Axis II disorders: Avoidant, Paranoid, Schizoid, and Schizotypal. Having a first degree relative with
a psychotic disorder was another exclusion criterion in the HC
group. All SCID interviewers were trained to administer the SCID in

The Journal of Nervous and Mental Disease Volume 197, Number 9, September 2009

The Journal of Nervous and Mental Disease Volume 197, Number 9, September 2009

TABLE 1. Demographic Characteristics and Social

Perception Performance in Participants With Schizophrenia
(SZ) and in Healthy Control Participants (HC); and Clinical
Characteristics in SZ
SZ (n 72) HC (n 58)
Mean (SD)
Mean (SD)
Demographics
Age
Education
Mothers education
Gender (M/F)
Clinical status
Positive symptoms
Negative symptoms
Affective flattening
Alogia
Avolition-apathy
Anhedonia-asociality
Inattention
Social perception
IPT-15

Challenges of Ecological Validity

toms (Andreasen, 1984) is an interview-based measure that assesses

negative symptoms. The Scale for the Assessment of Negative
Symptoms offers global ratings for affective flattening, alogia,
avolition-apathy, anhedonia-asociality, and inattention.

Community Functioning
t/x2

46.7 (9.6)
12.9 (1.7)a
11.8 (2.8)b
61/11

40.9 (7.6)
13.4 (1.0)
12.7 (2.8)c
34/24

3.7
1.7
1.7
11.1

0.001
0.099
0.086
0.001

2.4 (1.5)

1.8 (1.2)
0.7 (1.0)
2.7 (1.1)
2.6 (1.3)
1.6 (1.3)a

8.7 (1.7)

10.2 (1.7)

4.9

0.001

n 71 due to missing data.

n 61 due to missing data.
n 56 due to missing data.
Positive symptoms indicates BPRS, items Hallucinations, Unusual Thought Content and Conceptual Disorganization; Negative symptoms, SANS global scores.
a

b
c

The Role Functioning Scale (Goodman et al., 1993) yields

scores for 4 domains of everyday life functioning: work productivity, independent living, relationships with family/spouse, and relationships with friends. The scale has satisfactory psychometric
properties (Goodman et al., 1993) and has been used extensively in
schizophrenia research (e.g., Kee et al., 2003; Brekke et al., 2009).
Ratings were completed by trained interviewers after a 30-minute
interview with the participant.

Statistical Analyses
All statistical analyses were conducted using the SPSS software, version 16.0 (SPSS Inc., Chicago, IL). Independent samples
t-tests were used to evaluate group differences for social perception
performance and demographic variables, except differences in the
gender distribution which was assessed with the chi square test.
Veridicality was assessed by conducting bivariate correlations
(Pearsons r) between IPT-15 scores and community functioning in
the SZ group. The association between social perception and symptoms was explored through correlational analyses (Pearsons r). The
internal consistency of the IPT-15 was examined using Cronbachs
alpha for the SZ and the HC samples separately. Variables were
normally distributed and thus parametric analyses used.

RESULTS
the VISN 22 Mental Illness Research, Education and Clinical Center
Treatment Unit and demonstrated agreement between their ratings
and the consensus ratings of expert diagnosticians (minimum Kappa
coefficient 0.75). The demographic characteristics of the 2 study
groups are displayed in Table 1.

Measures
Social Perception
The Interpersonal Perception Task-15 (IPT-15; Costanzo and
Archer, 1993) is a videotape-based measure of social perception.
The 15 scenes of the IPT-15 were selected from the 30-item
Interpersonal Perception Task (IPT; Costanzo and Archer, 1989) to
improve the reliability and validity of the measure, as healthy
persons performed at or near chance on several items of the 30-item
IPT. Each scene of the IPT-15 lasts 30 to 90 seconds and involves
1 to 4 persons, diverse in age, race, and gender. The persons in the
scenes are not actors; but real persons in real situations. The scenes
of the IPT-15 were videotaped on a university campus and nearby
community. Each scene is followed by 1 multiple-choice question
about either the status of the persons (e.g., who won the racquetball
game?), the veracity of a person making 2 separate statements (e.g.,
which was the truth, the first statement or the second statement?), or
the intimacy level between persons (e.g., how long have they been
dating, 2 weeks or 2 years?). The IPT-15 yields a score between 0
and 15. Costanza and Archer (1989) have reported on the test-rest
reliability (r 0.70) and internal consistency (Kuder-Richardson
formula 20 statistic 0.52) of the 30-item IPT.

Clinical Symptoms
The Brief Psychiatric Rating Scale ( Ventura et al., 1993) is
an interview-based measure that assesses varied psychiatric symptoms. A positive symptom factor score was generated from the 3
items Hallucinations, Unusual Thought Content, and Conceptual
Disorganization. The Scale for the Assessment of Negative Symp 2009 Lippincott Williams & Wilkins

Table 1 displays the demographic features, symptoms, and

social perception performance of the 2 groups. An analysis of
covariance with age and gender as covariates was used to examine
group differences in social perception because the SZ group was
older and had more males than the HC group. The SZ group
displayed less social perception skill than their healthy counterparts
after controlling for age and gender (F (1, 129) 17.1, p 0.001).
Correlational analyses did not support the veridicality of the
IPT-15. Small and nonsignificant associations were observed between social perception assessed with the IPT-15 and functional
status assessed with the Role Functioning Scale. The IPT-15 showed
a statistically significant association with 1 symptom (affective
flattening). Correlational analyses are presented in Table 2. The
internal consistency (Cronbachs alpha) of the IPT-15 was very poor
for both groups (SZ 0.03, HC 0.07).

DISCUSSION
The main goal of this study was to examine the ecological
validity (verisimilitude and veridicality) of the IPT-15 as a measure
of social perception used in schizophrenia research. The verisimilitude (apparent and expert-based judgment of closeness to reality of
both the stimuli and its social cognitive processing demands) of the
IPT-15 is quite strong. Unfortunately, the veridicality (statistical
associations suggestive of real world performance) of the IPT-15 is
quite weak in this sample. Social perception assessed by the IPT-15
was not correlated with functional status assessed with the Role
Functioning Scale in persons with schizophrenia. However, it discriminated well between the 2 groups of participants.
This study shows that verisimilitude is no guarantee of
veridicality. In fact, the features of the IPT-15 illustrate that these 2
aspects of ecological validity can be quite independent. The relatively lengthy, 30 to 90 second long, videotape scenes of the IPT-15
add to the verisimilitude of the measure by making them look very
contextual and real. Partly due to the length of the scenes, the
measure has fewer items. The lower number of items reduces
www.jonmd.com | 701

Vaskinn et al.

The Journal of Nervous and Mental Disease Volume 197, Number 9, September 2009

CONCLUSIONS

TABLE 2. Bivariate Associations Between IPT-15 Scores and

Community Functioning (Veridicality) and Symptoms in SZ
IPT-15
Community functioning
Work productivity
Independent living
Family network relationships
Immediate social relationships
Symptoms
Positive symptoms
Negative symptoms
Affective flattening
Alogia
Avolition-apathy
Anhedonia-asociality
Inattention

0.03
0.02
0.04
0.09
0.00
0.28
0.22
0.13
0.10
0.17

a
Significant at the 0.05 level.
Community Functioning indicates role functioning scale; Positive symptoms,
BPRS, items hallucinations, unusual thought content and conceptual disorganization;
Negative symptoms, SANS global scores.

veridicality through the restricted range of scores and reduced

sampling of items in the relevant domain, which makes it more
difficult to identify associations with measures of functional status.
Measures with a larger number of items would likely have better
veridicality.
The low internal consistency is concerning as it raises questions about whether the IPT-15 is assessing a single domain or
construct. Only one type of social judgment (of status, veracity or
intimacy) is required to answer any one of the IPT multiple-choice
questions. It is possible that understanding these different facets of
social interactions involves somewhat different subprocesses, and
that this is reflected in the low internal consistency. Hence, the IPT
may actually assess more than one social perceptive domain. It
should be noted that many tests used in schizophrenia research (e.g.
the continuous performance test for vigilance, or some tests of
episodic memory) are not seen as suitable for internal consistency
calculation, but are nevertheless highly informative regarding risk
factors and community functioning. A low internal consistency in
this case simply means that test does not measure one domain. By
itself, it does not necessarily mean that a test is uninformative.
The results draw attention to a trade off. The IPT-15 is a
measure highly similar to daily tasks. With its objective accuracy
criterion it has correct answers. It is well-tolerated by test takers and
differentiates between people with and without presumed social
cognitive difficulties. But the low veridicality and internal consistency show that these features do not guarantee that a measure is a
useful research instrument for psychopathology.

702 | www.jonmd.com

The present study shows that the 2 approaches to ecological

validity (verisimilitude and veridicality) can dissociate. This underscores the importance of performing psychometric investigations of
measures borrowed from other fields, such as social psychology,
before utilizing them in schizophrenia research.
ACKNOWLEDGMENTS
The authors thank William P. Horan, PhD, for useful advice
during the planning of the study, and Mark McGee, BA, for his help
in preparing the data set.
REFERENCES
Andreasen NC (1984) Scale for the Assessment of Negative Symptoms (SANS).
Iowa City, IA: University of Iowa College of Medicine.
Brekke JS, Hoe M, Green MF (2009) Neurocognitive change, functional change
and service intensity during community-based psychosocial rehabilitation for
schizophrenia. Psychol Med. In press.
Chaytor N, Schmitter-Edgecombe M (2003) The ecological validity of neuropsychological tests: A review of the literature on everyday cognitive skills.
Neuropsych Rev.13:181197.
Corrigan PW, Toomey R (1995) Interpersonal problem solving and information
processing in schizophrenia. Schiz Bull. 21:395 403.
Costanzo M, Archer D (1989) Interpreting the expressive behavior of others: the
interpersonal perception task. J Nonverbal Behav. 13:225244.
Costanzo M, Archer D (1993) The Interpersonal Perception Task - 15. A Guide for
Researchers and Teachers. Berkeley, CA: University of California Extension.
Couture SM, Penn DL, Roberts DL (2006) The functional significance of social
cognition in schizophrenia: A review. Schiz Bull. 32:S44 S63.
First MB, Spitzer RI, Gibbon M, Williams JBW, Benjamin L (1996) The
Structured Clinical Interview for DSM-IV Axis II Personality Disorders. New
York: Biometrics Research.
First MB, Spitzer RI, Gibbon M, Williams JBW (1997) The Structured Clinical
Interview for DSM-IV Axis I Disorders - Patient Edition. New York: Biometrics
Research.
Franzen MD, Arnett PA (1997) The validity of neuropsychological assessment
procedures. In HW Reese, MD Franzen (Eds), Biological and Neuropsychological Mechanisms: Life-Span Developmental Psychology (pp 51 69). Mahwah, NJ: Erlbaum.
Goodman SH, Sewell DR, Cooley EL, Leavitt N (1993) Assessing levels of
adaptive functioning: the role functioning scale. Community Mental Health J.
29:119 131.
Green MF, Penn DL, Bentall R, Carpenter WT, Gaebel W, Gur RC, Kring AM,
Park S, Silverstein SM, Heinssen R (2008) Social cognition in schizophrenia:
An NIMH workshop on definitions, assessment and research opportunities.
Schiz Bull. 34:12111220.
Kee KS, Green MF, Mintz J, Brekke JS (2003) Is emotion perception a predictor
of functional outcome in schizophrenia? Schiz Bull. 29:487 497.
Sergi MJ, Rassovsky Y, Nuechterlein KH, Green MF (2006) Social perception as
a mediator of the influence of early visual processing on functional status in
schizophrenia. Am J Psych. 163:448 454.
Vaskinn A, Sundet K, Friis S, Simonsen C, Birkenaes AB, Jonsdottir H, Ringen
PA, Andreassen OA (2008) Emotion perception and learning potential: Mediators between neurocognition and social problem-solving in schizophrenia?
J Int Neurops Soc. 14:279 288.
Ventura J, Green MF, Shaner A, Lieberman RP (1993) Training and quality
assurance with the brief psychiatric rating scale: The drift busters. Int J
Methods Psychiatr Res. 3:221244.

2009 Lippincott Williams & Wilkins