BRAIN INJURY,

2002,

VOL.

16,

NO.

6, 509 516

Cutoff score on the apathy evaluation scale in

subjects with traumatic brain injury
MEL B. GLENNy , DAVID T. BURKEy ,
THERESE O NEIL-PIROZZI},
RICH ARD GOLDSTEIN y , LO YAL
JACOB and JENNIFER KETTELL}
y Department of Physical Medicine and Rehabilitation, Harvard Medical School,
Boston, MA, USA
Spaulding Rehabilitation Hospital, Boston, MA, USA
} Section Department of Speech-Language Pathology and Audiology, Northeastern
University, Boston, MA, USA
} Hebrew Rehabilitatio n Center for the Aged, Boston, MA, USA
(Received 15 February 2001; accepted 3 December 2001 )
This cross-sectional study was designed to determine a cutoff score on the Apathy Evaluation Scale
(AES) that predicts a clinicians designation of a subject with TBI as apathetic or not. Forty-five
outpatients with TBI completed the AES-S, and 37 family members, friends, or significant others
filled out the AES-I. Three clinicians prospectively gave their impressions of the presence or absence of
apathy and retrospectively chose the degree of apathy on a 7-point subjective rating scale. The data
was analysed by logistic regression and Receiver Operating Characteristic (ROC) curve. Sensitivity
and specificity were calculated. No cutoff score on the AES-S or AES-I was found to have reasonable
sensitivity and specificity with respect to the ability to predict the clinician s designation of a subject as
apathetic. The AES requires further study if it is to be used to measure apathy following TBI.

Introduction
Apathy is a common consequence of traumatic brain injury (TBI). Apathy has been
represented as a state characterized by decreased initiative and akinesia [1 3]. Marin
[3] defined apathy as a `lack of motivation not attributable to diminished level of
consciousness, cognitive impairment, or emotional distress. Marin describes three
domains that lead one to recognize apathy: (1) `deficits in goal-directed behaviour,
(2) `a decrement in goal-related thought content, and (3) emotional indifference
with flat affect [4].
Van Zomeran and Van den Burg [5] found that 23% of subjects with TBI
complained of decreased initiative 2 years after injury. In a survey of people with
TBI 6 months after injury, 21% reported `difficulty in becoming interested ([6], p.
615). In a follow-up survey 7 years from the time of injury, 28% cited `difficulty in
becoming interested as a problem, while 43% of their relatives endorsed this item as
an issue for the subjects with TBI ([7], p. 565). Despite its frequent occurrence and

Correspondence to: Mel B. Glenn, MD, Spaulding Rehabilitation Hospital, 125 Nashua St.,
Boston, MA 02114, USA. e-mail: mglenn@partners.org
Brain Injury ISSN 0269 9052 print/ISSN 1362 301X online # 2002 Taylor & Francis Ltd
http://www.tandf.co.uk/journals
DOI: 10.1080/02699050110119132

510

M. B. Glenn et al.

its propensity to interfere with efforts at rehabilitation, there is a scarcity of literature

on apathy following TBI.
Depression can be difficult to distinguish from neurologically-based apathy in
some patients [8]. Among young adult clients of the Commonwealth of
Massachusetts Statewide Head Injury Programme (mean age 28) who received
special education services in Massachusetts, 63% reported depression, 42% apathy
and 42% lack of initiative on a problem checklist [9]. Using the Apathy Evaluation
Scale (AES) [10] and the Beck Depression Inventory (BDI) [11], Kant et al. [12]
reported that 11% of subjects with TBI seen in a neuropsychiatric clinic were
apathetic but not depressed, while 60% were both apathetic and depressed.
The AES, developed by Marin et al. [10], is an 18-item tool that was first used to
measure the presence/absence of apathy in individuals with Alzheimers disease,
stroke, and depression, as well as in normal elderly (see table 1). It can be administered directly to the subject (AES-S) as a paper and pencil test, to a family member
or other informant (AES-I), or completed by a clinician after a semi-structured
interview with the subject (AES-C). The AES has been used to measure apathy
with and without depression in individuals with TBI [12].
In their study of the incidence of apathy in subjects with TBI, Kant et al. [12]
arbitrarily used an AES score that was 2 SD below the mean to designate a subject as
apathetic. Because this cutoff score may not accurately reflect a clinicians assessment
of the range of apathy that may interfere with a subjects daily function, the authors
conducted this study to determine such a cutoff score on the AES-S, to evaluate the
internal consistency of the AES for subjects with TBI, and to confirm Kant et al.s
findings with respect to the incidence of apathy and its overlap with depression in
people with TBI.

Table 1.
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.

Apathy evaluation scale, informant version (AES-I)

He/she is interested in things.

He/she gets things done during the day.
Getting things started on his/her own is important to him/her.
He/she is interested in having new experiences.
He/she is interested in learning new things.
He/she puts little effort into anything.
He/she approaches life with intensity.
Seeing a job through to the end is important to him/her.
He/she spends time doing things that interest him/her.
Someone has to tell him/her what to do each day.
He/she is less concerned about his/her problems than he/she should be.
He/she has friends.
Getting together with friends is important to him/her.
When something good happens, he/she gets excited.
He/she has an accurate understanding of his/her problems.
Getting things done during the day is important to him/her.
He/she has initiative.
He/she has motivation.

Items are rated and assigned points as follows: not at all true 0, slightly true 1, somewhat true 2, very
true 3.
Adapted with permission from Marin [4].

Cutoff score on the apathy evaluation scale

511

Methods
Subjects
Subjects were outpatients with TBI who were included if they were 16 years of age
and over, presented with Rancho Los Amigos Levels of Cognitive Functioning
Scale (RLAS) [13] level of V or higher at time of study participation, and did not
have other significant neurologic diagnoses or acute medical or behavioural conditions. Subjects were excluded if they had had psychostimulant or dopaminergic
medication dosage changes over the previous 6 weeks or antidepressant changes
over the past 8 weeks.
Subjects were classified as mild TBI if there was a change in mental status with
initial Glasgow Coma Scale (GCS) score of 13 15 following a loss of consciousness
(LOC) of 0 30 minutes, and if the period of post-traumatic amnesia was less than 24
hours. Subjects were classified as moderate TBI if the initial GCS score was 9 12 or
if LOC was 30 minutes to 6 hours. Those classified as severe TBI had an initial GCS
score of less than 9 with LOC greater than 6 hours.
Thirty-two (70%) subjects were male and 14 (30%) female. The mean age across
all subjects was 43.1 years (17.7 74.3, SD 14.9). The mean time since injury was
43.9 months (4.3 260.2, SD 57.7). GCS information, in some cases obtained retrospectively, was available on 41 subjects: 23 (52%) were classified as mild, eight (18%)
as moderate, and 13 (30%) as severe TBI. The mean Disability Rating Scale (DRS)
[14] score at the time of study participation was 3.0 (0 10.5, SD 2.6).
Instruments
The AES is an 18-item scale with the following scoring system for each item: not at
all true 0, slightly true 1, somewhat true 2, very true 3 (see table 1).
Validity has been studied in several populations, but not in subjects with TBI
[10]. Reliability has been studied in TBI [12] and in several other populations [10].
The RLAS is an 8-level observational scale describing a continuum of cognitive
and behavioural function following TBI. Inter-rater and test re-test reliability, as
well as concurrent and predictive validity have been reported [13].
The DRS is a 30-point rating scale measuring a continuum of outcomes following TBI. It covers a spectrum of impairment, disability, and handicap from basic
responsivity to employability. Inter-rater reliability, as well as concurrent and predictive validity have been established [14].
The BDI-II is a 21-item self-report inventory used to assess the severity of
depression. Test re-test reliability and convergent validity have been reported [15].
Procedures
Of 53 consecutive outpatients with TBI seen in follow-up visits with the studys
three clinician investigators, 46 consented to participate. Forty-five completed the
AES-S at the time of an outpatient visit. One subject did not complete the AES-S
due to aphasia. The Beck Depression Inventory-II (BDI-II) [15] was filled out as
well. Thirty-seven family members, friends, or significant others were available at
the time of the outpatient visit or within several days, and filled out the AES-I.
Subjects were assisted with reading the form when necessary.

512

M. B. Glenn et al.
Table 2.

(1)
(2)
(3)
(4)
(5)
(6)
(7)

The authors 7-point apathy scale

Initiates moderately or frequently, eukinetic or hyperkinetic, emotionally demonstrative or

well-modulated.
Initiates moderately or frequently, eukinetic or hyperkinetic, emotionally demonstrative or
well-modulated, but c/o apathy.
Initiates modestly, but regularly; mildly akinetic; mild emotional indifference.
Initiates infrequently; moderately akinetic; moderate emotional indifference.
Rarely initiates; voluntary movement only in response to cues from environment or bodily needs; rare
or no display of even minor emotional response.
Rarely initiates; rare voluntary movement; rare or no display of even minor emotional response.
Never initiates; completely akinetic, even when uncomfortable; no evidence of emotional
responsiveness (essentially akinetic mute or vegetative state).

Each subjects physician and/or speech-language pathologist prospectively gave

a blind impression of the presence or absence of apathy, scored the subject on the
DRS, and, after the initial data analysis, retrospectively described his/her perception
of the subjects apathy on a 7-point subjective rating scale (table 2). In order to study
the relationship between apathy and depression, the clinicians were instructed to use
the following variation on Marins definition of apathy, designed to be inclusive of
depressed individuals and to be more operational than the original definition: `a lack
of motivation not attributable to diminished level of consciousness, cognitive
impairment, or motor dysfunction; and manifested by decreased initiative, akinesia,
emotional indifference, and flat affect.
Data analysis
Data were analysed using descriptive and quantitative statistics. Specific statistics
included t-tests, analyses of variance, logistic regression (and subsequent Receiver
Operating Characteristics (ROC) curve), and Cronbachs result.
Results
The data was analysed by logistic regression and ROC curve. No cutoff score on
the AES-S or AES-I was found to have reasonable sensitivity and specificity with
respect to the ability to predict the clinicians designation of `apathetic or `not
apathetic. For the AES-I, the area under the ROC curve was 0.62. For the
AES-S, the area under the ROC curve was 0.74.
It was thought that a binary choice may have resulted in an imprecise application
of the definition of apathy that had been provided to the clinicians. Therefore, the
authors created a 7-point scale, which the three clinicians applied to the subjects
retrospectively (see table 2). A ROC curve found that a score of > 32 on the AES-S
produced the best combination of sensitivity and specificity (area under ROC
curve 0:81). Sensitivity was 95%, but specificity was 0% with respect to prediction of a rating of at least 2 (the lowest score consistent with any apathy) on a 7point apathy rating scale applied by the clinician (positive predictive value (PPV)
81%, negative predictive value (NPV) 0%). A score of > 41 provided a sensitivity of
68% (PPV 61%) and a specificity of 45% (NPV 53%) for the prediction of a rating of
at least 3 on a 7-point apathy scale (area under ROC curve 0:61) (see table 3). A
logistic model that included AES-S, gender, DRS score, age, and time since injury

Cutoff score on the apathy evaluation scale

Table 3.

Sensitivity and specificity of best cutoff on AES-S indicated by ROC curves

Rating on scale

Best cutoff by
ROC curve

2 or higher
3 or higher

AES-S > 32
AES-S > 41

Table 4.
Rating on scale
2 or higher
3 or higher

Sensitivity
(%)

Specificity
(%)

0.81
0.61

95
68

0
45

Best cutoff by
ROC curve

Area under
ROC curve

Sensitivity
(%)

Specificity
(%)

AES-I > 24
AES-I > 35

0.77
0.71

100
77

17
60

Severity category

AES-S scoresa by severity category

Mean AES-S

SD

38.5
35.0
35.3
37.0
38.8
24.4

8.3
10.7
8.1
8.6
9.8
4.5

23
7
13
43
83
127

Mild
Moderate
Severe
Total
Kant et al.s subjectsb
Kant et al.s healthy control subjects
b

Area under
ROC curve

Sensitivity and specificity of best cutoff on AES-I indicated by ROC curves

Table 5.

513

Higher AES scores indicate more severe apathy.

75% classified as mild TBI.

improved the ROC curve such that the sensitivity was 86% and the specificity 97%
(area under ROC 0:97).
On the AES-I, the best ROC curve (area under ROC curve 0:77) was at a
cutoff score of > 24 for a prediction of at least 2 on the 7-point scale (sensitivity
100%, specificity of 17%; PPV 86%, NPV 100%). For a prediction of at least 3 on
the scale, the best area under the ROC curve was 0.71 for a cutoff score of > 35,
with a sensitivity of 77% and specificity of 60% (PPV 74%, NPV 64%) (see table 4).
Given an inability to find a cutoff score that resulted in reasonable sensitivity and
specificity, the authors are unable to report the prevalence of apathy in this group.
The mean AES-S score was 37.3 (SD 8:8, n 45) and the mean AES-I score was
39.4 (SD 9:1, n 37) The difference was not significant. The correlation (r)
between AES-S and AES-I was 0.55. There was no significant difference in scores
among the mild, moderate and severe groups on the AES-S or AES-I (see tables 5
and 6). The greatest difference was between moderate severe and mild groups, the
latter having slightly higher scores (p 0:21).
Correlation coefficients (r) for the association between BDI-II scores with AESS and AES-I scores were 0.56 and 0.52, respectively. The results of the depression
survey are reported elsewhere [16]. Cronbachs was 0.90 for the AES-S, 0.92 for
the AES-I, and 0.93 for the BDI-2.

514

M. B. Glenn et al.
Table 6.

Severity category
Mild
Moderate
Severe
Total
Kant et al.s subjectsb
a
b

AES-I scoresa by severity category

Mean AES-I

SD

38.0
39.7
40.1
38.9
50.5

9.8
5.9
8.7
8.8
6.6

19
6
10
35
28

Higher AES scores indicate more severe apathy.

75% classified as mild TBI.

Discussion
The authors were unable to find a cutoff score on the AES-S or AES-I that
provided reasonable sensitivity and specificity with respect to a clinicians designation or rating of a subject as apathetic. When the AES-S was included in a logistic
model with several other variables, good sensitivity and specificity resulted, which
indicates that the clinicians were making a predictable distinction among the subjects, but that the AES did not capture this characteristic. This may reflect a discrepancy between the authors use of the term `apathy and that which the AES
measures or a problem with the validity and/or reliability of the AES-S. This studys
explicit allowance of depression as a cause of apathy could have contributed to the
discrepancy. Marin et al. [10] did not include many items reflecting emotional
aspects of apathy in the AES, as their scale development procedures tended to
eliminate such items. These procedures included item total correlations, factor
analysis, and the choice of seven items that did not correlate with the Hamilton
Rating Scale for Depression in order to allow the scale to discriminate apathy from
depression [10]. Nonetheless, Kant et al. [12] found that 85% of subjects who were
apathetic according to their AES-S criteria also met their BDI criteria for depression.
In the analysis of the BDI-II scores in these subjects, the authors found a greater
prevalence of depression among those with mild TBI [16]. Although one might
expect to find greater apathy among those with severe TBI, on both the AES-S and
AES-I, scores were similar among those with mild, moderate and severe TBI. In
fact, AES-S scores were somewhat higher among those with mild TBI. This could
be a result of the influence of depression on AES scores, which correlated modestly
with BDI-II scores. These factors argue against the possibility that this studys
allowance of depression as a cause of apathy interfered with the AESs ability to
predict the clinicians designation of apathy, but does indicate that the AES may not
discriminate between neurologically-based apathy and apathy caused by depression.
It is also possible that the AES is not a psychometrically sound measurement
tool, i.e. that there are reliability problems, or that it does not accurately reflect
either Marin et al.s or the authors definition of apathy. Test re-test reliability for
the AES-I and AES-C (clinician as rater based upon a single interview) have been
found to be good, but the test re-test reliability of the AES-S for subjects with
Alzheimers disease was not adequate [10]. AES test re-test reliability has not been
assessed in subjects with TBI. The authors did find good internal consistency for
both the AES-S and AES-I.
Intercorrelations among the AES-S, AES-I and AES-C ranged from 0.43 0.72,
the lowest correlation being between the AES-S and AES-I [10]. There was a

Cutoff score on the apathy evaluation scale

515

modest correlation between the two in this study (r 0:55). Kant et al. [12] found a
discrepancy between AES-S and AES-I results, with subjects rating themselves as
less apathetic. As hypothesized, this may be the result of having a subject with
frontal lobe disorders attempt to self-evaluate. However, the authors did not find
a significant difference between AES-S and AES-I results (p 0:31).
Marin et al. [10] did two types of predictive validity studies, with several items in
each study. These studies resulted in non-significant correlations between AES
scores and more than half of the predictive measures. For those that were significant,
correlations (Pearsons r) were no better than 0.45 between the scores and any single
predictive measure.
Stuss et al. [17] reviewed the various definitions of apathy seen in the literature.
They argue that it is problematic to rely upon the concept of motivation to define
apathy, as do Marin et al. [10], since motivation is an internal state and cannot be
directly measured. Marin [4] addresses this issue by providing behavioural anchors to
his definition. However, Stuss et al. [17] have also presented evidence that apathy is
not a single neuroanatomic nor neuropsychologic entity, which certainly could lead
to problems with validity for any tool that attempts to measure it as such.
The authors are unable to report the prevalence of apathy in this population.
However, the AES-S scores in the study population were in the vicinity of those
described by Kant et al. [12] in an outpatient population (see tables 4 and 5) with
TBI. Although, as noted above, this studys AES-I scores were not significantly
above the AES-S scores.
This studys findings suggest that the AES requires further study before it can be
confidently used to measure apathy. The AES-I may turn out to be a better measure
of apathy than the AES-S.

Acknowledgements
Supported by grant H133A980034-00 from the National Institute on Disability and
Rehabilitation Research, United States Department of Education.

References
1. Mattson, A. J. and Levin, H. S.: Frontal lobe dysfunction following closed head injury: a review
of the literature. Journal of Nervous and Mental Disease, 178: 282 291, 1990.
2. Okada, K., Kobayashi, S., Yamagata, S. et al.: Poststroke apathy and regional cerebral blood
flow. Stroke, 28: 2437 2441, 1997.
3. Marin, R. S.: Apathy: a neuropsychiatric syndrome. Journal of Neuropsychiatry and Clinical
Neurosciences, 3: 243 254, 1991.
4. Marin, R. S.: Apathy: concept, syndrome, neural mechanisms, and treatment. Seminars in Clinical
Neuropsychiatry, 1: 304 314, 1996.
5. Van Zomeran, A. and Van den Burg, W.: Residual complaints of patients two years after severe
head injury. Journal of Neurology, Neurosurgery and Psychiatry, 48: 21 28, 1985.
6. Oddy, M., Humphrey, M. and Uttley, D.: Subjective impairment and social recovery after
closed head injury. Journal of Neurology, Neurosurgery and Psychiatry, 41: 611 616, 1978.
7. Oddy, M., Coughlan, T., Tyerman, A. et al.: Social adjustment after closed head injury: a
further follow-up seven years after injury. Journal of Neurology, Neurosurgery and Psychiatry, 48: 564
568, 1985.
8. Corey, M. R.: A comprehensive model for psychosocial assessment of individuals with closed
head injury. Cognitive Rehabilitation, 5: 28 33, 1987.

516

Cutoff score on the apathy evaluation scale

9. LaVecchio, F. A.: Final report of the Massachusetts Traumatic Brain Injury Transition Project: the special
educational and transitional experiences of adult survivors of traumatic brain injury in childhood: a retrospective
analysis of functional outcome determinants and quality of life (Commonwealth of Massachusetts
Executive Office of Health & Human Services, Massachusetts Rehabilitatio n Commission
Statewide Head Injury Program), 1996.
10. Marin, R. S., Biedrzycki, R. C. and Firinciogullari, S.: Reliability and validity of the
Apathy Evaluation Scale. Psychiatry Research, 38: 143 162, 1991.
11. Beck, A. T., Steer, R. A. and Garbin, M. G.: Psychometric properties of the Beck Depression
Inventory: twenty-five years of evaluation. Clinical Psychology Review, 8: 77 100, 1988.
12. Kant, R., Duffy, J. D. and Pivovarnik, A.: Prevalence of apathy following head injury. Brain
Injury, 12: 87 92, 1998.
13. Gouvier, W. D., Blanton, P. D., LaPorte, K. K. et al.: Reliability and validity of the
Disability Rating Scale and the Levels of Cognitive Functioning Scale in monitoring recovery
from severe head injury. Archives of Physical Medicine and Rehabilitation, 68: 94 97, 1987.
14. Rappaport, M., Hall, K. M., Hopkins, K. et al.: Disability rating scale for severe head trauma:
coma to community. Archives of Physical Medicine and Rehabilitation, 63: 118 123, 1982.
15. Beck, A. T., Steer, R. A . and Brown, G. K.: Beck Depression Inventory, 2nd edn manual (San
Antonio: The Psychological Corporation, Harcourt Brace & Company), 1996.
16. Glenn, M. B., ONeil-Pirozzi, T., Goldstein, R. et al.: Depression amongst outpatients with
traumatic brain injury. Brain Injury, 15: 811 818, 2001.
17. Stuss, D. T., Van Reekum, R. and Murphy, K. J.: Differentiation of states and causes of
apathy. In: J. Borod (editor) Neuropsychology of Emotion (New York: Oxford University Press)
pp. 340 363, 2000.