Current Topics in Research

American Journal of Alzheimers

Disease & Other Dementias
The TMA-93: A New Memory Test 1-7
The Author(s) 2017
Reprints and permission:
for Alzheimers Disease in Illiterate sagepub.com/journalsPermissions.nav
DOI: 10.1177/1533317517722630
and Less Educated People journals.sagepub.com/home/aja

Didier Maillet, MSc1,2, Pauline Narme, PhD1,3,

Helene Amieva, PhD4, Fanny Matharan, MSc4, Olivier Bailon, MD1,
Herve Le Clesiau, MD5, and Catherine Belin, MD, MA1,3

Abstract
The present study aimed at validating the Memory Associative Test of the district of Seine-Saint-Denis (TMA)-93, a new test of
episodic memory. The TMA-93 was proposed to mostly less educated and multicultural elderly population composed of 376
healthy controls (HC) and 94 patients with Alzheimers disease (AD). The construct validity was checked by studying correlations
with a widely used memory test (the Free and Cued Selective Reminding Test [FCSRT]) in the subsample of literate patients.
Results showed that (i) all the TMA-93 scores of the patients with AD were lower than those of the HC, (ii) the TMA-93 total
score identified patients with AD with a high sensitivity (88%) and very high specificity (97%), and (iii) the TMA-93 total score was
strongly correlated with both free recall and total recall scores of the FCSRT. Taken together, results showed that the TMA-93 is
a reliable tool to assess episodic memory in a multicultural, less educated, or illiterate population, with good construct validity for
AD diagnostic accuracy.

Keywords
cognitive assessment, memory test, associative memory, dementia, cross-cultural neuropsychology, literacy

Introduction in memory clinics in France,12 the words chosen were con-

Clinical diagnosis of Alzheimers disease (AD) is based on the
establishment of cognitive decline, especially of episodic mem-
ory in the case of typical amnestic forms, sufficient to interfere
with daily living activities.1,2 To objectify this memory decline,
neuropsychological testing is required. However, in some cases
in clinical routine, the usual neuropsychological assessment
procedures are not possible or reliable; for example, when
patients are illiterate or have a low level of education, neurop-
sychological evaluation represents a real challenge due to the
lack of suitable tools.3,4 Normative data and the validation of
classical cognitive tests have been studied in literate partici-
pants only and often require abilities that have been acquired at
school such as reading and writing.5
Neuropsychological tests to assess episodic memory usually
involve learning and recalling a list of words or stories.6-8
When demographic effects on these tests are taken into
account, it appears that performance is more impacted by par-
ticipants educational level than by their age.9 Moreover, the
words used in these tests might not be appropriate when parti-
cipants are from other cultures and/or have a native language
that differs from that of the normalization/validation sample.
For example, in the French adaptation of the Free and Cued
Selective Reminding Test (FCSRT),10,11 which is widely used
trolled for their phonological complexity, length, and typicality
rank in French (as being familiar enough but not prototypical).
Added to the fact that the French FCSRT requires reading
abilities, not being a native French speaker and having a cul-
tural background that differs from the metropolitan French, one
will intrinsically change the typicality rank of words and con-
sequently the difficulty of the task (eg, a daffodil is not a
familiar flower for African people). Lastly, participants with
1
Unite Fonctionnelle Memoire et Maladies Neurodegeneratives, Service de
Neurologie, CHU Avicenne, Assistance Publique des Hopitaux de Paris
(AP-HP), Bobigny, France
2
Laboratoire PSITEC, EA 4072, UFR de psychologie, Universite Lille 3, Pont de
Bois, Villeneuve dAscq, France
3
Equipe Neuropsychologie du Vieillissement (EA 4468), Institut de Psychologie,
Universite Paris Descartes, Boulogne-Billancourt, France
4
INSERM U 897, EpidemiologieBiostatistique, ISPED, Universite de
Bordeaux, Bordeaux, France
5
Centre dExamens de Sante de la Caisse primaire dAssurance Maladie de la
Seine-Saint-Denis, Bobigny, France
Corresponding Author:
Didier Maillet, MSc, UF Memoire et Maladies Neurodegeneratives, Service de
Neurologie, Hopital Avicenne, 125 rue de Stalingrad, 93000 Bobigny, France.
Email: didier.maillet@aphp.fr
2 American Journal of Alzheimers Disease & Other Dementias XX(X)
no or less education preferentially learn by heart rather than by
using encoding strategies.13 For example, semantic clustering is
rarely implemented while encoding or retrieving information.9
Taking these limitations together, it seems inappropriate to
use classical memory tests for illiterate and/or multicultural
patients since a lower performance might be expected in these
patients, increasing the risk of false-positive errors in the diag-
nosis of AD.14 Yet, illiteracy and a low educational level are
now well known to be important risk factors for AD.15-17 It is
thus crucial to develop new memory tests for these populations.
A few previous studies suggested that procedures using the
recall of drawings or images of real objects might be more
suitable, especially when drawings refer to objects that are
familiar to patients.18,19 Based on these findings, we recently
validated a new episodic memory test (TNI-93 or Test des
Neuf Images du 93, ie, Nine Images test of the district of
Seine-Saint-Denis)20 for the screening of dementia in illiterate
and less educated patients. Although psychometric properties
of this first test were found to be good for the screening of
dementia in general, a test for the screening of AD in particular
was still needed. We designed a new test dedicated to this objec-
tive in illiterate and less educated people: the Memory Associa-
tive Test of the district of Seine-Saint-Denis (TMA-93)21
assessing episodic memory. The TMA-93 was based on the
principle of the associative learning strategy used in the subtest
of verbal paired associates from the Wechsler Memory Scale.8
Drawings of familiar objects of everyday life are displayed in
semantically related pairs during the encoding phase. Then,
participants have to recall the missing drawing when the asso-
ciated drawing is provided. This retrieval phase is repeated
3 times successively to assess participants learning abilities.
Although demographic effects have already been studied
to provide normalization data in healthy illiterate people,21 a
clinical validation study in pathology was needed. The present
study aimed at investigating (i) whether the TMA-93 shows
good properties for AD diagnostic accuracy in a population
living in France but with a low level of literacy or education
and (ii) the construct validity of the TMA-93 using correla-
tions between the TMA-93 and the French adaptation of the
FCSRT scores in a sample of native French speakers and
literate patients.
Methods
Participants
A total of 470 participants (healthy controls [HC]: n 376;
patients: n 94), aged older than 60 years, were included. All
participants have signed informed consent, and the study was
carried out following the Declaration of Helsinki principles.
The whole sample was multicultural, composed of native
French speakers with different levels of education and of par-
ticipants with an immigrant background, mainly less educated
with variable levels of French language skills, especially in its
written form. The HC were included over a 32-month period at
the Centre dExamen de Sante (CES; Center of Health Exams)
Table 1. Participants Characteristics.a
Healthy Controls Patients With
(n 376) AD (n 94) P Value
Age (years) 68.7 (5.9) 77.9 (6.5) <.001
Sex (male/female) 180/196 30/64 .002
Education level .25
Illiterates 68 19
1 to 7 years 148 41
8 to 11 years 97 15
12 years 63 19
MMSE (of 30) - 19 (5.1)b
Abbreviations: AD, Alzheimers disease; MMSE, Mini-Mental State Examination.
a
The values are expressed as mean (standard deviation).
b
Missing data for 14 patients.
in Bobigny. All of them lived in the Seine-Saint-Denis district
(France). They were recruited on a voluntary basis during a free
health checkup proposed by the French National Health Ser-
vice. During the checkup, they were examined by a neurologist
who excluded a diagnosis of dementia or psychiatric illness
based on the clinical criteria of the Diagnostic and Statistical.
Manual of Mental Disorders (Fourth Edition).1 Participants
with severe visual and/or auditory disorders that could interfere
with neuropsychological testing were excluded, as were parti-
cipants with a history of neurological or psychiatric disease.
When memory disorders were suspected, the participant was
systematically excluded and addressed to the Memory Clinic
for further investigations.
Patients attending the Memory Clinic at the Department of
Neurology of Avicenne Hospital (Assistance Publique des
Hopitaux de Paris) over the same period and who met the clinical
diagnosis of AD according to the DSM-IV criteria1 were included.
The diagnosis of AD was made by a neurologist, who was blind
to the patients TMA-93 results. When the patients level of
French permitted (85% of the patients), the French version of the
Mini-Mental State Examination (MMSE)22,23 was proposed.
Demographic characteristics are summarized in Table 1.
The 2 groups differed in age (t468 13.2; P < .001; Cohen
d 1.32) and sex distribution (w21;N470 5.8; P .02; Cramer
V 0.13). Patients with AD were older than HC, and there was
an overrepresentation of women compared with HC. However,
education level was similar between groups (w21;N470 4;
P > .2; Cramer V 0.09).
Material and Procedure
The TMA-93 comprised black and white drawings from the
Snodgrass and Vanderwart database.24 Twenty drawings of
familiar objects of everyday life were selected (Figure 1) and
pretested on a sample of healthy volunteer controls from the
CES. This sample, aged 60 years or older, shared the same
sociocultural characteristics as participants included in the vali-
dation study. All drawings were easily identified and orally
named in French by the whole pretested sample, even when
they had no formal education and/or did not master the French
Maillet et al
Figure 1. Illustration of the TMA-93 procedure: (A) examples of paired associates to name and memorize and (B) presentation of the first
stimulus during the first cued recall.
language. Paired associates were then constituted such that
each paired drawing was semantically related (eg, tree and
bird; knife and apple; Figure 1), but all the associates volunteers at the CES. Demographic characteristics were,
refer to different semantic categories.
The TMA-93 is a test assessing episodic memory, with the
following advantages: (i) instructions are easily understandable
including for participants with a low level of French; (ii) the
test is short, taking around 10 minutes to administer; and (iii) it
is not necessary to master written French or even to have a high
level of spoken French since responses can be made in the
participants mother tongue. During the encoding phase, parti-
cipants were asked to orally name the 10 paired drawings suc-
cessively (the tree and the bird) and were explicitly told to
memorize them. After this encoding phase, the first cued recall
was proposed: For each pair of associates, only 1 drawing was
displayed (eg, the tree) and participants had to recall the
missing associated drawing (eg, the bird; Figure 1). After the
participants response, whatever its accuracy in a 5-second
delay, the previously encoded paired drawing was presented
again. If no response was given, the participant was asked to
name the missing drawing again. The procedure was the same
for the 9 other paired associates.
When participants did not accurately recall the drawing of
the 10 paired associates during the first cued recall (ie, giving a
score <10), a second and a third cued recall were proposed,
following the same procedure, leading to a total score of about
30 (by summing the number of correct responses on the 3 recall
phases). On the contrary, when participants obtained the max-
imum score of 10 after the first cued recall (or the maximum
score of 20 after the second cued recall), the procedure was
stopped, and 20 points (or the remaining 10 points) were auto-
matically credited for a total score of 30.
Several kinds of errors were distinguished: (i) we called
errors all responses corresponding to an object that belonged
to 1 of the 9 other paired associates, (ii) intrusions corre-
sponded to responses that did not belong to the 10 paired
associates, and (iii) perseverations referred to repeatedly pro-
duced errors during the whole procedure.
3
Normative data of the TMA-93 were established in a multi-
cultural population, mainly of a low level of education among
therefore, comparable to those of the sample included in the
present study who came from the same center. We briefly
report here the main results that have been previously described
elsewhere.21 Normative data were obtained based on the results
of 433 participants (68.5 + 6 years), of whom 59.4% were
native French speakers. Concerning participants education
level, 15.7% had had no schooling and were illiterate, 20.6%
had a primary school level (between 1 and 7 years of educa-
tion), 46.5% had a secondary school level (between 8 and
11 years of education), and only 17.2% had completed second-
ary education, with or without university degrees. This strati-
fication has been chosen following the French educational
system (primary school, secondary school corresponding to the
middle and high school until the baccalaureate, then higher
education after the completion of secondary school, that is,
12 years of education) and the usual stratification found in
French norms for neuropsychological tests. The study of demo-
graphic effects showed that age and sex did not impact the
TMA-93 total score, but that education level did. Thus, the
scores corresponding to the 5th percentile differed according
to the participants education level, with a total score of 23 of
30 for illiterates, 24 of 30 for less educated, and 25 of 30 for
secondary and university levels.21 Intrusions were rare since
6.7% produced 1 intrusion or more and only 1.8% produced 2
intrusions (no healthy participant produced more than 2).
In order to investigate correlations between the TMA and
the FCSRT, we selected a subgroup of patients with AD
(n 55) who underwent the FCSRT during their neuropsycho-
logical testing. This subgroup included 20 males and
35 females, aged 76.6 + 8 years, 93% of whom were native
French speakers (due to the high level of written French
required for this test). Only 4 had no schooling, 28 had a pri-
mary school level, 8 a secondary, and 15 a university level. In
the French version of the FCSRT,11 16 printed words have to be
learned. After a semantic cued encoding phase, followed by an
4 American Journal of Alzheimers Disease & Other Dementias XX(X)
immediate recall, patients were asked to free recall the 16
words. Then a cued recall (using the semantic category) was
proposed for words not retrieved by free recall. This procedure
was repeated 3 times, leading to a total free recall (maximum
score of 48) and a total recall (total free recall added to the total
cued recall; maximum score of 48).
Statistical Analyses
We first conducted between-group comparisons using Student t
tests to verify that age and sex did not affect the TMA-93
performances in HC. We then compared performances of
patients with AD to those obtained by HC on each TMA-93
score using univariate analysis of variances with group (HC;
patients with AD) as between-group factor separately on each
score (total score, errors, intrusions, and perseverations). Prop-
erties of the TMA-93 for the detection of AD were then
assessed by calculating its sensitivity, specificity, and positive
and negative predictive values. The Youden index was used to
identify the cutoff score corresponding to the best compromise
between sensitivity and specificity. Finally, Pearson correlation
analyses were conducted between the TMA-93 total score and
FCSRT scores.
Performances are expressed by mean + standard error of
the mean. Analyses were performed using SPSS version 18.0
(SPSS Inc, Chicago, Illinois), and the threshold for statistical
significance was set at P < .05.
Results
Preliminary Analyses
Considering that our study groups differed in age and sex dis-
tribution, we first checked whether these variables impacted
performances in HC. Male and female performance did not
significantly differ on the total score (28.2 + 0.14 and 28.5
+ 0.14, respectively; t374 1.5; P > .1; Cohen d 0.16),
errors produced (0.4 + 0.06 and 0.28 + 0.06; t374 1.5;
P > .1; Cohen d 0.15), or intrusions (0.1 + 0.03 and 0.06
+ 0.02; t374 1.7; P > .08; Cohen d 0.18). Controls did not
produce perseverations. To control for a potential age effect,
we constituted 2 groups based on the median age (60-68 years
vs 69 years). The 2 groups did not differ on the total score
(28.4 + 0.14 and 28.4 + 0.15, respectively; t374 0.1;
P > .9; Cohen d 0.001), errors produced (0.3 + 0.05 and
0.38 + 0.06; t374 1.2; P > .2; Cohen d 0.12), or intru-
sions (0.08 + 0.03 and 0.09 + 0.02; t374 0.3; P > .7;
Cohen d 0.03).
Effect of Pathology on TMA-93 Performances
The scores obtained on the TMA-93 in both groups are sum-
marized in Table 2. They revealed that performances of
patients with AD were lower regardless of the score consid-
ered. They recalled fewer drawings than HC (F1, 468 1109;
P < .001; Z2 .7) and produced more errors (F1, 468 43.3;
P < .001; Z2 .09), intrusions (F1, 468 176; P < .001;
Table 2. TMA-93 Performances in Healthy Controls Versus Patients
With Alzheimers Disease (AD).a
Healthy Controls Patients With
(n 376) AD (n 94) P Value
Total score (of 30) 28.4 + 0.2 12.2 + 0.4 <.001
Errors 0.34 + 0.05 1.14 + 0.11 <.001
Intrusions 0.09 + 0.07 2.11 + 0.14 <.001
Perseverations 0 0.91 + 0.07 <.001
a
The values are expressed as mean + standard error of the mean.
Z2 .27), and perseverations (F 1, 468 123; P < .001;
Z2 .21) than HC did.
Properties of TMA-93 for the Diagnosis of AD
Discriminative properties for patients with AD were studied
using the TMA-93 total score. The Youden index was calcu-
lated to identify the best threshold value of the total score for
the diagnosis of AD. It revealed that the highest value was
obtained with the cutoff of 24 of 30 (Youden index of 0.84).
When considering the total score <24 of 30 as a threshold
value, the performance of 83 of the 94 patients was considered
as abnormal (88% sensitivity) and the performance of 365 of
the 376 HC was considered as normal (97% specificity). The
positive predictive value of the TMA-93 total score was 88%
and its negative predictive value was 97%. Overall, the TMA-93
total score showed good properties for the diagnosis of AD.
Correlation Analyses
To verify the construct validity of the TMA-93 total score, we
then studied correlations between this index and performances
on the widely used FCSRT in a subgroup of patients with AD
(n 55, mean age: 77.9 + 6.5 and mean MMSE: 19.1 + 5.2).
In this subgroup, the mean TMA-93 total score was 15.4 + 1.2,
the FCSRT total free recall (of 48) was 7.8 + 0.8, and the
FCSRT total recall (of 48) was 25.4 + 1.4.
The TMA-93 total score was positively and strongly corre-
lated with the FCSRT total free recall (r .63; P < .001) and
with the FCSRT total recall (r .51; P < .001). We also
checked how many patients were considered as normal on the
TMA-93 (based on the previously calculated threshold value)
but abnormal using the French normative data of the FCSRT.11
It was found that 9 patients (ie, 16%) were in this case. When
looking more closely at their characteristics, it appeared that
most of them (8 of 9 patients) had a secondary or university
level of education, suggesting that the TMA-93 total score
might not be sensitive enough when participants had a
moderate to high level of education.
Discussion
The main aim of the present study was to validate a tool21
previously proposed to assess episodic memory in illiterate or
less educated people and to investigate its sensitivity and
Maillet et al
specificity for the diagnosis of AD. First of all, it is noteworthy
that the TMA-93 was well understood in our sample, even
when participants had no schooling, had a poor level of spoken
French, and/or presented cognitive disorders. Added to the fact
that this is a short test with no minimum level of reading and
writing, this result confirms that it is easily understandable and
well adapted for such a population. Second, we showed that the
TMA-93 total score has good sensitivity and specificity for the
detection of AD. The present clinical validation study indicates
a good sensitivity and specificity to detect AD in a multicul-
tural population mainly composed of less educated people.
Interestingly, although we did not specifically study this score
in the present control sample, normative data of the TMA-93
showed that intrusions were extremely rare in HC. This sug-
gests that the production of more than 2 intrusions might be a
sign of memory disorder, sometimes considered as a pathog-
nomonic sign of AD.25
The TMA-93 is based on associative learning and uses
visual presentation. These 2 properties might account for our
interesting results. It has been found that deficits in visual
memory are a good predictor of AD occurrence several years
before diagnosis.26 Consistently with the present findings,
Lindeboom et al27 reported good properties of another test in
the detection of AD in less educated people. The Visual Asso-
ciation Test used also consisted of associative material: Pairs of
semantically unrelated objects were displayed (eg, an ape hold-
ing an umbrella) during the encoding phase, then participants
were asked to recall the missing objects of each pair (only 1
object was presented as a cue for each pair).
In a validation perspective, the present study also aimed at
investigating the construct validity of the TMA-93. Correla-
tions between this test and the French adaptation of the FCSRT
scores were conducted in a sample of native French speakers
and literate patients (with moderate to high level of education).
As expected, strong and positive correlations were found
between the TMA-93 total score and both free recall and total
recall scores of the FCSRT. These results argue strongly in
favor of the interest of TMA-93 for the detection of AD since
it has been previously found that impairment of free recall and
total recall scores of the FCSRT (i) correlated with hippocam-
pal atrophy in AD28 and (ii) can identify prodromal AD in
patients with mild cognitive impairment with a high sensitivity
and specificity.12 This suggests that the TMA-93 might be
interesting not only in the detection of AD at the dementia
stage (as was the case for the patients included in the present
study). However, future studies are needed to further investi-
gate its properties in the detection of early stages and to ascer-
tain whether it could be extended to atypical presentations of
AD29 or to other dementias, which are frequently associated
with memory disorders.30 When we compared the TMA-93 and
FCSRT scores to detect patients with AD, we showed that the
FCSRT scores detected 16% of them better than the TMA-93
total score did. This might be considered as a limitation. How-
ever, when looking more closely at the characteristics of the
patients who were not correctly classified by the TMA-93, it
was found that 89% of them had a secondary or university level
5
of education. Taken together, these results indicate that the
TMA-93 might be an alternative to the FCSRT when neuro-
psychological testing concerns a less educated and/or illiterate
person (the population for whom the test was designed) but
cannot replace the FCSRT when the education level is higher.
Correlations between the TMA-93 and the picture version of the
FCSRT31 might have been more appropriate to investigate the
construct validity of our new tool, since both tests use drawings
while the French FCSRT only use words. However, norms for
the picture version of the FCSRT in a French sample are not
available. Furthermore, the word version of the FCSRT is the
most frequently used in France, and recent studies showed that
scores from the picture and word versions are correlated.32,33
The main limitation of the study is that our 2 groups were
not well matched on demographic variables other than educa-
tional level (ie, age and gender). We checked that these 2
variables did not impact the TMA-93 scores in the HC group.
Consistently, a large epidemiologic study found that memory
skills were quite well preserved in normal aging.34 Further
studies are required to confirm the absence of an age effect
on performances since this point would be of interest to assess
older participants (ie, the oldest-old). This specific popula-
tion is highly concerned by neuropsychological testing, given
that age is a main risk factor for AD.35,36 Yet, diagnosis in the
oldest-old is also challenging in view of several sources of
vulnerability (eg, physical, sensorial, and cognitive) and the
lack of normative data for people older than 85 for most neu-
ropsychological tests.37-40
In summary, the TMA-93 appears to have several advan-
tages for memory assessment in a less educated and/or multi-
cultural population. French normative data are available for
this test,21 and the present study strongly argues in favor of its
interest in pathology given its good properties for the detection
of AD. Several global scales have been published worldwide to
assess such a population, especially in India,41,42 China,43
Mexico,44 Tunisia,45 Senegal,46 and Australia.47 However,
there are few tools available to specifically assess cognitive
functions that are suitable for the neuropsychological evalua-
tion of less educated or illiterate populations.48-52 To assess
episodic memory, a few tools have been adapted or specifically
developed, for example, in India,19 the Netherlands,27 and
Brazil,18,53 but very few in French-speaking countries.54-56
To fill this gap, we also developed the TNI-93. 20,21 The
TNI-93 also assessed episodic memory. Our validation study
showed its high sensitivity and specificity to screen for demen-
tia (not specifically AD). Although the TNI-93 was validated in
mostly multicultural, illiterate, and less educated people, both
normative data21 and the validation study20 indicated that the
TNI-93 scores were not influenced by the level of education.
On the contrary, normative data showed a significant effect of
education level on the TMA-93 scores.21 This was confirmed in
the present study by the TMA-93s lower detection power of
patients with AD compared to the FCSRT scores when educa-
tion level was higher. Thus, we specifically recommend using
the TMA-93 to assess episodic memory in less educated people
and when looking specifically for the detection of AD in this
6 American Journal of Alzheimers Disease & Other Dementias XX(X)
population. Finally, to complete the neuropsychological testing
of illiterate and less educated populations, further studies are
needed to develop and validate tools adapted to assess other cog-
nitive functions, which are also frequently impaired and may con-
tribute to the diagnosis of dementia, such as executive
functions57,58 and social cognition.59,60
Acknowledgments
The authors thank Frederic Dessi, Karema Soufi, Valerie Haziza, and
Sarra Le who contributed to the data collection. They also thank
Elizabeth Rowley-Jolivet for editing the English expression of the
manuscript.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to
the research, authorship, and/or publication of this article.
Funding
The authors received no financial support for the research, authorship,
and/or publication of this article.
References
1. American Psychiatric Association. Diagnostic and Statistical
Manual of Mental Disorders (4th ed, Text Revised). Washington,
DC: American Psychiatric Association Press; 2000.
2. McKhann GM, Knopman DS, Chertkow H, et al. The diagnosis of
dementia due to Alzheimers disease: recommendations from the
National Institute on AgingAlzheimers Association workgroups
on diagnostic guidelines for Alzheimers disease. Alzheimers
Dement. 2011;7(3):263-269.
3. Brucki SM.Illiteracy and dementia. Dement. Neuropsychol. 2010;
4(3):153-157.
4. Ardila A, Bertolucci PH, Braga LW, et al. Illiteracy: the neurop-
sychology of cognition without reading. Arch Clin Neuropsychol.
2010;25(8):689-712.
5. Mitrushina MN, Boone KB, DElia LF. Handbook of Normative
Data for Neuropsychological Assessment. Oxford, UK: Oxford
University Press; 1998.
6. Buschke H. Selective reminding for analysis of memory and
learning. J Verbal Learn Verbal Behav. 1973;12(5):543-550.
7. Buschke H, Sliwinski MJ, Kuslansky G, Lipton RB. Diagnosis of
early dementia by the Double Memory Test: encoding specificity
improves diagnostic sensitivity and specificity. Neurology. 1997;
48(4):989-997.
8. Wechsler D. Wechsler Memory Scale. Revised Edition.
New York, NY: The Psychological Corporation; 1987.
9. Ardila A, Rosselli M, Rosas P. Neuropsychological assessment in
illiterates: visuospatial and memory abilities. Brain Cogn. 1989;
11(2):147-166.
10. Grober E, Lipton RB, Hall C, Crystal H. Memory impairment on
free and cued selective reminding predicts dementia. Neurology.
2000;54(4):827-832.
11. Van Der Linden M, Coyette F, Poitrenaud J, et al. Lepreuve
Rappel libre/Rappel indice a` 16 items (RL/RI-16) [The Rappel
libre/Rappel indice a 16 items (RL/RI-16) test]. Levaluation des
troubles de la memoire. Marseille, France: Solal; 2004:5-47.
12. Sarazin M, Berr C, De Rotrou J, et al. Amnestic syndrome of the
medial temporal type identifies prodromal AD: a longitudinal
study. Neurology. 2007;69(19):1859-1867.
13. Bartlett F. Remembering: A Study in Experimental and Social
Psychology. Cambridge, UK: Cambridge University Press; 1932.
14. Teng EL. Cultural and educational factors in the diagnosis of
dementia. Alzheimer Dis Assoc Disord. 2002;16(suppl 2):S77-S79.
15. Zhang MY, Katzman R, Jin H, et al. The prevalence of dementia
and Alzheimers disease (AD) in Shanghai, China: impact of age,
gender and education. Ann Neurol. 1990;27(4):428-437.
16. Stern Y, Gurland B, Tatemichi TK, Tang MX, Wilder D, Mayeux
R. Influence of education and occupation on the incidence of
Alzheimers disease. JAMA. 1994;271(13):1004-1010.
17. Barnes DE, Yaffe K. The projected effect of risk factor reduction
on Alzheimers disease prevalence. Lancet Neurol. 2011;10(9):
819-828.
18. Nitrini R, Caramelli P, Herrera Junior E, et al. Performance of
illiterate and literate nondemented elderly subjects in two tests of
long-term memory. J Int Neuropsychol Soc. 2004;10(4):634-638.
19. Verghese J, Noone ML, Johnson B, et al. Picture-based memory
impairment screen for dementia. J Am Geriatr Soc. 2012;60(11):
2116-2120.
20. Maillet D, Matharan F, Le Clesiau H, et al. TNI-93: a new mem-
ory test for dementia detection in illiterate and low-educated
patients [published online September 1, 2016]. Arch Clin
Neuropsychol.
21. Dessi F, Maillet D, Metivet E, et al. Assessment of episodic
memory in illiterate elderly [in French]. Psychol Neuropsychiatr
Vieil. 2009;7(4):287-296.
22. Folstein MF, Folstein SE, McHugh PR. Mini-Mental State. A
practical method for grading the cognitive state of patients for the
clinician. J Psychiatr Res. 1975;12(3):189-198.
23. Kalafat M, Hugonot-Diener L, Poitrenaud J. Standardization of
the Mini Mental State (MMS), GRECOs version [in French]. Rev
Neuropsycol. 2003;13(2):209-236.
24. Snodgrass JG, Vanderwart M. A standardized set of 260 pictures:
norms for name agreement, image agreement, familiarity, and
visual complexity. J Exp Psychol Hum Learn. 1980;6(2):174-215.
25. Fuld PA, Katzman R, Davies P, Terry RD. Intrusions as a sign of
Alzheimer dementia: chemical and pathological verification. Ann
Neurol. 1982;11(2):155-159.
26. Kawas CH, Corrada MM, Brookmeyer R, et al. Visual memory
predicts Alzheimers disease more than a decade before diagno-
sis. Neurology. 2003;60(7):1089-1093.
27. Lindeboom J, Schmand B, Tulner L, Walstra G, Jonker C. Visual
association test to detect early dementia of the Alzheimer type.
J Neurol Neurosurg Psychiatry. 2002;73(2):126-133.
28. Sarazin M, Chauvire V, Gerardin E, et al. The amnestic syndrome
of hippocampal type in Alzheimers disease: an MRI study.
J Alzheimers Dis. 2010;22(1):285-294.
29. Alladi S, Xuereb J, Bak T, et al. Focal cortical presentations of
Alzheimers disease. Brain. 2007;130(pt 10):2636-2645.
30. Galton CJ, Patterson K, Xuereb JH, Hodges JR. Atypical and
typical presentations of Alzheimers disease: a clinical, neurop-
sychological, neuroimaging and pathological study of 13 cases.
Brain. 2000;123(pt 3):484-498.
Maillet et al
31. Grober E, Buschke H. Genuine memory deficits in dementia. Dev
Neuropsychol. 1987;3(1):13-36.
32. Delgado C, Munoz-Neira C, Soto A, et al. Comparison of the
psychometric properties of the Word and Picture versions of 47. LoGiudice D, Smith K, Thomas J, et al. Kimberley indigenous
the Free and Cued Selective Reminding Test in a Spanish-speaking
cohort of patients with Alzheimers disease and cognitively healthy
controls. Arch Clin Neuropsychol. 2016;31(2):165-175.
33. Zimmerman ME, Katz MJ, Wang C, et al. Comparison of Word
and Picture version of the Free and Cued Selective Reminding
Test (FCSRT) in older adults. Alzheimers Dement (Amst). 2015;
1(1):94-100.
34. Hoogendam YY, Hofman A, van der Geest JN, van der Lugt A,
Ikram MA. Patterns of cognitive function in aging: the Rotterdam
study. Eur J Epidemiol. 2014;29(2):133-140.
35. Corrada MM, Brookmeyer R, Paganini-Hill A, Berlau D, Kawas
CH. Dementia incidence continues to increase with age in the
oldest old: the 90 study. Ann Neurol. 2010;67(1):114-121.
36. Helmer C, Pasquier F, Dartigues JF. Epidemiology of Alzheimer
disease and related disorders [in French]. Med Sci(Paris). 2006;
22(3):288-296.
37. Miller LS, Mitchell MB, Woodard JL, et al. Cognitive perfor-
mance in centenarians and the oldest old: norms from the Georgia
Centenarian Study. Neuropsychol Dev Cogn B Aging Neuropsy-
chol Cogn. 2010;17(5):575-590.
38. Ivnik R, Malec JF, Smith GE, Tangalos EG, Peterson RC. Neu-
ropsychological tests norms above age 55: COWAT, BNT, MAE
Token, WRAT-R Reading, AMNART, STROOP, TMT, and JLO.
Clin Neuropsychol. 1996;10(3):262-278.
39. Kozora E, Cullum CM. Generative naming in normal aging: total
output and qualitative changes using phonemic and semantic con-
straints. Clin Neuropsychol. 1995;9(4):313-320.
40. Giulioli C, Meillon C, Gonzalez-Colaco Harmand M, Dartigues
JF, Amieva H. Normative scores for standard neuropsychological
tests in the oldest old from the French population-based PAQUID
study. Arch Clin Neuropsychol. 2016;31(1):58-65.
41. Ganguli M, Ratcliff G, Chandra V, et al. A Hindi version of the
MMSE: the development of a cognitive screening instrument for a
largely illiterate rural elderly population in India. Int J Geriatr
Psychiatry. 1995;10(5):367-377.
42. Mathuranath PS, Hodges HR, Mathew R, Cherian JP, George A,
Bak TH. Adaptation of the ACE for a Malayalam speaking pop-
ulation in southern India. Int J Geriatr Psychiatry. 2004;19(12):
1188-1194.
43. Xu G, Meyer JS, Huang Y, Du F, Chowdhury M, Quach M.
Adapting Mini-Mental State Examination for dementia screening
among illiterate or minimally educated elderly Chinese. Int J
Geriatr Psychiatry. 2003;18(7):609-616.
44. Ostrosky-Solis F, Esther Gomez-Perez M, Matute E, Rosselli M,
Ardila A, Pineda D. Neuropsi Attention and Memory: a neurop-
sychological test battery in Spanish with norms by age and edu-
cational level. Appl Neuropsychol. 2007;14(3):156-170.
45. Bellaj T, Ben Jemaa S, Attia-Romdhane N, et al. Mini Mental
State Examination (A-MMSE): fidelity, validity and normative
data [in French]. Tunis Med. 2008;86(7):768-776.
7
46. Toure K, Coume M, Ndiaye NND, et al. The test of Senegal:
a valid and reliable screening tool to assess for dementia in a
Senegalese elderly population. Afr J Neurol Sci. 2008;27(1):4-13.
cognitive assessment tool (KICA): development of a cognitive
assessment tool for older indigenous Australians. Int Psychoger-
iatr. 2006;18(2):269-280.
48. Dansilio S, Charamelo A. Constructional functions and figure
copying in illiterates or low-schooled Hispanics. Arch Clin
Neuropsychol. 2005;20(8):1105-1112.
49. Brucki SM, Nitrini R. Cancellation task in very low educated
people. Arch Clin Neuropsychol. 2008;23(2):139-147.
50. Paula JJ, de Avila RT, de Souza Costa D, et al. Assessing pro-
cessing speed and executive functions in low educated older
adults: the use of the Five Digits Test in patients with Alzheimers
disease, mild cognitive impairment and major depressive disor-
ders. Clin Neuropsychiatry. 2011;8(6):339-346.
51. Paula JJ, Costa MV, Andrade GF, Avila RT, Malloy-Diniz LF.
Validity and reliability of a simplified version of the Taylor
Complex Figure Test for the assessment of older adults with low
formal education. Dement Neuropsychol. 2016;10(1):52-57.
52. Cassimiro L, Fuentes D, Nitrini R, Yassuda MS. Decision-making
in cognitively unimpaired illiterate and low-educated older
women: results on the Iowa gambling task. Arch Clin Neuropsy-
chol. 2017;32(1):71-80.
53. Bertolucci PHF, Okamoto IH, Brucki SMD, Siviero MO,
Toniolo Neto J, Ramos LR. Applicability of the CERAD neu-
ropsychological battery to Brazilian elderly. Arq Neuropsi-
quiatr. 2001;59(3-A):532-536.
54. Mokri H, Matharan F, Peres K, et al. The Goblets test: norms in
the elderly population and properties in the detection of cognitive
impairment in elderly individuals selected from the general
population. Rev Neurol (Paris). 2013;169(11):871-878.
55. Noel M, Dumez K, Recher C, Luyat M, Dujardin S. Assessment
of verbal episodic memory by a new memory test with self-
initiated items (MAI test) [in French]. Geriatr Psychol Neurop-
sychiatr Vieil. 2014;12(4):440-447.
56. Vanderaspoilden V, Nury D, Frisque J, Peigneux P. The Brumory
test, an incidental long-term memory task designed for foreign,
non-French-speaking people with low educational level [in
French]. Rev Neurol(Paris). 2015;171(12):876-881.
57. Harciarek M, Cosentino S. Language, executive function and
social cognition in the diagnosis of frontotemporal dementia syn-
dromes. Int Rev Psychiatry. 2013;25(2):178-196.
58. Possin KL, Feigenbaum D, Rankin KP, et al. Dissociable execu-
tive functions in behavioral variant frontotemporal and Alzheimer
dementias. Neurology. 2013;80(24):2180-2185.
59. Elamin M, Pender N, Hardiman O, Abraham S. Social cognition
in neurodegenerative disorders: a systematic review. J Neurol
Neurosurg Psychiatry. 2012;83(11):1071-1079.
60. Narme P, Mouras H, Roussel M, Devendeville A, Godefroy O.
Assessment of socioemotional processes facilitates the distinction
between frontotemporal lobar degeneration and Alzheimers dis-
ease. J Clin Exp Neuropsychol. 2013;35(7):728-744.